hc3-original-docs/README.txt, Version 1.0, 2013-08-20 This directory contains the original metadata files which describe the hc3 data set. These original files should NOT be used as the main source of information about the hc3 data set because they have not been "cleaned up" and documented. Specifically, they contain extra information which is not documented, make reference to some data which is not included in the hc3 data set, and are harder to understand than the metadata tables provided with the data set. These are provided in case the extra non-documented information is useful and to document how the metadata tables provided with the data set were generated. Included in this directory are the original data files: KenjiData.mat and ElePosition.txt file. KenjiData.mat is a MatLab file. When loading into MatLab, you will see total　8569 cells there. Each row of PyrIntMap.Map, Clean, iCell{NN}, Region is the same cell. size(PyrIntMap.Map) = [8569 43]; What dose each column say? see PyrIntMap.content PyrIntMap.Map(:,18) = File ID, See /u12/kenji/data/ElePosition.txt, 4th column. PyrIntMap.Map(:,3) = electrode # PyrIntMap,Map(:,4) = clu # Map(:,5): number of cells excited by this cell. Map(:,6): number of cells inhibited by this cell. Stated again, contents of Map columns: > 1 file_ID file ID # > 2 cell_ID This is cell ID # in the session, starting from 1 to number of cells. Not useful at all. > 3 ele electrode # > 4 clu ID # in clu files > 5 exciting Based on cross-correlogram analysis, the cell monosynaptically excites other cells. > 6 inhibiting Based on cross-correlogram analysis, the cell monosynaptically inhibits other cells. > 7 excited Based on cross-correlogram analysis, the cell is monosynaptically excited other cells. > 8 inhibited Based on cross-correlogram analysis, the cell is monosynaptically excited other cells. > > > 19 RegionID > 20 AnimalID 20 corresponds to PhyIntMap.WhichFile. > 21 FireRate > 22 totalFireRate difference bwt 21 and 22? 21. MeanISI = mean(bootstrp(100,'mean',ISI)); FireRate = SampleRate/MeanISI ISI is interspike intervals. This is what i did but I'm not so sure if the second method is valid. 22. # of spikes divided by total recording length. > 43 Big_Which_Region(EC=1,CA1=2,CA3,DG=3) 19 and 43, correspond to PhyIntMap.whichRegion. ------------------- Region; 1=CA1,2=CA3,3=DG,4=EC2,5=EC3,6=EC5 Clean: 0=bad cells iCell; iCell{1}==1,pyr cells, iCell{2}==1,interneurons Beh 1st column: Merged session name 2nd column: Merged session name 3rd column: Name of HDD where you can find the data 4th column: Individual session name 5th column: behavior Mwheel: alternation task with wheel running. See Pastalkova et al., 2008 Open: open maze. Tmaze: t-maze. Zigzag: zigzag maze. See Royer et al., 2010 JNS. bigSquare: 180 cm by 180 cm bigSquarePlus: 180 cm by 180 cm, divided by plus shaped walls. circle: Circle maze. See Montgomery et al., 2008 linear: linear maze, 250 cm linearOne; linear maze LinearTwo: linear maze midSquare: 120 cm by 120 cm plus: plus maze. sleep wheel: Operant wheel running task, See Mizuseki et al., 2009 wheel_home: Homecage wheel running, animal run just for fun. 6th column: familiarity of the task. 1=animal did this task for the first time. 2=second time, 3=third time, ...., 10=10 times or more. 7th column: length recording in secound. 8th column: If this column is not empty, the session was to short or the animal did not do the task well, or video tape did not work. Please do not use those data. --------------------------- Also included is file: DecidePyrIntRegion.m It has code that was used to calculate values in iCell and Region. Contents of iCell are: iCell{1}==1,pyr cells, iCell{2}==1,interneurons iCell{3} = ~iCell{1} & ~iCell{2} iCell{4}: physiologically identified exciting cells based on CCG analysis. iCell{5}: physiologically identified inhibiting cells based on CCG analysis. Detailed method can be found in Mizuseki Sirota Pastalkova and Buzsaki., 2009 Neuron paper. ---------------- Comparing Region and ElePosition.txt DGCA3: not sure sure if the electrode is DG or CA3. Ctx: somewhere in the cortex (above the hippocampus) CA: somewhere in the hippocampus (we do not know if it is CA1, CA3 or DG) So basically these differences do not matter. -------------------- Data files removed: > 4) Following directories in Beh first column (and ElePosition.txt first column) but are not in the data set: > > KM011203, KM011204, KM011205, KM011206 > > gor01-6-8 > > km01.01, km01.02, km01.04, km01.05, km01.06, km01.07, km01.08, km01.09 > > Should the data in these directories be included in the data set? gor01-6-8 is Kamran Diba's data. Other data are not so informative. Please remove from the data list. ================================= From the above original metadata files and the files in the data set, and above information an SQLite data base was created which contains all the information in the originally provided files, loaded as tables into SQLite. hc3.db is the SQLite database containing the tables. tables.sql has the create table statements used to create tables in hc3.db. ---------- The following select statements were used to generate the tables which are the metadata tables (cell, session, epos, file) distributed with the data set. ---------------------- hc3-files.csv sqlite> .separator ',' sqlite> .mode csv sqlite> .output hc3-files.csv sqlite> select s.topdir, s.session, s.size, substr(ifnull(v.file,'-'),-3) as video_type, ifnull(v.size, 0) as video_size from hc3_session s left join video_files v on s.session=v.session order by s.topdir, s.session; sqlite> .quit jt-2:mlexport jt$ wc hc3-files.csv 442 442 17531 hc3-files.csv ---------------------- hc3-session.csv sqlite> .output hc3-session.csv sqlite> select b.id, b.topdir, b.session, b.behavior, b.familiarity, b.duration from beh b, hc3_session s where b.topdir=s.topdir and b.problem = '' order by b.id; sqlite> .quit jt-2:mlexport jt$ wc hc3-session.csv 6457 6457 270966 hc3-session.csv --------------------- hc3-cell.csv sqlite> .separator ',' sqlite> .mode csv sqlite> .output hc3-cell.csv sqlite> select m.id, e.topdir, t.animal, m.ele, m.clu, n.region_name, m.exciting, m.inhibiting, m.excited, m.inhibited, m.FireRate, m.totalFireRate, case (i.pyramidal*100+i.interneuron*10+i.not_pyin) when 100 then 'p' when 10 then 'i' when 1 then 'n' else 'error' end as celltype from pmMap m, elepos e, hc3_topdir t, region r, region_name n, iCell i, clean c where m.FileID=e.fileid and e.topdir=t.topdir and m.id=r.id and r.region_id=n.region_id and m.id=i.id and m.id=c.id and c.good=1 order by m.id; sqlite> .quit jt-2:mlexport jt$ wc hc3-cell.csv 7736 7736 570638 hc3-cell.csv jt-2:mlexport jt$ grep error hc3-cell.csv jt-2:mlexport jt$ sqlite3 hc3.db ------------------- epos sqlite> .output hc3-epos.csv sqlite> select e.topdir, t.animal, e1,e2,e3,e4,e5,e6,e7,e8,e9,e10,e11,e12,e13,e14,e15,e16 ...> from elepos e, hc3_topdir t ...> where e.topdir = t.topdir order by e.topdir; sqlite> .quit jt-2:mlexport jt$ wc hc3-epos.csv 119 119 8702 hc3-epos.csv