THINGS behavioural & eye-tracking data analyses

Image Recognition Performance Metrics

Metrics of behavioural performance on CNeuroMod-THINGS continuous image recognition task (hits, false alarm, d', reaction times) are computed per run, per session (6 runs except for session 1) and per subject (across all sessions) from run-specific *events.tsv files.

Scores are saved as .tsv files in sub-0*/beh. Output columns are described in task-things_beh.json

Launch the following script to process all subjects & sessions

DATADIR="cneuromod-things/THINGS/fmriprep/sourcedata/things"
OUTDIR="cneuromod-things/THINGS/behaviour"

python code/behav_data_memoperformance.py --idir="${DATADIR}" --odir="${OUTDIR}" --clean

Note: a handful of sessions had their planned patterns of repetition disrupted by a few sessions administered out of order (sub-03's sessions 24, 25 and 26; sub-06's sessions 19 to 26 inclusively). These sessions included "atypical trials" (e.g., images shown more than 3 times), and were excluded from the computation of behavioural performance metrics (they were flagged with not_for_memory == True in their *events.tsv file).

Input:

• All four subjects' *events.tsv files, across sessions (~36) and runs (6 per session), e.g., sub-03_ses-17_task-thingsmemory_run-2_events.tsv

Output:

• na_report.txt, a text file that lists every run for which at least one behavioural response is missing (no button press recorded for at least one trial)
• sub-0*/beh/sub-0*_task-things_desc-perTrial_beh.tsv, a concatenation of all events.tsv trials (excludes all trials with no button press). Columns and their values correspond to those of events.tsv files, as described in cneuromod-things/THINGS/fmriprep/sourcedata/things/task-things_events.json
• sub-0*/beh/sub-0*_task-things_desc-perRun_beh.tsv, performance metrics per run (excludes runs from session 1). Columns are described in task-things_beh.json
• sub-0*/beh/sub-0*_task-things_desc-perSession_beh.tsv, performance metrics per session (includes session 1). Columns are described in task-things_beh.json
• sub-0*/beh/sub-0*_task-things_desc-global_beh.tsv, subject's overall performance metrics on the entire task (excludes session 1 in the averaging). Columns are described in task-things_beh.json

Note: By default, the script derives metrics of performance from the first key press logged for a given trial. To derive metrics from the last logged key press instead (which accounts for subject self-correction), use the lastkp flag like this: python code/behav_data_memoperformance.py --idir="${DATADIR}" --odir="${OUTDIR}" --clean --lastkp.

---

Trial-Wise Image Ratings and Annotations

Image ratings from the THINGS and THINGSplus datasets, and manual image annotations produced specifically for CNeuroMod-THINGS, are assigned to each trial to perform representation analyses. Annotated trials are outputted as .tsv files per subject under sub-0*/beh.

The following image annotation files were downloaded from the THINGS object concept and object image database, and saved directly under cneuromod-things/THINGS/fmriprep/sourcedata/things/stimuli/annotations/THINGSplus:

• THINGS/things_concepts.tsv
• THINGSplus/Metadata/Concept-specific/arousal_meanRatings.tsv
• THINGSplus/Metadata/Concept-specific/category53_wideFormat.tsv
• THINGSplus/Metadata/Concept-specific/objectProperties_meanRatings.tsv
• THINGSplus/Metadata/Concept-specific/size_meanRatings.tsv
• THINGSplus/Metadata/Image-specific/imageLabeling_imageWise.tsv
• THINGSplus/Metadata/Image-specific/imageLabeling_objectWise.tsv

See the THINGSplus preprint for more information about these annotations.

Launch the following script to process a subject's sessions

EVDIR="cneuromod-things/THINGS/fmriprep/sourcedata/things"
ANDIR="cneuromod-things/THINGS/fmriprep/sourcedata/things/stimuli/annotations"
OUTDIR="cneuromod-things/THINGS/behaviour"

python code/behav_data_annotate.py --events_dir="${EVDIR}" --annot_dir="${ANDIR}" --out_dir="${OUTDIR}" --sub="01"

Input:

• A subject's *events.tsv files, across sessions (~36) and runs (6 per session), e.g., sub-03_ses-17_task-thingsmemory_run-2_events.tsv
• Various annotation files saved under stimuli/annotations. E.g., task-things_desc-manual_annotation.tsv

Output:

• sub-0*/beh/sub-0*_task-things_desc-perTrial_annotation.tsv, trialwise annotations for a concatenation of all events.tsv files. Columns are described in task-things_desc-perTrial_annotation.json
• sub-0*/beh/sub-0*_task-things_catNum.tsv, list of categories of images shown to the subject throughout the experiment
• sub-0*/beh/sub-0*_task-things_imgNum.tsv, list of image numbers for images shown to the subject throughout the experiment

---

Fixation Compliance Metrics

The script code/analyze_fixations.py processes eye-tracking data and its derivatives. It concatenates trial-wise fixation compliance metrics across all runs. It also converts drift-corrected gaze from normalized scores (as proportions of the projector screen) into degrees of visual angles (relative distance in x and y from central fixation mark, and absolute distance from central fixation) for trials with recorded button presses (gaze points are filtered above a set threshold of pupil detection confidence and then downsampled to 1 every 5 points, corresponding to a maximum of 50 Hz).

DATADIR="path/to/cneuromod-things"

python code/analyze_fixations.py --sub_num="01" --data_dir="${DATADIR}"

Input:

• A subject's sub-*_ses-*_task-things_run-*_events.tsv files, across sessions (~36) and runs (6 per session)
• sub-0*_ses-*_task-things_run-*_eyetrack.tsv.gz, files of drift-corrected gaze per run

Output:

• sub-0*_task-things_desc-driftCor_gaze.tsv, drift-corrected gaze (in degrees of visual angle from the central fixation mark) sampled during the image-viewing portion of each trials, concatenated across all runs. Includes 1 out of 5 gaze points derived from pupils detected above a set confidence threshold, from trials for which an answer (button press) was recorded.
• sub-0*_task-things_desc-fixCompliance_statseries.tsv, trial-wise metrics of fixation compliance concatenated across all events.tsv files (all runs). Columns are described in task-things_desc-fixCompliance_statseries.json