run xcp-d on hcp datasets can only process one subject #1471
Description
Summary
In the xcp-d/ingression/hcpya.py, engine create dataset_description.json at the end of function: convert_hcp_to_bids_single_subject(), but engine check dataset_description.json at the beginning of function: convert_hcp_to_bids_single_subject() and quit if it exist.
Additional details
- xcp_d version: 0.11.0rc1
- Docker version:
- Apptainer version:
What were you trying to do?
I had 800+ subjects download from HCP datasets in F:\hcp_datasets\raw\, and I run docker run --rm -it -v "F:\hcp_datasets\raw\:/home/hcp/raw" -v "F:\hcp_datasets\xcpd_output\:/home/hcp/xcpd_output" pennlinc/xcp_d:latest /home/hcp/raw /home/hcp/xcpd_output participant --mode "linc" --input_type "hcp" --atlases Tian
What did you expect to happen?
Engine can process every sunject in the folder correctly.
What actually happened?
Engine can only process one subject, and then log: 'Converted dataset already exists. Skipping conversion.') and quit every other subjects processing pipeline.
Reproducing the bug
`dataset_description.json` should create after this for loop:
`xcp-d/ingression/hcpya.py line:191`
for subject_id in participant_ids:
LOGGER.info(f'Converting {subject_id}')
convert_hcp_to_bids_single_subject(
in_dir=in_dir,
out_dir=out_dir,
sub_ent=subject_id,
)
and delete the following two line:
`xcp-d/ingression/hcpya.py line:370`
if not os.path.isfile(dataset_description_fmriprep):
write_json(dataset_description_dict, dataset_description_fmriprep)