Documentation for field map file layout and IntendedFor field.

[arokem]

Summary

@maouw and I are trying to run qsiprep on a dataset with AP/PA scans, where one scan (the AP) are 10 b=0 volumes to be used for SDC. We are encountering a mysterious error.

Additional details

• QSIPrep version: v1.0.2.dev0+gfc89945.d20250405, run with singularity.

What were you trying to do?

Running qsiprep with:

../qsiprep-latest.sif --skip-bids-validation --b0-threshold 50 --output-resolution 1 --participant-label=006423 wd o4 participant

What actually happened?

We get the following stack trace and the program terminates without producing the expected outputs:

Node: qsiprep_1_0_wf.sub_006423_ses_1_wf.dwi_preproc_ses_1_wf.pre_hmc_wf.merge_minus.merge_dwis
Working directory: /mmfs1/gscratch/escience/altan/proj/chiv/scaffold/work/qsiprep_1_0_wf/sub_006423_ses_1_wf/dwi_preproc_ses_1_wf/pre_hmc_wf/merge_minus/merge_dwis

Node inputs:

b0_refs = <undefined>
b0_threshold = 50
bids_dwi_files = ['/mmfs1/gscratch/escience/altan/proj/chiv/scaffold/wd/sub-006423/ses-1/dwi/sub-006423_ses-1_dir-AP_dwi.nii.gz']
bval_files = ['/mmfs1/gscratch/escience/altan/proj/chiv/scaffold/wd/sub-006423/ses-1/dwi/sub-006423_ses-1_dir-AP_dwi.bval']
bvec_files = ['/mmfs1/gscratch/escience/altan/proj/chiv/scaffold/wd/sub-006423/ses-1/dwi/sub-006423_ses-1_dir-AP_dwi.bvec']
carpetplot_data = <undefined>
denoising_confounds = ['/mmfs1/gscratch/escience/altan/proj/chiv/scaffold/work/qsiprep_1_0_wf/sub_006423_ses_1_wf/dwi_preproc_ses_1_wf/pre_hmc_wf/merge_minus/dwi_denoise_ses_1_dir_AP_dwi_wf/hstack_confounds/confounds.csv']
dwi_files = ['/mmfs1/gscratch/escience/altan/proj/chiv/scaffold/work/qsiprep_1_0_wf/sub_006423_ses_1_wf/dwi_preproc_ses_1_wf/pre_hmc_wf/merge_minus/dwi_denoise_ses_1_dir_AP_dwi_wf/denoiser/sub-006423_ses-1_dir-AP_dwi_denoised.nii.gz']
harmonize_b0_intensities = True
raw_concatenated_files = <undefined>
scan_metadata = {'/mmfs1/gscratch/escience/altan/proj/chiv/scaffold/wd/sub-006423/ses-1/dwi/sub-006423_ses-1_dir-AP_dwi.nii.gz': {'AcquisitionDuration': 48.1512, 'AcquisitionMatrixPE': 109, 'AcquisitionNumber': 6, 'AcquisitionTime': '14:42:17.330000', 'BidsGuess': ['dwi', '_acq-SKSEDwiSEp13_run-601_dwi'], 'BodyPart': 'BRAIN', 'CoilString': 'MULTI COIL', 'ConversionSoftware': 'dcm2niix', 'ConversionSoftwareVersion': 'v1.0.20250505', 'Dcm2bidsVersion': '3.2.0', 'DeidentificationMethodCodeSequence': [{'CodeValue': '3', 'CodingSchemeDesignator': 'XNAT', 'CodingSchemeVersion': '1.0', 'CodeMeaning': 'XNAT DicomEdit 6 Script'}], 'DeviceSerialNumber': '45179', 'EchoTime': 0.08, 'EchoTrainLength': 85, 'EstimatedEffectiveEchoSpacing': 0.000563042, 'EstimatedTotalReadoutTime': 0.0624976, 'FlipAngle': 90, 'FrequencyEncodingSteps': 112, 'ImageOrientationPatientDICOM': [1, 0, 0, 0, 1, 0], 'ImageType': ['ORIGINAL', 'PRIMARY', 'DIFFUSION', 'NONE', 'MAGNITUDE'], 'ImagingFrequency': 127.766704, 'InPlanePhaseEncodingDirectionDICOM': 'COL', 'InstitutionAddress': '1959 NE Pacific  Street', 'InstitutionName': 'University of washington', 'InstitutionalDepartmentName': 'MRI', 'MRAcquisitionType': '2D', 'MTState': False, 'MagneticFieldStrength': 3, 'Manufacturer': 'Philips', 'ManufacturersModelName': 'Ingenia Elition X', 'Modality': 'MR', 'ParallelAcquisitionTechnique': 'MBSENSE', 'ParallelReductionFactorInPlane': 1.3, 'ParallelReductionFactorOutOfPlane': 3, 'PartialFourierDirection': 'PHASE', 'PartialFourierEnabled': 'YES', 'PatientPosition': 'HFS', 'PercentPhaseFOV': 100, 'PercentSampling': 98.2143, 'PhaseEncodingAxis': 'j', 'PhaseEncodingDirection': 'j-', 'PhaseEncodingStepsNoPartialFourier': 109, 'PhaseEncodingStepsOutOfPlane': 1, 'PhilipsRescaleIntercept': 0, 'PhilipsRescaleSlope': 0.815873, 'PhilipsScaleSlope': 0.0741405, 'PixelBandwidth': 2326.17, 'ProcedureStepDescription': 'CHIV', 'ProtocolName': 'WIP DTI_10b0_BlipA', 'PulseSequenceName': 'DwiSE', 'ReconMatrixPE': 112, 'RepetitionTime': 4, 'ScanOptions': 'FS', 'ScanningSequence': 'SE', 'SequenceVariant': 'SK', 'SeriesDescription': 'DTI_10b0_BlipA', 'SeriesNumber': 601, 'SliceThickness': 2, 'SoftwareVersions': '5.7.1\\5.7.1.2', 'SpacingBetweenSlices': 2, 'StationName': 'PHILIPS-2JLMOH3', 'StudyDescription': 'CHIV', 'TablePosition': [0, 0, 0.6], 'UsePhilipsFloatNotDisplayScaling': 1, 'WaterFatShift': 27.4688}}

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 66, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 525, in run
    result = self._run_interface(execute=True)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 643, in _run_interface
    return self._run_command(execute)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 769, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node merge_dwis.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 401, in run
	    runtime = self._run_interface(runtime)
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/interfaces/dwi_merge.py", line 76, in _run_interface
	    to_concat, b0_means, corrections = harmonize_b0s(
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/interfaces/dwi_merge.py", line 716, in harmonize_b0s
	    harmonized_niis.append(math_img(f'img*{correction:.32f}', img=nii_img))
	  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/nilearn/image/image.py", line 1058, in math_img
	    result = eval(formula, data_dict)
	  File "<string>", line 1, in <module>
	NameError: ("Input formula couldn't be processed, you provided 'img*nan',", "name 'nan' is not defined")

FWIW, the input data does not have any NaNs in it.

Are we supposed to somehow tell qsiprep that the AP scans are our reverse phase encoded scans through the metadata or in the call to qsiprep?

[arokem]

An update is that following a conversation with @kellychang4, we organized the data as follows:

.
└── ses-1
    ├── anat
    │   ├── sub-006423_ses-1_T1w.json
    │   └── sub-006423_ses-1_T1w.nii.gz
    ├── dwi
    │   ├── sub-006423_ses-1_dir-AP_dwi.bval
    │   ├── sub-006423_ses-1_dir-AP_dwi.bvec
    │   ├── sub-006423_ses-1_dir-AP_dwi.json
    │   └── sub-006423_ses-1_dir-AP_dwi.nii.gz
    └── fmap
        ├── sub-006423_ses-1_dir-PA_dwi.bval
        ├── sub-006423_ses-1_dir-PA_dwi.bvec
        ├── sub-006423_ses-1_dir-PA_dwi.json
        └── sub-006423_ses-1_dir-PA_dwi.nii.gz

And added an IntendedFor field in the sidecar for the json file, running the call to qsiprep with the --distortion-group-merge none flag, but we're still getting the same outcome. Same for version 1.0.1, which we also tried.

[arokem]

Looks like we should have kept our BIDS validation on. On some further experiments (this time without skipping BIDS validation), we found that the right way to organize the files is:

.
└── ses-1
    ├── anat
    │   ├── sub-006423_ses-1_T1w.json
    │   └── sub-006423_ses-1_T1w.nii.gz
    ├── dwi
    │   ├── sub-006423_ses-1_dir-AP_dwi.bval
    │   ├── sub-006423_ses-1_dir-AP_dwi.bvec
    │   ├── sub-006423_ses-1_dir-AP_dwi.json
    │   └── sub-006423_ses-1_dir-AP_dwi.nii.gz
    └── fmap
        ├── sub-006423_ses-1_dir-PA_epi.json
        └── sub-006423_ses-1_dir-PA_epi.nii.gz

That is, the reverse phase data was saved with the _epi suffix and without the b-values and b-vectors.

Also, the IntendedFor field was rather finicky. The only way we could make it pass validation is by making it relative to the subject directory (that is, something like "ses-1/dwi/sub-006423_ses-1_dir-AP_dwi.nii.gz")

Since this is a pretty common use-case, maybe a step-by-step description as above might be useful in the docs? I only preprocess data occasionally and have to relearn this every time I do it. At least I will now have this issue to refer back to 😄

[arokem]

OK - further question here: dcm2bids creates an IntendedFor field that looks like this:

"IntendedFor": "bids::sub-<###>/dwi/sub-<###>_dir-PA_dwi.nii.gz",

This validates just find in the website BIDS validator, but the validator in the qsiprep container raises:

1: [ERR] 'IntendedFor' field needs to point to an existing file. (code: 37 - INTENDED_FOR)
		./sub-018536/fmap/sub-<###>_dir-AP_epi.nii.gz
			Evidence: bids::sub-<###>/dwi/sub-<###>_dir-PA_dwi.nii.gz

And so on for every subject in the data. Does someone here understand why this passes validation in the web-based validator, but not in the qsiprep validator? Should I just skip BIDS validation, or will that come to bite me later on?

[arokem]

Looks like the latter is a duplication of #957 and #658, so perhaps just a warning somewhere in the documentation is needed? Altogether, I think that this issue is more about documentation than any features of the software (but it would be nice to have BIDS URIs work eventually).