error when using option --distortion-group-merge

[jhauneuro]

Summary

When testing the latest version of qsiprep I get an error when using this option: --distortion-group-merge average.

Additional details

• QSIPrep version: 0.22.0
• Docker version: 25.0.4

What were you trying to do?

Preprocess my dwi data.

What did you expect to happen?

Successfully complete preprocessing of my dwi data.

What actually happened?

I received an error.

Reproducing the bug

Here is the command I ran:

docker run -ti --rm --gpus all \
	-v /usr/local/freesurfer/7.4.1/license.txt:/opt/freesurfer/license.txt:ro \
	-v $bids_dir:/data:ro \
	-v $eddy_config_file:/sngl/eddy/eddy_config.json:ro \
	-v $out_dir:/out \
	-v $out_dir:/scratch \
	qsiprep:latest /data /out participant \
	--eddy-config /sngl/eddy/eddy_config.json \
	--output-resolution $res --pepolar-method TOPUP \
	--distortion-group-merge average \
	--bids-database-dir $bids_dir \
	-w /scratch --omp-nthreads 1 --nthreads 1 --participant_label $subjID

Error message:

Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/opt/conda/envs/qsiprep/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/cli/workflow.py", line 160, in build_workflow
    retval["workflow"] = workflow_builder()
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/workflows/base.py", line 82, in init_qsiprep_wf
    single_subject_wf = init_single_subject_wf(subject_id)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/workflows/base.py", line 295, in init_single_subject_wf  
    merging_group_workflows[merged_group] = init_distortion_group_merge_wf(
TypeError: init_distortion_group_merge_wf() got an unexpected keyword argument 'template'

[DaveOC90]

I have also had this error, with QSIPrep 0.22.0 and Docker version 24.0.7

[julfou81]

Same error here with qsiprep v 0.22.0 and Apptainer version 1.1.6-1.el7.

[julfou81]

I want to mention that the same error (TypeError: init_distortion_group_merge_wf() got an unexpected keyword argument 'template') occurs also with --distortion-group-merge concat. The same dataset was successfully processed with qsiprep 0.20.0 with the argument --distortion-group-merge average.
QSIPREP v 0.22.0 seems to go further with --distortion-group-merge average. I will keep you posted.

[mattcieslak]

would anyone be willing to test out a patch? I don't have any test data on hand where you can use this feature.

[julfou81]

I would be willing to try out, I have test data on hand. Could this patch be tested as a singularity image?

[jhauneuro]

@mattcieslak I'd be happy to test it out on my data.

[mattcieslak]

If you follow the instructions here you can clone the qsiprep repo and check out the fix/distortion_group_merge branch. I think this will fix it

[julfou81]

Thank you @mattcieslak for the fix and the instructions! I am not very good at the GitHub repo manipulation.

Here is what I did:

1. Download the fixed repo from qsiprep to my $HOME directory:

git clone https://github.com/PennLINC/qsiprep.git --branch fix/distortion_group_merge

2. Launch my singularity call in this way:

singularity run -e --containall --writable-tmpfs --cleanenv -B /scratch/jsein/BIDS:/work,$HOME/.templateflow:/opt/templateflow,$HOME/qsiprep/qsiprep:/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep \
 --nv /scratch/jsein/my_images/qsiprep-0.22.1.sif  /work/$study  \
 /work/$study/derivatives/qsiprep24 participant --participant_label $sub     \
 -w /work/temp_data24_${study} --output-resolution 1.8  --fs-license-file /work/freesurfer/license.txt \
  --eddy-config /work/$study/code/qsiprep/eddy_params24.json   \
 --b0-threshold 50 --unringing-method mrdegibbs  --denoise-method dwidenoise  \
 --distortion-group-merge average  

Unfortunately I got this error:

Process Process-2:
Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/opt/conda/envs/qsiprep/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/cli/workflow.py", line 48, in build_workflow
    from ..reports.core import generate_reports
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/reports/core.py", line 25, in <module>
    from nireports.assembler.report import Report
ModuleNotFoundError: No module named 'nireports'

[mattcieslak]

Interesting- I wonder if the nireports dependency is only in the unstable tag. Could you try using pennlinc/qsiprep:unstable to build the singularity image?

[julfou81]

Thank you for those suggestions. I did build the singularity image with the unstable tag.

Here is the command I launched:

singularity run -e --containall --writable-tmpfs --cleanenv -B /scratch/jsein/BIDS:/work \
 --nv /scratch/jsein/my_images/qsiprep-unstable.sif  /work/$study  \
 /work/$study/derivatives/qsiprep_unstable participant --participant_label $sub     \
 -w /work/temp_data24_${study} --output-resolution 1.8  --fs-license-file /work/freesurfer/license.txt \
  --eddy-config /work/$study/code/qsiprep/eddy_params24.json   \
 --b0-threshold 50 --unringing-method mrdegibbs  --denoise-method dwidenoise  \
 --distortion-group-merge average 

Without the patch, I got the same error:

Process Process-2:
Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/opt/conda/envs/qsiprep/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/cli/workflow.py", line 122, in build_workflow
    retval["workflow"] = init_qsiprep_wf()
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/workflows/base.py", line 82, in init_qsiprep_wf
    single_subject_wf = init_single_subject_wf(subject_id)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/workflows/base.py", line 303, in init_single_subject_wf
    merging_group_workflows[merged_group] = init_distortion_group_merge_wf(
TypeError: init_distortion_group_merge_wf() got an unexpected keyword argument 'template'

And the output log was saying:

subject is CTR09
base is /scratch/jsein/BIDS/EcriPark
study is EcriPark
bids-validator@1.8.4

	�[33m1: [WARN] The recommended file /README is very small. Please consider expanding it with additional information about the dataset. (code: 213 - README_FILE_SMALL)�[39m

�[36m	Please visit https://neurostars.org/search?q=README_FILE_SMALL for existing conversations about this issue.�[39m

        �[34m�[4mSummary:�[24m�[39m                   �[34m�[4mAvailable Tasks:�[24m�[39m        �[34m�[4mAvailable Modalities:�[24m�[39m 
        4030 Files, 88.42GB        ecriture                MRI                   
        72 - Subjects                                                            
        3 - Sessions                                                             


�[36m	If you have any questions, please post on https://neurostars.org/tags/bids.�[39m
241021-20:48:58,367 nipype.workflow IMPORTANT:
	 Running QSIPrep version 0.23.1.dev4+g2967323.d20241011

         License NOTICE ##################################################
         QSIPrep 0.23.1.dev4+g2967323.d20241011
         Copyright The PennLINC Developers.
         
         This product includes software developed by
         the NiPreps Community (https://nipreps.org/).
         
         Portions of this software were developed at the Department of
         Psychology at Stanford University, Stanford, CA, US.
         
         This software is also distributed as a Docker container image.
         The bootstrapping file for the image ("Dockerfile") is licensed
         under the MIT License.
         
         This software may be distributed through an add-on package called
         "Docker Wrapper" that is under the BSD 3-clause License.
         #################################################################
241021-20:49:00,203 nipype.workflow IMPORTANT:
	 Building QSIPrep's workflow:
           * BIDS dataset path: /work/EcriPark.
           * Participant list: ['CTR09'].
           * Run identifier: 20241021-204150_67e5b85d-2e02-4d65-a10e-db912e494ff8.
241021-20:49:00,275 nipype.workflow INFO:
	 Running nonlinear normalization to template
241021-20:49:00,303 nipype.workflow INFO:
	 Combining all dwi files within each available session:
241021-20:49:00,303 nipype.workflow INFO:
	 	- 2 scans in session 01
241021-20:49:00,334 nipype.workflow INFO:
	 [{'dwi_series': ['/work/EcriPark/sub-CTR09/ses-01/dwi/sub-CTR09_ses-01_dir-PA_dwi.nii.gz'], 'dwi_series_pedir': 'j', 'fieldmap_info': {'suffix': 'rpe_series', 'rpe_series': ['/work/EcriPark/sub-CTR09/ses-01/dwi/sub-CTR09_ses-01_dir-AP_dwi.nii.gz']}, 'concatenated_bids_name': 'sub-CTR09_ses-01'}]

[julfou81]

With the patch, launching the command with the unstable image this way:

singularity run -e --containall --writable-tmpfs --cleanenv -B /scratch/jsein/BIDS:/work,$HOME/.templateflow:/opt/templateflow,$HOME/qsiprep/qsiprep:/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep \
 --nv /scratch/jsein/my_images/qsiprep-unstable.sif  /work/$study  \
 /work/$study/derivatives/qsiprep_unstable participant --participant_label $sub     \
 -w /work/temp_data24_${study} --output-resolution 1.8  --fs-license-file /work/freesurfer/license.txt \
  --eddy-config /work/$study/code/qsiprep/eddy_params24.json   \
 --b0-threshold 50 --unringing-method mrdegibbs  --denoise-method dwidenoise  \
 --distortion-group-merge average  \

Here is the error I got:

Process Process-2:
Traceback (most recent call last):
  File "/opt/conda/envs/qsiprep/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/opt/conda/envs/qsiprep/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/cli/workflow.py", line 122, in build_workflow
    retval["workflow"] = init_qsiprep_wf()
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/workflows/base.py", line 82, in init_qsiprep_wf
    single_subject_wf = init_single_subject_wf(subject_id)
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/workflows/base.py", line 303, in init_single_subject_wf
    merging_group_workflows[merged_group] = init_distortion_group_merge_wf(
  File "/opt/conda/envs/qsiprep/lib/python3.10/site-packages/qsiprep/workflows/dwi/distortion_group_merge.py", line 193, in init_distortion_group_merge_wf
    SeriesQC(output_file_name=config.execution.output_prefix), name="series_qc"
AttributeError: type object 'execution' has no attribute 'output_prefix'

[samtorrisi]

i also built the unstable version yesterday and get the same
AttributeError: type object 'execution' has no attribute 'output_prefix'
error