Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Region based nnunet scripts #66

Merged
merged 50 commits into from
Nov 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
b77ea9f
conversion to nnunet region-based format
plbenveniste Oct 9, 2023
a5f82a1
modified for training or inference
plbenveniste Oct 13, 2023
00e437f
added exclusion based on exclude.yml file
plbenveniste Oct 13, 2023
ea37122
created file for evaluation of lesion seg pred
plbenveniste Oct 17, 2023
6548a4d
added anima function in script
plbenveniste Oct 17, 2023
ca5e645
added readme file for anima installation
plbenveniste Oct 17, 2023
e945a7e
changed folder structure and adding save to csv
plbenveniste Oct 18, 2023
87c3766
removed voxel labels above the first disc level
plbenveniste Oct 18, 2023
f3038d8
changed file termination to seg-manual and removed break
plbenveniste Oct 18, 2023
fb6c94e
added dictionnary for inference
plbenveniste Oct 18, 2023
24c83bb
changed encoding to np.int16 for masks for nnunet compatibility
plbenveniste Oct 18, 2023
2a7e39e
Created file for conversion from nnunet predictions to BIDS format
plbenveniste Oct 18, 2023
d6d870b
modified script to run on entire folder (not single prediction)
plbenveniste Oct 19, 2023
3149d73
removed two useless prints
plbenveniste Oct 20, 2023
373b0be
fixed selection of file in dictionnary problematic because of 000 at…
plbenveniste Oct 20, 2023
d62c7f5
created file for nnunet dataset generation with multiplication by -1 …
plbenveniste Oct 20, 2023
c00e7a9
fixed site value and added contrast value to dictionnary
plbenveniste Oct 20, 2023
edc3a50
create notebook to compare fold results
plbenveniste Oct 20, 2023
22965ca
finished analysis of first nnUnet predictions
plbenveniste Oct 21, 2023
dcc18d8
added analysis of 3d nnunet on multiplied -1 images
plbenveniste Oct 23, 2023
73a5fc5
fixed formatting for nnunet inference dataset (added 0000)
plbenveniste Oct 24, 2023
b8df433
added exclude file for M0 and M12 comparison
plbenveniste Oct 24, 2023
461d5e0
modified script to multiply inference images by -1 as well
plbenveniste Oct 24, 2023
2f042a6
updated with performances of 2d models
plbenveniste Oct 24, 2023
01e6303
added image with 4 dimensions
plbenveniste Oct 25, 2023
02bfc48
added flag for case files are not in imageTs
plbenveniste Oct 25, 2023
2950157
add binarization to lesion mask and sc mask
plbenveniste Oct 25, 2023
379e9bb
fixed typo in os command
plbenveniste Oct 25, 2023
586baee
added inference of fold 3 and 4 from 2d nnUNet mul -1
plbenveniste Oct 25, 2023
9c1bce5
added subjects only present at M0
plbenveniste Oct 25, 2023
bd58e1a
added file with parent folder but no PSIR file
plbenveniste Oct 25, 2023
45b901c
added subject sub-mon164 (only present at M12)
plbenveniste Oct 25, 2023
44b9e22
added final plots for abstract figures
plbenveniste Oct 26, 2023
89b35d0
new figures created
plbenveniste Oct 29, 2023
4d8f54a
formatted the plot
plbenveniste Oct 30, 2023
3a94315
formated reference
plbenveniste Oct 30, 2023
7fe19c3
finalised figures
plbenveniste Oct 31, 2023
b50f925
fixed label per site
plbenveniste Nov 2, 2023
5a6a516
added requirements file for nnunet
plbenveniste Nov 9, 2023
cac4d31
formatted convert to nnunet format file
plbenveniste Nov 9, 2023
ff58af7
formatted convert with mul psir
plbenveniste Nov 9, 2023
9a8e3ce
formatted convert predictions
plbenveniste Nov 9, 2023
446a88a
added readme for nnunet
plbenveniste Nov 9, 2023
328ebcb
fixed typo in nnunet readme
plbenveniste Nov 9, 2023
0dfca72
added sct installation in readme
plbenveniste Nov 9, 2023
ee705f3
formatted titles
plbenveniste Nov 9, 2023
6e7f06c
Update nnunet/README.md
plbenveniste Nov 10, 2023
2d94058
Update nnunet/README.md
plbenveniste Nov 10, 2023
37cb04c
Added venv creation for nnunet
plbenveniste Nov 10, 2023
a4b20f4
added comment about train ratio
plbenveniste Nov 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
171 changes: 171 additions & 0 deletions exclude_M0_M12_comparison.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
# PSIR
- sub-mon118_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/43
- sub-mon006_ses-M0 # FOV covers the whole brain. The SC has only C1-C6; see https://github.com/ivadomed/canproco/issues/22#issue-1568568207
- sub-mon009_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1758684880
- sub-mon032_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1758686509
- sub-mon097_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1758686772
- sub-mon113_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1760175861
- sub-mon148_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1758686965
- sub-mon152_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1760248384
- sub-mon168_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1758687112
- sub-mon191_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1758687352
- sub-van134_ses-M0 # No disc labeling file : impossible to do it : https://github.com/ivadomed/canproco/issues/53#issuecomment-1761698648
- sub-van135_ses-M0 # No disc labeling file : impossible to do it : https://github.com/ivadomed/canproco/issues/53#issuecomment-1761702591
- sub-van171_ses-M0 # No disc labeling file : impossible to do it : https://github.com/ivadomed/canproco/issues/53#issuecomment-1761910863
- sub-van176_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1758687602
- sub-van181_ses-M0 # No disc labeling file : impossible to do it : https://github.com/ivadomed/canproco/issues/53#issuecomment-1767235494
- sub-van201_ses-M0 # No disc labeling file : impossible to do it : https://github.com/ivadomed/canproco/issues/53#issuecomment-1767238882
- sub-van206_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1758688851
- sub-van207_ses-M0 # No disc labeling file : impossible to do it : https://github.com/ivadomed/canproco/issues/53#issuecomment-1761973064
- sub-tor014_ses-M0 # Poor data quality and no disc labeling file; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1760420705
- sub-tor133_ses-M0 # Poor data quality; see https://github.com/ivadomed/canproco/issues/53#issuecomment-1758689928
# STIR
- sub-cal149_ses-M0 # FOV does not cover the whole SC in sagittal plane; see https://github.com/ivadomed/canproco/issues/22#issue-1568568207
# Similar accross timepoints
- sub-edm005_ses-M0
- sub-edm031_ses-M0
- sub-edm035_ses-M0
- sub-edm054_ses-M0
- sub-edm065_ses-M0
- sub-edm088_ses-M0
- sub-mon002_ses-M0
- sub-mon003_ses-M0
- sub-mon005_ses-M0
- sub-mon006_ses-M0
- sub-mon007_ses-M0
- sub-mon009_ses-M0
- sub-mon010_ses-M0
- sub-mon011_ses-M0
- sub-mon014_ses-M0
- sub-mon031_ses-M0
- sub-mon032_ses-M0
- sub-mon041_ses-M0
- sub-mon052_ses-M0
- sub-mon060_ses-M0
- sub-tor010_ses-M0
- sub-tor012_ses-M0
- sub-tor014_ses-M0
- sub-tor016_ses-M0
- sub-tor021_ses-M0
- sub-tor029_ses-M0
- sub-tor036_ses-M0
- sub-tor038_ses-M0
- sub-tor039_ses-M0
- sub-tor043_ses-M0
- sub-tor044_ses-M0
- sub-tor051_ses-M0
# Subjects only present at M12
- sub-cal064_ses-M12
- sub-edm161_ses-M12
- sub-mon164_ses-M12
# Subject with 4 dimensions
- sub-cal123_ses-M12
# Subject only present at M0
- sub-cal080_ses-M0
- sub-cal113_ses-M0
- sub-cal125_ses-M0
- sub-cal130_ses-M0
- sub-cal131_ses-M0
- sub-cal136_ses-M0
- sub-cal140_ses-M0
- sub-cal151_ses-M0
- sub-cal153_ses-M0
- sub-cal154_ses-M0
- sub-cal156_ses-M0
- sub-cal162_ses-M0
- sub-cal173_ses-M0
- sub-cal185_ses-M0
- sub-cal191_ses-M0
- sub-cal197_ses-M0
- sub-cal198_ses-M0
- sub-cal199_ses-M0
- sub-cal200_ses-M0
- sub-cal206_ses-M0
- sub-cal207_ses-M0
- sub-cal209_ses-M0
- sub-cal210_ses-M0
- sub-cal213_ses-M0
- sub-cal214_ses-M0
- sub-edm008_ses-M0
- sub-edm010_ses-M0
- sub-edm018_ses-M0
- sub-edm023_ses-M0
- sub-edm029_ses-M0
- sub-edm034_ses-M0
- sub-edm038_ses-M0
- sub-edm053_ses-M0
- sub-edm064_ses-M0
- sub-edm067_ses-M0
- sub-edm089_ses-M0
- sub-edm095_ses-M0
- sub-edm098_ses-M0
- sub-edm107_ses-M0
- sub-edm138_ses-M0
- sub-edm157_ses-M0
- sub-edm162_ses-M0
- sub-edm164_ses-M0
- sub-edm169_ses-M0
- sub-edm178_ses-M0
- sub-edm181_ses-M0
- sub-edm183_ses-M0
- sub-mon013_ses-M0
- sub-mon027_ses-M0
- sub-mon064_ses-M0
- sub-mon107_ses-M0
- sub-mon108_ses-M0
- sub-mon109_ses-M0
- sub-mon139_ses-M0
- sub-mon163_ses-M0 # parent folder exist, missing only PSIR contrast
- sub-mon180_ses-M0
- sub-mon181_ses-M0
- sub-tor004_ses-M0
- sub-tor018_ses-M0
- sub-tor024_ses-M0
- sub-tor025_ses-M0
- sub-tor031_ses-M0
- sub-tor032_ses-M0
- sub-tor041_ses-M0
- sub-tor055_ses-M0
- sub-tor063_ses-M0
- sub-tor066_ses-M0
- sub-tor073_ses-M0
- sub-tor076_ses-M0
- sub-tor077_ses-M0
- sub-tor106_ses-M0
- sub-tor109_ses-M0
- sub-tor110_ses-M0
- sub-tor112_ses-M0
- sub-tor124_ses-M0
- sub-tor125_ses-M0
- sub-tor147_ses-M0
- sub-van009_ses-M0
- sub-van093_ses-M0
- sub-van112_ses-M0
- sub-van116_ses-M0
- sub-van123_ses-M0
- sub-van125_ses-M0
- sub-van129_ses-M0
- sub-van131_ses-M0
- sub-van134_ses-M0
- sub-van144_ses-M0
- sub-van145_ses-M0
- sub-van147_ses-M0
- sub-van149_ses-M0
- sub-van150_ses-M0
- sub-van151_ses-M0
- sub-van154_ses-M0
- sub-van168_ses-M0
- sub-van172_ses-M0
- sub-van173_ses-M0
- sub-van178_ses-M0
- sub-van184_ses-M0
- sub-van186_ses-M0
- sub-van204_ses-M0
- sub-van210_ses-M0
- sub-van211_ses-M0
- sub-van214_ses-M0
- sub-van215_ses-M0
- sub-van217_ses-M0
- sub-van219_ses-M0
- sub-van221_ses-M0
- sub-van222_ses-M0
132 changes: 132 additions & 0 deletions nnunet/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
# Installation instructions

## Installation of Anima metrics

Installation of

```
cd ~
mkdir anima/
cd anima/
wget -q https://github.com/Inria-Empenn/Anima-Public/releases/download/v4.2/Anima-macOS-4.2.zip # for MACOS
unzip Anima-macOS-4.2.zip
rm Anima-macOS-4.2.zip
git lfs install
git clone --depth 1 https://github.com/Inria-Visages/Anima-Scripts-Public.git
git clone --depth 1 https://github.com/Inria-Visages/Anima-Scripts-Data-Public.git
```

Configure directories

```
cd ~
mkdir .anima/
touch .anima/config.txt

echo "[anima-scripts]" >> .anima/config.txt
echo "anima = ${HOME}/anima/Anima-Binaries-4.2/" >> .anima/config.txt
echo "anima-scripts-public-root = ${HOME}/anima/Anima-Scripts-Public/" >> .anima/config.txt
echo "extra-data-root = ${HOME}/anima/Anima-Scripts-Data-Public/" >> .anima/config.txt
```

## Installation of required libraries

Create a virtual invironment:
~~~
conda create -n venv_nnunet python=3.9
~~~

Activate the environment with the following command:
~~~
conda activate venv_nnunet
~~~

To install required libraries to train an nnUNet v2:

```
valosekj marked this conversation as resolved.
Show resolved Hide resolved
pip install -r requirements_nnunet.txt
```

Install SpinalCordToolbox 6.0 :

Installation link : https://spinalcordtoolbox.com/user_section/installation.html


# Data preparation

Create the following folders:

~~~
mkdir nnUNet_raw
mkdir nnUNet_preprocessed
mkdir nnUNet_results
~~~

We are training a region-based nnUNet taking an image from contrasts PSIR or STIR and creating a mask with 0=background, 1=spinal cord and 2=MS lesion.

Convert the data to the nnUNet format :

~~~
python convert_BIDS_to_nnunet.py --path-data /path/to/BIDS/dataset --path-out /path/to/nnUNet_raw --taskname TASK-NAME --tasknumber DATASET-ID --contrasts PSIR,STIR --test-ratio XX --time-point ses-XX --type training --exclude-file /path/to/exclude_file.yml
~~~

> **Note**
> The test ratio is 0.2 for 20% (train ratio is therefore 80%). For M0 images, the time point is ses-M0.

To mutliply PSIR images by -1 before training and convert the data to the nnUNet format :

~~~
python convert_BIDS_to_nnunet_with_mul_PSIR.py --path-data /path/to/BIDS/dataset --path-out /path/to/nnUNet_raw --taskname TASK-NAME --tasknumber DATASET-ID --contrasts PSIR,STIR --test-ratio XX --time-point ses-XX --type training --exclude-file /path/to/exclude_file.yml
~~~

# Model training

Before training the model, nnU-Net performs data preprocessing and checks the integrity of the dataset:

~~~
export nnUNet_raw="/path/to/nnUNet_raw"
export nnUNet_preprocessed="/path/to/nnUNet_preprocessed"
export nnUNet_results="/path/to/nnUNet_results"

nnUNetv2_plan_and_preprocess -d DATASET-ID --verify_dataset_integrity
~~~

You will get the configuration plan for all four configurations (2d, 3d_fullres, 3d_lowres, 3d_cascade_fullres).

To train the model, use the following command:
~~~
CUDA_VISIBLE_DEVICES=XXX nnUNetv2_train DATASET-ID CONFIG FOLD --npz
~~~

> **Note**
> Example for Dataset 101, on 2d config on fold 0: CUDA_VISIBLE_DEVICES=2 nnUNetv2_train 101 2d 0 --npz

# Model inference

Convert data to nnUNet format for inference using `convert_BIDS_to_nnunet.py` or `convert_BIDS_to_nnunet_with_mul_PSIR.py` with `--type=inference`.

Then perform inference:
~~~
CUDA_VISIBLE_DEVICES=XXX nnUNetv2_predict -i /path/to/image/folder -o /path/to/predictions -d DATASET_ID -c CONFIG --save_probabilities -chk checkpoint_best.pth -f FOLD
~~~

# Inference evaluation

First, convert the predictions back the BIDS format ; this only keeps the lesion segmentation and discards spinal cord segmentation :

~~~
python convert_predictions_to_BIDS.py --pred-folder /path/to/predictions --out-folder /path/to/output/folder --conversion-dict /path/to/conversion/dict
~~~

If you are converting predictions which are not the evaluations set from nnUNet, use the flag `--not-imageTs`.

Then, you can evaluate the lesion prediction with Anima metrics

~~~
python evaluate_lesion_seg_prediction.py --pred-folder path/to/predictions --dataset path/to/dataset --animaPath path/to/animaSegPerfAnalyzer --output-folder path/to/output_folder
~~~

# Evaluation analysis

The following Notebook `nnUNet_inference_analysis.ipynb` was used to perform analysis of the nnUNet segmentations.
To use it, change the path to the csv files.
Loading