How to choose the best checkpoint?

[hajlyx]

Choose final checkpoint or checkpoint with lowest val loss, which is better?

Assume the val set has 20 patients.

Best.

[FabianIsensee]

There is no hard rule to that. You will never really know because you need to pick one and then evaluate on the test set only once. If I were you I would look at the plot. If the evaluation metric goes up and then back down again I would use the best. If not I would use final.
Best,
Fabian

[hajlyx]

There is no hard rule to that. You will never really know because you need to pick one and then evaluate on the test set only once. If I were you I would look at the plot. If the evaluation metric goes up and then back down again I would use the best. If not I would use final.
Best,
Fabian

Thanks for reply.

Another question is about ensemble strategy. It seems that in nnUNet we just use the mean value of outputs from five fold trained model. Is there a more robust and elegant ensemble strategy?

Best.
Wayen.

[FabianIsensee]

There are certainly better ways to do it. Here I just average and it works well enough. You can choose to do this differently
Best,
Fabian

[hajlyx]

Thanks.

I'm going to look for different ways.