Question about model selection for the evaluation in FMoW-wilds

[LFhase]

Hi Fish authors,

Thanks for the nice work! When I was trying to reproduce the results of Fish in FMoW-wilds, I found the function save_best_model (code) always saves the last epoch model. It seems Fish would use the last epoch model selection instead of selecting the models according to the validation performance, which appears to be different from the wilds evaluation protocol (I also failed to find a corresponding description in the paper).

Maybe I missed something, could you help look into this problem? Thank you very much 😄.

Best, Andrew