Move instance suffix to correct position in restart file names

[alperaltuntas]

This PR partially addresses ESCOMP/MOM_interface#238 by including the instance number in the component name portion of the restart filenames, rather than appending it to the end of the filename.

This behavior is controlled by a newly added runtime parameter RESTARTFILE_APPENDIX_PREFIX. If this prefix is provided (and found in the restart file name), the ensemble instance number (appendix) is added right after the first occurrence of the prefix (RESTARTFILE_APPENDIX_PREFIX). If not found, the appendix is added to the end of the file name (as before).

[alperaltuntas]

This PR is ready for review. @kdraeder, Could you confirm that this PR fixes the restart filename consistency issue. Note that this PR needs to be evaluated in conjunction with the changes from ESCOMP/MOM_interface#261

[kdraeder]

This PR is ready for review. @kdraeder, Could you confirm that this PR fixes the restart filename consistency issue.

@alperaltuntas What's the preferred (or acceptable) MOM6 way to get this code for testing?
I don't have much experience with this, but I'm happy to do it. Thanks!

[alperaltuntas]

This PR is ready for review. @kdraeder, Could you confirm that this PR fixes the restart filename consistency issue.

@alperaltuntas What's the preferred (or acceptable) MOM6 way to get this code for testing? I don't have much experience with this, but I'm happy to do it. Thanks!

@kdraeder, I've already tested this thoroughly, so no formal testing is needed on your end. If you could confirm that these changes resolve the issue you reported in ESCOMP/MOM_interface#238 (the part related to the restart file name) using any relevant test of your choice (for example, by rerunning a case where you previously observed the issue), that should be sufficient.

[kdraeder]

@alperaltuntas I've exported the changes in this PR into the CESM
I've been using to test st_archive changes (cesm3_0_beta03-121-g0a50f32).
The components/mom/MOM6/config_src/drivers/nuopc_cap/mom_cap.F90
has the same git log as the dev/ncar branch of the MOM6 I checked out from this PR.
The branch being used to develop the restart file name changes
(fix_multiinstance_restartname2) has commits after the last dev/ncar, which is 2024-11-7.
MOM6 lives within my MOM_interface branch mom_st-arch_dart, which was branched
from the default tag for this CESM (headless state): mi_250210.

I built a HISTC_CAM70%LT_CLM60%BGC-CROP_CICE_MOM6%MARBL-BIO_MOSART_SGLC_SWAV case,
but the run failed in MOM with
FATAL from PE 0: check_nml_error in fms_mod:
Unknown namelist, or mistyped namelist variable in namelist DIAG_MANAGER_NML
I don't see that namelist in components/mom/MOM6/..., but it is in the MOM_interface:
param_templates/json/input_nml.json
param_templates/input_nml.yaml

So it seems that my question is "is there a MOM_interface that will work
with this modified MOM6? and with this CESM?"
Or would I need a different CESM to test this way?

[alperaltuntas]

Since beta03 is quite outdated, I suggest we go ahead and merge these changes now, and include them in the upcoming MOM6-centric alpha tag, cesm3_0_alpha07d. Once that tag is available, you can test both this and other related changes in a more up-to-date CESM context. If you still run into issues at that point, we can iterate further based on that.
Does that approach work for you?

[kdraeder]

That sounds good.
Thanks for pointing out my outdated CESM. Time flies . . .

[alperaltuntas]

@mnlevy1981 Just checking if you'd like to review this PR and ESCOMP/MOM_interface#261. If not, I'll go ahead and merge them.

[mnlevy1981]

If you've verified there's no effect on single-instance cases, I'm okay with these changes