Skip to content

GitHub: FMS-MOM6 build jobs, tarball artifacts#915

Merged
adcroft merged 1 commit into
NOAA-GFDL:dev/gfdlfrom
marshallward:gh-merge-fms-mom
Jun 17, 2025
Merged

GitHub: FMS-MOM6 build jobs, tarball artifacts#915
adcroft merged 1 commit into
NOAA-GFDL:dev/gfdlfrom
marshallward:gh-merge-fms-mom

Conversation

@marshallward
Copy link
Copy Markdown
Member

Inconsistencies in jobs within a workflow were leading to differences in compiler versions across jobs. This was breaking GitHub Actions workflows.

We want to continue using *-latest versioning, so this patch merges the FMS build into each MOM6 build. Although there is greater compute time, it should be happening in parallel. It may even be shorter, since we do not need to wait on the additional OS setup.

This patch also packs executables and other data into tarballs. This preserves most of the file data (file mode permissions and modified-time, mainly) so that we don't have to do Makefile tricks or use chmod to get things working in the next job. We also don't need to resort to horrible awk tricks to generate executable lists.

There may be a slight speedup here if the builds happen in parallel, since we are doing one less image setup, but CI runtimes are very variable so I don't think we'll be able to know right away.

(@Hallberg-NOAA noted that different MOM6 builds may still have different versions. Eventually we may need to have one job set the version and then force the others to use this prescribed version. But this is an improvement over the current state.)

@marshallward
Copy link
Copy Markdown
Member Author

There is some issue here where FMS is being rebuilt in the test jobs, even though the executables are already built. (Also a minor typo in build-opt-target and a problem building the tc4 grid data in test-regressions.)

@marshallward marshallward force-pushed the gh-merge-fms-mom branch 4 times, most recently from 7b818e5 to 99100ea Compare June 8, 2025 04:28
Inconsistencies in jobs within a workflow were leading to differences in
compiler versions across jobs.  This was breaking GitHub Actions
workflows.  We want to continue using `*-latest` runners, so we have
merged the FMS stage into each of the MOM6 builds.

There are additional changes which improve the overall workflow/job
decomposition.

* FMS builds included in MOM6 stage

* Artifacts are now packed into tarballs, which preserve most of the
  file state (mtime, file mode, etc).

  * No longer any need to chmod the executables.

  * `*.gnco` timestamp warnings are removed.

  * Overall simplification of generated executable lists; we now
    assume that build/* only contains executables.

* build.timing and build.unit are now correctly tagged as .NOTPARALLEL

* Macros for defining the target URL are now environment variables,
  stored in the workspace `env`.
@marshallward
Copy link
Copy Markdown
Member Author

I have many multiple adjustments to this PR:

  • The "oldest" -o make flags used to prevent executable rebuilds have been restored where needed

  • Some missing DO_REGRESSION_TESTS flags have been restored (also fixing some redundant builds)

  • Perf profiling output has been repaired

  • compare.timing output has been repaired

  • Several rules which must be run serially now have .NOTPARALLEL directives

  • the URL-tracking macros (MOM_TARGET_SLUG, MOM_TARGET_LOCAL_BRANCH) are now defined at the workspace environment variable level

  • The unused testing-setup macro has been deleted

  • tarball unpacking is now verbose, for output inspection

  • .gcno files are now "touched" to disable modified-time warnings

  • Dimension tests are now packed as a single matrix of tests

  • the TARGET_CODEBASE checkout rule is now wrapped in a DO_REGRESSION_TESTS, to avoid an empty : rule.

    However the whole DO_REGRESSION_TESTS usage has shown itself to be very problematic and did not properly consider the "parsing" vs "execution" stages of make. This needs a tidy up, so that we can remove the DO_REGRESSION_TESTS flags from the yaml file.

While there are even more improvements to be made, it's probably best that I stop while I am ahead.

@adcroft adcroft merged commit 28e5651 into NOAA-GFDL:dev/gfdl Jun 17, 2025
52 checks passed
@marshallward marshallward deleted the gh-merge-fms-mom branch July 28, 2025 14:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants