Matching packages between main and develop branches and removal of outdated pull request template by fmalatino · Pull Request #86 · NOAA-GFDL/pace

fmalatino · 2024-06-04T16:04:52Z

Description
The pytest-subtests package was added to requirements to enable full testing suite.
The python version in github workflows was rolled back to 3.11.7 to match environments in submodules.
The submodules, NDSL, pyFV3, and pySHiELD are also brought to the latest commits in their respective develop branches

How Has This Been Tested?
Tested via current test suite

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
Any dependent changes have been merged and published in downstream modules
New check tests, if applicable, are included

…irments_dev.txt

bensonr · 2024-06-04T22:13:28Z

@fmalatino - before formally reviewing, wanted to make sure the submodule changes for NDSL, pyFV3, and pySHiELD is expected as part of the rollback to Python 3.11.7

fmalatino · 2024-06-05T13:56:36Z

@bensonr I figured I would bring the develop branch up to date with the current versions of the submodules at the same time. I will ammend the initial comment as well.

oelbert

Looks like you'll have to update any calls to constraints.txt now that it's been split into constraints_38 and constaints_311, though it's probably better to just bump the python version to 3.11 and keep one constraint file. If we want to keep support for python 3.8 it's probably better to use environment markers I think?

Also while we're updating the install environment can we delete lines 20-23 from pace/Dockerfile? It seems redundant with lines 24-27

fmalatino · 2024-06-13T14:36:08Z

Looks like you'll have to update any calls to constraints.txt now that it's been split into constraints_38 and constaints_311, though it's probably better to just bump the python version to 3.11 and keep one constraint file. If we want to keep support for python 3.8 it's probably better to use environment markers I think?

Also while we're updating the install environment can we delete lines 20-23 from pace/Dockerfile? It seems redundant with lines 24-27

My vote is that we just move to support for 3.11.7. Can do with the lines in the Dockerfile; will be in next push.

…ts file

oelbert

Great thanks. One more change I think and we should be ok

oelbert · 2024-06-13T17:04:25Z

Thanks for the change. We should probably update the python image in here from 3.8.3 to 3.11.7 as well

oelbert

Have you successfully built the container and run a notebook in it?

…pdating constraints.txt

oelbert

👍 👍 👍 👍 👍

bensonr · 2024-07-08T18:38:26Z

 stencil_config:
  compilation_config:
-    backend: numpy
+    backend: dace:cpu


Do we need both numpy and dace:cpu here? Or do we have other tests for numpy that provide adequate coverage?

Sorry this was a relic from the testing the orchestration issues. Will keep it at numpy only

* working baroclinic test case example (#81) * fixing dockerfile overriding netcdf, mpi4py, and numpy versions with mutually incompatible versions (#84) * Update notebooks for eta file creations (#85) * Update pace to use python 3.11.9 (#82) * Updated container for Github Action and reusable workflow (#87) * Updated container for Github Action and reusable workflow - Redo (#89) * Matching packages between main and develop branches and removal of outdated pull request template (#86) * Changed workflows to use python 3.11.7, added pytest-subtests to requirments_dev.txt * Updating constraints file * Updating Dockerfile to use 3.11-slim-bookworm base python image * reusable workflow CI - checkout corrrect repository (#90) * pace/tests/main/grid/test_eta.py moved to NDSL/tests/grid/test_eta.py (#92) * Updated README to include directions for generating eta files (#99) * Updated submodules to most recent versions as of 3 Jan 2025 15:35 EST * Updated c5 build script to match new methods --------- Co-authored-by: Oliver Elbert <oliver.elbert36@gmail.com> Co-authored-by: MiKyung Lee <58964324+mlee03@users.noreply.github.com> Co-authored-by: mlee03 <Mikyung.Lee@lscamd50-d.gfdl.noaa.gov> Co-authored-by: Xingqiu Yuan <xingqiu.yuan@noaa.gov>

* working baroclinic test case example (#81) * fixing dockerfile overriding netcdf, mpi4py, and numpy versions with mutually incompatible versions (#84) * Update notebooks for eta file creations (#85) * updated generate_eta_files * remove notebook, add script * lint --------- Co-authored-by: mlee03 <Mikyung.Lee@lscamd50-d.gfdl.noaa.gov> * Update pace to use python 3.11.9 (#82) * Updates to allow use of python3.11 * Removed main_unit_tests.yml as it is covered by tests in containers * Updated images for unit tests with openmpi and mpich * Updated container for Github Action and reusable workflow (#87) * updated generate_eta_files * remove notebook, add script * lint * correct path * update constraints * python3 execute * Update main_unit_tests_mpich.yaml * Update main_unit_tests_mpich.yaml * update input directory * get ..s correct * lint --------- Co-authored-by: mlee03 <Mikyung.Lee@lscamd50-d.gfdl.noaa.gov> * Updated container for Github Action and reusable workflow - Redo (#89) * test * tet * fix test dir * test * checkout pace * rename * test * rm * remove dir * fix name * test * pyshield * test * fix trigger mistake * test * change lint version * remove push * lint --------- Co-authored-by: mlee03 <Mikyung.Lee@lscamd50-d.gfdl.noaa.gov> * Matching packages between main and develop branches and removal of outdated pull request template (#86) * Changed workflows to use python 3.11.7, added pytest-subtests to requirments_dev.txt * Matched package dependencies between main and develop branches * Updating constraints file * Updating Dockerfile to use 3.11-slim-bookworm base python image * reusable workflow CI - checkout corrrect repository (#90) * fix trigger mistake * fix spacking * update yaml files * rename lint.yaml * remove backup yaml file * simply pull request trigger * beloved lint --------- Co-authored-by: mlee03 <Mikyung.Lee@lscamd50-d.gfdl.noaa.gov> * add build script for intel oneapi and gcc compiler separately (#88) * add build script for intel oneapi and gcc compiler separately * update the python version and remove the link time optimization option for gcc * pace/tests/main/grid/test_eta.py moved to NDSL/tests/grid/test_eta.py (#92) * Removed test_eta.py, moved to NDSL/tests/grid * Updated constraints to reflect incoming new versions of subrepositories (#94) * Updated README to include directions for generating eta files (#99) * Updated README to include directions for generating eta files * Linting * Updated baroclinic_c2.yaml to generate zarr output * Update submodules and addition of baroclinic run to workflow (#101) * Updated submodules as of 21 Jan 2025, and added baroclinic run to github workflow * Hotfix/update netcdf version for docker files (#102) * Update netcdf version * also upgrade pip * update ndsl * `xarray` to `2025.01.2` to grab the `DataTree` (#103) * Update submodules (#104) * Updated submodules to versions as of 15 Jan 2025 * Updated submodules as of 21 Jan 2025, and added baroclinic run to github workflow * Updated submodules as of 5 09:49 March 2025 * Reverting changes to build_gaea_c5_gcc.sh * Reverting changes to build_gaea_c5_gcc.sh one more thing --------- Co-authored-by: Oliver Elbert <oliver.elbert36@gmail.com> Co-authored-by: MiKyung Lee <58964324+mlee03@users.noreply.github.com> Co-authored-by: mlee03 <Mikyung.Lee@lscamd50-d.gfdl.noaa.gov> Co-authored-by: Xingqiu Yuan <xingqiu.yuan@noaa.gov> Co-authored-by: Florian Deconinck <deconinck.florian@gmail.com>

Changed workflows to use python 3.11.7, added pytest-subtests to requ…

da98b4d

…irments_dev.txt

fmalatino requested review from bensonr, mlee03, oelbert and xyuan June 4, 2024 16:05

fmalatino changed the title ~~Addition of pytest-subtests to requirements and python workflow using 3.11.7~~ Matching packages between main and develop branches and removal of outdated pull request template Jun 13, 2024

fmalatino added 2 commits June 12, 2024 20:27

Matched package dependencies between main and develop branches

f86e143

Removed singular constraint file

01b31b7

oelbert requested changes Jun 13, 2024

View reviewed changes

Removed redudant lines from Dockerfile and created singular constrain…

6eb9f27

…ts file

fmalatino requested a review from oelbert June 13, 2024 15:35

Fixing requirements_dev.txt

6c7ff9f

oelbert requested changes Jun 13, 2024

View reviewed changes

Updated Dockerfile base image to python 3.11.7-slim-bullseye

14866fd

fmalatino requested a review from oelbert June 13, 2024 20:17

oelbert reviewed Jun 18, 2024

View reviewed changes

fmalatino added 3 commits June 27, 2024 10:39

Working on Dockerfile

b90b02e

Merged in changes from PR 89

4f54c75

Updating constraints file

9652fca

fmalatino marked this pull request as draft June 27, 2024 18:06

mlee03 reviewed Jul 1, 2024

View reviewed changes

Comment thread Makefile Outdated

mlee03 reviewed Jul 1, 2024

View reviewed changes

Comment thread constraints.txt Outdated

fmalatino added 2 commits July 1, 2024 14:09

Reverting changes in Makefile for docker builds, updating NDSL, and u…

0642982

…pdating constraints.txt

Updating platformdirs to v3.10.0 in constraints.txt

763250e

fmalatino marked this pull request as ready for review July 1, 2024 18:58

fmalatino mentioned this pull request Jul 2, 2024

reusable workflow CI - checkout corrrect repository #90

Merged

7 tasks

Updating constraints.txt

9f68b76

fmalatino requested review from mlee03 and oelbert July 3, 2024 14:23

fmalatino added 2 commits July 3, 2024 14:04

Updating Dockerfile to use 3.11-slim-bookworm base python image

2c7bedb

Updating constraints to use platformdirs v3.10.0

5ed4ae9

oelbert approved these changes Jul 4, 2024

View reviewed changes

bensonr reviewed Jul 8, 2024

View reviewed changes

Removing dace:cpu backend from baroclinic_c12 example

ecb7c65

fmalatino requested a review from bensonr July 8, 2024 19:01

bensonr approved these changes Jul 8, 2024

View reviewed changes

fmalatino merged commit e7f5a48 into NOAA-GFDL:develop Jul 8, 2024

fmalatino deleted the fix/python_ver_tests branch August 22, 2024 02:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Matching packages between main and develop branches and removal of outdated pull request template#86

Matching packages between main and develop branches and removal of outdated pull request template#86
fmalatino merged 15 commits into
NOAA-GFDL:developfrom
fmalatino:fix/python_ver_tests

fmalatino commented Jun 4, 2024 •

edited

Loading

Uh oh!

bensonr commented Jun 4, 2024

Uh oh!

fmalatino commented Jun 5, 2024

Uh oh!

oelbert left a comment

Uh oh!

fmalatino commented Jun 13, 2024

Uh oh!

oelbert left a comment

Uh oh!

oelbert Jun 13, 2024

Uh oh!

oelbert left a comment

Uh oh!

Uh oh!

Uh oh!

oelbert left a comment

Uh oh!

bensonr Jul 8, 2024

Uh oh!

fmalatino Jul 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

fmalatino commented Jun 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bensonr commented Jun 4, 2024

Uh oh!

fmalatino commented Jun 5, 2024

Uh oh!

oelbert left a comment

Choose a reason for hiding this comment

Uh oh!

fmalatino commented Jun 13, 2024

Uh oh!

oelbert left a comment

Choose a reason for hiding this comment

Uh oh!

oelbert Jun 13, 2024

Choose a reason for hiding this comment

Uh oh!

oelbert left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

oelbert left a comment

Choose a reason for hiding this comment

Uh oh!

bensonr Jul 8, 2024

Choose a reason for hiding this comment

Uh oh!

fmalatino Jul 8, 2024

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fmalatino commented Jun 4, 2024 •

edited

Loading