Skip to content

Optional and non-optional dependencies#154

Merged
FlorianDeconinck merged 8 commits into
NOAA-GFDL:developfrom
romanc:romanc/non-optional-imports
May 30, 2025
Merged

Optional and non-optional dependencies#154
FlorianDeconinck merged 8 commits into
NOAA-GFDL:developfrom
romanc:romanc/non-optional-imports

Conversation

@romanc
Copy link
Copy Markdown
Collaborator

@romanc romanc commented May 28, 2025

Description

It looks like in earlier versions, a couple packages were optional. In particular this includes the following packages

dace, gt4py, mpi4py, and xarray

which are now all installed by default given the current setup.py. We can thus simplify their imports (in case they were still imported conditionally). In addition, a couple code paths didn't import cupy from ndsl.optional_imports. This has been corrected too.

Doing all this, I discovered a bug in ndsl/checkpointer/snapshots.py, where a snapshot's dataset would only ever contain the first savepoint. This was unearthed by tests in tests/checkpointer/test_snapshot.py, which suddenly started failing after cleaning up the imports.

As a follow-up, PR #156 will look into re-enabling tests with zarr installed. Looks like these tests will need some love/work before we can re-enable them.

How Has This Been Tested?

Covered by existing test cases.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas: N/A
  • I have made corresponding changes to the documentation: N/A
  • My changes generate no new warnings
  • Any dependent changes have been merged and published in downstream modules: N/A
  • New check tests, if applicable, are included: N/A

@romanc romanc changed the title Romanc/non optional imports WIP: non optional imports May 28, 2025
@romanc romanc force-pushed the romanc/non-optional-imports branch 5 times, most recently from 3ec64a3 to 4d16ef2 Compare May 28, 2025 10:17
@romanc romanc changed the title WIP: non optional imports Optional and non-optional imports May 28, 2025
@romanc romanc changed the title Optional and non-optional imports Optional and non-optional dependencies May 28, 2025
@romanc romanc force-pushed the romanc/non-optional-imports branch 3 times, most recently from 381e314 to 5b877f7 Compare May 28, 2025 12:37
@romanc romanc mentioned this pull request May 28, 2025
7 tasks
@romanc romanc force-pushed the romanc/non-optional-imports branch from 5b877f7 to 816d28b Compare May 28, 2025 12:43
@romanc romanc mentioned this pull request May 28, 2025
@romanc romanc marked this pull request as ready for review May 28, 2025 12:52
fmalatino
fmalatino previously approved these changes May 28, 2025
Copy link
Copy Markdown
Collaborator

@FlorianDeconinck FlorianDeconinck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of extra import clean up since we are there

Comment thread ndsl/initialization/allocator.py Outdated
Comment thread ndsl/quantity/quantity.py Outdated
@romanc romanc force-pushed the romanc/non-optional-imports branch 3 times, most recently from 9f06d39 to 3d50671 Compare May 30, 2025 10:00
Copy link
Copy Markdown
Collaborator Author

@romanc romanc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some inline comments on the latest changes. New changes are only in the last commit. The rest just has new commit hashes because I rebased.

Comment thread tests/dsl/test_caches.py
import shutil

import pytest
from gt4py.cartesian import config as gt_config
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both tests (and there are only two) were requiring the same imports, so I moved them up here

Comment on lines +107 to +112
@function
def dim(a, b):
"""
Performs a check on the difference
between the values in arguments
a and b. The variable diff is set
to the difference between a and b
when the difference is positive,
otherwise it is set to zero. The
function returns the diff variable.
Calculates a - b, camped to 0, i.e. max(a - b, 0).
"""
diff = a - b if a - b > 0 else 0
return diff
return max(a - b, 0)
Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@oelbert I did sneak in some cleanups here. FYI. Also, given the simplicity of the expression, I wonder if we should rename the function or if we need it at all...

Comment thread ndsl/quantity/field_bundle.py Outdated
oelbert
oelbert previously approved these changes May 30, 2025
Copy link
Copy Markdown
Collaborator

@oelbert oelbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

fmalatino
fmalatino previously approved these changes May 30, 2025
Copy link
Copy Markdown
Collaborator

@FlorianDeconinck FlorianDeconinck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove iskip and we should be good to go

@romanc romanc dismissed stale reviews from oelbert and fmalatino via acfad0a May 30, 2025 15:35
@romanc romanc force-pushed the romanc/non-optional-imports branch from ff38735 to acfad0a Compare May 30, 2025 15:35
@romanc romanc requested a review from FlorianDeconinck May 30, 2025 15:35
@romanc
Copy link
Copy Markdown
Collaborator Author

romanc commented May 30, 2025

Removed #isort: skip (and rebased to develop). Happy weekend everyone!

Copy link
Copy Markdown
Collaborator

@FlorianDeconinck FlorianDeconinck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@FlorianDeconinck FlorianDeconinck merged commit 24ca703 into NOAA-GFDL:develop May 30, 2025
5 checks passed
@romanc romanc deleted the romanc/non-optional-imports branch June 2, 2025 07:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants