Refactor: QuantityFactory data dimensions operations#272
Conversation
Deprecate `set_extra_dim_lengths` + utest
romanc
left a comment
There was a problem hiding this comment.
I like the naming and I think we should propagate it down into the sizer.
… (dependencies might die)
|
Blocked by NOAA-GFDL/pace#154 |
romanc
left a comment
There was a problem hiding this comment.
That turned out bigger then I expected. Thanks for the cleanup! 🧹 ✨
| extra_dim_lengths: dict[str, int], | ||
| layout: tuple[int, int], | ||
| *, | ||
| data_dimensions: dict[str, int] = {}, |
There was a problem hiding this comment.
It is a python anti-pattern to have an empty dict/list as default argument. The reason is that this will coupke function invocations because that empty dict might not be empty anymore once we get here in the next invocation. People usually use dict | None as type with None as default and the test for Non inside and assign a {} in that case (wich is then local to that call).
There was a problem hiding this comment.
We can also do a follow-up, but basically, as far as I can see, the last commit, bcea590, wouldn't be necessary if we had
data_dimensions: dict[str, int] | None = None,in the argument list and then do some more fiddling inside the function. This is surprising, especially coming form C++ where default arguments behave differently. Here's a simple example of what goes wrong.
I think there's mypy or flake rules to catch those. I remember seeing them (probably in gt4py). I'll search tomorrow.
There was a problem hiding this comment.
Ah it's in flake8-bugbear
B006: Do not use mutable data structures for argument defaults. They are created during function definition time. All calls to the function reuse this one instance of that data structure, persisting changes between them.
There was a problem hiding this comment.
Another way to fix this is actually use a dataclasses.field(default_factory) on the base dataclass
Pull Request is not mergeable
"extra dims" have been renamed to "data dimensions" in NDSL [1] and after doing so we figured that we need to be careful with mutable default values [2]. Togehter, these two PRs make specifying `extra_dim_lengths` unnecessary if they are the default empty dictionary. [1]: NOAA-GFDL/NDSL#272 [2]: NOAA-GFDL/NDSL#277
Description
The
set_extra_dim_lengthsfunction ofQuantityFactoryis badly named: it updates the dictionary of data dimensions, rather than set.Usage of this function is also a bit odd, you mostly want to add dimensions. But because a
QuantityFactorycan be shared (should be shared) across many codes you want to make sure to not override previous dimensions set.Introducing:
QuantityFactory.add_data_dimensions: check for existing names and error out.QuantityFactory.update_data_dimensions: update existing or add new ones.Deprecating
set_extra_dim_lengthsfor later removal.How has this been tested?
New utest.
Checklist