You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Following up from #307, this PR suggests to make gt4py_backend a required argument for a Quantity object. The gt4py backend name is used to ensure an optimal data layout, copying data into a better layout if needed. The backend name is then stored with other metadata in quantity.metadata.
Directly creating Quantity objects (like without using the QuantityFactory) is done mostly in tests. There's sporadic usage in the halo code and in the legacy restart system.
Related to issue #230 (like not quite there - just a step in the right direction).
Questions:
For me, this seems to be a step in the right direction, especially in context of refactor: Quantities and data allocation #230. Is that really true? Or do we (e.g. in the halo code) fundamentally rely on the fact that we can build a Quantity without an associated backend?
Assuming we go ahead with this: Should we rename gt4py_backend -> backend?
Should we stop allowing basic data types in the __init__ of Quantity? I mean, the type-hint of data is already set to np.ndarray | cupy.ndarray. Not that this would stop anyone in the python world ...
What are quantity.attrs? From usage in test_transpose.py, I assume it's meant as a dumping ground for additional information stored on (and propagated with) quantities. If so, should we allow to set those attrs in the constructor and "pull them up" into the ones(), zeros(), empty(), ... of QuantityFactory? That way users don't need to do the two-step q = quantity_factory.zeros(); q.attrs = {"my": "description here"}
How has this been tested?
Please describe the tests that you ran to verify your changes. Please also note
any relevant details for your test configuration (e.g. compiler, OS). Include
enough information so someone can reproduce your tests.
Checklist
My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation (e.g. add new modules to docs/docstrings/)
My changes generate no new warnings
Any dependent changes have been merged and published in downstream modules
romanc
changed the title
WIP: refactor: quantity make gt4py_backend mandatory in the future
RFC: refactor: quantity make gt4py_backend mandatory in the future
Nov 5, 2025
Halo exchange don't allocate Quantity. Allocations are in a buffer pool.
I am not against removing the capacity to allocate Quantity by hand. QuantityFactory are near-trivial to build, taking only a GridSizer and a backend. We can think on how to lighten even more this API.
We should rename everything backend, unless it's an internal concept which it isn't here.
We don't use Quantity for scalar, though they were designed to be transparent. Boils down to our native support for scalars, which seems relatively solid. Could make automatic data -> Quantity conversion a bit more tedious. Minor, would remove.
Those are used for interoperability with xarray (see NetcdfMonitor, ZarrMonitor, test_translate.py). I agree they are not in right level, those are metadata and should be stored there. I'd have to look again at exactly what those carry and if we need to cache them at all or if they can be re-computed on demand
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Following up from #307, this PR suggests to make
gt4py_backenda required argument for aQuantityobject. The gt4py backend name is used to ensure an optimal data layout, copying data into a better layout if needed. The backend name is then stored with other metadata inquantity.metadata.Directly creating
Quantityobjects (like without using theQuantityFactory) is done mostly in tests. There's sporadic usage in the halo code and in the legacy restart system.Related to issue #230 (like not quite there - just a step in the right direction).
Questions:
Quantitywithout an associated backend?gt4py_backend->backend?__init__ofQuantity? I mean, the type-hint ofdatais already set tonp.ndarray | cupy.ndarray. Not that this would stop anyone in the python world ...NDSL/ndsl/quantity/quantity.py
Lines 81 to 83 in 2c9946c
quantity.attrs? From usage intest_transpose.py, I assume it's meant as a dumping ground for additional information stored on (and propagated with) quantities. If so, should we allow to set thoseattrsin the constructor and "pull them up" into theones(),zeros(),empty(), ... ofQuantityFactory? That way users don't need to do the two-stepq = quantity_factory.zeros(); q.attrs = {"my": "description here"}How has this been tested?
Please describe the tests that you ran to verify your changes. Please also note
any relevant details for your test configuration (e.g. compiler, OS). Include
enough information so someone can reproduce your tests.
Checklist