Fix casting rules changing in Numpy 2#10897
Merged
Merged
Conversation
This fixes several places in `quantum_info` that will be affected by the casting-rule changes coming in Numpy 2.0[^1]. There are several other places where the casting rules will change the output types of certain arrays due to the removal of Python-scalar value-dependent casting, but those do not appear to have an observable effect on our behaviour, so I have left them. It is also becoming invalid to set a scalar entry in a Numpy array with an array-like with a non-zero number of dimensions, even if the sequence is of length 1. [^1]: https://numpy.org/neps/nep-0050-scalar-promotion.html
jakelishman
added
type: qa
Collaborator
|
One or more of the the following people are requested to review this:
|
Closed
ikkoham
approved these changes
Sep 29, 2023
Contributor
ikkoham
left a comment
ikkoham
left a comment
There was a problem hiding this comment.
Thanks. I didn't know the NumPy 2. Note that pauli_table.py will be removed soon, but this PR is fine.
Member
Author
|
Thanks Hamamura-san. Yeah, I'm hoping that If it helps - we're tracking our complete progress on NumPy 2 support in #10889. |
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
rupeshknn
pushed a commit
to rupeshknn/qiskit
that referenced
this pull request
Oct 9, 2023
This fixes several places in `quantum_info` that will be affected by the casting-rule changes coming in Numpy 2.0[^1]. There are several other places where the casting rules will change the output types of certain arrays due to the removal of Python-scalar value-dependent casting, but those do not appear to have an observable effect on our behaviour, so I have left them. It is also becoming invalid to set a scalar entry in a Numpy array with an array-like with a non-zero number of dimensions, even if the sequence is of length 1. [^1]: https://numpy.org/neps/nep-0050-scalar-promotion.html
jakelishman
added a commit
to jakelishman/qiskit-terra
that referenced
this pull request
Mar 12, 2024
This commit brings the Qiskit test suite to a passing state (with all optionals installed) with Numpy 2.0.0b1, building on previous commits that handled much of the rest of the changing requirements: - Qiskitgh-10890 - Qiskitgh-10891 - Qiskitgh-10892 - Qiskitgh-10897 - Qiskitgh-11023 Notably, this commit did not actually require a rebuild of Qiskit, despite us compiling against Numpy; it seems to happen that the C API stuff we use via `rust-numpy` (which loads the Numpy C extensions dynamically during module initialisation) hasn't changed.
jakelishman
added a commit
to jakelishman/qiskit-terra
that referenced
this pull request
Apr 15, 2024
This commit brings the Qiskit test suite to a passing state (with all optionals installed) with Numpy 2.0.0b1, building on previous commits that handled much of the rest of the changing requirements: - Qiskitgh-10890 - Qiskitgh-10891 - Qiskitgh-10892 - Qiskitgh-10897 - Qiskitgh-11023 Notably, this commit did not actually require a rebuild of Qiskit, despite us compiling against Numpy; it seems to happen that the C API stuff we use via `rust-numpy` (which loads the Numpy C extensions dynamically during module initialisation) hasn't changed. The main changes are: - adapting to the changed `copy=None` and `copy=False` semantics in `array` and `asarray`. - making sure all our implementers of `__array__` accept both `dtype` and `copy` arguments. Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com>
jakelishman
added a commit
to jakelishman/qiskit-terra
that referenced
this pull request
Apr 16, 2024
This commit brings the Qiskit test suite to a passing state (with all optionals installed) with Numpy 2.0.0b1, building on previous commits that handled much of the rest of the changing requirements: - Qiskitgh-10890 - Qiskitgh-10891 - Qiskitgh-10892 - Qiskitgh-10897 - Qiskitgh-11023 Notably, this commit did not actually require a rebuild of Qiskit, despite us compiling against Numpy; it seems to happen that the C API stuff we use via `rust-numpy` (which loads the Numpy C extensions dynamically during module initialisation) hasn't changed. The main changes are: - adapting to the changed `copy=None` and `copy=False` semantics in `array` and `asarray`. - making sure all our implementers of `__array__` accept both `dtype` and `copy` arguments. Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com>
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Apr 25, 2024
* Finalise support for Numpy 2.0 This commit brings the Qiskit test suite to a passing state (with all optionals installed) with Numpy 2.0.0b1, building on previous commits that handled much of the rest of the changing requirements: - gh-10890 - gh-10891 - gh-10892 - gh-10897 - gh-11023 Notably, this commit did not actually require a rebuild of Qiskit, despite us compiling against Numpy; it seems to happen that the C API stuff we use via `rust-numpy` (which loads the Numpy C extensions dynamically during module initialisation) hasn't changed. The main changes are: - adapting to the changed `copy=None` and `copy=False` semantics in `array` and `asarray`. - making sure all our implementers of `__array__` accept both `dtype` and `copy` arguments. Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com> * Update `__array__` methods for Numpy 2.0 compatibility As of Numpy 2.0, implementers of `__array__` are expected and required to have a signature def __array__(self, dtype=None, copy=None): ... In Numpys before 2.0, the `copy` argument will never be passed, and the expected signature was def __array__(self, dtype=None): ... Because of this, we have latitude to set `copy` in our implementations to anything we like if we're running against Numpy 1.x, but we should default to `copy=None` if we're running against Numpy 2.0. The semantics of the `copy` argument to `np.array` changed in Numpy 2.0. Now, `copy=False` means "raise a `ValueError` if a copy is required" and `copy=None` means "copy only if required". In Numpy 1.x, `copy=False` meant "copy only if required". In _both_ Numpy 1.x and 2.0, `ndarray.astype` takes a `copy` argument, and in both, `copy=False` means "copy only if required". In Numpy 2.0 only, `np.asarray` gained a `copy` argument with the same semantics as the `np.array` copy argument from Numpy 2.0. Further, the semantics of the `__array__` method changed in Numpy 2.0, particularly around copying. Now, Numpy will assume that it can pass `copy=True` and the implementer will handle this. If `copy=False` is given and a copy or calculation is required, then the implementer is required to raise `ValueError`. We have a few places where the `__array__` method may (or always does) calculate the array, so in all these, we must forbid `copy=False`. With all this in mind: this PR sets up all our implementers of `__array__` to either default to `copy=None` if they will never actually need to _use_ the `copy` argument within themselves (except perhaps to test if it was set by Numpy 2.0 to `False`, as Numpy 1.x will never set it), or to a compatibility shim `_numpy_compat.COPY_ONLY_IF_NEEDED` if they do naturally want to use it with those semantics. The pattern def __array__(self, dtype=None, copy=_numpy_compat.COPY_ONLY_IF_NEEDED): dtype = self._array.dtype if dtype is None else dtype return np.array(self._array, dtype=dtype, copy=copy) using `array` instead of `asarray` lets us achieve all the desired behaviour between the interactions of `dtype` and `copy` in a way that is compatible with both Numpy 1.x and 2.x. * fixing numerical issues on mac-arm * Change error to match Numpy --------- Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com> Co-authored-by: Sebastian Brandhofer <148463728+sbrandhsn@users.noreply.github.com>
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Jun 18, 2024
* Finalise support for Numpy 2.0 This commit brings the Qiskit test suite to a passing state (with all optionals installed) with Numpy 2.0.0b1, building on previous commits that handled much of the rest of the changing requirements: - gh-10890 - gh-10891 - gh-10892 - gh-10897 - gh-11023 Notably, this commit did not actually require a rebuild of Qiskit, despite us compiling against Numpy; it seems to happen that the C API stuff we use via `rust-numpy` (which loads the Numpy C extensions dynamically during module initialisation) hasn't changed. The main changes are: - adapting to the changed `copy=None` and `copy=False` semantics in `array` and `asarray`. - making sure all our implementers of `__array__` accept both `dtype` and `copy` arguments. Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com> * Update `__array__` methods for Numpy 2.0 compatibility As of Numpy 2.0, implementers of `__array__` are expected and required to have a signature def __array__(self, dtype=None, copy=None): ... In Numpys before 2.0, the `copy` argument will never be passed, and the expected signature was def __array__(self, dtype=None): ... Because of this, we have latitude to set `copy` in our implementations to anything we like if we're running against Numpy 1.x, but we should default to `copy=None` if we're running against Numpy 2.0. The semantics of the `copy` argument to `np.array` changed in Numpy 2.0. Now, `copy=False` means "raise a `ValueError` if a copy is required" and `copy=None` means "copy only if required". In Numpy 1.x, `copy=False` meant "copy only if required". In _both_ Numpy 1.x and 2.0, `ndarray.astype` takes a `copy` argument, and in both, `copy=False` means "copy only if required". In Numpy 2.0 only, `np.asarray` gained a `copy` argument with the same semantics as the `np.array` copy argument from Numpy 2.0. Further, the semantics of the `__array__` method changed in Numpy 2.0, particularly around copying. Now, Numpy will assume that it can pass `copy=True` and the implementer will handle this. If `copy=False` is given and a copy or calculation is required, then the implementer is required to raise `ValueError`. We have a few places where the `__array__` method may (or always does) calculate the array, so in all these, we must forbid `copy=False`. With all this in mind: this PR sets up all our implementers of `__array__` to either default to `copy=None` if they will never actually need to _use_ the `copy` argument within themselves (except perhaps to test if it was set by Numpy 2.0 to `False`, as Numpy 1.x will never set it), or to a compatibility shim `_numpy_compat.COPY_ONLY_IF_NEEDED` if they do naturally want to use it with those semantics. The pattern def __array__(self, dtype=None, copy=_numpy_compat.COPY_ONLY_IF_NEEDED): dtype = self._array.dtype if dtype is None else dtype return np.array(self._array, dtype=dtype, copy=copy) using `array` instead of `asarray` lets us achieve all the desired behaviour between the interactions of `dtype` and `copy` in a way that is compatible with both Numpy 1.x and 2.x. --------- Co-authored-by: Lev S. Bishop <18673315+levbishop@users.noreply.github.com> Co-authored-by: Raynel Sanchez <87539502+raynelfss@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This fixes several places in
quantum_infothat will be affected by the casting-rule changes coming in Numpy 2.01. There are several other places where the casting rules will change the output types of certain arrays due to the removal of Python-scalar value-dependent casting, but those do not appear to have an observable effect on our behaviour, so I have left them.It is also becoming invalid to set a scalar entry in a Numpy array with an array-like with a non-zero number of dimensions, even if the sequence is of length 1.
Details and comments
For reference, I found these by running the test suite with Numpy 1.26.0 and the environment variable
NPY_PROMOTION_STATE=weakset. I also ran with that set toweak_and_warnto see if there were other places that looked suspicious, but that is (predictably) very noisy, and none of the related changes jumped out at me as particularly problematic.Running with Numpy 1.26.0 (on macOS) runs afoul of #10305, so it's possible I missed a couple of things, but I don't think so. Unfortunately, a bug in Numpy 1.24.4 means that
np.complex128(1j) * opflow.Z(as a MWE) segfaults the Python interpreter when running inNPY_PROMOTION_STATE=weak, so it's pretty hard work to disentangle the test suite from that well enough to use the old Numpy version with the new casting rules to find errors (plus they've been changing a little over time).Footnotes
https://numpy.org/neps/nep-0050-scalar-promotion.html ↩