Relax tolerance of UnitaryOverlap tests#11069
Merged
Merged
Conversation
The failure rate for the parametric identity tests was about 0.2%, which at our CI scale corresponds to approximately one spurious failure every two days. The alternative to this is to fix the random seed of the test, but similar to other `quantum_info` tests, we would like some degree of extended coverage. Lifting the tolerance by three orders of magnitude at this circuit size should ensure a zero false-positive rate. This also changes the test to actually display the failure tolerance, so it's easier to tell from a CI run if a failure was real.
jakelishman
added
type: qa
Collaborator
|
One or more of the the following people are requested to review this:
|
Pull Request Test Coverage Report for Build 6588620468
💛 - Coveralls |
mtreinish
approved these changes
Oct 20, 2023
mergify Bot
pushed a commit
that referenced
this pull request
Oct 20, 2023
The failure rate for the parametric identity tests was about 0.2%, which at our CI scale corresponds to approximately one spurious failure every two days. The alternative to this is to fix the random seed of the test, but similar to other `quantum_info` tests, we would like some degree of extended coverage. Lifting the tolerance by three orders of magnitude at this circuit size should ensure a zero false-positive rate. This also changes the test to actually display the failure tolerance, so it's easier to tell from a CI run if a failure was real. (cherry picked from commit 80018cc)
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Oct 20, 2023
The failure rate for the parametric identity tests was about 0.2%, which at our CI scale corresponds to approximately one spurious failure every two days. The alternative to this is to fix the random seed of the test, but similar to other `quantum_info` tests, we would like some degree of extended coverage. Lifting the tolerance by three orders of magnitude at this circuit size should ensure a zero false-positive rate. This also changes the test to actually display the failure tolerance, so it's easier to tell from a CI run if a failure was real. (cherry picked from commit 80018cc) Co-authored-by: Jake Lishman <jake.lishman@ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The failure rate for the parametric identity tests was about 0.2%, which at our CI scale corresponds to approximately one spurious failure every two days. The alternative to this is to fix the random seed of the test, but similar to other
quantum_infotests, we would like some degree of extended coverage. Lifting the tolerance by three orders of magnitude at this circuit size should ensure a zero false-positive rate.This also changes the test to actually display the failure tolerance, so it's easier to tell from a CI run if a failure was real.
Details and comments