Skip to content

Refactor of USPS proofing job spec to eliminate flakey tests#8081

Merged
JackRyan1989 merged 17 commits intomainfrom
jryn/usps-proofing-spec-testing
Mar 29, 2023
Merged

Refactor of USPS proofing job spec to eliminate flakey tests#8081
JackRyan1989 merged 17 commits intomainfrom
jryn/usps-proofing-spec-testing

Conversation

@JackRyan1989
Copy link
Contributor

@JackRyan1989 JackRyan1989 commented Mar 27, 2023

Refactor of USPS Proofing job spec to hopefully eliminate flakey tests.

Changes:

  1. Remove "minutes_since_establishment" from "it" blocks and instead place in single location within the shared example.
  2. Wrap needed tests in freeze_time blocks
  3. Add specific fields necessary for analytics checks
  4. Move some other checks from separate expectations into one expectation, where appropriate.

📜 Testing Plan

Provide a checklist of steps to confirm the changes.

  • Run tests locally many (~20) times and be sure that the tests do not error unexpectedly.

issuer: pending_enrollment.issuer,
minutes_since_last_status_check: 15.0,
minutes_since_last_status_update: 2.days.in_minutes,
minutes_to_completion: range_approximating(3.days.in_minutes, vary_left: -60, vary_right: 5),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i'm unfamiliar w/ vary_left and vary_right - are they the bounds of the range?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep. So this line says to expect a value between a range going from 1 hour before 3.days.in_minutes to five minutes after that time.

@JackRyan1989 JackRyan1989 marked this pull request as draft March 29, 2023 15:23
@JackRyan1989 JackRyan1989 requested a review from tomas-nava March 29, 2023 15:23
@JackRyan1989 JackRyan1989 marked this pull request as ready for review March 29, 2023 17:48
@JackRyan1989 JackRyan1989 changed the title Draft: Refactor of usps proofing job spec to eliminate flakey tests. Refactor of USPS proofing job spec to eliminate flakey tests Mar 29, 2023
Copy link
Contributor

@tomas-nava tomas-nava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approve!

I made a bunch of tiny suggestions of additional parameters to include; you can ignore them if they're redundant or irrelevant.

Copy link
Contributor

@eileen-nava eileen-nava left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests passed when I repeatedly ran them on my local machine. LGTM.

jack-ryan-nava-pbc and others added 9 commits March 29, 2023 14:22
Co-authored-by: Tomas Apodaca <thomas.apodaca@gsa.gov>
Co-authored-by: Tomas Apodaca <thomas.apodaca@gsa.gov>
Co-authored-by: Tomas Apodaca <thomas.apodaca@gsa.gov>
Co-authored-by: Tomas Apodaca <thomas.apodaca@gsa.gov>
Co-authored-by: Tomas Apodaca <thomas.apodaca@gsa.gov>
Co-authored-by: Tomas Apodaca <thomas.apodaca@gsa.gov>
Co-authored-by: Tomas Apodaca <thomas.apodaca@gsa.gov>
Co-authored-by: Tomas Apodaca <thomas.apodaca@gsa.gov>
Co-authored-by: Tomas Apodaca <thomas.apodaca@gsa.gov>
@JackRyan1989 JackRyan1989 merged commit d387ce2 into main Mar 29, 2023
@JackRyan1989 JackRyan1989 deleted the jryn/usps-proofing-spec-testing branch March 29, 2023 19:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants