Reset cookie when validating HTML markup in accessibility test to improve reliability#12428
Merged
mitchellhenke merged 1 commit intomainfrom Aug 15, 2025
Conversation
mitchellhenke
force-pushed
the
mitchellhenke/fix-flaky-accessiblity-testing-matcher
branch
from
fe4087b to
d509152
Compare
mitchellhenke
force-pushed
the
mitchellhenke/fix-flaky-accessiblity-testing-matcher
branch
from
cdf4dad to
2e9d9b8
Compare
…rove reliability changelog: Internal, Testing, Reset cookie when validating HTML markup in accessibility test to improve reliability Co-authored-by: Davi (she/they) <davida.marion@gsa.gov>
mitchellhenke
force-pushed
the
mitchellhenke/fix-flaky-accessiblity-testing-matcher
branch
from
2e9d9b8 to
41c5270
Compare
Sgtpluck
approved these changes
Aug 14, 2025
mitchellhenke
force-pushed
the
mitchellhenke/fix-flaky-accessiblity-testing-matcher
branch
from
dcadb84 to
41c5270
Compare
mitchellhenke
deleted the
mitchellhenke/fix-flaky-accessiblity-testing-matcher
branch
This was referenced Aug 18, 2025
mitchellhenke
pushed a commit
that referenced
this pull request
Aug 19, 2025
* LG-16538: sms one account notifications (#12408) * changelog: Upcoming Features, One Account, Notify users via text message * add changes * fix translation error and rubocop * Reset cookie when validating HTML markup in accessibility test to improve reliability (#12428) changelog: Internal, Testing, Reset cookie when validating HTML markup in accessibility test to improve reliability Co-authored-by: Davi (she/they) <davida.marion@gsa.gov> * Bump libphonenumber-js from 1.12.10 to 1.12.12 (#12431) Bumps [libphonenumber-js](https://gitlab.com/catamphetamine/libphonenumber-js) from 1.12.10 to 1.12.12. - [Changelog](https://gitlab.com/catamphetamine/libphonenumber-js/blob/master/CHANGELOG.md) - [Commits](https://gitlab.com/catamphetamine/libphonenumber-js/compare/v1.12.10...v1.12.12) --- updated-dependencies: - dependency-name: libphonenumber-js dependency-version: 1.12.12 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Update Rails dependencies to patch vulnerabilities (#12433) changelog: Internal, Maintenance, Update Rails dependencies to patch vulnerabilities * Update rubyzip (#12435) changelog: Internal, Maintenance, Update rubyzip --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Malick Diarra <malick.diarra@gsa.gov> Co-authored-by: Davi (she/they) <davida.marion@gsa.gov> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🛠 Summary of changes
An additional check was added in #7640 to better ensure HTML markup validity. It requires some more complex uses of Capybara drivers, and one of the (unintuitive) side effects is that the cookie that is set in the rack driver can persist beyond the current test. For a consistent reproduction, I have been running:
The result of the existing behavior is a test that follows one that calls
expect_page_to_have_no_accessibility_violationsmay have an existing session. It is not a more consistent issue as the data in the session is frequently overwritten (especially warden-based elements). The specific issue for this more common failure is the SP request may still be persisted in the session, but the test expects the content to be presented based on no SP being present.The proposed fix is to set the cookie to an empty string, which seems to address the issue in my testing, at least for now. We may want to consider different approaches as the behavior is deeper in the Capybara driver and harder to reason about.