-
Notifications
You must be signed in to change notification settings - Fork 202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add baseline functionality for running Playwright tests against live environments #5075
Conversation
10f49eb
to
50f0eef
Compare
Latest k6 run output1
Footnotes
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you so much for adding this!
I like the idea of adding this in the test
fixture.
|
||
const encoder = new TextEncoder() | ||
|
||
const signingSecret = process.env.HMAC_SIGNING_SECRET |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to add this env variable to the infrastructure repository to get this to work, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's the same secret used for signing requests in k6, so no need for a new secret (though you could want to change the name to make it generic instead of k6 specific). See here in the new workflow file that I passed K6_SIGNING_SECRET
for HMAC_SIGNING_SECRET
:
38e4301
to
e9a39b9
Compare
…environments (WordPress#5075) * Add HMAC signing capability for playwright tests * Add workflow to run Playwright smoketests in CI * Remove testing code for new workflow
Fixes
Related to #4706 by @sarayourfriend
Description
This PR introduces the same HMAC request signing strategy used for #4908 tests against live environments to use with Playwright. This allows Playwright tests to run against the live environment and bypass Cloudflare's WAF, otherwise it gets blocked as automated traffic.
To enable the signing transparently, I've created an override of the
page
fixture that automatically adds the HMAC headers to requests. This requires usingtest
imported from~~/test/playwright/utils/test
so the fixture is always available. I've updated all imports oftest
to use the new location and added an ESLint rule to encourage importing from the fixture-enabled location.Additionally, I've added a new workflow file similar to the k6 workflow for running the Playwright smoke tests. The smoke tests are identified via the test annotation
@deployment-smoketest
. The workflow is dispatched after staging deployments, before the load test runs.For this initial integration, I've just tagged some keyboard navigation tests with
@deployment-smoketest
, but to fully close the issue, more tests should be included.Testing Instructions
CI should pass, regular Playwright test runs should be unaffected.
You should be able to run the deployment smoke tests locally using the following command:
Maintainers can pull the signing secret as needed.
I also added temporary triggers for the new workflow file to run on pull_requests, and you can check out the successful run of that at this link. I've removed the pull_request trigger now, so it will only run when dispatched, just like the k6 workflow.
Checklist
Update index.md
).main
) or a parent feature branch.ov just catalog/generate-docs
for catalogPRs) or the media properties generator (
ov just catalog/generate-docs media-props
for the catalog or
ov just api/generate-docs
for the API) where applicable.Developer Certificate of Origin
Developer Certificate of Origin