Skip to content

CI: Automate the "Tested up to" bump in readme.txt - #82415

Closed
ramonjd wants to merge 5 commits into
trunkfrom
add/tested-up-to-bump-workflow
Closed

ramonjd wants to merge 5 commits into
trunkfrom
add/tested-up-to-bump-workflow

Conversation

@ramonjd

@ramonjd ramonjd commented Sep 4, 2026

Copy link
Copy Markdown
Member

What

A daily scheduled workflow that opens a pull request bumping the Tested up to field in readme.txt when a new WordPress version becomes the latest stable release.

Why

The field only changes today inside the manual "bump minimum required WordPress version" chore, which lands weeks after the WordPress release. In between, nothing notices the gap: the performance workflow reads the field on every trunk push and release, so a stale value quietly pins those runs to the old wp/X.Y branch. Right now the version check API already reports 7.1 while readme.txt says 7.0, so the first run after merge will open the 7.1 bump.

What changed

  • New bump-tested-up-to.yml workflow: daily cron plus manual dispatch, restricted to WordPress/gutenberg.
  • The check runs first against a sparse checkout of readme.txt with no credentials. The write-scoped app token is only minted when there is something to push, and the app-created pull request triggers CI (the default GITHUB_TOKEN would not).
  • readme.txt may legitimately run ahead of the API (it can name an upcoming release), so only an older value triggers a bump, and only once the wp/X.Y branch exists.
  • Deduplication keys on the pull request record rather than the branch: a closed pull request counts as a human decision and is never re-proposed, while a leftover branch from an interrupted run is overwritten.
  • A network failure in the branch checks fails the run instead of reading as "branch missing".
  • The generated pull request body names the merge prerequisites: CI on the new WordPress version, and the REFERENCE_COMMIT update that docs/explanations/architecture/performance.md requires for every Tested up to change.

Manual testing

The workflow only runs on WordPress/gutenberg, so exercise the check logic locally:

  1. Run curl -sf https://api.wordpress.org/core/stable-check/1.0/ | jq -r 'with_entries(select(.value=="latest"))|keys[]' | cut -d. -f1,2 and note it prints 7.1 while readme.txt says Tested up to: 7.0.
  2. Run git ls-remote --exit-code --heads origin wp/7.1 and confirm it exits 0.
  3. Run gh pr list --head update/tested-up-to-7.1 --state all and confirm it is empty, meaning a scheduled run today would open the 7.1 bump pull request.
  4. After merge, trigger the workflow from the Actions tab and check it opens that pull request with both labels and the prerequisites in its body.

Notes

  • The app needs pull request write permission. The token step fails cleanly if the installed app does not grant it.
  • A release-time staleness warning for build-plugin-zip.yml was drafted and reverted on this branch to keep concerns separate. It can follow as its own pull request.

Poll the WordPress version check API once a day. When a new WordPress
version becomes the latest stable release, readme.txt still names an
older version, and the matching wp/X.Y branch exists, open a pull
request against trunk that bumps the "Tested up to" field.

The pull request is opened with the release app token so CI runs on it.
A branch-existence guard keeps the workflow from opening the same pull
request twice, and the readme value may run ahead of the API without
triggering anything, since it can legitimately name an upcoming release.
Add an informational job to the release workflow that compares the
"Tested up to" field in readme.txt with the latest stable WordPress
version from the version check API. When readme.txt names an older
version, the run shows a warning annotation telling the releaser to
merge the pending bump pull request first.

The job is deliberately incapable of disrupting a release: no other job
depends on it and it runs with continue-on-error, so even a broken
check only shows as an annotation.
Apply the review findings to the scheduled bump workflow:

- Skip scheduled runs on forks, which have no app credentials.
- Run the check first against a sparse checkout of readme.txt, and only
  mint the write-scoped app token when there is something to push.
- Deduplicate on the pull request record instead of branch existence, so
  a closed pull request counts as a human decision and a leftover branch
  from an interrupted run is overwritten instead of blocking forever.
- Distinguish a missing ref (exit code 2) from an infrastructure failure
  in the ls-remote guard, so outages fail loudly instead of reading as
  "branch missing".
- Apply labels on gh pr create so a labeling failure is loud, instead of
  a swallowed gh pr edit that would trip the label enforcement check.
- Pipe curl into jq, removing the versions.json temp file that survived
  a parse failure in the working tree.
- Cross-reference the stable-check query (unit-test.yml) and the readme
  parser (getTestedUpToMajor in resolve-performance-branches.mjs) that
  must stay in sync with this file.
- Reword the pull request body: merging is the compatibility
  attestation, and the REFERENCE_COMMIT update that performance.md
  requires for every "Tested up to" change is named as a merge
  prerequisite.
@ramonjd ramonjd added [Type] Build Tooling Issues or PRs related to build tooling GitHub Actions Pull requests that update GitHub Actions code labels Sep 4, 2026
@coderabbitai

coderabbitai Bot commented Sep 4, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

Replace the sed extraction of the "Tested up to" value with the
unit-tested getTestedUpToMajor() from the performance branch resolver,
so the workflow that writes the line and the resolver that reads it
share one parser. The module imports cleanly on a bare runner (Node
builtins only, no install, no side effects), and it throws on a
malformed line, failing the step with a clear error instead of an
empty-string fallback.

Add tools/release to the sparse checkout and anchor both patterns so
readme.txt files deeper in the tree are not materialized.
@Mamaduka

Mamaduka commented Sep 4, 2026

Copy link
Copy Markdown
Member

Maybe we should keep this process manual; we also do a lot of cleanup after bumping versions, which might be hard to automate - #82404.

@ramonjd

ramonjd commented Sep 4, 2026

Copy link
Copy Markdown
Member Author

Fair enough. The bump was missed in the last release: https://wordpress.org/support/topic/latest-version-of-gutenberg-2/

Maybe there could just be a non-blocking check with warnings when building the RC instead?

Happy to close this, was just a YOLO experiment

Thanks!

@Mamaduka

Mamaduka commented Sep 4, 2026 •

Copy link
Copy Markdown
Member

Maybe we should add this as a post-release task for someone who will handle the plugin after the WP release.

IMO, it's fine if version bumps happen a week or two later.

@ramonjd ramonjd closed this Sep 4, 2026
@Mamaduka
Mamaduka deleted the add/tested-up-to-bump-workflow branch September 4, 2026 05:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

GitHub Actions Pull requests that update GitHub Actions code [Type] Build Tooling Issues or PRs related to build tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants