diff --git a/.agents/skills/_shared/git-github-hard-stop.md b/.agents/skills/_shared/git-github-hard-stop.md new file mode 100644 index 00000000000..eb3cb4aa6bc --- /dev/null +++ b/.agents/skills/_shared/git-github-hard-stop.md @@ -0,0 +1,20 @@ + + + +# Git and GitHub Access Hard Stop + +Use this guardrail from any workflow that runs `git`, `ssh`, or `gh` commands. + +If a Git/GitHub command fails because of authentication, authorization, missing credentials, SSO, token scope, SSH key setup, remote access, or push permissions, stop and ask the user to resolve access. + +Do **not** work around access failures by: + +- switching remote protocols or remotes; +- editing credentials, tokens, or SSH config; +- generating new tokens or SSH keys; +- rewriting remotes to bypass permissions; +- force-pushing or bypassing branch protections/required checks. + +Report the exact command, the relevant error output, and the next action needed from the user, then wait. + +This hard stop is for access/authentication/authorization problems only. Normal Git workflow problems such as merge conflicts, stale branches, dirty worktrees, or mechanical rebase conflicts should be handled by the relevant workflow. Stop for user guidance only when conflict resolution would change behavior, alter contributor intent, or require a design decision. diff --git a/.agents/skills/_shared/pr-follow-up.md b/.agents/skills/_shared/pr-follow-up.md new file mode 100644 index 00000000000..b86d66a60a5 --- /dev/null +++ b/.agents/skills/_shared/pr-follow-up.md @@ -0,0 +1,42 @@ + + + +# PR CI and Automated Review Follow-Up + +Use this workflow after creating a PR and after every push to an open PR. + +## Watch checks + +```bash +PR_NUMBER=${PR_NUMBER:-$(gh pr view --json number -q .number)} +gh pr checks "$PR_NUMBER" --watch +``` + +Then inspect the settled check state: + +```bash +gh pr view "$PR_NUMBER" --json url,statusCheckRollup,comments,reviews,reviewDecision +``` + +## Inspect automated feedback + +Check sticky PR comments and inline review comments from CodeRabbit and the PR Review Advisor: + +```bash +gh api "repos/NVIDIA/NemoClaw/issues/${PR_NUMBER}/comments" --paginate \ + --jq '.[] | select((.body // "") | test("CodeRabbit|coderabbit|PR Review Advisor|nemoclaw-pr-review-advisor"; "i")) | {author: .user.login, updated_at, body}' + +gh api "repos/NVIDIA/NemoClaw/pulls/${PR_NUMBER}/comments" --paginate \ + --jq '.[] | select((.body // "") | test("CodeRabbit|coderabbit|PR Review Advisor|nemoclaw-pr-review-advisor"; "i")) | {author: .user.login, path, line, updated_at, body}' +``` + +## Triage + +- **CI failure:** inspect the failing job logs, fix the root cause, rerun relevant local checks, commit, push, and monitor again. +- **CodeRabbit or PR Review Advisor correctness/security/test-coverage finding:** address it when valid, rerun relevant checks, commit, push, and monitor again. +- **Style nits or false positives:** avoid unnecessary churn. Note the rationale in your final report or comment on the PR when reviewer-visible context is useful. +- **Ambiguous, risky, broad, or design-changing feedback:** stop and consult the user before changing code. + +Repeat until required CI is green and there are no unresolved actionable CodeRabbit or PR Review Advisor findings, or until the user tells you to stop. + +If any follow-up push or `gh`/GitHub query hits SSH, authentication, remote access, authorization, or permission problems, follow [Git and GitHub Access Hard Stop](git-github-hard-stop.md). Do not stop for ordinary merge conflicts or dirty-worktree state; resolve mechanical conflicts in the relevant workflow and ask the user only when resolution would change behavior or contributor intent. diff --git a/.agents/skills/nemoclaw-contributor-create-pr/SKILL.md b/.agents/skills/nemoclaw-contributor-create-pr/SKILL.md index 3a7636c4077..abbd50b3d53 100644 --- a/.agents/skills/nemoclaw-contributor-create-pr/SKILL.md +++ b/.agents/skills/nemoclaw-contributor-create-pr/SKILL.md @@ -1,6 +1,6 @@ --- name: nemoclaw-contributor-create-pr -description: Create GitHub pull requests that follow the NemoClaw PR template. Use when the user wants to create a new PR, submit code for review, open a pull request, or push changes for review. Trigger keywords - create PR, pull request, new PR, submit for review, open PR, push for review. +description: Create GitHub pull requests that follow the NemoClaw PR template, then monitor CI and automated review feedback. Use when the user wants to create a new PR, submit code for review, open a pull request, or push changes for review. Trigger keywords - create PR, pull request, new PR, submit for review, open PR, push for review. --- # Create GitHub Pull Request @@ -14,6 +14,10 @@ Create pull requests on the NemoClaw GitHub repository using the `gh` CLI. This - You must have commits on a branch that is pushed to the remote. - The PR description must include a valid DCO `Signed-off-by:` declaration, and every commit that will appear in the PR must appear as `Verified` in GitHub. +## Hard Stop: Git, SSH, and Authentication Problems + +Follow the shared [Git and GitHub Access Hard Stop](../_shared/git-github-hard-stop.md) guardrail for SSH, authentication, remote access, authorization, or permission failures. Resolve ordinary Git workflow problems such as merge conflicts or dirty worktrees in the current workflow. + ## Step 1: Verify Branch State Before creating a PR, verify the branch. @@ -86,6 +90,8 @@ Ensure the branch is pushed to the remote. git push -u origin HEAD ``` +If the push fails because of SSH, authentication, remote access, authorization, or permission problems, follow [Git and GitHub Access Hard Stop](../_shared/git-github-hard-stop.md). Resolve ordinary non-access Git failures, such as merge conflicts or dirty worktrees, in the current workflow. + ## Step 4: Prepare DCO Declaration and Verify GitHub Commits Before creating the PR, prepare the DCO declaration for the PR body and verify every commit in `main..HEAD`. @@ -215,7 +221,7 @@ gh pr create \ --title "(): " \ --assignee "@me" \ --body "$(cat <<'EOF' - + EOF )" ``` @@ -237,17 +243,23 @@ For work-in-progress that is not ready for review: gh pr create --draft --title "..." --assignee "@me" --body "..." ``` -## Step 8: Report the Result +## Step 8: Monitor CI and Review Feedback + +After creating the PR, do not stop at the URL. Follow the shared [PR CI and Automated Review Follow-Up](../_shared/pr-follow-up.md) workflow: watch required CI, inspect CodeRabbit and PR Review Advisor feedback, address valid findings, and consult the user when feedback is ambiguous or design-changing. + +## Step 9: Report the Result -After the PR is created, display the PR URL as a clickable markdown link: +After the PR is created and the initial CI/reviewer follow-up is handled, display the PR URL as a clickable markdown link and summarize the status: ```text Created PR [#NNN](https://github.com/NVIDIA/NemoClaw/pull/NNN) +CI: passing/pending/failing +Automated review: no actionable findings / addressed findings / waiting on user ``` ## Common Mistakes to Avoid -- **Do not invent your own PR body format.** Use the template from Step 5 exactly. +- **Do not invent your own PR body format.** Use the template from Step 6 exactly. - **Do not omit sections.** Even if a section is not applicable, keep it with the "Skip if..." comment. - **Do not check boxes for steps you did not run.** If you did not run `npm run docs`, leave that box unchecked. - **Do not rerun hook-covered checks by default.** Normal commit and push hooks are valid verification. Use `npx prek run --from-ref main --to-ref HEAD` as the fallback when hooks were skipped, missing, or uncertain. @@ -257,3 +269,5 @@ Created PR [#NNN](https://github.com/NVIDIA/NemoClaw/pull/NNN) - **Do not rely on maintainers to repair contributor signature history.** If force-push is not allowed and the branch contains an unverified commit, use a fresh branch and fresh PR. - **Do not forget `--assignee @me`.** Every PR must be assigned to its creator. - **Do not create PRs from main.** Always use a feature branch. +- **Do not troubleshoot Git/GitHub access in-agent.** If SSH, `gh`, authentication, remote access, authorization, or push permissions fail, stop and ask the user to fix access. Do resolve ordinary merge conflicts and dirty-worktree state when the workflow calls for it. +- **Do not abandon the PR immediately after creation.** Watch CI and automated feedback from CodeRabbit and the PR Review Advisor, address valid findings, and consult the user when feedback is ambiguous. diff --git a/.agents/skills/nemoclaw-contributor-update-docs/SKILL.md b/.agents/skills/nemoclaw-contributor-update-docs/SKILL.md index c1007e12c9d..8539f859273 100644 --- a/.agents/skills/nemoclaw-contributor-update-docs/SKILL.md +++ b/.agents/skills/nemoclaw-contributor-update-docs/SKILL.md @@ -239,6 +239,7 @@ Commit changes and open a pull request with a concise summary of the doc updates Apply the `area: docs`, `area: skills`, and next-patch release label so reviewers can identify doc-only changes for the next train and generated skill updates. When creating the PR with `gh pr create`, pass all labels, for example a post-release docs refresh for `0.0.63` uses `--label "area: docs" --label "area: skills" --label v0.0.64`. If the release label does not exist, report that instead of substituting another label. +Follow `nemoclaw-contributor-create-pr` for the PR mechanics, including [Git and GitHub Access Hard Stop](../_shared/git-github-hard-stop.md) and [PR CI and Automated Review Follow-Up](../_shared/pr-follow-up.md). ## Tips @@ -270,3 +271,4 @@ User says: "Catch up the docs for everything merged since v0.1.0." ``` If the next-patch release label does not exist, report that the PR was created without the release label or that PR creation failed because the label was missing. + Follow up after PR creation using [PR CI and Automated Review Follow-Up](../_shared/pr-follow-up.md); use [Git and GitHub Access Hard Stop](../_shared/git-github-hard-stop.md) if access or authentication blocks progress. diff --git a/.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md b/.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md index a1db694bac1..100c5b0878d 100644 --- a/.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md +++ b/.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md @@ -1,6 +1,6 @@ --- name: nemoclaw-maintainer-cut-release-tag -description: Creates deterministic NemoClaw semver release tags on origin/main and drafts release notes. Use when cutting a release, tagging a version, shipping a build, creating vX.Y.Z tags, or preparing release announcements. +description: Creates deterministic NemoClaw semver release tags on origin/main, handles release housekeeping, and drafts release notes. Use when cutting a release, tagging a version, shipping a build, creating vX.Y.Z tags, or preparing release announcements. user_invocable: true --- @@ -9,9 +9,9 @@ user_invocable: true # Cut Release Tag -Use the release scripts only. Do not run raw `git tag`, `git push`, `gh api`, or version-bump commands by hand for the normal release flow. +Use the release scripts for normal release operations. Do not run raw `git tag`, `git push`, `gh api`, or version-bump commands by hand for the normal release flow. -The release is one annotated semver tag on an already-merged `origin/main` commit. The GitHub workflow moves `latest`; release admins promote `lkg` manually after validation. +The release is one annotated semver tag on an already-merged `origin/main` commit. The GitHub workflow moves `latest`; release admins promote `lkg` manually after validation. After the tag is cut, finish version-label housekeeping for remaining open issues/PRs and draft release notes for the maintainer to post. ## Hard Rules @@ -21,6 +21,7 @@ The release is one annotated semver tag on an already-merged `origin/main` commi - Never push `latest` or `lkg` from this skill. - Never move, delete, or force-push an existing remote semver tag unless the maintainer explicitly starts protected-tag remediation. - Draft release notes locally. Do not create the GitHub Discussion; the maintainer does that. +- Follow the shared [Git and GitHub Access Hard Stop](../_shared/git-github-hard-stop.md) for SSH, authentication, remote access, authorization, or permission failures. ## Workflow @@ -32,8 +33,9 @@ Release Progress: - [ ] Step 2: Show plan and exact confirmation phrase - [ ] Step 3: Cut the semver tag from the confirmed plan - [ ] Step 4: Wait for workflow-managed latest -- [ ] Step 5: Generate release-note data and draft Markdown -- [ ] Step 6: Hand off announcement steps +- [ ] Step 5: Housekeep remaining open issues/PRs +- [ ] Step 6: Generate release-note data and draft Markdown +- [ ] Step 7: Hand off announcement steps ``` ### Step 1: Generate Release Plan @@ -63,7 +65,8 @@ Read the generated `plan.json` and show the maintainer: - target `origin/main` commit and headline, - plan hash, - forbidden operations, -- exact confirmation phrase. +- exact confirmation phrase, +- open issue/PR housekeeping plan for the release label. Ask the maintainer to paste the exact phrase: @@ -87,7 +90,7 @@ The script verifies a clean worktree, unchanged `origin/main`, tag availability, /cut-result.json ``` -If the script fails, stop and report the error. Do not improvise git commands. +If the script fails because of SSH, authentication, remote access, authorization, or permissions, follow [Git and GitHub Access Hard Stop](../_shared/git-github-hard-stop.md). For other precondition failures, report the failed precondition and use the recovery guidance below. Do not improvise git commands. ### Step 4: Wait for Workflow-Managed `latest` @@ -105,7 +108,28 @@ The script waits until `vX.Y.Z^{}` and `latest^{}` both peel to the planned comm If it fails, report the failed workflow/status. Do not manually move `latest`. -### Step 5: Generate Release-Note Data and Draft Markdown +### Step 5: Housekeep Remaining Open Issues/PRs + +Move any remaining open issues or PRs labeled with the released version to the next patch label: + +```bash +node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-day/scripts/bump-stragglers.ts +``` + +Then verify the released version has no open stragglers: + +```bash +gh issue list --repo NVIDIA/NemoClaw --state open --label --limit 100 +gh pr list --repo NVIDIA/NemoClaw --state open --label --limit 100 +``` + +Summarize: + +- shipped/closed items that remain associated with ``; +- open issues/PRs bumped to ``; +- any items that need manual maintainer attention. + +### Step 6: Generate Release-Note Data and Draft Markdown Collect deterministic release-note input: @@ -129,7 +153,7 @@ Draft release notes from `notes-data.json` using the style from `nemoclaw-mainta Do not create or update a GitHub Discussion. -### Step 6: Hand Off Announcement +### Step 7: Hand Off Announcement Return: @@ -138,6 +162,7 @@ Return: - plan path and plan hash, - `cut-result.json`, `latest-result.json`, and `notes-data.json` paths, - Markdown draft path, +- issue/PR housekeeping summary, - suggested discussion title: `NemoClaw is out`, - reminder: maintainer creates the Announcement discussion and shares its link in external channels. @@ -149,3 +174,4 @@ Return: - `latest` workflow fails or times out: report the workflow/status; do not move `latest` manually. - `latest` workflow rejects a rollback: keep `latest` unchanged, inspect the plan target commit, and regenerate the plan for the current `origin/main` tip if appropriate. - `lkg` changed: stop and escalate to a release admin. +- Housekeeping finds open items that should still ship in the released version: stop and ask the maintainer whether to leave the label or bump them. diff --git a/.agents/skills/nemoclaw-maintainer-day/MERGE-GATE.md b/.agents/skills/nemoclaw-maintainer-day/MERGE-GATE.md index eb26449ba3b..46ab27b43b8 100644 --- a/.agents/skills/nemoclaw-maintainer-day/MERGE-GATE.md +++ b/.agents/skills/nemoclaw-maintainer-day/MERGE-GATE.md @@ -9,7 +9,8 @@ For the full priority list see [PR-REVIEW-PRIORITIES.md](PR-REVIEW-PRIORITIES.md 1. **CI green** — all required checks in `statusCheckRollup`. 2. **No conflicts** — `mergeStateStatus` clean. 3. **No major CodeRabbit** — ignore style nits; block on correctness/security bugs. -4. **Risky code tested** — see [RISKY-AREAS.md](RISKY-AREAS.md). Confirm tests exist (added or pre-existing). +4. **No unresolved actionable PR Review Advisor findings** — correctness, security, acceptance, and test-depth findings block until addressed or explicitly judged false-positive. +5. **Risky code tested** — see [RISKY-AREAS.md](RISKY-AREAS.md). Confirm tests exist (added or pre-existing). ## Step 1: Run the Gate Checker @@ -17,7 +18,7 @@ For the full priority list see [PR-REVIEW-PRIORITIES.md](PR-REVIEW-PRIORITIES.md node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-day/scripts/check-gates.ts ``` -This checks all 4 gates programmatically and returns structured JSON with `allPass` and per-gate `pass`/`details`. +This checks the deterministic gates programmatically and returns structured JSON with `allPass` and per-gate `pass`/`details`. PR Review Advisor follow-up remains a manual review step; use [PR CI and Automated Review Follow-Up](../_shared/pr-follow-up.md) for the shared triage loop. ## Step 2: Interpret Results @@ -28,15 +29,16 @@ The script handles the deterministic checks. You handle judgment calls: - **CI failing but narrow:** Follow the salvage workflow in [SALVAGE-PR.md](SALVAGE-PR.md). - **CI pending:** Wait and re-check. Do not approve while checks are still running. - **CodeRabbit:** Script flags unresolved major/critical threads. Review the `snippet` to confirm it's a real issue vs style nit. If doubt, leave unapproved. +- **PR Review Advisor:** Read the latest sticky advisor comment and apply [PR CI and Automated Review Follow-Up](../_shared/pr-follow-up.md). Valid correctness, security, acceptance-coverage, and test-depth findings block approval unless explicitly judged false-positive. - **Tests:** If `riskyCodeTested.pass` is false, follow [TEST-GAPS.md](TEST-GAPS.md). ## Step 3: Approve or Report -**Approve only when:** `allPass` is true AND `mergeStateStatus` is not DIRTY. Approving a PR with conflicts is wasted effort — the rebase will invalidate the approval. +**Approve only when:** `allPass` is true, `mergeStateStatus` is not DIRTY, and the latest PR Review Advisor comment has no unresolved actionable findings. Approving a PR with conflicts is wasted effort — the rebase will invalidate the approval. The correct sequence for a conflicted PR: **salvage (rebase) → CI green → approve → report ready for merge.** -**All pass + no conflicts:** Approve and summarize why. +**All pass + no conflicts + no actionable PR Review Advisor findings:** Approve and summarize why. **Any fail:** diff --git a/.agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md b/.agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md index 1c9ea7b3a17..81585c0a629 100644 --- a/.agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md +++ b/.agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md @@ -8,7 +8,8 @@ Ordered list of what NemoClaw maintainers look for in a pull request. Higher ite 2. **CI green** — all required checks in `statusCheckRollup` must pass. 3. **No merge conflicts** — `mergeStateStatus` must be clean. 4. **No unresolved major/critical CodeRabbit findings** — correctness and safety findings block; style nits do not. Use judgment on borderline cases. -5. **Tests for touched risky code** — risky areas must have test coverage, either added in the PR or pre-existing. No exceptions. +5. **No unresolved actionable PR Review Advisor findings** — correctness, security, acceptance-coverage, and test-depth findings block unless explicitly judged false-positive. Ask the user before acting on ambiguous or design-changing advice. +6. **Tests for touched risky code** — risky areas must have test coverage, either added in the PR or pre-existing. No exceptions. ## Quality expectations (block if violated, but fixable via salvage) @@ -29,7 +30,7 @@ The team follows a daily ship cycle. All maintainer skills operate within this r 1. **Morning** (`/nemoclaw-maintainer-morning`) — triage the backlog, pick items for the day, label them with the target version (e.g., `v0.0.8`). 2. **During the day** (`/nemoclaw-maintainer-day`) — land PRs using the maintainer loop. Version labels make progress visible on dashboards. -3. **Evening** (`/nemoclaw-maintainer-evening`) — check what shipped, bump open items to the next version (`v0.0.9`), generate a QA-focused summary, and cut the tag. +3. **Evening** (`/nemoclaw-maintainer-evening`) — check what shipped, bump open items to the next version (`v0.0.9`), generate a QA-focused summary, cut the tag, and prepare release notes for posting. 4. **Overnight** — QA team (different timezone) tests the tag. Any issues they file enter the next morning's triage like any other issue. Version labels are living markers: they always mean "ship in this version." If an item doesn't make the cut, the label moves to the next patch version. diff --git a/.agents/skills/nemoclaw-maintainer-day/SALVAGE-PR.md b/.agents/skills/nemoclaw-maintainer-day/SALVAGE-PR.md index 29cb952d8cd..3154a419ef6 100644 --- a/.agents/skills/nemoclaw-maintainer-day/SALVAGE-PR.md +++ b/.agents/skills/nemoclaw-maintainer-day/SALVAGE-PR.md @@ -1,3 +1,6 @@ + + + # Salvage PR Workflow Take one near-mergeable PR and make the smallest safe change to unblock it. @@ -57,7 +60,9 @@ Use only commands matching the changed area. ## Step 8: Push -Push when: fix is small, improves mergeability, validation passed, you have push permission. Never force-push. If you cannot push, prepare a comment describing the fix. +Push when: fix is small, improves mergeability, validation passed, you have push permission. Never force-push. + +If the push fails because of SSH, authentication, remote access, authorization, or permission problems, follow [Git and GitHub Access Hard Stop](../_shared/git-github-hard-stop.md). Resolve ordinary merge conflicts or dirty-worktree state in the salvage workflow. **Fork PRs:** Most PRs come from contributor forks. Check where to push: @@ -73,9 +78,13 @@ git push git@github.com:/.git : Do **not** push to `origin` — that creates a separate branch on NVIDIA/NemoClaw that won't appear in the PR. -## Step 9: Route to Merge Gate +## Step 9: Monitor After Push + +After any maintainer push, follow [PR CI and Automated Review Follow-Up](../_shared/pr-follow-up.md) before routing onward. Keep salvage narrow: address valid correctness, security, and test-coverage findings with the smallest safe follow-up; consult the user when feedback is ambiguous, design-changing, or outside the salvage scope. + +## Step 10: Route to Merge Gate -If PR looks ready, follow [MERGE-GATE.md](MERGE-GATE.md). +If PR looks ready after CI and automated feedback settle, follow [MERGE-GATE.md](MERGE-GATE.md). ## Notes diff --git a/.agents/skills/nemoclaw-maintainer-day/SECURITY-SWEEP.md b/.agents/skills/nemoclaw-maintainer-day/SECURITY-SWEEP.md index dab2ab59359..8d52b25a331 100644 --- a/.agents/skills/nemoclaw-maintainer-day/SECURITY-SWEEP.md +++ b/.agents/skills/nemoclaw-maintainer-day/SECURITY-SWEEP.md @@ -1,3 +1,6 @@ + + + # Security Sweep Workflow Review a security-sensitive item before it enters the normal fast path. @@ -8,7 +11,7 @@ The morning triage and `find-review-pr` already surface security-labeled PRs. St ## Step 2: Gather Context -Read the PR or issue, all comments, linked items, changed files, diff, current checks, and recent relevant `main` commits. +Read the PR or issue, all comments, CodeRabbit findings, PR Review Advisor feedback, linked items, changed files, diff, current checks, and recent relevant `main` commits. ## Step 3: Classify Risk diff --git a/.agents/skills/nemoclaw-maintainer-day/SKILL.md b/.agents/skills/nemoclaw-maintainer-day/SKILL.md index ba02b8191d3..e6b3259d2c2 100644 --- a/.agents/skills/nemoclaw-maintainer-day/SKILL.md +++ b/.agents/skills/nemoclaw-maintainer-day/SKILL.md @@ -29,7 +29,7 @@ The first script determines the target version. The second shows shipped vs open From the open version-targeted items, pick the highest-value one: -1. **Ready-now PR** — green CI, no conflicts, no major CodeRabbit, has tests → follow [MERGE-GATE.md](MERGE-GATE.md) +1. **Ready-now PR** — green CI, no conflicts, no major CodeRabbit or PR Review Advisor findings, has tests → follow [MERGE-GATE.md](MERGE-GATE.md) 2. **Salvage-now PR** — close to ready, needs small fix → follow [SALVAGE-PR.md](SALVAGE-PR.md) 3. **Security item** — touches risky areas → follow [SECURITY-SWEEP.md](SECURITY-SWEEP.md) 4. **Test-gap item** — risky code with weak tests → follow [TEST-GAPS.md](TEST-GAPS.md) diff --git a/.agents/skills/nemoclaw-maintainer-day/scripts/bump-stragglers.ts b/.agents/skills/nemoclaw-maintainer-day/scripts/bump-stragglers.ts index 8150a01e2ab..2fb7b08bcc8 100644 --- a/.agents/skills/nemoclaw-maintainer-day/scripts/bump-stragglers.ts +++ b/.agents/skills/nemoclaw-maintainer-day/scripts/bump-stragglers.ts @@ -10,7 +10,9 @@ * Usage: node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-day/scripts/bump-stragglers.ts [--repo OWNER/REPO] */ -import { run, parseStringArg } from "./shared.ts"; +import { execFileSync } from "node:child_process"; + +import { parseStringArg } from "./shared.ts"; interface BumpedItem { number: number; @@ -35,23 +37,12 @@ function main(): void { const repo = parseStringArg(args, "--repo", "NVIDIA/NemoClaw"); - // Create target label if needed - run("gh", [ - "label", - "create", - to, - "--repo", - repo, - "--description", - "Release target", - "--color", - "1d76db", - ]); + ensureReleaseLabel(repo, to); const bumped: BumpedItem[] = []; // Bump open PRs - const prOut = run("gh", [ + const prs = ghJsonArray<{ number: number; title: string }>([ "pr", "list", "--repo", @@ -65,30 +56,23 @@ function main(): void { "--limit", "100", ]); - if (prOut) { - try { - const prs = JSON.parse(prOut) as Array<{ number: number; title: string }>; - for (const pr of prs) { - run("gh", [ - "pr", - "edit", - String(pr.number), - "--repo", - repo, - "--remove-label", - from, - "--add-label", - to, - ]); - bumped.push({ number: pr.number, title: pr.title, type: "pr" }); - } - } catch { - /* ignore */ - } + for (const pr of prs) { + gh([ + "pr", + "edit", + String(pr.number), + "--repo", + repo, + "--remove-label", + from, + "--add-label", + to, + ]); + bumped.push({ number: pr.number, title: pr.title, type: "pr" }); } // Bump open issues - const issueOut = run("gh", [ + const issues = ghJsonArray<{ number: number; title: string }>([ "issue", "list", "--repo", @@ -102,30 +86,86 @@ function main(): void { "--limit", "100", ]); - if (issueOut) { - try { - const issues = JSON.parse(issueOut) as Array<{ number: number; title: string }>; - for (const issue of issues) { - run("gh", [ - "issue", - "edit", - String(issue.number), - "--repo", - repo, - "--remove-label", - from, - "--add-label", - to, - ]); - bumped.push({ number: issue.number, title: issue.title, type: "issue" }); - } - } catch { - /* ignore */ - } + for (const issue of issues) { + gh([ + "issue", + "edit", + String(issue.number), + "--repo", + repo, + "--remove-label", + from, + "--add-label", + to, + ]); + bumped.push({ number: issue.number, title: issue.title, type: "issue" }); } const output: BumpOutput = { from, to, bumped }; console.log(JSON.stringify(output, null, 2)); } +function ensureReleaseLabel(repo: string, label: string): void { + const labels = ghJsonArray<{ name: string }>([ + "label", + "list", + "--repo", + repo, + "--search", + label, + "--json", + "name", + "--limit", + "100", + ]); + if (labels.some((entry) => entry.name === label)) return; + + gh([ + "label", + "create", + label, + "--repo", + repo, + "--description", + "Release target", + "--color", + "1d76db", + ]); +} + +function ghJsonArray(args: string[]): T[] { + const output = gh(args); + try { + const parsed = JSON.parse(output) as unknown; + if (!Array.isArray(parsed)) { + throw new Error(`expected JSON array, got ${typeof parsed}`); + } + return parsed as T[]; + } catch (error) { + const reason = error instanceof Error ? error.message : String(error); + throw new Error(`Failed to parse gh JSON output for gh ${args.join(" ")}: ${reason}`); + } +} + +function gh(args: string[]): string { + try { + return execFileSync("gh", args, { + encoding: "utf-8", + maxBuffer: 10 * 1024 * 1024, + stdio: ["ignore", "pipe", "pipe"], + }).trim(); + } catch (error) { + const errorObject = typeof error === "object" && error !== null ? error : null; + const stdout = readStringProperty(errorObject, "stdout")?.trim(); + const stderr = readStringProperty(errorObject, "stderr")?.trim(); + throw new Error([`gh ${args.join(" ")} failed`, stdout, stderr].filter(Boolean).join("\n")); + } +} + +function readStringProperty(value: object | null, key: string): string | undefined { + if (!value || Array.isArray(value)) return undefined; + const property = Reflect.get(value, key); + return typeof property === "string" ? property : undefined; +} + main(); diff --git a/.agents/skills/nemoclaw-maintainer-evening/SKILL.md b/.agents/skills/nemoclaw-maintainer-evening/SKILL.md index f8aa9411f07..5da3743934d 100644 --- a/.agents/skills/nemoclaw-maintainer-evening/SKILL.md +++ b/.agents/skills/nemoclaw-maintainer-evening/SKILL.md @@ -6,7 +6,7 @@ user_invocable: true # NemoClaw Maintainer Evening -Wrap up the day: check progress, bump stragglers, summarize for QA, cut the tag. +Wrap up the day: check progress, bump stragglers, summarize for QA, cut the tag, and prepare release notes for posting. See [PR-REVIEW-PRIORITIES.md](../nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md) for the daily cadence. @@ -35,15 +35,16 @@ node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer This lists commits since the last tag, identifies risky areas touched, and suggests QA test focus areas. Format the output as a concise summary the user can paste into the tag annotation or a handoff channel. -## Step 4: Cut the Tag +## Step 4: Cut the Tag and Publish Release Notes -Load `cut-release-tag`. The version is already known — default to patch bump, but still show the commit and changelog for confirmation. +Load `cut-release-tag`. The version is already known — default to patch bump, but still show the commit, changelog, housekeeping plan, and release notes draft for confirmation. NemoClaw releases are tag-based: tag `main`, let the workflow move `latest`, verify remaining open issues/PRs have been bumped, and prepare the release notes announcement for the maintainer to post. ## Step 5: Confirm and Share -After the tag is cut, present the final summary: +After the tag is cut and release notes are drafted or posted by the maintainer, present the final summary: - **Tag**: `v0.0.8` at commit `abc1234` +- **Release notes draft**: `../nemoclaw-release-v0.0.8/release-note-draft.md` - **Shipped**: 4 items (#1234, #1235, #1236, #1237) - **Bumped to v0.0.9**: 1 item (#1238 — still needs CI fix) - **QA focus areas**: installer changes, new onboard preset @@ -58,6 +59,6 @@ node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer ## Notes -- Never cut a tag without user confirmation. +- Never cut a tag or hand off release notes without user confirmation. - If nothing was labeled or nothing shipped, ask whether to skip the tag today. - Version labels are living markers: they always mean "ship in this version." If an item slips, the label moves forward. diff --git a/.agents/skills/nemoclaw-skills-guide/SKILL.md b/.agents/skills/nemoclaw-skills-guide/SKILL.md index c264a58148e..e1dd13eb8d3 100644 --- a/.agents/skills/nemoclaw-skills-guide/SKILL.md +++ b/.agents/skills/nemoclaw-skills-guide/SKILL.md @@ -30,7 +30,7 @@ Covers the daily maintainer cadence (morning standup, daytime loop, evening hand ### `nemoclaw-contributor-*` (3 skills) For contributors to the NemoClaw codebase. -Covers creating pull requests that follow the project template, drafting documentation updates from recent commits, and onboarding new messaging channels. +Covers creating pull requests that follow the project template, monitoring CI and automated review feedback after pushing, drafting documentation updates from recent commits, and onboarding new messaging channels. ## Skill Catalog @@ -60,7 +60,7 @@ Covers creating pull requests that follow the project template, drafting documen | `nemoclaw-maintainer-policies` | Read-only maintainer workflow policy reference for Issue Type, labels, Project fields, daily release labels, triage, duplicates, blocked items, and workflow decisions. | | `nemoclaw-maintainer-cross-issue-sweep` | Scan open issues for adjacent fixes or contradiction risks when reviewing a PR. | | `nemoclaw-maintainer-day` | Daytime loop: pick the highest-value version-targeted item and execute the right workflow (merge gate, salvage, security sweep, test gaps, hotspot cooling, or sequencing). Designed for `/loop`. | -| `nemoclaw-maintainer-evening` | End-of-day handoff: check version progress, bump stragglers to the next patch, generate a QA handoff summary, and cut the release tag. | +| `nemoclaw-maintainer-evening` | End-of-day handoff: check version progress, bump stragglers to the next patch, generate a QA handoff summary, cut the release tag, and hand off release notes. | | `nemoclaw-maintainer-cut-release-tag` | Cut an annotated semver tag on a maintainer-confirmed `origin/main` commit; the GitHub workflow moves `latest`, and `lkg` stays manual. | | `nemoclaw-maintainer-release-notes` | Draft release notes from live tag/compare data, with the three-paragraph narrative, categorized change list, and external-only contributor thanks. | | `nemoclaw-maintainer-find-review-pr` | Find open PRs labeled security + priority-high, link each to its issue, detect duplicates, and present a review summary. | @@ -73,7 +73,7 @@ Covers creating pull requests that follow the project template, drafting documen | Skill | Summary | |-------|---------| -| `nemoclaw-contributor-create-pr` | Create GitHub pull requests that follow the NemoClaw PR template, including pre-PR checks, conventional commit titles, and DCO sign-off. | +| `nemoclaw-contributor-create-pr` | Create GitHub pull requests that follow the NemoClaw PR template, including pre-PR checks, conventional commit titles, DCO sign-off, post-push CI monitoring, and CodeRabbit/PR Review Advisor follow-up. | | `nemoclaw-contributor-onboard-messaging-channel` | Add or review a new messaging channel with manifest-first implementation, upstream source analysis, plugin install confirmation, reachability checks, policies, docs, and tests. | | `nemoclaw-contributor-update-docs` | Scan recent git commits for user-facing changes, draft or update documentation pages, and refresh generated user skills during release prep. | diff --git a/AGENTS.md b/AGENTS.md index ab9b6ce2942..5882b9a01b8 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -153,6 +153,14 @@ All hooks managed by [prek](https://prek.j178.dev/) (installed via `npm install` 2. Run `make check` to verify your environment is set up correctly 3. Check that `npm test` passes before starting +### Git and GitHub Access Failures + +Follow `.agents/skills/_shared/git-github-hard-stop.md`: if SSH, `gh`, authentication, authorization, remote access, or push permission fails, stop and ask the user instead of working around access. Do not stop for ordinary merge conflicts or dirty-worktree state; resolve mechanical conflicts in the relevant workflow and ask the user only when resolution would change behavior or contributor intent. + +### Pull Request Follow-Up + +Follow `.agents/skills/_shared/pr-follow-up.md`: after opening or pushing to a PR, monitor required CI and automated review comments, address valid CodeRabbit and PR Review Advisor findings, and consult the user when feedback is ambiguous or design-changing. + ### Common Patterns **Adding a CLI command:** diff --git a/scripts/bump-version.ts b/scripts/bump-version.ts index f8d94ee05ce..950e7d19e27 100644 --- a/scripts/bump-version.ts +++ b/scripts/bump-version.ts @@ -8,7 +8,7 @@ import process from "node:process"; import { pathToFileURL } from "node:url"; import YAML from "yaml"; -type Options = { +export type Options = { version: string; push: boolean; commit: boolean; @@ -16,8 +16,6 @@ type Options = { dryRun: boolean; skipTests: boolean; docsMode: "latest" | "versioned"; - createPr: boolean; - branchName: string; }; type PackageJson = { @@ -111,32 +109,23 @@ function main(): void { runTypecheckAndTests(); } - if (options.createPr) { - createReleasePr(options, previousVersion, tagName); - } else { - if (options.commit) { - git(["add", ...FILES_TO_STAGE]); - git(["commit", "-m", `chore(release): bump version to ${tagName}`]); - } + if (options.commit) { + git(["add", ...FILES_TO_STAGE]); + git(["commit", "-m", `chore(release): bump version to ${tagName}`]); + } - if (options.tag) { - git(["tag", "-a", tagName, "-m", tagName]); - updateLatestTag(tagName); - } + if (options.tag) { + git(["tag", "-a", tagName, "-m", tagName]); + } - if (options.push) { - git(["push", "origin", "HEAD"]); - if (options.tag) { - git(["push", "origin", tagName]); - git(["push", "origin", "latest", "--force"]); - } - } + if (options.push) { + git(buildReleasePushArgs(tagName, options.tag)); } log(`Version bump complete: ${previousVersion} -> ${options.version}`); } -function parseArgs(args: string[]): Options { +export function parseArgs(args: string[]): Options { let version = ""; let push = false; let commit = true; @@ -144,20 +133,12 @@ function parseArgs(args: string[]): Options { let dryRun = false; let skipTests = false; let docsMode: "latest" | "versioned" = "versioned"; - let createPr = true; - let branchName = ""; for (const arg of args) { switch (arg) { case "--push": push = true; break; - case "--create-pr": - createPr = true; - break; - case "--no-create-pr": - createPr = false; - break; case "--no-commit": commit = false; break; @@ -181,10 +162,6 @@ function parseArgs(args: string[]): Options { printUsageAndExit(0); break; default: - if (arg.startsWith("--branch=")) { - branchName = arg.slice("--branch=".length); - break; - } if (arg.startsWith("-")) { throw new Error(`Unknown flag: ${arg}`); } @@ -212,17 +189,7 @@ function parseArgs(args: string[]): Options { throw new Error("--tag requires committing; do not combine --tag with --no-commit"); } - if (createPr && push) { - throw new Error( - "--push cannot be combined with --create-pr; PR mode pushes a release branch instead", - ); - } - - if (!branchName) { - branchName = `release/${version}`; - } - - return { version, push, commit, tag, dryRun, skipTests, docsMode, createPr, branchName }; + return { version, push, commit, tag, dryRun, skipTests, docsMode }; } function printUsageAndExit(code: number): never { @@ -230,12 +197,9 @@ function printUsageAndExit(code: number): never { "Usage: npm run bump:version -- [options]", "", "Options:", - " --push Push the commit and tags to origin (non-PR mode only)", - " --create-pr Create a release PR branch and open a PR (default)", - " --no-create-pr Update the current branch directly instead of opening a PR", - " --branch=NAME Use a custom PR branch name (default: release/)", + " --push Push the commit and semver tag to origin", " --no-commit Update files but do not create a commit", - " --no-tag Update files but do not create vX.Y.Z/latest tags", + " --no-tag Update files but do not create the vX.Y.Z tag", " --dry-run Print the release plan and checks without writing files", " --skip-tests Skip npm test and typecheck verification", " --docs-latest Keep public docs URLs pointed at /latest/", @@ -469,127 +433,12 @@ function git(args: string[]): void { run("git", args); } -function createReleasePr(options: Options, previousVersion: string, tagName: string): void { - if (!options.commit) { - throw new Error("--create-pr requires commits; do not combine it with --no-commit"); - } - - ensureGhCliAvailable(); - ensureBranchDoesNotExist(options.branchName); - - git(["checkout", "-b", options.branchName]); - git(["add", ...FILES_TO_STAGE]); - git(["commit", "-m", `chore(release): bump version to ${tagName}`]); - git(["push", "-u", "origin", options.branchName]); - - const prBody = buildPrBody(previousVersion, options.version, { - ranTests: !options.skipTests, - ranFormat: false, - }); - const prUrl = run("gh", [ - "pr", - "create", - "--base", - "main", - "--head", - options.branchName, - "--title", - `chore(release): bump version to ${tagName}`, - "--body", - prBody, - ]).trim(); - - log(`Release PR created: ${prUrl}`); - log(`Review and merge the PR before creating release tags on main.`); -} - -function ensureGhCliAvailable(): void { - run("gh", ["--version"]); -} - -function ensureBranchDoesNotExist(branchName: string): void { - if (gitRefExists(`refs/heads/${branchName}`) || gitRemoteBranchExists(branchName)) { - throw new Error(`Branch already exists: ${branchName}`); - } -} - -function gitRemoteBranchExists(branchName: string): boolean { - return ( - run("git", ["ls-remote", "--exit-code", "--heads", "origin", branchName], { - allowFailure: true, - }).exitCode === 0 - ); -} - -type PrBodyOptions = { - ranTests: boolean; - ranFormat: boolean; -}; - -function buildPrBody(previousVersion: string, nextVersion: string, options: PrBodyOptions): string { - const gitUserName = run("git", ["config", "user.name"]).trim(); - const gitUserEmail = run("git", ["config", "user.email"]).trim(); - - if (!gitUserName) { - throw new Error("git config user.name is required to build the PR sign-off"); - } - if (!gitUserEmail) { - throw new Error("git config user.email is required to build the PR sign-off"); - } - - return [ - "## Summary", - `Bump NemoClaw from ${previousVersion} to ${nextVersion} across the CLI package, plugin package,`, - "blueprint manifest, installer defaults, and versioned docs references.", - "", - "## Changes", - `- bump release version from ${previousVersion} to ${nextVersion}`, - "- update installer and docs version references to match the npm/package version", - "- keep release changes isolated in a PR branch instead of updating main directly", - "", - "## Type of Change", - "- [x] Code change for a new feature, bug fix, or refactor.", - "- [ ] Code change with doc updates.", - "- [ ] Doc only. Prose changes without code sample modifications.", - "- [ ] Doc only. Includes code sample changes.", - "", - "## Testing", - `- [${options.ranFormat ? "x" : " "}] \`npx prek run --all-files\` passes (or equivalently \`make check\`).`, - `- [${options.ranTests ? "x" : " "}] \`npm test\` passes.`, - "- [ ] `npm run docs` builds without warnings. (for doc-only changes)", - "", - "## Checklist", - "", - "### General", - "", - "- [x] I have read and followed the [contributing guide](https://github.com/NVIDIA/NemoClaw/blob/main/CONTRIBUTING.md).", - "- [ ] I have read and followed the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md). (for doc-only changes)", - "", - "### Code Changes", - `- [${options.ranFormat ? "x" : " "}] Formatters applied — \`npx prek run --all-files\` auto-fixes formatting (or \`make format\` for targeted runs).`, - "- [ ] Tests added or updated for new or changed behavior.", - "- [x] No secrets, API keys, or credentials committed.", - "- [x] Doc pages updated for any user-facing behavior changes (new commands, changed defaults, new features, bug fixes that contradict existing docs).", - "", - "### Doc Changes", - '- [ ] Follows the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md). Try running the `update-docs` agent skill to draft changes while complying with the style guide. For example, prompt your agent with "`/update-docs` catch up the docs for the new changes I made in this PR."', - "- [ ] New pages include SPDX license header and frontmatter, if creating a new page.", - "- [x] Cross-references and links verified.", - "", - "---", - `Signed-off-by: ${gitUserName} <${gitUserEmail}>`, - ].join("\n"); -} - -// Keep automated releases scoped to the semver tag and `latest`. -// The public installer's `lkg` tag is promoted manually by release admins after validation. -function updateLatestTag(tagName: string): void { - log(`Updating mutable 'latest' tag to ${tagName}`); - if (gitRefExists("refs/tags/latest")) { - git(["tag", "-fa", "latest", "-m", `latest -> ${tagName}`]); - } else { - git(["tag", "-a", "latest", "-m", `latest -> ${tagName}`]); +export function buildReleasePushArgs(tagName: string, includeTag: boolean): string[] { + const args = ["push", "--atomic", "origin", "HEAD"]; + if (includeTag) { + args.push(`refs/tags/${tagName}:refs/tags/${tagName}`); } + return args; } function gitRefExists(ref: string): boolean { @@ -709,7 +558,7 @@ function printDryRunPlan( } else { log("Checks: installer version, build:cli, typecheck:cli, npm test"); } - log("No files were written. No commit, PR, tags, or pushes were performed."); + log("No files were written. No commit, tag, or push was performed."); } function log(message: string): void { diff --git a/test/bump-stragglers.test.ts b/test/bump-stragglers.test.ts new file mode 100644 index 00000000000..62dce392bcf --- /dev/null +++ b/test/bump-stragglers.test.ts @@ -0,0 +1,131 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { spawnSync } from "node:child_process"; +import fs from "node:fs"; +import os from "node:os"; +import path from "node:path"; + +import { describe, expect, it } from "vitest"; + +function runBumpStragglers(fakeGh: string) { + const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "bump-stragglers-")); + const bin = path.join(tmp, "bin"); + fs.mkdirSync(bin); + const ghPath = path.join(bin, "gh"); + fs.writeFileSync(ghPath, fakeGh); + fs.chmodSync(ghPath, 0o755); + try { + return spawnSync( + process.execPath, + [ + "--experimental-strip-types", + "--no-warnings", + ".agents/skills/nemoclaw-maintainer-day/scripts/bump-stragglers.ts", + "v1.2.3", + "v1.2.4", + ], + { + cwd: process.cwd(), + encoding: "utf-8", + env: { ...process.env, PATH: `${bin}${path.delimiter}${process.env.PATH ?? ""}` }, + }, + ); + } finally { + fs.rmSync(tmp, { recursive: true, force: true }); + } +} + +describe("bump-stragglers release housekeeping", () => { + it("fails visibly when gh label lookup fails", () => { + const result = runBumpStragglers(`#!/usr/bin/env bash +set -euo pipefail +case "$*" in + "label list"*) echo 'auth failed' >&2; exit 4 ;; + *) echo "unexpected gh args: $*" >&2; exit 9 ;; +esac +`); + + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("auth failed"); + expect(result.stdout).toBe(""); + }); + + it("fails visibly when gh returns non-array JSON", () => { + const result = runBumpStragglers(`#!/usr/bin/env bash +set -euo pipefail +case "$*" in + "label list"*) printf '{}' ;; + *) echo "unexpected gh args: $*" >&2; exit 9 ;; +esac +`); + + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("expected JSON array"); + expect(result.stdout).toBe(""); + }); + + it("does not create the target label when it already exists", () => { + const result = runBumpStragglers(`#!/usr/bin/env bash +set -euo pipefail +case "$*" in + "label list"*) printf '[{"name":"v1.2.4"}]' ;; + "label create"*) echo 'label create should not run' >&2; exit 9 ;; + "pr list"*) printf '[]' ;; + "issue list"*) printf '[]' ;; + *) echo "unexpected gh args: $*" >&2; exit 9 ;; +esac +`); + + expect(result.status).toBe(0); + expect(JSON.parse(result.stdout)).toEqual({ from: "v1.2.3", to: "v1.2.4", bumped: [] }); + }); + + it("fails visibly when gh returns invalid JSON", () => { + const result = runBumpStragglers(`#!/usr/bin/env bash +set -euo pipefail +case "$*" in + "label list"*) printf '[{"name":"v1.2.4"}]' ;; + "pr list"*) printf 'not-json' ;; + *) echo "unexpected gh args: $*" >&2; exit 9 ;; +esac +`); + + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("Failed to parse gh JSON output"); + expect(result.stdout).toBe(""); + }); + + it("fails visibly when a GitHub PR edit command fails", () => { + const result = runBumpStragglers(`#!/usr/bin/env bash +set -euo pipefail +case "$*" in + "label list"*) printf '[{"name":"v1.2.4"}]' ;; + "pr list"*) printf '[{"number":42,"title":"needs more work"}]' ;; + "pr edit"*) echo 'auth failed' >&2; exit 7 ;; + *) echo "unexpected gh args: $*" >&2; exit 9 ;; +esac +`); + + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("auth failed"); + expect(result.stdout).toBe(""); + }); + + it("fails visibly when a GitHub issue edit command fails", () => { + const result = runBumpStragglers(`#!/usr/bin/env bash +set -euo pipefail +case "$*" in + "label list"*) printf '[{"name":"v1.2.4"}]' ;; + "pr list"*) printf '[]' ;; + "issue list"*) printf '[{"number":84,"title":"still open"}]' ;; + "issue edit"*) echo 'permission denied' >&2; exit 7 ;; + *) echo "unexpected gh args: $*" >&2; exit 9 ;; +esac +`); + + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("permission denied"); + expect(result.stdout).toBe(""); + }); +}); diff --git a/test/bump-version.test.ts b/test/bump-version.test.ts index cf915849a12..ed76809478a 100644 --- a/test/bump-version.test.ts +++ b/test/bump-version.test.ts @@ -3,7 +3,32 @@ import { describe, expect, it } from "vitest"; -import { collectDocsVersionSegments, rewriteDocsPublicUrls } from "../scripts/bump-version"; +import { + buildReleasePushArgs, + collectDocsVersionSegments, + parseArgs, + rewriteDocsPublicUrls, +} from "../scripts/bump-version"; + +describe("bump-version release contract", () => { + it("rejects removed release PR flags", () => { + for (const flag of ["--create-pr", "--no-create-pr", "--branch=release/1.2.3"]) { + expect(() => parseArgs(["1.2.3", flag])).toThrow(`Unknown flag: ${flag}`); + } + }); + + it("builds semver-tag-only push args without latest", () => { + expect(buildReleasePushArgs("v1.2.3", true)).toEqual([ + "push", + "--atomic", + "origin", + "HEAD", + "refs/tags/v1.2.3:refs/tags/v1.2.3", + ]); + expect(buildReleasePushArgs("v1.2.3", false)).toEqual(["push", "--atomic", "origin", "HEAD"]); + expect(buildReleasePushArgs("v1.2.3", true).join(" ")).not.toContain("latest"); + }); +}); describe("bump-version docs URL helpers", () => { it("rewrites latest and versioned docs URL segments while preserving suffix delimiters", () => { diff --git a/test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts b/test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts index 30f264782e8..2e3e22e0a7c 100644 --- a/test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts +++ b/test/e2e-scenario/support-tests/e2e-scenarios-workflow.test.ts @@ -8,6 +8,8 @@ import path from "node:path"; import { describe, expect, it } from "vitest"; import YAML from "yaml"; + +import { testTimeoutOptions } from "../../helpers/timeouts"; import { evaluateE2eVitestWorkflowDispatchSelectors, readFreeStandingJobsInventory, @@ -75,483 +77,493 @@ describe("e2e-vitest-scenarios workflow boundary", () => { expect(validateE2eVitestScenariosWorkflowBoundary()).toEqual([]); }); - it("evaluates high-risk dispatch selector behavior before secret-bearing jobs run", () => { - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "network-policy,../escape", - }), - ).toMatchObject({ - valid: false, - liveScenariosRuns: false, - selectedFreeStandingJobs: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "network-policy-vitest", - scenarios: "network-policy", - }), - ).toMatchObject({ - valid: false, - liveScenariosRuns: false, - selectedFreeStandingJobs: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "network-policy", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["network-policy-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "network-policy,ubuntu-repo-cloud-openclaw", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: true, - selectedFreeStandingJobs: ["network-policy-vitest"], - registryScenarios: ["ubuntu-repo-cloud-openclaw"], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "openshell-version-pin", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["openshell-version-pin-vitest"], - registryScenarios: [], - }); - expect(evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "skill-agent" })).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["skill-agent-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "skill-agent-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["skill-agent-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "openclaw-skill-cli", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["openclaw-skill-cli-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "openclaw-skill-cli-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["openclaw-skill-cli-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "credential-sanitization", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["credential-sanitization-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "credential-sanitization-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["credential-sanitization-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "sessions-agents-cli", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["sessions-agents-cli-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "sessions-agents-cli-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["sessions-agents-cli-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "runtime-overrides-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["runtime-overrides-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "runtime-overrides", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["runtime-overrides-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "messaging-compatible-endpoint", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["messaging-compatible-endpoint-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "messaging-compatible-endpoint-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["messaging-compatible-endpoint-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "inference-routing", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["inference-routing-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "inference-routing-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["inference-routing-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "cloud-inference", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["cloud-inference-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "cloud-inference-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["cloud-inference-vitest"], - registryScenarios: [], - }); - expect(evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "hermes-e2e" })).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["hermes-e2e-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "hermes-root-entrypoint-smoke", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["hermes-root-entrypoint-smoke-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "hermes-root-entrypoint-smoke-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["hermes-root-entrypoint-smoke-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "common-egress-agent", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["common-egress-agent-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "common-egress-agent-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["common-egress-agent-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "shields-config", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["shields-config-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "shields-config-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["shields-config-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "rebuild-openclaw", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["rebuild-openclaw-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "rebuild-openclaw-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["rebuild-openclaw-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "state-backup-restore", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["state-backup-restore-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "state-backup-restore-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["state-backup-restore-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "model-router-provider-routed-inference", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["model-router-provider-routed-inference-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "model-router-provider-routed-inference-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["model-router-provider-routed-inference-vitest"], - registryScenarios: [], - }); - expect(evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "diagnostics" })).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["diagnostics-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ jobs: "diagnostics-vitest" }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["diagnostics-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "gateway-drift-preflight", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["gateway-drift-preflight-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "gateway-drift-preflight-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["gateway-drift-preflight-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "openclaw-inference-switch", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["openclaw-inference-switch-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "openclaw-inference-switch-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["openclaw-inference-switch-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "bedrock-runtime-compatible-anthropic", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["bedrock-runtime-compatible-anthropic-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "bedrock-runtime-compatible-anthropic-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["bedrock-runtime-compatible-anthropic-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - scenarios: "issue-2478-crash-loop-recovery", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["issue-2478-crash-loop-recovery-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "issue-2478-crash-loop-recovery-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["issue-2478-crash-loop-recovery-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "gateway-health-honest" }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["gateway-health-honest-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ jobs: "gateway-health-honest-vitest" }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["gateway-health-honest-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "concurrent-gateway-ports" }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["concurrent-gateway-ports-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ jobs: "concurrent-gateway-ports-vitest" }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["concurrent-gateway-ports-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "channels-add-remove" }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["channels-add-remove-vitest"], - registryScenarios: [], - }); - expect( - evaluateE2eVitestWorkflowDispatchSelectors({ - jobs: "channels-add-remove-vitest", - }), - ).toMatchObject({ - valid: true, - liveScenariosRuns: false, - selectedFreeStandingJobs: ["channels-add-remove-vitest"], - registryScenarios: [], - }); - }); + it( + "evaluates high-risk dispatch selector behavior before secret-bearing jobs run", + testTimeoutOptions(30_000), + () => { + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "network-policy,../escape", + }), + ).toMatchObject({ + valid: false, + liveScenariosRuns: false, + selectedFreeStandingJobs: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "network-policy-vitest", + scenarios: "network-policy", + }), + ).toMatchObject({ + valid: false, + liveScenariosRuns: false, + selectedFreeStandingJobs: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "network-policy", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["network-policy-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "network-policy,ubuntu-repo-cloud-openclaw", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: true, + selectedFreeStandingJobs: ["network-policy-vitest"], + registryScenarios: ["ubuntu-repo-cloud-openclaw"], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "openshell-version-pin", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["openshell-version-pin-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "skill-agent" }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["skill-agent-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "skill-agent-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["skill-agent-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "openclaw-skill-cli", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["openclaw-skill-cli-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "openclaw-skill-cli-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["openclaw-skill-cli-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "credential-sanitization", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["credential-sanitization-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "credential-sanitization-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["credential-sanitization-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "sessions-agents-cli", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["sessions-agents-cli-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "sessions-agents-cli-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["sessions-agents-cli-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "runtime-overrides-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["runtime-overrides-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "runtime-overrides", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["runtime-overrides-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "messaging-compatible-endpoint", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["messaging-compatible-endpoint-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "messaging-compatible-endpoint-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["messaging-compatible-endpoint-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "inference-routing", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["inference-routing-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "inference-routing-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["inference-routing-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "cloud-inference", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["cloud-inference-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "cloud-inference-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["cloud-inference-vitest"], + registryScenarios: [], + }); + expect(evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "hermes-e2e" })).toMatchObject( + { + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["hermes-e2e-vitest"], + registryScenarios: [], + }, + ); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "hermes-root-entrypoint-smoke", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["hermes-root-entrypoint-smoke-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "hermes-root-entrypoint-smoke-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["hermes-root-entrypoint-smoke-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "common-egress-agent", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["common-egress-agent-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "common-egress-agent-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["common-egress-agent-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "shields-config", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["shields-config-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "shields-config-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["shields-config-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "rebuild-openclaw", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["rebuild-openclaw-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "rebuild-openclaw-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["rebuild-openclaw-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "state-backup-restore", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["state-backup-restore-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "state-backup-restore-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["state-backup-restore-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "model-router-provider-routed-inference", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["model-router-provider-routed-inference-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "model-router-provider-routed-inference-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["model-router-provider-routed-inference-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "diagnostics" }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["diagnostics-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ jobs: "diagnostics-vitest" }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["diagnostics-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "gateway-drift-preflight", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["gateway-drift-preflight-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "gateway-drift-preflight-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["gateway-drift-preflight-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "openclaw-inference-switch", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["openclaw-inference-switch-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "openclaw-inference-switch-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["openclaw-inference-switch-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "bedrock-runtime-compatible-anthropic", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["bedrock-runtime-compatible-anthropic-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "bedrock-runtime-compatible-anthropic-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["bedrock-runtime-compatible-anthropic-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + scenarios: "issue-2478-crash-loop-recovery", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["issue-2478-crash-loop-recovery-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "issue-2478-crash-loop-recovery-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["issue-2478-crash-loop-recovery-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "gateway-health-honest" }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["gateway-health-honest-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ jobs: "gateway-health-honest-vitest" }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["gateway-health-honest-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "concurrent-gateway-ports" }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["concurrent-gateway-ports-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ jobs: "concurrent-gateway-ports-vitest" }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["concurrent-gateway-ports-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ scenarios: "channels-add-remove" }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["channels-add-remove-vitest"], + registryScenarios: [], + }); + expect( + evaluateE2eVitestWorkflowDispatchSelectors({ + jobs: "channels-add-remove-vitest", + }), + ).toMatchObject({ + valid: true, + liveScenariosRuns: false, + selectedFreeStandingJobs: ["channels-add-remove-vitest"], + registryScenarios: [], + }); + }, + ); it("derives the free-standing inventory from workflow job metadata", () => { const inventory = readFreeStandingJobsInventory(); diff --git a/tools/pr-review-advisor/README.md b/tools/pr-review-advisor/README.md index 02060d85204..8c56e6e390a 100644 --- a/tools/pr-review-advisor/README.md +++ b/tools/pr-review-advisor/README.md @@ -35,6 +35,10 @@ It intentionally does not report GitHub mergeability, branch protection, CI stat The workflow is advisory and must not be configured as a required status check. Making it required can create circular wait behavior and defeats the goal of letting it observe settled required-check state. +## Author and agent follow-up + +Authors and coding agents should follow the shared [PR CI and Automated Review Follow-Up](../../.agents/skills/_shared/pr-follow-up.md) workflow after opening a PR or pushing follow-up commits. If SSH, authentication, remote access, authorization, or permission problems prevent reading comments or pushing fixes, follow [Git and GitHub Access Hard Stop](../../.agents/skills/_shared/git-github-hard-stop.md). + ## Safety model - Static analysis only.