diff --git a/.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md b/.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md index a086f0ebb9b..8a42058bf6b 100644 --- a/.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md +++ b/.agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md @@ -1,373 +1,139 @@ --- name: nemoclaw-maintainer-cut-release-tag -description: Creates deterministic NemoClaw semver release tags on origin/main after verifying the pre-tag dated changelog entry, handles release housekeeping, drafts announcement release notes, and verifies the maintainer-published Announcement. Use when cutting a release, tagging a version, shipping a build, creating vX.Y.Z tags, publishing release announcements, or completing release communication. +description: Operates NemoClaw's frozen daily release edition, including the scheduled 4 PM candidate plan, the 4 AM signed semver tag for a non-empty edition regardless of E2E state, workflow-managed latest and release-label housekeeping, manual recovery, release-note data, and Announcement handoff. Use when closing an edition, cutting or recovering a release tag, verifying latest, or explaining the daily release timeline. user_invocable: true --- -# Cut Release Tag +# Cut a NemoClaw Release Tag -Use the release scripts for normal release operations. Do not run raw `git tag`, `git push`, `gh api`, or version-bump commands by hand for the normal release flow. +Use the canonical repository `NVIDIA/NemoClaw`. Read the maintainer policy +[release train](../nemoclaw-maintainer-policies/references/release-train.md) before a release operation. -The release is one signed annotated semver tag on an already-merged `origin/main` commit. The GitHub workflow requires that tag to be GitHub-Verified, points `latest` at the exact verified tag object, carries remaining open issues/PRs to the next patch label, and deletes the released label while holding the shared release-label coordination queue; release admins promote `lkg` manually after validation. After the workflow is verified, draft release notes, then verify the maintainer-published Announcement before final handoff. +## Daily Contract -## LKG Production Image Dispatch +Use `America/Los_Angeles` for every boundary: -When a release admin creates or moves `lkg` to a commit carrying a `vX.Y.Z` tag, the `Release / LKG Brev Image` workflow dispatches the `Release Production Image` workflow in `brevdev/nemoclaw-image` on its `main` branch. -The dispatch passes the immutable semver tag instead of the mutable `lkg` tag. -The source workflow requires the `NEMOCLAW_IMAGE_DISPATCH_TOKEN` Actions secret with Actions read/write access to `brevdev/nemoclaw-image`; a missing secret fails before the API request, and the workflow summary never includes its value. -The trigger summary records the selected release tag, full commit SHA, target workflow, dispatch result, downstream run ID, and a direct link to the downstream run. -After `lkg` promotion, find and wait for the source trigger run using the promoted commit: +| Time | Contract | +|---|---| +| 8:00 AM–4:00 PM | Merge reviewed PRs. Each `main` advance asynchronously dispatches an agent review for its exact SHA range and selects every workflow E2E. | +| 4:00 PM | Stop merging. `.github/workflows/release-edition-close.yaml` freezes the latest GitHub-recorded `main` push at or before the cutoff. | +| 4:00 PM–4:00 AM | Consume the E2E runs already triggered by the edition's `main` pushes; diagnose, rerun selectively, or prepare fix PRs. Do not merge during the freeze. | +| 4:00 AM | `.github/workflows/release-edition-cut.yaml` tags the frozen candidate regardless of E2E state. | +| 4:00 AM–8:00 AM | Continue advisory E2E work and prepare next-edition fixes. | +| 8:00 AM | Hand off state to the next release doula and reopen merging. | -```bash -gh run list --repo NVIDIA/NemoClaw --workflow release-lkg-brev-image.yaml --commit --event push --limit 1 --json databaseId,status,conclusion,url -gh run watch --repo NVIDIA/NemoClaw --exit-status -gh run view --repo NVIDIA/NemoClaw --log -``` - -Extract the exact `https://github.com/brevdev/nemoclaw-image/actions/runs/` URL printed by the source run, give that link to the maintainer immediately, and tell them to follow it to terminal success. -Treat dispatch acceptance as an intermediate state, not proof of production image promotion: the downstream run must succeed and its summary must show successful runtime E2E validation and promotion of the `nemoclaw-brev-cpu` image family. -A rejected dispatch fails the trigger run but does not move or roll back `lkg`. -Deleting `lkg` does not dispatch an image build. -The downstream scheduled reconciliation remains available if the event-driven dispatch fails or is delayed. - -## Hard Rules - -- Tag only the commit captured in a generated release plan. -- Do not generate the release plan until the release-prep docs PR containing `docs/changelog/YYYY-MM-DD.mdx` and the exact planned `## vX.Y.Z` heading is merged or explicitly waived. -- Treat the dated MDX entry as the canonical release history. A conventional Release Notes page or post-tag Announcement draft cannot replace it. -- If `origin/main` changes after plan generation, regenerate the plan before cutting the tag. -- Before asking for release confirmation, satisfy the canonical [pre-tag E2E evidence policy](../nemoclaw-maintainer-policies/references/release-train.md#pre-tag-e2e-evidence) for that commit. -- Run full mode unless one existing full run for the candidate SHA contains complete workflow E2E and `Exact staging Brev Launchable` evidence. -- Ask the maintainer to paste the confirmation phrase from the plan before cutting the tag. -- Push only the semver tag (`vX.Y.Z`) from the agent-controlled step. -- Never push `latest` or `lkg` from this skill. -- Never move, delete, or force-push an existing remote semver tag unless the maintainer explicitly starts protected-tag remediation. -- Delete the released version label only after open work moves forward and a final query finds no open stragglers. Never rename or reuse a released label. -- Keep label retirement inside the `release-latest-tag` workflow so it cannot overlap the post-merge labeler. Do not run the retirement script directly. -- Draft release notes locally. Do not create the GitHub Discussion; the maintainer does that. -- Do not mark the announcement step complete until the maintainer provides a valid Discussion URL and the published Announcement is verified. -- Follow the shared [Git and GitHub Access Hard Stop](../_shared/git-github-hard-stop.md) for SSH, authentication, remote access, authorization, or permission failures. - -## Workflow - -Copy this checklist and update it as you proceed: - -```text -Release Progress: -- [ ] Step 1: Preflight and generate release plan -- [ ] Step 2: Show plan, E2E evidence, and confirmation phrase -- [ ] Step 3: Cut the semver tag from the confirmed plan -- [ ] Step 4: Wait for workflow-managed latest -- [ ] Step 5: Carry open work forward and retire the released label -- [ ] Step 6: Generate release-note data and draft Markdown -- [ ] Step 7: Wait for maintainer-published Announcement -- [ ] Step 8: Verify Announcement and hand off sharing -``` - -### Step 1: Preflight and Generate Release Plan +A merge after 4:00 PM belongs to the next edition. Never regenerate or advance the frozen candidate merely because `main` moved. -Start with one read-only pass that checks these prerequisites together: +## Authority and Invariants -- refresh `origin/main` and resolve its full SHA; -- check the target changelog heading and release-prep docs state; and -- inventory existing E2E runs for the same SHA before deciding what to dispatch. +- E2E is advisory. Never inspect E2E state to authorize, delay, cancel, or select the tag. +- Keep the required deterministic `checks` aggregate, including `npm run test:smoke` and installer integration, as the PR merge floor while live E2E runs asynchronously after merge. +- Require exactly one direct `docs/changelog/*.mdx` entry with the exact `## vX.Y.Z` heading at a non-empty candidate. No changelog waiver exists. +- Require the frozen candidate to remain an ancestor of `origin/main` and the planned previous tag to remain the newest remote semver tag. +- Treat trusted workflow-run and artifact provenance, plan consistency, tag collision checks, signing, GitHub verification, `latest`, `lkg`, carry-forward, and label retirement as fail-closed controls. +- Skip the edition without creating a tag when the frozen candidate contains no commits after the latest semver tag. +- Never move `lkg`; only a release admin may do that through a separate operation. +- Never create or update a GitHub Discussion. A maintainer publishes the Announcement. -Do not wait for merges to stop. The plan captures one candidate SHA for evidence; a late drift check advances it when `origin/main` moves. -Do not dispatch or poll a workflow during this pass. - -Before this step, confirm release-prep docs are merged or explicitly waived. -Return to `nemoclaw-maintainer-evening` if docs are still pending. - -For the planned version, inspect `origin/main` before generating the plan: - -```bash -git grep -n '^## vX\.Y\.Z$' origin/main -- 'docs/changelog/*.mdx' -``` +## Normal Scheduled Path -Require exactly one match in a dated file directly under `docs/changelog/`. -Confirm that a newly created file begins with the parser-safe MDX SPDX comment and that the entry contains its summary and detailed bullets. -If the entry is missing or malformed, return to `nemoclaw-contributor-update-docs`; do not substitute the post-tag announcement workflow. -If the maintainer explicitly waives the entry, preserve the reason in the release-plan presentation and confirmation handoff. +### 1. Prepare Before 4 PM -Run one of: +Run `/nemoclaw-contributor-update-docs for vX.Y.Z` early enough for its dated changelog PR to merge before cutoff. Confirm the exact heading at the intended candidate: ```bash -npm run release:plan -- --bump patch -npm run release:plan -- --bump minor -npm run release:plan -- --bump major -``` - -Patch is the default if the maintainer says "yes", "go", or similar without choosing. - -The script writes a plan outside the checkout root, for example: - -```text -../nemoclaw-release-v0.0.58/plan.json -``` - -### Step 2: Show Plan, E2E Evidence, and Ask for Confirmation - -Read the generated `plan.json` and show the maintainer: - -- previous tag, -- next tag, -- target `origin/main` commit and headline, -- plan hash, -- forbidden operations, -- confirmation phrase, -- open issue/PR housekeeping plan for the release label, including deletion of the released label after carry-forward succeeds. - -Unless Step 1 records an explicit waiver, verify that the plan's next tag matches the H2 version heading in the dated changelog entry at the candidate SHA. -When the entry is waived, show the recorded waiver reason in the plan presentation and confirmation handoff instead. - -For the plan's full `origin/main` SHA, review `.github/workflows/e2e.yaml` at that commit and build the evidence ledger required by the canonical [pre-tag E2E evidence policy](../nemoclaw-maintainer-policies/references/release-train.md#pre-tag-e2e-evidence). The workflow is the sole source of truth; do not substitute or maintain a separate release-gating test list. - -From a checkout whose `HEAD` is the plan candidate SHA and whose `git status --short` is empty, generate one release E2E preflight: - -```bash -CANDIDATE_SHA="" -npm run release:e2e-evidence -- \ - --candidate-sha "$CANDIDATE_SHA" \ - >"$EVIDENCE_DIR/preflight.json" -``` - -The preflight derives every required execution from one empty-selector dispatch. -The full run includes every workflow E2E and `Exact staging Brev Launchable`. -Each job that declares `RELEASE_E2E_ACTIVATION_PATH` requires that path at the candidate SHA. -A missing activation path is a preflight failure. - -Check whether one existing full run for the candidate SHA contains complete evidence. If it does not, load `nemoclaw-maintainer-e2e` and dispatch one full run. Do not combine evidence from different workflow run IDs. Do not substitute a selective run for full-run evidence. - -Monitor the dispatched correlation ID with one bounded status query. - -Before accepting full-mode exact Brev evidence, require: - -- the workflow `head_sha` to equal the plan candidate SHA; -- the trusted dispatch receipt to prove empty selectors and `include_staging_brev_launchable=true`; -- the workflow conclusion to be `success`; -- the `Exact staging Brev Launchable` job conclusion to be `success`; -- the job URL and selected successful Launchable job attempt; -- Launchable E2E identity for the same SHA; and -- cleanup evidence that reports the qualified workspace as `ABSENT`. - -Treat a skipped job as missing evidence even when the workflow concludes `success`. -If the plan candidate SHA changes, discard the run and Launchable E2E evidence. -Run full mode again for the new candidate SHA. -No release-note-only delta exception is currently defined. - -For the accepted full run, reuse `run-$RUN_ID.json` and `jobs-$RUN_ID.json` returned by `nemoclaw-maintainer-e2e`, and collect the workflow-produced dispatch receipt. -If those files were not returned, collect them once: - -```bash -gh api "repos/NVIDIA/NemoClaw/actions/runs/$RUN_ID" \ - >"$EVIDENCE_DIR/run-$RUN_ID.json" -gh api --paginate --slurp \ - "repos/NVIDIA/NemoClaw/actions/runs/$RUN_ID/jobs?filter=all&per_page=100" \ - >"$EVIDENCE_DIR/jobs-$RUN_ID.json" -ARTIFACT_PAGES="$(gh api --paginate --slurp \ - "repos/NVIDIA/NemoClaw/actions/runs/$RUN_ID/artifacts?per_page=100")" -DISPATCH_ARTIFACT_NAME="$(jq -r --arg prefix "e2e-dispatch-$RUN_ID-" \ - '[.[] | .artifacts[] | select(.expired != true and (.name | startswith($prefix)))] - | sort_by(.created_at) | last | .name // empty' <<<"$ARTIFACT_PAGES")" -test -n "$DISPATCH_ARTIFACT_NAME" -gh run download "$RUN_ID" \ - --repo NVIDIA/NemoClaw \ - --name "$DISPATCH_ARTIFACT_NAME" \ - --dir "$EVIDENCE_DIR/dispatch-$RUN_ID" -``` - -Use the latest existing receipt artifact, not the run's latest attempt number. A partial rerun can leave `generate-matrix` successful and therefore reuse its earlier receipt; the ledger permits that earlier receipt only when it binds the same run and its attempt does not exceed the run's latest attempt. - -Successful workflow E2E and `Exact staging Brev Launchable` evidence may accumulate across rerun attempts of that workflow run. Evidence from another workflow run does not satisfy the ledger. - -Create `manifest.json` in the private evidence directory: - -```json -{ - "candidateSha": "", - "runs": [ - { - "runJson": "run-123.json", - "jobsJson": "jobs-123.json", - "dispatchJson": "dispatch-123/dispatch.json" - } - ] -} +git grep -n -E '^## vX\.Y\.Z$' origin/main -- ':(glob)docs/changelog/*.mdx' ``` -Do not type empty-selector claims or selector lists into the manifest. The helper derives them from the workflow-produced receipt and rejects a receipt whose selector fields disagree with its empty-selector flag. -Build the ledger with `npm run release:e2e-evidence -- --manifest "$EVIDENCE_DIR/manifest.json"`. -The helper derives the denominator from the workflow, preserves matrix rows as separate semantic identifiers, binds every run and its actual dispatch inputs to the candidate SHA, and keeps an earlier successful attempt when a later attempt fails. -The manifest and helper cover the workflow-derived test execution ledger only. They do not replace exact Brev Launchable E2E acceptance: keep the raw `dispatch.json`, `launchable-e2e.json`, and `cleanup.json` validation in `nemoclaw-maintainer-e2e`, and carry its validated return beside this ledger or record the required Launchable E2E exception. +The output must contain exactly one match from a direct child of `docs/changelog/`. -Reject a failed workflow run before presenting the ledger. Rerun its failed jobs until the same workflow run concludes with `success`. Exceptions apply only to missing or skipped executions in that otherwise successful run. +### 2. Close the Edition at 4 PM -Before showing the confirmation prompt, present: +The scheduled close workflow: -- the candidate SHA; -- the number of tests with successful evidence out of the number required by the workflow; -- each required test mapped to a successful run or job URL and attempt; and -- when accepted full-mode exact Brev evidence exists, its workflow URL, `Exact staging Brev Launchable` job URL, selected evidence attempt, Launchable E2E identity, and cleanup result; and -- a separate itemized maintainer exception for each missing or skipped execution in the accepted successful workflow run, including its test identifier, run links, current result, and rationale; and -- a separate itemized maintainer exception for missing or invalid exact Brev Launchable E2E evidence in the accepted successful workflow run, including run and job URLs, the missing or invalid receipt, and rationale. +1. derives the edition date in `America/Los_Angeles`; +2. resolves the latest `post-merge-agent-review.yaml` push run that GitHub recorded at or before exactly 4:00 PM; +3. validates ancestry, version progression, tag absence, and the changelog entry; +4. writes a schema-v2 plan with a SHA-256 consistency hash; and +5. uploads `release-edition-plan-YYYY-MM-DD` for three days. -Do not ask for the phrase until the workflow run concludes with `success` and each test and the exact Brev Launchable E2E job has successful evidence or its own permitted itemized exception. -Immediately before asking, refresh `origin/main` once and compare its full SHA with the plan. If it moved, discard all prior candidate-bound evidence, regenerate the plan, rerun preflight and the full E2E workflow for the new SHA, capture a new manifest, and rebuild the ledger before requesting confirmation. +If a merge creates its `main` push run after 4:00 PM, the cutoff selection excludes it even when the close workflow starts late. -Exercise the configured Git signing backend before asking for confirmation: +### 3. Use the Freeze for Advisory Validation -```bash -npm run release:cut -- --plan --preflight-only -``` +Assign one agent to the complete 4:00 PM–8:00 AM loop. Load `nemoclaw-maintainer-e2e` to inventory every immutable E2E run triggered by an edition `main` push, including every selected workflow E2E, its automatic retry evidence, and any selective reruns. Classify failures as product regressions, flaky tests, infrastructure failures, or stale tests. After each result, immediately select the next actionable failure, rerun, or prepared fix. Prepare and validate fix PRs, but do not merge them until 8:00 AM. Keep the loop running after the 4:00 AM tag; stop only at handoff or transfer the same state to a replacement agent. -Require status 0. This preflight creates and deletes one local temporary tag. It does not push a ref. Git selects the maintainer's configured OpenPGP, SSH, or X.509 signer. +Keep this state for the morning handoff: -Ask the maintainer to paste this phrase: +- frozen candidate SHA and next tag; +- exact-SHA post-merge agent-review runs; +- E2E failures, classifications, reruns, and prepared fix PRs; +- unresolved release risks; and +- cut, `latest`, housekeeping, and release-note artifact URLs. -```text -CONFIRM RELEASE vX.Y.Z -``` - -Do not proceed on a generic "yes" at this step. +Do not put internal E2E classifications or rerun details in the public Announcement. -### Step 3: Cut the Semver Tag +### 4. Cut at 4 AM -Run the cut script with the plan and the maintainer's phrase: +The scheduled cut workflow selects the successful scheduled close artifact for the previous edition date. It validates the artifact-to-edition binding and invokes: ```bash -npm run release:cut -- --plan --confirm "CONFIRM RELEASE vX.Y.Z " +scripts/release-cut-tag.sh --plan --scheduled ``` -The script verifies a clean worktree, unchanged `origin/main`, tag availability, target reachability, and remote peeled tag state, then creates and pushes the signed annotated tag using the configured signing key. It writes: - -```text -/cut-result.json -``` +The script accepts unattended authority only when all of these values match: -If the script fails because of SSH, authentication, remote access, authorization, or permissions, follow [Git and GitHub Access Hard Stop](../_shared/git-github-hard-stop.md). For other precondition failures, report the failed precondition and use the recovery guidance below. Do not improvise git commands. +- `GITHUB_ACTIONS=true`; +- `GITHUB_REPOSITORY=NVIDIA/NemoClaw`; +- `GITHUB_EVENT_NAME=schedule`; and +- `GITHUB_WORKFLOW_REF=NVIDIA/NemoClaw/.github/workflows/release-edition-cut.yaml@refs/heads/main`. -### Step 4: Wait for Workflow-Managed `latest` +The `release-tag` GitHub environment must provide: -Run: +- secret `NEMOCLAW_RELEASE_TAG_SIGNING_KEY`, containing only the dedicated SSH private signing key; +- variable `NEMOCLAW_RELEASE_TAG_SIGNER_NAME`; and +- variable `NEMOCLAW_RELEASE_TAG_SIGNER_EMAIL`. -```bash -npm run release:wait-latest -- --plan -``` +Register the corresponding public key as a signing key on the dedicated GitHub release identity. Restrict the environment to `main` and do not add a human approval wait: the schedule is the release authority. `GITHUB_TOKEN` pushes the tag; the signing key never authenticates a network request. -The script waits until `vX.Y.Z` and `latest` reference the same tag object, verifies both peel to the planned commit, and verifies `lkg` did not change from the plan. It writes: +The workflow writes the private key to a mode-`0600` file under `RUNNER_TEMP` only for the cut job. The always-run step deletes that file during normal and cancellation cleanup; disposal of the ephemeral GitHub-hosted runner is the fallback if abrupt runner loss prevents the step from executing. GitHub supplies `GITHUB_TOKEN` only to the job, limits it to the declared `actions: read` and `contents: write` permissions, and expires it when the job finishes. -```text -/latest-result.json -``` +Because a tag pushed with `GITHUB_TOKEN` does not emit a second workflow run, the cut workflow directly calls `release-latest-tag.yaml`. That reusable workflow verifies the exact signed annotated tag object through GitHub, moves `latest`, carries open items to the next patch label, verifies none remain, and deletes the released label. -If it fails, report the failed workflow/status. Do not manually move `latest`. +### 5. Verify and Hand Off at 8 AM -### Step 5: Verify Carry-Forward and Label Retirement +Require the cut workflow's handoff artifact to contain: -The `release-latest-tag` workflow continues after moving `latest`: it moves every remaining open issue or PR carrying the released version to the next patch label, verifies none remain, and deletes the released label. The workflow and post-merge labeler share one queued concurrency group, so assignment cannot overlap the verification-and-delete window. +- `plan.json` and its plan hash; +- `cut-result.json` with `tagged` or `no-changes`; +- `latest-result.json` for a tagged edition; and +- `notes-data.json` for a tagged edition. -Find the workflow run started by Step 3 and wait for it to finish: +Report failed non-E2E controls immediately. Report E2E state separately as advisory next-edition work. Load `nemoclaw-maintainer-release-notes` to draft Markdown, then ask the maintainer to publish it in the `Announcements` Discussion category and return the URL for read-only verification. -```bash -RELEASE_SHA="" -mapfile -t RELEASE_RUN_IDS < <( - gh run list --repo NVIDIA/NemoClaw --workflow release-latest-tag.yaml --limit 20 \ - --event push --commit "$RELEASE_SHA" --json databaseId --jq '.[].databaseId' -) -if (( ${#RELEASE_RUN_IDS[@]} != 1 )); then - echo "Expected exactly one release-latest-tag push run for $RELEASE_SHA" >&2 - exit 1 -fi -gh run watch "${RELEASE_RUN_IDS[0]}" --repo NVIDIA/NemoClaw --exit-status -``` +## Manual Preflight and Recovery -This automatic post-tag housekeeping is covered by the release plan and confirmation in Step 2. Do not run `scripts/retire-release-label.mts` directly; doing so would bypass the coordination boundary. +A manual dispatch of `release-edition-cut.yaml` accepts an `edition_date` and downloads the trusted close artifact. For a ready edition it performs signing preflight only; for a `no-changes` edition it records that no tag is needed. It cannot publish a tag. -Then verify the released version label no longer exists: +For local recovery of the already frozen edition, download its trusted plan artifact, inspect its SHA, edition date, plan hash, changelog entry, and operations, then run: ```bash -gh label list --repo NVIDIA/NemoClaw --search --json name \ - --jq '.[] | select(.name == "")' +scripts/release-cut-tag.sh --plan --preflight-only +scripts/release-cut-tag.sh \ + --plan \ + --confirm 'CONFIRM RELEASE vX.Y.Z ' ``` -The command must return no output. Never rename the released label into a future version; a future target must be a separately created label with its own GitHub identity. +Confirmed local recovery may cut a scheduled plan whose candidate is behind current `origin/main`; ancestry must still hold. A newly generated maintainer plan remains bound to the current `origin/main` tip and becomes stale if `main` moves. -Summarize: - -- open issues/PRs moved to ``; -- released label deleted; -- any items that need manual maintainer attention. - -### Step 6: Generate Release-Note Data and Draft Markdown - -Collect deterministic release-note input: +After a locally authenticated tag push, wait for the tag-triggered `release-latest-tag.yaml` run and verify: ```bash -npm run release:notes-data -- --plan +scripts/release-wait-latest.sh --plan +node --experimental-strip-types --no-warnings scripts/release-notes-data.mts --plan ``` -This writes: - -```text -/notes-data.json -``` - -If `notes-data.json` has `status: "partial"` or non-empty `pullRequestWarnings`, report the warnings and ask the maintainer whether to fetch/fill the missing PR metadata before drafting. - -Load and follow `nemoclaw-maintainer-release-notes`, then use its output as the draft. Save only Markdown, outside the checkout root: - -```text -/release-note-draft.md -``` - -Before continuing to Step 7, verify the draft has three lead paragraphs, categorized shipped changes, one what-changed-and-why-it-matters bullet with a visible `#NNNN` link for every included change, and thanks for external contributors only. - -Do not create or update a GitHub Discussion. -Do not edit `docs/changelog/` in this post-tag step; the canonical entry must already be present in the tagged commit. - -### Step 7: Wait for Maintainer-Published Announcement - -Return: - -- release tag, -- confirmed release commit, -- plan path and plan hash, -- `cut-result.json`, `latest-result.json`, and `notes-data.json` paths, -- Markdown draft path, -- issue/PR housekeeping summary, -- suggested discussion title: `NemoClaw is out`. - -Ask the maintainer to publish the draft in the `Announcements` Discussion category and return the resulting Discussion URL. Do not create or update the Discussion. Keep Step 7 in progress until the maintainer provides the URL. - -### Step 8: Verify Announcement and Hand Off Sharing - -Before making any network request, reject the maintainer-provided URL unless it matches `https://github.com/NVIDIA/NemoClaw/discussions/` with no query string or fragment. Only then open it using a read-only GitHub or web capability and verify: - -- the title is `NemoClaw is out`; -- the category is `Announcements`; -- the body preserves the draft's three lead paragraphs, category headings, every included PR link, comparison URL, and external contributor usernames; formatting-only edits are acceptable; -- the comparison link targets `...` and visible PR links target `github.com/NVIDIA/NemoClaw/pull/`. - -If the Announcement is valid, return its URL with the release artifacts and mark the release workflow complete. Remind the maintainer to share that Discussion URL in the appropriate external channels. Do not create a duplicate Announcement. - -## Recovery - -- Plan generation fails: fix the named precondition, then regenerate the plan. -- Planned changelog entry is missing or malformed: stop before plan generation and run the pre-tag `nemoclaw-contributor-update-docs` workflow. Use post-release recovery only when the tag already exists. -- Full-mode E2E waits in the Launchable concurrency queue: keep the run pending until the earlier Launchable E2E job finishes. -- Full-mode E2E ran for another SHA: reject the run and dispatch full mode for the plan candidate SHA. -- `Exact staging Brev Launchable` was skipped in an otherwise successful candidate run: dispatch full mode again or record the required itemized maintainer exception. -- Launchable E2E or cleanup evidence is missing or invalid in an otherwise successful candidate run: dispatch full mode again or record the separate itemized maintainer exception. Do not infer Launchable E2E success from the workflow conclusion. -- `origin/main` moved after plan generation: regenerate the plan and ask for the new confirmation phrase. -- Remote semver tag already exists: stop; do not retag unless the maintainer explicitly starts protected-tag remediation. -- Signing preflight fails: fix the reported Git signer or signing-key failure. Run the preflight again before requesting confirmation. -- `latest` workflow fails or times out: report the workflow/status; do not move `latest` manually. -- `latest` workflow rejects a rollback: keep `latest` unchanged, inspect the plan target commit, and regenerate the plan for the current `origin/main` tip if appropriate. -- `lkg` changed: stop and escalate to a release admin. -- Post-tag housekeeping fails: report the workflow error and list items still carrying the released label. After the failure is fixed, rerun `release-latest-tag.yaml` with `` through `workflow_dispatch`; the promotion and retirement steps are idempotent, already-moved items no longer match the source label, and an already-deleted released label is treated as success. Do not run the retirement script outside the workflow. -- Announcement is not published yet: keep Step 7 in progress and return the draft path and suggested title; the tag and housekeeping remain complete. -- Announcement title, category, body, or links are wrong: ask the maintainer to edit the existing Discussion, then verify the same URL again. Do not create a replacement. After three failed verification attempts for the same Discussion, stop and escalate to a release admin. -- Announcement cannot be inspected: report the read failure and ask the maintainer to confirm access or provide a public URL; do not mark Step 8 complete. +If tag push, GitHub verification, `latest`, signing, changelog, ancestry, collision, or housekeeping fails, stop and report the exact invariant. Do not retag, move `latest`, retire labels directly, or use E2E state as a workaround. diff --git a/.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts b/.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts deleted file mode 100644 index ef5685e2009..00000000000 --- a/.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts +++ /dev/null @@ -1,587 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -import { execFileSync } from "node:child_process"; -import { readFileSync } from "node:fs"; -import path from "node:path"; -import { fileURLToPath } from "node:url"; - -import YAML from "yaml"; -import { readFreeStandingJobsInventory } from "../../../../tools/e2e/workflow-boundary.mts"; -import { - buildE2eWorkflowPlan, - type E2eWorkflowPlan, -} from "../../../../tools/e2e/workflow-plan.mts"; - -type JsonRecord = Record; -type ExecutionGroup = "default"; - -export type ReleaseE2eExecution = { - id: string; - jobId: string; - expectedName: string; - group: ExecutionGroup; -}; - -export type ReleaseE2ePreflight = { - candidateSha: string; - dispatches: { - completeRun: { - includeStagingBrevLaunchable: true; - jobs: ""; - mode: "full"; - targets: ""; - }; - }; - exceptionsRequired: string[]; - executions: ReleaseE2eExecution[]; - launchableE2eJobId: string; - requiredExecutionCount: number; -}; - -export type ReleaseE2eRunEvidence = { - dispatch: unknown; - jobs: unknown; - run: unknown; -}; - -export type ReleaseE2eLedgerEntry = ReleaseE2eExecution & { - attempts: Array<{ - attempt: number; - conclusion: string; - status: string; - jobUrl: string; - runUrl: string; - }>; - successfulEvidence?: { - attempt: number; - jobUrl: string; - runUrl: string; - }; - status: "missing" | "successful"; -}; - -export type ReleaseE2eLedger = { - candidateSha: string; - entries: ReleaseE2eLedgerEntry[]; - successfulCount: number; - missingCount: number; - requiredCount: number; -}; - -type ReleaseEvidenceManifest = { - candidateSha: string; - runs: Array<{ - dispatchJson: string; - jobsJson: string; - runJson: string; - }>; -}; - -type CliOptions = { - candidateSha?: string; - manifest?: string; - workflowPath: string; -}; - -const REPO_ROOT = path.join(path.dirname(fileURLToPath(import.meta.url)), "..", "..", "..", ".."); -const DEFAULT_WORKFLOW_PATH = path.join(REPO_ROOT, ".github", "workflows", "e2e.yaml"); -const SHA_PATTERN = /^[a-f0-9]{40}$/u; -const SAFE_REPO_PATH_PATTERN = /^(?!\/)(?!.*(?:^|\/)\.\.(?:\/|$))[^\\]+$/u; -const MATRIX_EXPRESSION_PATTERN = /\$\{\{\s*matrix\.([A-Za-z0-9_-]+)\s*\}\}/gu; - -function record(value: unknown, label: string): JsonRecord { - if (!value || typeof value !== "object" || Array.isArray(value)) { - throw new Error(`${label} must be an object`); - } - return value as JsonRecord; -} - -function stringField(value: JsonRecord, field: string, label: string): string { - const result = value[field]; - if (typeof result !== "string" || result.length === 0) { - throw new Error(`${label}.${field} must be a non-empty string`); - } - return result; -} - -function numberField(value: JsonRecord, field: string, label: string): number { - const result = value[field]; - if (!Number.isInteger(result) || (result as number) < 1) { - throw new Error(`${label}.${field} must be a positive integer`); - } - return result as number; -} - -function booleanField(value: JsonRecord, field: string, label: string): boolean { - const result = value[field]; - if (typeof result !== "boolean") { - throw new Error(`${label}.${field} must be a boolean`); - } - return result; -} - -function requireEqual(actual: unknown, expected: unknown, label: string): void { - if (actual !== expected) { - throw new Error(`${label} must equal ${JSON.stringify(expected)}`); - } -} - -function matrixRows(rawMatrix: unknown, jobId: string): JsonRecord[] { - const matrix = record(rawMatrix, `${jobId}.strategy.matrix`); - if (typeof matrix.include === "string") { - throw new Error(`${jobId} has a dynamic matrix that needs a planner-specific expansion`); - } - - const axes = Object.entries(matrix).filter( - ([key]) => - key !== "exclude" && key !== "include" && key !== "fail-fast" && key !== "max-parallel", - ); - let rows: JsonRecord[] = [{}]; - for (const [key, rawValues] of axes) { - if (!Array.isArray(rawValues) || rawValues.length === 0) { - throw new Error(`${jobId} matrix axis ${key} must be a non-empty array`); - } - rows = rows.flatMap((row) => - rawValues.map((value) => { - if (typeof value !== "string" && typeof value !== "number" && typeof value !== "boolean") { - throw new Error(`${jobId} matrix axis ${key} contains an unsupported value`); - } - return { ...row, [key]: value }; - }), - ); - } - - const excludes = Array.isArray(matrix.exclude) - ? matrix.exclude.map((row) => record(row, "exclude")) - : []; - rows = rows.filter( - (row) => - !excludes.some((excluded) => - Object.entries(excluded).every(([key, value]) => row[key] === value), - ), - ); - - if (Array.isArray(matrix.include)) { - if (axes.length > 0) { - throw new Error( - `${jobId} combines matrix axes and include rows; add explicit expansion support`, - ); - } - rows = matrix.include.map((row) => record(row, `${jobId}.strategy.matrix.include`)); - } - return rows; -} - -function renderMatrixJobName(jobId: string, rawJob: JsonRecord, row: JsonRecord): string { - const configuredName = rawJob.name; - if (configuredName !== undefined && typeof configuredName !== "string") { - throw new Error(`${jobId}.name must be a string when set`); - } - if (configuredName) { - const rendered = configuredName.replace(MATRIX_EXPRESSION_PATTERN, (_match, key: string) => { - if (!Object.hasOwn(row, key)) { - throw new Error(`${jobId}.name references missing matrix dimension ${key}`); - } - return String(row[key]); - }); - if (rendered.includes("${{ matrix.")) { - throw new Error(`${jobId}.name contains an unsupported matrix expression`); - } - return rendered; - } - return `${jobId} (${Object.values(row) - .map((value) => String(value)) - .join(", ")})`; -} - -function executionId(jobId: string, row: JsonRecord): string { - if (typeof row.id === "string" && row.id.length > 0) return `${jobId}[id=${row.id}]`; - const dimensions = Object.entries(row) - .map(([key, value]) => `${key}=${JSON.stringify(value)}`) - .join(","); - return `${jobId}[${dimensions}]`; -} - -function jobExecutions( - jobId: string, - rawJob: JsonRecord, - group: ExecutionGroup, - plan: E2eWorkflowPlan, -): ReleaseE2eExecution[] { - let rows: JsonRecord[] = []; - if (jobId === "live") rows = plan.matrix as unknown as JsonRecord[]; - else if (jobId === "shared-e2e") rows = plan.testMatrix as unknown as JsonRecord[]; - else { - const strategy = record(rawJob.strategy ?? {}, `${jobId}.strategy`); - if (strategy.matrix !== undefined) rows = matrixRows(strategy.matrix, jobId); - } - - if (rows.length === 0) { - const configuredName = rawJob.name; - return [ - { - expectedName: typeof configuredName === "string" ? configuredName : jobId, - group, - id: jobId, - jobId, - }, - ]; - } - return rows.map((row) => ({ - expectedName: renderMatrixJobName(jobId, rawJob, row), - group, - id: executionId(jobId, row), - jobId, - })); -} - -function workflowJobs(workflowPath: string): JsonRecord { - const workflow = record(YAML.parse(readFileSync(workflowPath, "utf8")), "workflow"); - return record(workflow.jobs, "workflow.jobs"); -} - -function isLaunchableE2eJob(jobId: string, job: JsonRecord): boolean { - const condition = job.if; - return ( - jobId === "staging-brev-launchable" && - typeof condition === "string" && - condition.includes("inputs.include_staging_brev_launchable") - ); -} - -function requiresConfirmedJetsonRunner(job: JsonRecord): boolean { - const runsOn = job["runs-on"]; - return typeof runsOn === "string" && runsOn.includes("inputs.allow_jetson_runner_queue"); -} - -function releaseActivationPath(job: JsonRecord, jobId: string): string | undefined { - const rawEnvironment = job.env; - if (rawEnvironment === undefined) return undefined; - const environment = record(rawEnvironment, `workflow.jobs.${jobId}.env`); - const activationPath = environment.RELEASE_E2E_ACTIVATION_PATH; - if (activationPath === undefined) return undefined; - if ( - typeof activationPath !== "string" || - activationPath.length === 0 || - !SAFE_REPO_PATH_PATTERN.test(activationPath) - ) { - throw new Error( - `${jobId}.env.RELEASE_E2E_ACTIVATION_PATH must be a nonempty relative repository path without backslashes or parent-directory segments`, - ); - } - return activationPath; -} - -function candidatePathExists(candidateSha: string, candidatePath: string): boolean { - try { - const output = execFileSync( - "git", - ["ls-tree", "--name-only", candidateSha, "--", candidatePath], - { - cwd: REPO_ROOT, - encoding: "utf8", - stdio: ["ignore", "pipe", "pipe"], - }, - ); - return output.trim() === candidatePath; - } catch (error) { - throw new Error( - `could not inspect release E2E activation path ${candidatePath} at candidate ${candidateSha}`, - { cause: error }, - ); - } -} - -export function buildReleaseE2ePreflight(input: { - candidateSha: string; - candidatePathExists?: (candidateSha: string, candidatePath: string) => boolean; - plan?: E2eWorkflowPlan; - workflowPath?: string; -}): ReleaseE2ePreflight { - if (!SHA_PATTERN.test(input.candidateSha)) { - throw new Error("candidateSha must be a lowercase 40-character commit SHA"); - } - const workflowPath = input.workflowPath ?? DEFAULT_WORKFLOW_PATH; - const jobs = workflowJobs(workflowPath); - const inventory = readFreeStandingJobsInventory(workflowPath); - const plan = input.plan ?? buildE2eWorkflowPlan(); - const pathExists = input.candidatePathExists ?? candidatePathExists; - const defaultJobIds = inventory.workflowJobs.filter((jobId) => jobId !== "shared-e2e"); - for (const jobId of defaultJobIds) { - const activationPath = releaseActivationPath( - record(jobs[jobId], `workflow.jobs.${jobId}`), - jobId, - ); - if (activationPath !== undefined && !pathExists(input.candidateSha, activationPath)) { - throw new Error( - `candidate commit is missing required E2E activation path ${activationPath} for ${jobId}`, - ); - } - } - const launchableE2eJobs = defaultJobIds.filter((jobId) => - isLaunchableE2eJob(jobId, record(jobs[jobId], `workflow.jobs.${jobId}`)), - ); - if (launchableE2eJobs.length !== 1) { - throw new Error(`expected exactly one Launchable E2E job, found ${launchableE2eJobs.length}`); - } - const launchableE2eJobId = launchableE2eJobs[0]!; - const executions = [ - ...defaultJobIds.flatMap((jobId) => - jobExecutions(jobId, record(jobs[jobId], `workflow.jobs.${jobId}`), "default", plan), - ), - ...jobExecutions("live", record(jobs.live, "workflow.jobs.live"), "default", plan), - ...jobExecutions( - "shared-e2e", - record(jobs["shared-e2e"], "workflow.jobs.shared-e2e"), - "default", - plan, - ), - ]; - const duplicateIds = executions - .map((execution) => execution.id) - .filter((id, index, ids) => ids.indexOf(id) !== index); - if (duplicateIds.length > 0) { - throw new Error(`release E2E execution identifiers are not unique: ${duplicateIds.join(",")}`); - } - - const exceptionsRequired: string[] = []; - - return { - candidateSha: input.candidateSha, - dispatches: { - completeRun: { - includeStagingBrevLaunchable: true, - jobs: "", - mode: "full", - targets: "", - }, - }, - exceptionsRequired, - executions, - launchableE2eJobId, - requiredExecutionCount: executions.length, - }; -} - -function flattenJobs(value: unknown): JsonRecord[] { - const pages = Array.isArray(value) ? value : [value]; - return pages.flatMap((page, pageIndex) => { - const jobs = record(page, `jobs page ${pageIndex}`).jobs; - if (!Array.isArray(jobs)) throw new Error(`jobs page ${pageIndex}.jobs must be an array`); - return jobs.map((job, jobIndex) => record(job, `jobs page ${pageIndex}.jobs[${jobIndex}]`)); - }); -} - -function matchesExpectedName(actual: string, expected: string): boolean { - if (actual === expected) return true; - if (!actual.endsWith("...")) return false; - return expected.startsWith(actual.slice(0, -3)); -} - -export function buildReleaseE2eLedger( - preflight: ReleaseE2ePreflight, - runs: readonly ReleaseE2eRunEvidence[], -): ReleaseE2eLedger { - if (runs.length !== 1) { - throw new Error( - `release E2E evidence requires exactly one workflow run, received ${runs.length}`, - ); - } - const attempts = new Map(); - - for (const [runIndex, evidence] of runs.entries()) { - const label = `runs[${runIndex}]`; - const run = record(evidence.run, `${label}.run`); - requireEqual(run.head_sha, preflight.candidateSha, `${label}.run.head_sha`); - requireEqual(run.head_branch, "main", `${label}.run.head_branch`); - requireEqual(run.event, "workflow_dispatch", `${label}.run.event`); - requireEqual(run.path, ".github/workflows/e2e.yaml", `${label}.run.path`); - requireEqual(run.status, "completed", `${label}.run.status`); - requireEqual(run.conclusion, "success", `${label}.run.conclusion`); - const runId = numberField(run, "id", `${label}.run`); - const runAttempt = numberField(run, "run_attempt", `${label}.run`); - const runUrl = stringField(run, "html_url", `${label}.run`); - - const dispatch = record(evidence.dispatch, `${label}.dispatch`); - requireEqual(dispatch.kind, "nemoclaw-e2e-dispatch-v1", `${label}.dispatch.kind`); - requireEqual(dispatch.candidateSha, preflight.candidateSha, `${label}.dispatch.candidateSha`); - requireEqual(dispatch.eventName, "workflow_dispatch", `${label}.dispatch.eventName`); - requireEqual(dispatch.workflowRunId, String(runId), `${label}.dispatch.workflowRunId`); - const receiptAttempt = numberField(dispatch, "workflowRunAttempt", `${label}.dispatch`); - if (receiptAttempt > runAttempt) { - throw new Error(`${label}.dispatch.workflowRunAttempt exceeds the workflow run attempt`); - } - const jobsInput = dispatch.jobs; - const targetsInput = dispatch.targets; - if (typeof jobsInput !== "string" || typeof targetsInput !== "string") { - throw new Error(`${label}.dispatch jobs and targets must be strings`); - } - requireEqual(jobsInput, "", `${label}.dispatch.jobs`); - requireEqual(targetsInput, "", `${label}.dispatch.targets`); - requireEqual( - booleanField(dispatch, "emptySelectors", `${label}.dispatch`), - true, - `${label}.dispatch.emptySelectors`, - ); - requireEqual( - booleanField(dispatch, "includeStagingBrevLaunchable", `${label}.dispatch`), - true, - `${label}.dispatch.includeStagingBrevLaunchable`, - ); - - const selectedExecutions = preflight.executions; - for (const job of flattenJobs(evidence.jobs)) { - const jobRunId = numberField(job, "run_id", `runs[${runIndex}].job`); - const jobAttempt = numberField(job, "run_attempt", `runs[${runIndex}].job`); - if (jobRunId !== runId || jobAttempt > runAttempt) continue; - const name = stringField(job, "name", `runs[${runIndex}].job`); - const matches = selectedExecutions.filter((execution) => - matchesExpectedName(name, execution.expectedName), - ); - if (matches.length > 1) { - throw new Error( - `GitHub job name ${JSON.stringify(name)} ambiguously matches ${matches - .map((execution) => execution.id) - .join(",")}`, - ); - } - if (matches.length === 0) continue; - const execution = matches[0]!; - const values = attempts.get(execution.id) ?? []; - values.push({ - attempt: jobAttempt, - conclusion: stringField(job, "conclusion", `runs[${runIndex}].job`), - status: stringField(job, "status", `runs[${runIndex}].job`), - jobUrl: stringField(job, "html_url", `runs[${runIndex}].job`), - runUrl, - }); - attempts.set(execution.id, values); - } - } - - const entries = preflight.executions.map((execution): ReleaseE2eLedgerEntry => { - const executionAttempts = [...(attempts.get(execution.id) ?? [])].sort( - (left, right) => right.attempt - left.attempt || right.jobUrl.localeCompare(left.jobUrl), - ); - const successful = executionAttempts.find( - (attempt) => attempt.status === "completed" && attempt.conclusion === "success", - ); - return { - ...execution, - attempts: executionAttempts, - ...(successful - ? { - successfulEvidence: { - attempt: successful.attempt, - jobUrl: successful.jobUrl, - runUrl: successful.runUrl, - }, - } - : {}), - status: successful ? "successful" : "missing", - }; - }); - const successfulCount = entries.filter((entry) => entry.status === "successful").length; - return { - candidateSha: preflight.candidateSha, - entries, - successfulCount, - missingCount: entries.length - successfulCount, - requiredCount: entries.length, - }; -} - -function parseArgs(argv: readonly string[]): CliOptions { - const options: CliOptions = { - workflowPath: DEFAULT_WORKFLOW_PATH, - }; - for (let index = 0; index < argv.length; index += 1) { - const arg = argv[index]; - const value = argv[index + 1]; - if (arg !== "--candidate-sha" && arg !== "--manifest" && arg !== "--workflow") { - throw new Error(`Unknown argument: ${arg}`); - } - if (value === undefined) throw new Error(`${arg} requires a value`); - if (arg === "--candidate-sha") options.candidateSha = value; - else if (arg === "--manifest") options.manifest = value; - else options.workflowPath = value; - index += 1; - } - return options; -} - -function readManifest(manifestPath: string): { - manifest: ReleaseEvidenceManifest; - runs: ReleaseE2eRunEvidence[]; -} { - const directory = path.dirname(path.resolve(manifestPath)); - const raw = record(JSON.parse(readFileSync(manifestPath, "utf8")), "manifest"); - const manifest = raw as ReleaseEvidenceManifest; - if (!SHA_PATTERN.test(manifest.candidateSha) || !Array.isArray(manifest.runs)) { - throw new Error("release E2E evidence manifest has an invalid schema"); - } - const runs = manifest.runs.map((entry, index) => { - if ( - typeof entry.dispatchJson !== "string" || - typeof entry.jobsJson !== "string" || - typeof entry.runJson !== "string" - ) { - throw new Error(`manifest.runs[${index}] has an invalid schema`); - } - return { - dispatch: JSON.parse(readFileSync(path.resolve(directory, entry.dispatchJson), "utf8")), - jobs: JSON.parse(readFileSync(path.resolve(directory, entry.jobsJson), "utf8")), - run: JSON.parse(readFileSync(path.resolve(directory, entry.runJson), "utf8")), - }; - }); - return { manifest, runs }; -} - -function requireCandidateCheckout(candidateSha: string): void { - const headSha = execFileSync("git", ["rev-parse", "HEAD"], { - cwd: REPO_ROOT, - encoding: "utf8", - }).trim(); - if (headSha !== candidateSha) { - throw new Error(`checkout HEAD ${headSha} does not match candidate SHA ${candidateSha}`); - } -} - -export function runReleaseE2eEvidenceCli(argv = process.argv.slice(2)): void { - const options = parseArgs(argv); - if (options.manifest) { - const { manifest, runs } = readManifest(options.manifest); - requireCandidateCheckout(manifest.candidateSha); - const preflight = buildReleaseE2ePreflight({ - candidateSha: manifest.candidateSha, - workflowPath: options.workflowPath, - }); - process.stdout.write(`${JSON.stringify(buildReleaseE2eLedger(preflight, runs), null, 2)}\n`); - return; - } - if (options.candidateSha === undefined) { - throw new Error("--candidate-sha is required for preflight"); - } - requireCandidateCheckout(options.candidateSha); - process.stdout.write( - `${JSON.stringify( - buildReleaseE2ePreflight({ - candidateSha: options.candidateSha, - workflowPath: options.workflowPath, - }), - null, - 2, - )}\n`, - ); -} - -const invokedFile = process.argv[1] ? path.resolve(process.argv[1]) : ""; -if (invokedFile === fileURLToPath(import.meta.url)) { - try { - runReleaseE2eEvidenceCli(); - } catch (error) { - console.error(error instanceof Error ? error.message : String(error)); - process.exitCode = 1; - } -} diff --git a/.agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md b/.agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md index 5257c16f5d1..5b0ce2b93e4 100644 --- a/.agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md +++ b/.agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md @@ -53,15 +53,13 @@ write a suggestion. The team follows a daily ship cycle. All maintainer skills operate within this rhythm. -1. **Morning** (`/nemoclaw-maintainer-morning`) — triage the backlog, pick items for the day, label them with the target version (e.g., `v0.0.8`). -2. **During the day** (`/nemoclaw-maintainer-day`) — land PRs using the maintainer loop. Version labels make progress visible on dashboards. -3. **Evening** (`/nemoclaw-maintainer-evening`) — Check shipped work and the pre-tag changelog PR. - Confirm that the pre-tag changelog PR contains `docs/changelog/YYYY-MM-DD.mdx` for the release. - Identify open items and prepare the QA summary. Record the release SHA and required E2E evidence. - Cut the tag after confirmation. Move open items to the next patch label and delete the released label. - Prepare the Announcement. -4. **Overnight** — A QA team in another time zone validates the tag. - Put new issues into the next morning's triage. +1. **Morning** (`/nemoclaw-maintainer-morning`): Triage the backlog, pick items for the day, and label them with the target version (for example, `v0.0.8`). +2. **During the day** (`/nemoclaw-maintainer-day`): Land PRs using the maintainer loop. Version labels make progress visible on dashboards. +3. **4 PM close** (`/nemoclaw-maintainer-evening`): Stop merging, require the dated changelog entry, freeze the exact candidate, identify stragglers, and start the advisory overnight handoff. +4. **4 PM–4 AM frozen loop**: Use exact-SHA post-merge agent reviews and the E2E runs from every edition `main` push to diagnose regressions, rerun selectively, and prepare fixes without merging. +5. **4 AM cut** (`/nemoclaw-maintainer-cut-release-tag`): Tag the frozen candidate regardless of E2E state. Keep changelog, ancestry, tag, signing, `latest`, `lkg`, and housekeeping controls fail-closed. +6. **4 AM–8 AM continuation**: Continue advisory diagnosis and fix preparation. +7. **8 AM handoff** (`/nemoclaw-maintainer-morning`): Hand state to the next release doula and reopen the merge window. Version labels activate release work. They do not show readiness. If an open item misses the tag, move its label to the next patch after the release. diff --git a/.agents/skills/nemoclaw-maintainer-e2e/SKILL.md b/.agents/skills/nemoclaw-maintainer-e2e/SKILL.md index a61a6f468b0..e8a637ffc9d 100644 --- a/.agents/skills/nemoclaw-maintainer-e2e/SKILL.md +++ b/.agents/skills/nemoclaw-maintainer-e2e/SKILL.md @@ -1,6 +1,6 @@ --- name: nemoclaw-maintainer-e2e -description: Dispatches and verifies trusted GitHub Actions E2E for NemoClaw maintainers, including manual PR E2E for the current PR head commit. Use for requests such as run E2E for PR #123, run the E2E suite, run the Launchable E2E, run the full E2E suite, deploy pre-release full E2E, run pre-tag full E2E, or run release-candidate E2E. +description: Dispatches and verifies trusted advisory GitHub Actions E2E for NemoClaw maintainers, including exact-revision manual PR E2E and per-main-push overnight diagnosis. Use for requests such as run E2E for PR #123, run the E2E suite, diagnose post-merge E2E, selectively rerun failures, or validate a release candidate without gating its tag. --- @@ -9,7 +9,7 @@ description: Dispatches and verifies trusted GitHub Actions E2E for NemoClaw mai # Run Maintainer E2E Use `.github/workflows/e2e.yaml` from trusted `main`. -Every push to `main` selects every workflow E2E. A selected job can remain queued until its configured runner is available. No E2E job is excluded from trusted `main` push selection. Pre-tag evidence still requires the full `workflow_dispatch` mode described below. +Every push to `main` selects every workflow E2E. A selected job can remain queued until its configured runner is available. No workflow E2E requires a separate explicit dispatch. Do not substitute local `npm run test:live-e2e` unless the maintainer explicitly requests local execution. ## Manual PR E2E @@ -144,8 +144,7 @@ A changed head repository, head SHA, or base SHA invalidates the evidence and re | “Run the E2E suite” | Ordinary | empty | `false` | | “Run the Launchable E2E” | Launchable | `staging-brev-launchable` | `false` | | “Run the full E2E suite” | Full | empty | `true` | -| “deploy pre-release full E2E” | Full | empty | `true` | -| “run pre-tag full E2E” | Full | empty | `true` | +| “diagnose post-merge E2E” | Main-push inspection | n/a | n/a | | “run release-candidate E2E” | Full | empty | `true` | A generic E2E request must not authorize the Brev Launchable path. @@ -156,6 +155,55 @@ Ordinary mode selects every workflow E2E except `Exact staging Brev Launchable`. Launchable mode runs only `Exact staging Brev Launchable`. Full mode selects every workflow E2E, including `Exact staging Brev Launchable`, in the same workflow run. +## Inspect the Edition's Main-Push Runs + +For overnight diagnosis, start from the trusted frozen plan. Every push to `main` already starts its own immutable run selecting every workflow E2E, so do not dispatch a duplicate full run merely to begin the overnight loop: + +```bash +export PLAN_PATH=/path/to/downloaded-release-edition-plan/plan.json +FROZEN_CANDIDATE_SHA="$(node -p "require(process.env.PLAN_PATH).candidateCommit")" +export EDITION_CUTOFF_AT="$(node -p "require(process.env.PLAN_PATH).authorization.cutoffAt")" +CUTOFF_SECONDS="$(node -p "Date.parse(process.env.EDITION_CUTOFF_AT) / 1000")" +WINDOW_START_SECONDS="$((CUTOFF_SECONDS - 8 * 60 * 60))" +RUNS="$(gh run list --repo NVIDIA/NemoClaw --workflow e2e.yaml \ + --event push --branch main --limit 200 \ + --json databaseId,createdAt,headSha,status,conclusion,url)" +MATCHES="$(jq -c --argjson start "$WINDOW_START_SECONDS" --argjson end "$CUTOFF_SECONDS" \ + '[.[] | select((.createdAt | fromdateiso8601) >= $start and + (.createdAt | fromdateiso8601) <= $end)] | sort_by(.createdAt)' <<<"$RUNS")" +jq -e 'length >= 1' <<<"$MATCHES" >/dev/null +jq -e 'all(.[]; (.headSha // "") | test("^[0-9a-f]{40}$"))' <<<"$MATCHES" >/dev/null +``` + +The inventory covers the 8:00 AM–4:00 PM merge window ending at the plan's exact cutoff. More than one run is expected. Fetch `origin/main` and require every selected `headSha` to be an ancestor of `FROZEN_CANDIDATE_SHA`; exclude and report any unrelated run instead of silently treating it as edition evidence. If a merge's push run is not visible yet, report it as pending and poll. Do not silently substitute full mode. + +Each run is keyed to its own `headSha`; a later push does not cancel an earlier main-push run. `.github/workflows/e2e-main-retry.yaml` may rerun failed jobs from a non-superseded main-push run up to two times. Keep the source run ID, attempt, SHA, job conclusion, artifacts, and retry evidence together. The tag remains bound only to the frozen plan. + +Watch and inspect each selected run even when it fails: + +```bash +gh run watch "" --repo NVIDIA/NemoClaw +gh run view "" --repo NVIDIA/NemoClaw \ + --json status,conclusion,headSha,jobs,url +gh run view "" --repo NVIDIA/NemoClaw --log-failed +``` + +Classify failures before choosing selective reruns. Dispatch full mode only for an explicit full or release-candidate rerun request. + +## Operate the Overnight Loop + +For the frozen edition, keep one agent session active from 4:00 PM through the 8:00 AM handoff. Do not start competing agents over the same failure set. Repeat this sequence until the handoff boundary: + +1. inspect newly completed main-push, automatic-retry, or selective E2E jobs and exact-SHA post-merge advisor findings; +2. choose the highest-impact unresolved failure that is not already owned by a prepared fix; +3. classify it as a product regression, flaky test, infrastructure failure, or stale test; +4. prepare the smallest focused fix or justified test cleanup and run its deterministic checks; +5. open or update a fix PR without merging it during the freeze; +6. dispatch only the selective rerun needed to test the diagnosis; and +7. update the shared handoff state, then immediately choose the next actionable item. + +Do not wait idly for an unrelated rerun when another unresolved failure can be diagnosed. Continue across the 4:00 AM tag without changing the frozen candidate or treating the tag as E2E success. At 8:00 AM, stop the loop and hand over every unresolved failure, rerun, and prepared PR. If the active agent cannot continue before then, transfer the same state to one replacement agent. + ## Resolve the Candidate Run from a trusted NemoClaw checkout: @@ -166,9 +214,7 @@ git fetch --prune origin main CANDIDATE_SHA="$(git rev-parse origin/main)" ``` -For a pre-tag request, use the full candidate SHA from the generated release plan. -Require that SHA to equal `origin/main` before dispatch. -Stop and regenerate the release plan when they differ. +For a frozen-edition request, read `FROZEN_CANDIDATE_SHA` from the generated release plan and compare it with `CANDIDATE_SHA`, which is the current trusted `origin/main` dispatch ref. When they match, the rerun is also bound to the frozen candidate. If `main` advanced after the cutoff, keep the plan frozen and describe new dispatches as current-main or next-edition validation; the current direct-main workflow does not dispatch an older ancestor. Do not use manual PR checkout inputs to bypass that boundary. Record `CANDIDATE_SHA` for every dispatch. Do not use a relative revision in the evidence report. @@ -276,7 +322,7 @@ gh api "repos/NVIDIA/NemoClaw/actions/runs/$RUN_ID/jobs?filter=latest&per_page=1 >"$EVIDENCE_DIR/jobs-latest-$RUN_ID.json" ``` -For full-mode or release evidence, collect every attempt for the matrix-preserving ledger: +For full mode, collect every attempt so the validator can retain successful evidence from an earlier attempt of the same workflow run: ```bash gh api --paginate --slurp \ @@ -284,7 +330,7 @@ gh api --paginate --slurp \ >"$EVIDENCE_DIR/jobs-$RUN_ID.json" ``` -Reuse `run-$RUN_ID.json` and `jobs-$RUN_ID.json` as the `nemoclaw-maintainer-cut-release-tag` manifest inputs and as the full-mode validator inputs. Do not fetch the same run again. `jobs-latest-$RUN_ID.json` is only for ordinary and Launchable modes. +Use `jobs-$RUN_ID.json` as the full-mode validator input. `jobs-latest-$RUN_ID.json` is only for ordinary and Launchable modes. For ordinary and Launchable modes, require `run-$RUN_ID.json` to report: @@ -332,12 +378,12 @@ The validator requires: - `cleanup.json` to report the same workspace as `ABSENT`. A skipped, cancelled, queued, or failed Launchable E2E job is not evidence. -A Launchable-mode run is not full-mode or pre-tag release evidence. +A Launchable-mode run is not full-mode evidence. A missing, mismatched, or failed cleanup receipt is not evidence. -## Bind Release Evidence +## Bind Advisory Evidence -If no release plan exists, label a successful full run against `origin/main` as provisional release evidence. +If no release plan exists, label a successful full run against `origin/main` as advisory E2E evidence. Return: - candidate SHA; @@ -347,13 +393,9 @@ Return: - Launchable E2E identity; and - cleanup result. -If the release candidate SHA changes, discard the earlier full run and dispatch full mode for the new SHA. -No release-note-only delta exception is currently defined. - -When `nemoclaw-maintainer-cut-release-tag` invokes this skill, return the validated fields for its pre-tag E2E evidence ledger. -The trusted `dispatch.json` receipt proves that full mode used empty selectors and included `Exact staging Brev Launchable`. -The release evidence ledger proves the result of each workflow E2E. -Do not ask for the release confirmation phrase in this skill. +Bind every result to the tested SHA. For the frozen edition, classify failures and prepare fixes for the next merge window. +Return the trusted dispatch, test, Launchable, and cleanup receipts needed for diagnosis. +Never treat success as tag authorization or failure as a reason to delay the 4 AM tag. ## Access Failures diff --git a/.agents/skills/nemoclaw-maintainer-evening/SKILL.md b/.agents/skills/nemoclaw-maintainer-evening/SKILL.md index 330f2a7327b..1e833190a11 100644 --- a/.agents/skills/nemoclaw-maintainer-evening/SKILL.md +++ b/.agents/skills/nemoclaw-maintainer-evening/SKILL.md @@ -1,95 +1,75 @@ --- name: nemoclaw-maintainer-evening -description: Runs the end-of-day NemoClaw release handoff, including the pre-tag dated changelog PR, version progress, straggler planning, QA summary, tag cut, and announcement draft. Use at the end of the workday. Trigger keywords - evening, end of day, EOD, wrap up, ship it, cut tag, handoff, done for the day, pre-tag release notes. +description: Closes the 4 PM NemoClaw release edition, verifies pre-cutoff changelog readiness, reports shipped work and stragglers, starts the advisory overnight E2E and fix loop, and prepares the frozen handoff. Use at end of day, edition close, EOD, release freeze, or overnight QA handoff. user_invocable: true --- + + + # NemoClaw Maintainer Evening -Wrap up the day: check progress, identify stragglers, summarize for QA, cut the tag, automatically carry stragglers to the next patch, retire the released label, and prepare release notes for posting. +Close the edition at 4:00 PM `America/Los_Angeles`. Do not cut the tag in this skill; the scheduled 4:00 AM workflow owns the normal cut. -See [PR-REVIEW-PRIORITIES.md](../nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md) for the daily cadence. +Read [PR-REVIEW-PRIORITIES.md](../nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md) and the [release train](../nemoclaw-maintainer-policies/references/release-train.md). -## Step 1: Check Progress +## 1. Check Progress ```bash node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-day/scripts/version-target.ts node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-day/scripts/version-progress.ts ``` -The first script determines the target version. The second shows shipped vs open. Present the progress summary to the user. +Present merged labeled PRs as the edition contents. List open labeled PRs and issues as stragglers that post-tag housekeeping will move to the next patch label. -## Step 2: Review Post-Tag Stragglers +## 2. Finish Pre-Tag Docs -```bash -gh pr list --repo NVIDIA/NemoClaw --state open --label --limit 100 \ - --json number,title,url,labels -gh issue list --repo NVIDIA/NemoClaw --state open --label --limit 100 \ - --json number,title,url,labels -``` +Run `/nemoclaw-contributor-update-docs for ` early enough for its PR to merge before cutoff. Require one direct `docs/changelog/YYYY-MM-DD.mdx` child containing the exact `## ` heading, parser-safe SPDX comment, summary, and detailed bullets. No waiver exists. -List open labeled PRs and issues as the post-tag housekeeping plan. Tell the maintainer that, after the tag and workflow-managed `latest` are verified, `cut-release-tag` will automatically move all of them to the next patch label and delete the released label. +## 3. Freeze and Stop Merging -If an item should leave the daily release flow instead of moving forward, remove it from the released-version label before asking for the release confirmation phrase. +At 4:00 PM, announce that the edition is closed and merging remains stopped until 8:00 AM. The scheduled `release-edition-close.yaml` run at 4:17 PM must upload `release-edition-plan-YYYY-MM-DD`. -## Step 3: Generate Handoff Summary +Inspect its summary and record: -```bash -node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-day/scripts/handoff-summary.ts -``` +- frozen candidate SHA; +- `ready` or `no-changes` status; +- previous and next tags; +- changelog match; and +- plan hash. -This lists commits since the last tag, identifies risky areas touched, and suggests QA test focus areas. Format the output as a concise summary the user can paste into the tag annotation or a handoff channel. +Do not advance the candidate if `main` moves after cutoff. A late merge belongs to the next edition. -## Pre-Tag Docs +## 4. Start the Advisory Overnight Loop -Run `/nemoclaw-contributor-update-docs for ` before loading `cut-release-tag`. -Confirm that the release-prep docs PR creates or updates one direct child of `docs/changelog/` for the planned date and contains the exact `## ` heading, a parser-safe MDX SPDX comment, the summary, and the detailed release bullets. -An ordinary docs refresh or a post-tag Discussion draft does not satisfy this step. -The release-prep docs PR, including the dated changelog entry, must be merged, or explicitly waived with a reason that names the missing changelog entry, before `release:plan` captures the release commit. -If a docs PR or any other intended PR merges after `release:plan`, regenerate the plan before cutting the tag. +Run the handoff summary: -## Step 4: Cut the Tag and Publish Release Notes - -Load `cut-release-tag`. -The version is already known, so use a patch bump unless the maintainer selects another bump. -Show the commit, changelog, carry-forward plan, label-retirement plan, and release notes draft. +```bash +node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-day/scripts/handoff-summary.ts +``` -After the release plan captures the candidate SHA, load `nemoclaw-maintainer-e2e`. -Run full mode unless one existing full run for the candidate SHA contains complete workflow E2E and `Exact staging Brev Launchable` evidence. -Review the pre-tag E2E evidence ledger from `.github/workflows/e2e.yaml` at that commit. -Require the accepted workflow run to conclude with `success`. Require successful `Exact staging Brev Launchable` evidence with matching Launchable E2E identity and verified workspace absence, or record the permitted itemized exception described below. -Each missing or skipped execution in that successful run requires its own itemized maintainer exception. -Missing or invalid Launchable E2E evidence in that successful run requires a separate itemized exception with run and job URLs, the missing or invalid receipt, and rationale. -Do not ask for the release confirmation phrase until the run succeeds and each required execution has successful evidence or a permitted exception. +Assign one agent to remain active through the 8:00 AM handoff. Load `nemoclaw-maintainer-e2e`. Inventory every E2E run triggered by the edition's `main` pushes and every workflow E2E selected in those runs, including automatic retry evidence and selective reruns, then classify failures, consolidate duplicates, delete stale tests when justified, and prepare focused fix PRs. Include exact-SHA post-merge advisor runs. After each diagnosis, rerun, or prepared fix, immediately select the next actionable failure. Do not merge fixes during the freeze and do not stop the loop when the 4:00 AM tag is cut. -Tag the confirmed release commit with `vX.Y.Z`. -Let the workflow move `latest`, carry open work forward, and delete the released label. -Prepare the Announcement draft for the maintainer to post. +E2E does not authorize or block the 4:00 AM tag. Do not build a release evidence ledger or request exceptions. -## Step 5: Confirm and Share +## 5. Publish the Overnight Handoff -After the tag is cut and release notes are drafted or posted by the maintainer, present the final summary: +Provide: -- **Tag**: `v0.0.8` at commit `abc1234` -- **Pre-tag E2E evidence**: 12/13 tests and exact Brev Launchable E2E passing for the candidate SHA; 1 itemized maintainer exception -- **Release notes draft**: `../nemoclaw-release-v0.0.8/release-note-draft.md` -- **Shipped**: 4 items (#1234, #1235, #1236, #1237) -- **Moved to v0.0.9**: 1 item (#1238 — still needs CI fix) -- **Retired label**: `v0.0.8` -- **QA focus areas**: installer changes, new onboard preset +- target version and frozen SHA; +- shipped PRs and open stragglers; +- QA focus areas; +- exact-SHA agent-review runs and actionable findings; +- E2E failures, classifications, reruns, and prepared fixes; and +- carry-forward and released-label retirement state; +- the expected 4:00 AM cut and 8:00 AM handoff checkpoints. -This summary can be shared in the team's handoff channel. +Name the overnight agent owner and the 8:00 AM handoff destination. If the agent cannot continue, transfer the same state to one replacement instead of starting competing loops. -## Step 6: Update State +Save state: ```bash -node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-day/scripts/state.ts history "tag-cut" "" "shipped N items, carried M forward" +node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-day/scripts/state.ts history "edition-closed" "" "froze ; shipped N; carried M" ``` -## Notes - -- Never cut a tag or hand off release notes without user confirmation. -- If nothing was labeled or nothing shipped, ask whether to skip the tag today. -- A PR version label activates release work; it is not a readiness claim. -- If an open item misses the tag, post-tag housekeeping moves its target to the next patch version. -- After carry-forward succeeds, post-tag housekeeping deletes the released label; never rename or reuse it. +Escalate a failed changelog, plan provenance, ancestry, signing, collision, `latest`, `lkg`, or housekeeping invariant. Report E2E failures as advisory work, not release blockers. diff --git a/.agents/skills/nemoclaw-maintainer-morning/SKILL.md b/.agents/skills/nemoclaw-maintainer-morning/SKILL.md index daaf4a6be86..3d4d5fec177 100644 --- a/.agents/skills/nemoclaw-maintainer-morning/SKILL.md +++ b/.agents/skills/nemoclaw-maintainer-morning/SKILL.md @@ -6,11 +6,17 @@ user_invocable: true # NemoClaw Maintainer Morning -Start the day: triage, pick a version target, label items, share the plan. +Start the day at 8:00 AM: receive the overnight release handoff, reopen merging, triage, pick a version target, label items, and share the plan. See [PR-REVIEW-PRIORITIES.md](../nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md) for the daily cadence and review priorities. -## Step 1: Determine Target Version and Stragglers +## Step 1: Receive the Release-Doula Handoff + +Confirm the 4:00 AM cut result, workflow-managed `latest`, carry-forward, released-label retirement, exact-SHA post-merge review findings, advisory E2E state, and prepared fix PRs. A failed non-E2E release invariant is the first priority. E2E regressions and flaky or stale tests enter normal triage; they do not retroactively invalidate the tag. + +Announce that the merge window is open from 8:00 AM to 4:00 PM. + +## Step 2: Determine Target Version and Stragglers ```bash node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-day/scripts/version-target.ts @@ -18,7 +24,7 @@ node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer This fetches tags, computes the next patch version, and finds open items still carrying older version labels. Surface stragglers first — they indicate post-tag housekeeping was interrupted or an item slipped across multiple cycles. Decide whether to relabel them to today's target or defer them out of the daily release flow. -## Step 2: Triage +## Step 3: Triage Run the triage script to rank the full backlog: @@ -30,7 +36,7 @@ If too few results, run without `--approved-only`. The script fetches open PRs t Also use `find-review-pr` to surface `security` PRs whose Project Priority is `Urgent` or `High`. Merge these into the candidate pool. -## Step 3: Label Version Targets +## Step 4: Label Version Targets Present the ranked queue to the user. After they confirm which items to target, label them: @@ -40,7 +46,7 @@ gh pr edit --repo NVIDIA/NemoClaw --add-label "" gh issue edit --repo NVIDIA/NemoClaw --add-label "" ``` -## Step 4: Save State and Output the Plan +## Step 5: Save State and Output the Plan Pipe triage output into state: @@ -61,6 +67,7 @@ Include: total items targeted, how many are PRs vs issues, how many are already ## Notes - This skill runs once at the start of the day. Use `/nemoclaw-maintainer-day` during the day to execute. +- Merge only during the 8:00 AM–4:00 PM window. A late merge belongs to the next edition. - On a PR, the target version label activates daily release work; actual release inclusion requires that PR to be merged with the label at cutoff. - On an issue, the target version label is tracking or "needs PR" coordination only. - Stragglers from previous versions should be addressed first — they already slipped once. diff --git a/.agents/skills/nemoclaw-maintainer-policies/references/daily-flow.md b/.agents/skills/nemoclaw-maintainer-policies/references/daily-flow.md index 104df850c66..cfc33f77e47 100644 --- a/.agents/skills/nemoclaw-maintainer-policies/references/daily-flow.md +++ b/.agents/skills/nemoclaw-maintainer-policies/references/daily-flow.md @@ -27,8 +27,9 @@ The daily flow is an outline, not a rigid ceremony. | 4 | Recommend | Publish a daily standup slate grouped by priority lane with suggested owners, next actions, and decision gaps. | | 5 | Assign | Maintainers convert the recommendation into assignments, deferrals, or follow-up questions. | | 6 | Execute | Engineers drive the assigned slate toward merge, fix, unblock, PR creation, or explicit deferral. | -| 7 | Release | The daily release is cut from merged PRs carrying the daily version label. | -| 8 | Handoff | Summarize what shipped, what slipped, what QA should focus on, and what should seed the next cycle. | +| 7 | Close | At 4 PM, freeze the edition and stop merging while asynchronous review and the edition's post-merge E2E runs continue. | +| 8 | Tag | At 4 AM, tag the frozen candidate regardless of E2E state. | +| 9 | Handoff | At 8 AM, summarize what shipped, what slipped, review and E2E state, prepared fixes, and next-cycle work. | ## Priority Order @@ -66,9 +67,11 @@ Agents may recommend labels, assignments, Project field changes, comments, merge ## Release Boundary - A PR daily version label activates daily release work; it is not a readiness claim. -- Release inclusion requires a PR to be both merged and carrying the relevant daily version label at release cutoff. +- Release inclusion requires a PR to be both merged and carrying the relevant daily version label in the frozen 4 PM candidate. - Issue daily version labels are tracking or coordination signals only. -- Before tag confirmation, capture the candidate SHA and review every E2E test declared by `.github/workflows/e2e.yaml` at that commit. Require the accepted workflow run's `head_sha` and all associated evidence to match the candidate SHA, and require the run to conclude with `success`. Each declared execution must have successful evidence from that run or an explicit itemized maintainer exception. Permit exceptions only for missing or skipped executions, not failed executions. If a late merge changes the candidate SHA, invalidate the prior run and evidence and repeat the complete workflow-wide validation. +- Stop merges from 4 PM to 8 AM. A merge after cutoff belongs to the next edition and never advances the frozen candidate. +- Dispatch an asynchronous agent review for each exact `main` SHA range. Every `main` push selects every workflow E2E; consume those runs and prepare fixes during the freeze. +- E2E is advisory: tag the frozen candidate at 4 AM regardless of its state, while keeping changelog, ancestry, signing, collision, `latest`, `lkg`, and housekeeping controls fail-closed. - Open PRs and issues that miss a tagged release carry forward by automatically moving from the released version label to the next patch label after the tag and `latest` are verified. - After carry-forward leaves no open item on the released label, delete that repository label. Never rename or reuse it. - Durable release history belongs in releases, release notes, or manifests, not in long-lived labels. diff --git a/.agents/skills/nemoclaw-maintainer-policies/references/release-train.md b/.agents/skills/nemoclaw-maintainer-policies/references/release-train.md index ed68a2dd11b..7ad83073ecb 100644 --- a/.agents/skills/nemoclaw-maintainer-policies/references/release-train.md +++ b/.agents/skills/nemoclaw-maintainer-policies/references/release-train.md @@ -3,95 +3,115 @@ # NemoClaw Release Train -Daily release labels coordinate release work. They do not classify issues and they do not promise readiness. +Daily release labels coordinate release work. They do not classify issues, promise readiness, or gate release tags. -## Rules +## Daily Timeline + +Use `America/Los_Angeles` for all boundaries. + +| Time | State | Work | +|---|---|---| +| 8:00 AM–4:00 PM | Merge window | Merge reviewed PRs. Dispatch an asynchronous agent review for every exact `main` SHA range, and select every workflow E2E for each `main` push. | +| 4:00 PM | Edition closed | Stop merging. Freeze the latest GitHub-recorded `main` push at or before the cutoff. | +| 4:00 PM–4:00 AM | Frozen | Consume the E2E runs from every edition `main` push, diagnose failures, selectively rerun, and prepare fix PRs. Do not merge. | +| 4:00 AM | Edition tagged | Tag the frozen candidate regardless of E2E state. Continue the advisory loop. | +| 8:00 AM | Handoff | Give tag, review, E2E, failure, and fix state to the next release doula. Reopen merging. | + +A merge after the cutoff belongs to the next edition even if the scheduled close workflow starts late. The planner selects the latest `post-merge-agent-review.yaml` push run that GitHub recorded at or before exactly 4:00 PM. It does not select the current tip at workflow execution time. + +## Release Labels - PRs own the release-inclusion meaning of daily version labels. - Engineers and agents may add the current `v0.0.x` label to open PRs to activate them for day work. -- After a PR merges to `main`, the trusted post-merge workflow adds the next patch label only when the merge is ahead of the latest release tag. A merge already contained in a release tag receives no release label. -- A scheduled and manually dispatchable reconciliation pass repairs missed or failed merge events only across the untagged interval from the latest release tag to `main`. -- Post-merge assignment and tag-triggered label retirement share one queued GitHub Actions concurrency group. Authorized automation cannot add a released label during the retirement verification-and-delete window. -- Issues may also carry daily version labels when they need a PR, fix, or regression follow-up for the daily tag. -- Applying a daily version label is not a readiness claim. -- Release includes PRs that both carry the daily version label and are merged by cutoff. -- Issue version labels are tracking signals. An issue label does not include work in the release without a merged, labeled PR. -- Open PRs and issues that miss a tagged release carry forward automatically by moving from the released version label to the next patch label. -- After the semver tag and workflow-managed `latest` are verified, post-tag housekeeping moves open stragglers and deletes the released version label. Tags and commit ancestry are the only durable release-membership record. -- Released version labels must be deleted, never renamed or reused for a later release. +- A PR is included only when it carries the target label and its merge is in the frozen candidate. +- Issue labels are tracking signals; they do not include code in an edition. +- Applying a version label is not a readiness claim. +- The trusted post-merge workflow labels untagged merges, and reconciliation repairs missed events. +- Open PRs and issues that miss the edition move to the next patch label after the tag succeeds. +- Released labels are deleted after carry-forward. Never rename, recreate, or reuse them. +- Tags and commit ancestry are the durable release-membership record. + +Post-merge assignment and release-label retirement share one queued concurrency group so authorized writes cannot race the verification-and-delete window. ## Release-Prep Docs -Run `/nemoclaw-contributor-update-docs for vX.Y.Z` before generating the final release plan for `vX.Y.Z`. -The pre-tag release-note docs PR must create or update `docs/changelog/YYYY-MM-DD.mdx`. -Use the required `## vX.Y.Z` heading, parser-safe MDX SPDX comment, summary, and detailed bullets. -This dated file is the release history for all documentation variants. Ordinary documentation pages and the post-tag Announcement do not replace it. -Release-prep docs, including that entry, must be merged or explicitly waived before `release:plan` captures the release commit. -If any merge lands after `release:plan`, generate a fresh plan before cutting the tag. +Run `/nemoclaw-contributor-update-docs for vX.Y.Z` early enough for the PR to merge before 4:00 PM. The PR must create or update `docs/changelog/YYYY-MM-DD.mdx` with the exact `## vX.Y.Z` heading, parser-safe MDX SPDX comment, summary, and detailed bullets. + +The frozen non-empty candidate must contain exactly one matching heading in a direct child of `docs/changelog/`. Ordinary documentation and the post-tag Announcement do not replace it. No changelog waiver exists. + +## Frozen Plan + +`.github/workflows/release-edition-close.yaml` is scheduled for 4:17 PM to avoid the busiest cron boundary while preserving the exact 4:00 PM cutoff. Because GitHub cron is UTC-only, the workflow registers both PST and PDT UTC times and admits only the schedule that resolves to 4:17 PM in `America/Los_Angeles`. It creates a schema-v2 release plan containing: + +- edition date and exact cutoff instant; +- planning-time `origin/main` and frozen candidate SHAs; +- candidate source workflow run ID and GitHub-recorded time; +- previous and next semver tags; +- untagged commit count and `ready` or `no-changes` status; +- exact changelog match for a non-empty edition; +- `latest` and `lkg` observations; +- scheduled authority and forbidden operations; and +- a SHA-256 consistency hash over the complete plan. + +The trusted scheduled run uploads `release-edition-plan-YYYY-MM-DD`. The workflow run and artifact provenance establish scheduled release authority. The hash provides a stable receipt identifier and detects changes that do not update it; it does not authenticate a file from another source. + +If no commits follow the latest semver tag at cutoff, the edition succeeds as `no-changes` and creates no tag. -## Cutoff +## Asynchronous Review and Advisory E2E -The daily cutoff is the maintainer-defined point where the release tag is prepared. +Every non-initial push to `main` dispatches the existing isolated PR Review Advisor with the exact immutable `before...after` SHA range. Advisor concurrency includes the head SHA, so a later merge does not cancel an earlier review. These post-merge reviews are asynchronous findings for the overnight loop and morning triage; they are not required status checks. -At cutoff: +Every push to `main` starts `.github/workflows/e2e.yaml` for that immutable head SHA and selects every workflow E2E. A selected protected job may remain queued for its runner or environment approval; it does not require a separate dispatch. Runs are queued rather than cancelled when later pushes arrive. `.github/workflows/e2e-main-retry.yaml` may retry failed jobs from a non-superseded main-push run up to two times. Assign one overnight agent to own the loop continuously from edition close through the 8:00 AM handoff. Keep the shared handoff state with that agent across the 4:00 AM tag boundary. The agent must use all edition main-push results, automatic retry evidence, exact-SHA Advisor artifacts, and selective reruns to: -1. List merged PRs carrying the target version label. -2. Confirm each is intended for the release. -3. List open PRs and issues still carrying the target label as post-tag stragglers. -4. Confirm the merged release-note docs PR contains the dated changelog entry for the target version, or record an explicit waiver that names the missing entry. -5. Generate QA handoff from merged PRs. -6. Generate the release plan to capture the candidate commit. Merges may continue; a late drift check advances the candidate and invalidates evidence for the older SHA. -7. Review the candidate commit's pre-tag E2E evidence. -8. Cut the release tag only with explicit maintainer confirmation. -9. After the tag and workflow-managed `latest` are verified, automatically move every open straggler to the next patch label, verify none remain, and delete the released version label. +1. classify product regressions, flaky tests, infrastructure failures, and stale tests; +2. consolidate duplicates; +3. prepare focused fix PRs or test cleanups; +4. validate those PRs without merging during the freeze; and +5. immediately select the next actionable failure after each diagnosis, rerun, or prepared fix; +6. preserve unresolved state for the 8:00 AM handoff. -## Pre-Tag E2E Evidence +Do not stop the loop when the 4:00 AM tag is cut. Stop only at the 8:00 AM handoff, or hand the same state to a replacement agent when the active agent cannot continue. -The release candidate is the full `origin/main` commit SHA captured by the generated release plan. At that commit, `.github/workflows/e2e.yaml` is the sole source of truth for the release E2E test set. Do not maintain a separate release-gating test list. +E2E never enters the tag authorization. Do not create an E2E waiver ledger, wait for a run, or advance the candidate in response to E2E results. This moves live E2E out of the merge path and into a methodical post-merge validation window while retaining exact-SHA agent review and the required deterministic `checks` aggregate on every code-changing PR. -Before asking for the release confirmation phrase, build and show an evidence ledger for that SHA: +The deterministic merge floor remains fail-closed. It includes static checks, build and typecheck, installer integration, CLI and plugin tests, package contracts, and hermetic E2E-support tests. The `build-typecheck` lane runs `npm run test:smoke`, which builds the CLI and plugin, starts the compiled CLI with an isolated home, and verifies the packed CLI and plugin entrypoints without requiring credentials, containers, GPUs, or external services. Do not waive, skip, or remove `checks`, `build-typecheck`, or `installer-integration` because live E2E moved after merge. -- Preflight the candidate workflow and existing candidate evidence before dispatching new work. -- Derive the denominator from the candidate workflow. Do not copy it into a second release test list. -- Require every declared `RELEASE_E2E_ACTIVATION_PATH` to exist at the candidate SHA. A missing path is a preflight failure. -- Require the workflow-produced trusted dispatch receipt to bind the accepted run candidate SHA, run ID, attempt, and selector inputs. -- Run `nemoclaw-maintainer-e2e` in full mode when the ledger lacks complete evidence for the candidate SHA. -- Require one completed, successful full workflow run that selects every workflow E2E, including `Exact staging Brev Launchable`. -- Require the trusted dispatch receipt to bind the workflow run and an attempt no later than the run's latest attempt. The receipt must record empty selectors and `include_staging_brev_launchable=true`. -- Require the Launchable E2E receipt to identify the candidate SHA in the repository and provision records. -- Require the cleanup receipt to identify the qualified workspace and report `ABSENT`. -- Every E2E execution declared by the workflow must have at least one completed, successful execution for the candidate SHA. -- Treat each expanded matrix execution as a separate ledger entry. Use its matrix `id`, or all distinguishing matrix dimensions when no single ID exists, in the test identifier so results for distinct expansions are never collapsed under the parent job. -- Successful evidence may accumulate across rerun attempts of that workflow run. Evidence from another workflow run does not satisfy the ledger. A later failure does not erase an earlier successful execution for the same test and SHA. -- Skipped, unexecuted, queued, in-progress, cancelled, and failing results do not count as successful evidence. -- Map each test with successful evidence to its successful run or job URL and attempt number. -- Each missing or skipped execution in the accepted successful workflow run requires its own itemized maintainer exception. Record the test identifier, relevant run links or available evidence, the current result, and the rationale. -- Missing or invalid exact Brev Launchable E2E evidence in the accepted successful workflow run requires a separate itemized maintainer exception. Record the run and job URLs, the missing or invalid receipt, and the rationale. +## Tag and Promotion -The accepted workflow run must be completed and have a `success` conclusion. A failed workflow run cannot supply the release ledger. Rerun its failed jobs until the workflow concludes with `success`. An itemized test exception applies only to a missing or skipped execution in that otherwise successful run. +`.github/workflows/release-edition-cut.yaml` uses the same PST/PDT schedule gate for 4:17 AM and selects the trusted plan artifact for the previous edition date. It must fail closed on: -Each test and the exact Brev Launchable E2E job in the accepted successful workflow run must have successful evidence or its own permitted itemized exception before release confirmation. Immediately before confirmation, compare `origin/main` with the planned SHA. If the candidate SHA changes, discard the ledger and its exceptions, including Launchable E2E evidence. Regenerate the release plan and repeat the review for the new SHA. This does not freeze `main` or prevent merges. No release-note-only delta exception is currently defined. +- missing, expired, ambiguous, or wrong-date plan provenance; +- plan hash or schema mismatch; +- changelog absence or duplication; +- candidate or previous-tag ancestry failure; +- a newer remote semver tag or a tag collision; +- wrong repository, event, workflow revision, or authorization mode; +- signing-key or signer-identity failure; +- remote tag mismatch or GitHub signature-verification failure; +- `latest` rollback or tag-object mismatch; +- unexpected `lkg` movement; and +- carry-forward or released-label retirement failure. -## Carry Forward +Do not fail, wait, or branch on E2E state. -Open PRs and issues that miss the cutoff remain active carry-forward work, but their target changes after the release succeeds. Post-tag housekeeping creates the next patch label if needed, removes the released-version label from every open straggler, adds the next patch label, verifies no open item remains on the released label, and deletes the released label. +The `release-tag` environment holds a dedicated SSH private signing key and non-secret signer name/email variables. The signing key signs only; `GITHUB_TOKEN` authenticates the tag push. Because `GITHUB_TOKEN` pushes do not trigger a new push workflow, the cut workflow directly calls the reusable `release-latest-tag.yaml` workflow. Local maintainer tag pushes still use its tag trigger. -The `release-latest-tag` workflow runs automatic carry-forward after moving `latest`. It shares the release-label coordination queue with post-merge assignment and must complete before housekeeping is considered successful. The release confirmation must include the housekeeping plan, so the post-tag label writes remain inside the authorized release operation. Do not run the retirement script directly or manually add a label whose semver tag already exists. +## Carry Forward and Label Retirement -Maintainers may: +After GitHub verifies the signed annotated semver tag, `release-latest-tag.yaml` moves `latest` to that exact tag object. Inside the shared release-label queue, it creates the next patch label if needed, moves every open straggler, verifies none remain on the released label, and deletes the released label. -- Add the current version label when they want the PR visible in the current day queue. -- Remove a version label without replacement when an item is deferred, superseded, closed, or no longer part of the daily cycle. -- Rerun post-tag housekeeping after a partial failure. Moved items no longer have the released label, so the operation can resume safely. +Do not invoke the retirement script directly. Rerun `release-latest-tag.yaml` through manual dispatch after fixing a partial failure; its operations are idempotent. -## Label Retirement +## 8 AM Handoff -Release labels are temporary planning state. Retire one only when all conditions are true: +Hand over: -1. The semver tag and workflow-managed `latest` both resolve to the confirmed release commit. -2. Every open PR and issue has moved to the next patch label or explicitly left the daily release cycle. -3. A final query finds no open item carrying the released label. -4. The release confirmation explicitly authorizes deletion of that released label. -5. Retirement runs inside the shared release-label coordination queue. +- edition date, tag, candidate SHA, and plan hash; +- `cut-result.json`, `latest-result.json`, and `notes-data.json` artifact locations; +- exact-SHA post-merge review runs and actionable findings; +- E2E classifications, reruns, flaky or broken tests, and unresolved risks; +- prepared fix PRs ready for the reopened merge window; +- carry-forward and released-label retirement state; and +- Announcement draft status. -Delete the repository label after those checks. Deletion removes it from merged and closed items without preserving a second, mutable release-membership signal. Never rename a released label into a future version, and never recreate a label whose semver tag already exists. +Keep candidate internals, review diagnostics, E2E classifications, rerun details, and failure rationale out of the public Announcement. diff --git a/.agents/skills/nemoclaw-maintainer-release-notes/SKILL.md b/.agents/skills/nemoclaw-maintainer-release-notes/SKILL.md index 497bc7ecf10..3bbc25e8ac5 100644 --- a/.agents/skills/nemoclaw-maintainer-release-notes/SKILL.md +++ b/.agents/skills/nemoclaw-maintainer-release-notes/SKILL.md @@ -87,7 +87,7 @@ Be careful with sensitive internal cleanup: - Do not count testing reverts or guardrail reversions as release value unless the user explicitly asks for a full raw changelog. - If a revert-like commit must be mentioned, use neutral language and do not frame it as someone else's mistake. - Avoid public wording that could embarrass a teammate. -- Keep the candidate SHA, E2E failure classifications, rerun ledger, and waiver rationale out of the public Announcement. Preserve that information in the release evidence ledger. +- Keep candidate internals, agent-review diagnostics, E2E classifications, rerun details, and failure rationale out of the public Announcement. Preserve that information in the overnight handoff. ## Step 4: Categorize the Changes @@ -179,7 +179,7 @@ Also return the suggested discussion title: `NemoClaw is out`. - Never draft from memory alone; use live `gh api compare` and PR metadata. - Never mention contributor affiliation unless the user explicitly asks. - Never thank internal contributors by default; keep thanks external-only. -- Never include the candidate SHA, internal E2E failure classifications, rerun details, or waiver rationale in the public Announcement. +- Never include candidate internals, agent-review diagnostics, internal E2E classifications, rerun details, or failure rationale in the public Announcement. - Never include testing reverts as release-value bullets unless explicitly asked for a raw changelog. - Never create duplicate release Discussions. - Never create or silently repair `docs/changelog/` from this post-tag Announcement workflow. If the canonical entry is missing, report the release-process defect and route recovery through `nemoclaw-contributor-update-docs`. diff --git a/.agents/skills/nemoclaw-skills-guide/SKILL.md b/.agents/skills/nemoclaw-skills-guide/SKILL.md index ab60fdbfa81..c17965af54c 100644 --- a/.agents/skills/nemoclaw-skills-guide/SKILL.md +++ b/.agents/skills/nemoclaw-skills-guide/SKILL.md @@ -56,9 +56,9 @@ automated review feedback, drafting documentation updates, and onboarding new me | `nemoclaw-maintainer-policies` | Answer maintainer workflow questions from the read-only policy references. | | `nemoclaw-maintainer-cross-issue-sweep` | Find open issues that a PR can also fix or conflict with. Report file and line evidence. | | `nemoclaw-maintainer-day` | Run one daytime maintainer pass for the release version. Select a merge, salvage, security, test, conflict, or sequencing workflow. Designed for `/loop`. | -| `nemoclaw-maintainer-evening` | End-of-day handoff: require the pre-tag dated changelog PR, check version progress, identify stragglers, generate a QA handoff summary, cut the release tag, carry stragglers forward, retire the released label, and hand off the Announcement. | -| `nemoclaw-maintainer-cut-release-tag` | Verify the dated changelog entry, cut an annotated semver tag on a maintainer-confirmed `origin/main` commit, wait for workflow-managed `latest`, carry remaining open items forward, and delete the released label; `lkg` stays manual. | -| `nemoclaw-maintainer-e2e` | Describe default E2E triggered by pushes to `main`, dispatch exact-revision manual PR E2E, and verify applicable workflow evidence. | +| `nemoclaw-maintainer-evening` | Close the 4 PM edition, require the dated changelog entry, freeze the candidate, identify stragglers, and start the advisory overnight handoff. | +| `nemoclaw-maintainer-cut-release-tag` | Operate the frozen 4 PM edition and 4 AM signed tag for a non-empty edition regardless of E2E state, verify workflow-managed `latest`, carry remaining open items forward, and delete the released label. `lkg` stays manual. | +| `nemoclaw-maintainer-e2e` | Inspect per-`main`-push E2E, dispatch exact-revision manual PR E2E or selective reruns, and verify applicable workflow evidence. | | `nemoclaw-maintainer-release-notes` | Draft the post-tag Announcement from live tag/compare data, with the three-paragraph narrative, categorized change list, and external-only contributor thanks. | | `nemoclaw-maintainer-find-review-pr` | Find open security PRs with Urgent or High Project Priority. Link each PR to its issue and identify competing PRs. | | `nemoclaw-maintainer-pr-comparator` | Compare open PRs for the same issue. Apply gates and score the eligible PRs before you recommend one to merge. | diff --git a/.github/actions/ci-build-typecheck/action.yaml b/.github/actions/ci-build-typecheck/action.yaml index 6e1b803a661..3e34bd07a06 100644 --- a/.github/actions/ci-build-typecheck/action.yaml +++ b/.github/actions/ci-build-typecheck/action.yaml @@ -19,13 +19,9 @@ runs: npm install --ignore-scripts cd nemoclaw && npm install --ignore-scripts - - name: Build TypeScript plugin + - name: Run deterministic smoke floor shell: bash - run: cd nemoclaw && npm run build - - - name: Build CLI TypeScript modules - shell: bash - run: npm run build:cli + run: npm run test:smoke - name: Verify compiled package contracts shell: bash diff --git a/.github/workflows/post-merge-agent-review.yaml b/.github/workflows/post-merge-agent-review.yaml new file mode 100644 index 00000000000..340a34e5a03 --- /dev/null +++ b/.github/workflows/post-merge-agent-review.yaml @@ -0,0 +1,53 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +name: PR Review / Post-Merge Dispatch + +on: + push: + branches: [main] + +# This trusted dispatcher cannot read repository contents or PR secrets. Its +# only write is an exact-SHA workflow_dispatch to the existing isolated advisor. +permissions: + actions: write + +concurrency: + group: post-merge-agent-review-${{ github.sha }} + cancel-in-progress: false + +jobs: + dispatch-exact-main-delta: + if: ${{ github.repository == 'NVIDIA/NemoClaw' && github.event.before != '0000000000000000000000000000000000000000' }} + runs-on: ubuntu-24.04 + timeout-minutes: 5 + steps: + - name: Dispatch exact-SHA agent review + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + BASE_SHA: ${{ github.event.before }} + HEAD_SHA: ${{ github.sha }} + with: + script: | + const base = process.env.BASE_SHA ?? ''; + const head = process.env.HEAD_SHA ?? ''; + const sha = /^[0-9a-f]{40}$/; + if (!sha.test(base) || !sha.test(head) || base === head) { + throw new Error(`Invalid post-merge review range: ${base}...${head}`); + } + const { owner, repo } = context.repo; + await github.rest.actions.createWorkflowDispatch({ + owner, + repo, + workflow_id: 'pr-review-advisor.yaml', + ref: context.payload.repository.default_branch, + inputs: { + base_ref: base, + head_ref: head, + run_analysis: 'true', + }, + }); + core.summary + .addHeading('Post-merge agent review dispatched') + .addRaw(`Exact range: \`${base}...${head}\``) + .write(); diff --git a/.github/workflows/pr-review-advisor.yaml b/.github/workflows/pr-review-advisor.yaml index 92ca3aa33c6..0a1ce6a1160 100644 --- a/.github/workflows/pr-review-advisor.yaml +++ b/.github/workflows/pr-review-advisor.yaml @@ -46,7 +46,7 @@ on: permissions: {} concurrency: - group: pr-review-advisor-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref }}-${{ inputs.target_repo || github.repository }}-${{ inputs.target_pr || '' }}-${{ github.event_name != 'pull_request_target' || github.event.action != 'edited' || github.event.changes.base != null }} + group: pr-review-advisor-${{ github.event_name }}-${{ github.event.pull_request.number || github.ref }}-${{ inputs.target_repo || github.repository }}-${{ inputs.target_pr || '' }}-${{ inputs.head_ref || '' }}-${{ github.event_name != 'pull_request_target' || github.event.action != 'edited' || github.event.changes.base != null }} cancel-in-progress: true jobs: diff --git a/.github/workflows/release-edition-close.yaml b/.github/workflows/release-edition-close.yaml new file mode 100644 index 00000000000..ad319872fd1 --- /dev/null +++ b/.github/workflows/release-edition-close.yaml @@ -0,0 +1,181 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +name: Release / Edition Close + +on: + schedule: + # GitHub cron is UTC-only. These cover 4:17 PM in PDT and PST; the + # schedule gate below admits only the offset active in Los Angeles. + - cron: "17 23 * * *" + - cron: "17 0 * * *" + +permissions: + actions: read + contents: read + +concurrency: + group: release-edition-close + cancel-in-progress: false + +jobs: + schedule-gate: + runs-on: ubuntu-24.04 + timeout-minutes: 5 + outputs: + active: ${{ steps.gate.outputs.active }} + steps: + - name: Select the active Los Angeles schedule + id: gate + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + SCHEDULE: ${{ github.event.schedule }} + with: + script: | + const schedule = process.env.SCHEDULE ?? ''; + const match = /^17 (0|23) \* \* \*$/.exec(schedule); + if (!match) throw new Error(`Unexpected edition-close schedule: ${schedule}`); + const now = new Date(); + const scheduledAt = new Date(Date.UTC( + now.getUTCFullYear(), + now.getUTCMonth(), + now.getUTCDate(), + Number(match[1]), + 17, + )); + if (scheduledAt.getTime() > now.getTime() + 60 * 60 * 1000) { + scheduledAt.setUTCDate(scheduledAt.getUTCDate() - 1); + } + const localTime = new Intl.DateTimeFormat('en-US', { + timeZone: 'America/Los_Angeles', + hour: '2-digit', + minute: '2-digit', + hourCycle: 'h23', + }).format(scheduledAt); + const active = localTime === '16:17'; + core.info(`${schedule} resolves to ${localTime} America/Los_Angeles`); + core.setOutput('active', String(active)); + + freeze-candidate: + needs: schedule-gate + if: ${{ github.repository == 'NVIDIA/NemoClaw' && needs.schedule-gate.outputs.active == 'true' }} + runs-on: ubuntu-24.04 + timeout-minutes: 15 + steps: + - name: Check out trusted main history + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: main + fetch-depth: 0 + persist-credentials: false + + - name: Set up Node + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: 22.19.0 + + - name: Resolve edition boundary + id: edition + shell: bash + run: | + TZ=America/Los_Angeles node - <<'NODE' >> "$GITHUB_OUTPUT" + const now = new Date(); + const cutoff = new Date(now.getFullYear(), now.getMonth(), now.getDate(), 16); + if (now < cutoff) cutoff.setDate(cutoff.getDate() - 1); + const editionDate = [cutoff.getFullYear(), cutoff.getMonth() + 1, cutoff.getDate()] + .map((part, index) => index === 0 ? String(part) : String(part).padStart(2, '0')) + .join('-'); + console.log(`date=${editionDate}`); + console.log(`cutoff_at=${cutoff.toISOString()}`); + NODE + + - name: Resolve latest server-recorded main push + id: candidate + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + CUTOFF_AT: ${{ steps.edition.outputs.cutoff_at }} + with: + script: | + const cutoff = Date.parse(process.env.CUTOFF_AT); + if (!Number.isFinite(cutoff)) { + throw new Error(`Invalid edition cutoff: ${process.env.CUTOFF_AT}`); + } + + let selected; + for (let page = 1; page <= 10 && !selected; page += 1) { + const response = await github.rest.actions.listWorkflowRuns({ + ...context.repo, + workflow_id: 'post-merge-agent-review.yaml', + branch: 'main', + event: 'push', + per_page: 100, + page, + }); + const eligible = response.data.workflow_runs + .filter((run) => + run.event === 'push' && + run.head_branch === 'main' && + /^[0-9a-f]{40}$/.test(run.head_sha ?? '') && + Number.isFinite(Date.parse(run.created_at)) && + Date.parse(run.created_at) <= cutoff) + .sort((left, right) => + Date.parse(right.created_at) - Date.parse(left.created_at) || right.id - left.id); + selected = eligible[0]; + if (response.data.workflow_runs.length < 100) break; + } + if (!selected) { + throw new Error( + `No server-recorded main push was found at or before ${process.env.CUTOFF_AT}`, + ); + } + + core.info(`Frozen candidate ${selected.head_sha} from ${selected.html_url}`); + core.setOutput('sha', selected.head_sha); + core.setOutput('run_id', String(selected.id)); + core.setOutput('recorded_at', selected.created_at); + core.setOutput('run_url', selected.html_url); + + - name: Freeze release candidate + env: + CANDIDATE_RECORDED_AT: ${{ steps.candidate.outputs.recorded_at }} + CANDIDATE_RUN_ID: ${{ steps.candidate.outputs.run_id }} + CANDIDATE_SHA: ${{ steps.candidate.outputs.sha }} + EDITION_DATE: ${{ steps.edition.outputs.date }} + PLAN_PATH: ${{ runner.temp }}/release-edition/plan.json + run: >- + node --experimental-strip-types --no-warnings scripts/release-plan.mts + --bump patch + --scheduled-edition "$EDITION_DATE" + --candidate-sha "$CANDIDATE_SHA" + --candidate-run-id "$CANDIDATE_RUN_ID" + --candidate-recorded-at "$CANDIDATE_RECORDED_AT" + --output "$PLAN_PATH" + + - name: Summarize frozen edition + env: + PLAN_PATH: ${{ runner.temp }}/release-edition/plan.json + run: | + node - <<'NODE' >> "$GITHUB_STEP_SUMMARY" + const fs = require('fs'); + const plan = JSON.parse(fs.readFileSync(process.env.PLAN_PATH, 'utf8')); + console.log('## Frozen release edition'); + console.log(''); + console.log(`- Edition: \`${plan.authorization.editionDate}\``); + console.log(`- Cutoff: \`${plan.authorization.cutoffAt}\``); + console.log(`- Candidate: \`${plan.candidateCommit}\``); + console.log(`- Candidate recorded: \`${plan.authorization.candidateSource.recordedAt}\``); + console.log(`- Candidate source run: \`${plan.authorization.candidateSource.runId}\``); + console.log(`- Status: \`${plan.status}\``); + console.log(`- Next tag: \`${plan.nextTag}\``); + console.log(`- Plan consistency ID: \`${plan.planHash}\``); + console.log('- The successful scheduled workflow run and its artifact are the authorization boundary.'); + console.log('- E2E is advisory and is not part of tag authorization.'); + NODE + + - name: Upload immutable edition plan + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: release-edition-plan-${{ steps.edition.outputs.date }} + path: ${{ runner.temp }}/release-edition/plan.json + if-no-files-found: error + retention-days: 3 diff --git a/.github/workflows/release-edition-cut.yaml b/.github/workflows/release-edition-cut.yaml new file mode 100644 index 00000000000..9c5b5d23d2d --- /dev/null +++ b/.github/workflows/release-edition-cut.yaml @@ -0,0 +1,295 @@ +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +name: Release / Edition Cut + +on: + schedule: + # GitHub cron is UTC-only. These cover 4:17 AM in PDT and PST; the + # schedule gate below admits only the offset active in Los Angeles. + - cron: "17 11 * * *" + - cron: "17 12 * * *" + workflow_dispatch: + inputs: + edition_date: + description: Frozen edition date (YYYY-MM-DD); manual runs only preflight signing + required: true + type: string + +permissions: {} + +concurrency: + group: release-edition-cut + cancel-in-progress: false + +jobs: + schedule-gate: + runs-on: ubuntu-24.04 + timeout-minutes: 5 + outputs: + active: ${{ steps.gate.outputs.active }} + steps: + - name: Select the active Los Angeles schedule + id: gate + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + EVENT_NAME: ${{ github.event_name }} + SCHEDULE: ${{ github.event.schedule }} + with: + script: | + if (process.env.EVENT_NAME !== 'schedule') { + core.setOutput('active', 'true'); + return; + } + const schedule = process.env.SCHEDULE ?? ''; + const match = /^17 (11|12) \* \* \*$/.exec(schedule); + if (!match) throw new Error(`Unexpected edition-cut schedule: ${schedule}`); + const now = new Date(); + const scheduledAt = new Date(Date.UTC( + now.getUTCFullYear(), + now.getUTCMonth(), + now.getUTCDate(), + Number(match[1]), + 17, + )); + if (scheduledAt.getTime() > now.getTime() + 60 * 60 * 1000) { + scheduledAt.setUTCDate(scheduledAt.getUTCDate() - 1); + } + const localTime = new Intl.DateTimeFormat('en-US', { + timeZone: 'America/Los_Angeles', + hour: '2-digit', + minute: '2-digit', + hourCycle: 'h23', + }).format(scheduledAt); + const active = localTime === '04:17'; + core.info(`${schedule} resolves to ${localTime} America/Los_Angeles`); + core.setOutput('active', String(active)); + + cut: + needs: schedule-gate + if: ${{ github.repository == 'NVIDIA/NemoClaw' && needs.schedule-gate.outputs.active == 'true' }} + runs-on: ubuntu-24.04 + timeout-minutes: 20 + environment: release-tag + permissions: + actions: read + contents: write + outputs: + edition_date: ${{ steps.edition.outputs.date }} + status: ${{ steps.result.outputs.status }} + tag: ${{ steps.result.outputs.tag }} + steps: + - name: Check out trusted release tooling + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: main + fetch-depth: 0 + persist-credentials: false + + - name: Resolve frozen edition + id: edition + env: + DISPATCH_EDITION_DATE: ${{ inputs.edition_date }} + shell: bash + run: | + if [ "$GITHUB_EVENT_NAME" = "schedule" ]; then + edition_date="$(TZ=America/Los_Angeles date --date='yesterday' +%F)" + else + edition_date="$DISPATCH_EDITION_DATE" + fi + if ! [[ "$edition_date" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]]; then + echo "::error::Invalid edition date: $edition_date" + exit 1 + fi + echo "date=$edition_date" >> "$GITHUB_OUTPUT" + echo "artifact=release-edition-plan-$edition_date" >> "$GITHUB_OUTPUT" + + - name: Find successful edition-close artifact + id: close-run + uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 + env: + ARTIFACT_NAME: ${{ steps.edition.outputs.artifact }} + with: + script: | + const artifactName = process.env.ARTIFACT_NAME ?? ''; + const { owner, repo } = context.repo; + const response = await github.rest.actions.listWorkflowRuns({ + owner, + repo, + workflow_id: 'release-edition-close.yaml', + event: 'schedule', + status: 'completed', + branch: 'main', + per_page: 30, + }); + const successful = response.data.workflow_runs + .filter((run) => run.conclusion === 'success' && run.head_branch === 'main') + .sort((a, b) => b.id - a.id); + const sources = []; + for (const run of successful) { + const artifacts = await github.paginate(github.rest.actions.listWorkflowRunArtifacts, { + owner, + repo, + run_id: run.id, + per_page: 100, + }); + const matches = artifacts.filter( + (artifact) => artifact.name === artifactName && artifact.expired === false, + ); + if (matches.length > 1) { + throw new Error(`Run ${run.id} has duplicate ${artifactName} artifacts`); + } + if (matches.length === 1) { + sources.push({ run, artifact: matches[0] }); + } + } + if (sources.length !== 1) { + throw new Error( + `Expected exactly one successful scheduled source for ${artifactName}, found ${sources.length}`, + ); + } + const [{ run }] = sources; + core.setOutput('run_id', String(run.id)); + core.info(`Using ${artifactName} from trusted scheduled run ${run.html_url}`); + + - name: Download frozen edition plan + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + name: ${{ steps.edition.outputs.artifact }} + path: ${{ runner.temp }}/release-edition + repository: NVIDIA/NemoClaw + run-id: ${{ steps.close-run.outputs.run_id }} + github-token: ${{ github.token }} + + - name: Bind plan to requested edition + env: + EDITION_DATE: ${{ steps.edition.outputs.date }} + PLAN_PATH: ${{ runner.temp }}/release-edition/plan.json + run: | + node - <<'NODE' + const fs = require('fs'); + const plan = JSON.parse(fs.readFileSync(process.env.PLAN_PATH, 'utf8')); + if (plan.authorization?.type !== 'scheduled-workflow') throw new Error('Plan is not scheduled'); + if (plan.authorization.editionDate !== process.env.EDITION_DATE) { + throw new Error(`Plan edition ${plan.authorization.editionDate} does not match ${process.env.EDITION_DATE}`); + } + NODE + + - name: Configure dedicated release signer + id: signer + env: + SIGNING_KEY: ${{ secrets.NEMOCLAW_RELEASE_TAG_SIGNING_KEY }} + SIGNER_EMAIL: ${{ vars.NEMOCLAW_RELEASE_TAG_SIGNER_EMAIL }} + SIGNER_NAME: ${{ vars.NEMOCLAW_RELEASE_TAG_SIGNER_NAME }} + PLAN_PATH: ${{ runner.temp }}/release-edition/plan.json + shell: bash + run: | + plan_status="$(node -p "require(process.env.PLAN_PATH).status")" + if [ "$plan_status" = "no-changes" ]; then + echo "configured=false" >> "$GITHUB_OUTPUT" + exit 0 + fi + if [ -z "$SIGNING_KEY" ] || [ -z "$SIGNER_EMAIL" ] || [ -z "$SIGNER_NAME" ]; then + echo "::error::The release-tag environment is missing its signing key or signer identity" + exit 1 + fi + key_path="$RUNNER_TEMP/nemoclaw-release-tag-signing-key" + umask 077 + printf '%s\n' "$SIGNING_KEY" > "$key_path" + git config gpg.format ssh + git config user.signingkey "$key_path" + git config user.email "$SIGNER_EMAIL" + git config user.name "$SIGNER_NAME" + echo "configured=true" >> "$GITHUB_OUTPUT" + + - name: Cut scheduled tag or preflight manual dispatch + env: + PLAN_PATH: ${{ runner.temp }}/release-edition/plan.json + PUSH_REMOTE_URL: https://x-access-token:${{ github.token }}@github.com/${{ github.repository }}.git + shell: bash + run: | + if [ "$GITHUB_EVENT_NAME" = "schedule" ]; then + scripts/release-cut-tag.sh --plan "$PLAN_PATH" --scheduled + else + scripts/release-cut-tag.sh --plan "$PLAN_PATH" --preflight-only + fi + + - name: Remove release signer + if: ${{ always() }} + run: rm -f -- "$RUNNER_TEMP/nemoclaw-release-tag-signing-key" + + - name: Export cut result + id: result + env: + RESULT_PATH: ${{ runner.temp }}/release-edition/cut-result.json + run: | + status="$(node -p "require(process.env.RESULT_PATH).status")" + tag="$(node -p "require(process.env.RESULT_PATH).tag || ''")" + echo "status=$status" >> "$GITHUB_OUTPUT" + echo "tag=$tag" >> "$GITHUB_OUTPUT" + + - name: Upload cut evidence + if: ${{ always() }} + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: release-edition-cut-${{ steps.edition.outputs.date }} + path: ${{ runner.temp }}/release-edition/ + if-no-files-found: error + retention-days: 14 + + promote: + needs: cut + if: ${{ needs.cut.outputs.status == 'tagged' }} + uses: ./.github/workflows/release-latest-tag.yaml + permissions: + contents: write + issues: write + pull-requests: write + with: + tag: ${{ needs.cut.outputs.tag }} + + verify-and-handoff: + needs: [cut, promote] + if: ${{ always() && (needs.cut.outputs.status == 'tagged' || needs.cut.outputs.status == 'no-changes') }} + permissions: + actions: read + contents: read + pull-requests: read + runs-on: ubuntu-24.04 + timeout-minutes: 15 + steps: + - name: Check out release verification tooling + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + ref: main + fetch-depth: 0 + persist-credentials: false + + - name: Set up Node + uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0 + with: + node-version: 22.19.0 + + - name: Download cut evidence + uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1 + with: + pattern: release-edition-cut-* + path: ${{ runner.temp }}/release-edition + merge-multiple: true + + - name: Verify latest and create handoff data + if: ${{ needs.cut.outputs.status == 'tagged' }} + env: + GH_TOKEN: ${{ github.token }} + PLAN_PATH: ${{ runner.temp }}/release-edition/plan.json + run: | + scripts/release-wait-latest.sh --plan "$PLAN_PATH" + node --experimental-strip-types --no-warnings scripts/release-notes-data.mts --plan "$PLAN_PATH" + + - name: Upload verified handoff + uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1 + with: + name: release-edition-handoff-${{ needs.cut.outputs.tag || format('no-changes-{0}', needs.cut.outputs.edition_date) }} + path: ${{ runner.temp }}/release-edition/ + if-no-files-found: error + retention-days: 14 diff --git a/.github/workflows/release-latest-tag.yaml b/.github/workflows/release-latest-tag.yaml index b7d81a16e9f..b40d391c835 100644 --- a/.github/workflows/release-latest-tag.yaml +++ b/.github/workflows/release-latest-tag.yaml @@ -7,6 +7,12 @@ on: push: tags: - "v*.*.*" + workflow_call: + inputs: + tag: + description: Semver release tag created by the scheduled edition cut + required: true + type: string workflow_dispatch: inputs: tag: @@ -40,7 +46,7 @@ jobs: id: verify-release-tag uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0 env: - RELEASE_TAG: ${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref_name }} + RELEASE_TAG: ${{ inputs.tag || github.ref_name }} with: script: | const releaseTag = process.env.RELEASE_TAG ?? ''; @@ -87,7 +93,7 @@ jobs: - name: Move latest to the verified release tag object env: EXPECTED_RELEASE_TAG_OBJECT: ${{ steps.verify-release-tag.outputs.tag_object_sha }} - RELEASE_TAG: ${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref_name }} + RELEASE_TAG: ${{ inputs.tag || github.ref_name }} PUSH_REMOTE_URL: https://x-access-token:${{ github.token }}@github.com/${{ github.repository }}.git run: scripts/release-latest-tag.sh @@ -99,7 +105,7 @@ jobs: - name: Retire the released target label env: GH_TOKEN: ${{ github.token }} - RELEASE_TAG: ${{ github.event_name == 'workflow_dispatch' && inputs.tag || github.ref_name }} + RELEASE_TAG: ${{ inputs.tag || github.ref_name }} run: >- node --experimental-strip-types --no-warnings scripts/retire-release-label.mts "$RELEASE_TAG" --repo "$GITHUB_REPOSITORY" diff --git a/ci/source-shape-test-budget.json b/ci/source-shape-test-budget.json index 6f4a5d81121..85032ac702f 100644 --- a/ci/source-shape-test-budget.json +++ b/ci/source-shape-test-budget.json @@ -606,6 +606,11 @@ "test": "keeps the installer verifier inside the trusted composite action", "category": "security" }, + { + "file": "test/deterministic-smoke-workflow.test.ts", + "test": "keeps the deterministic smoke floor in the required build and typecheck action", + "category": "security" + }, { "file": "test/pr-workflow-contract.test.ts", "test": "keeps the trusted test-size guard closed around budget policy changes", @@ -686,6 +691,36 @@ "test": "binds latest promotion to the exact GitHub-verified tag object", "category": "security" }, + { + "file": "test/release-edition-workflow.test.ts", + "test": "closes the edition after the exact Los Angeles cutoff with read-only permissions", + "category": "security" + }, + { + "file": "test/release-edition-workflow.test.ts", + "test": "admits only the UTC close schedule that is 4:17 PM in Los Angeles", + "category": "security" + }, + { + "file": "test/release-edition-workflow.test.ts", + "test": "cuts at 4 AM without E2E input and delegates exact-object promotion", + "category": "security" + }, + { + "file": "test/release-edition-workflow.test.ts", + "test": "admits the correct UTC cut schedule in PST, PDT, and manual mode", + "category": "security" + }, + { + "file": "test/release-edition-workflow.test.ts", + "test": "dispatches one asynchronous advisor run for the exact main SHA range", + "category": "security" + }, + { + "file": "test/release-edition-workflow.test.ts", + "test": "keeps independent merge reviews from cancelling each other", + "category": "security" + }, { "file": "test/release-lkg-brev-image.test.ts", "test": "keeps LKG dispatch inside the trusted secret boundary (#6772)", diff --git a/package.json b/package.json index 6de43ffcf79..3f0bcf0b2eb 100644 --- a/package.json +++ b/package.json @@ -34,6 +34,7 @@ "test:runtime-audit": "tsx scripts/audit-test-runtime.mts", "test:integration": "npm run clean:cli && npm run build:cli && vitest run --project integration --project installer-integration", "test:package": "npm run clean:cli && npm --prefix nemoclaw run clean && npm run build:cli && npm --prefix nemoclaw run build && vitest run --project package-contract", + "test:smoke": "npm run clean:cli && npm --prefix nemoclaw run clean && npm run build:cli && npm --prefix nemoclaw run build && vitest run --project package-contract test/package-contract/deterministic-smoke.test.ts", "test:coverage:cli": "npm run clean:cli && npm run build:cli && tsx scripts/check-dist-sourcemaps.mts dist && vitest run --project cli --project integration --coverage --coverage.reporter=text-summary --coverage.reporter=json-summary --coverage.reportsDirectory=coverage/cli --coverage.include=\"bin/**/*.js\" --coverage.include=\"src/**/*.ts\" --coverage.exclude=\"test/**/*.js\" --coverage.exclude=\"test/**/*.ts\" && tsx scripts/check-coverage-ratchet.mts coverage/cli/coverage-summary.json ci/coverage-threshold-cli.json \"CLI coverage\"", "test:coverage:plugin": "vitest run --project plugin --coverage --coverage.reporter=text-summary --coverage.reporter=json-summary --coverage.reportsDirectory=coverage/plugin --coverage.include=\"nemoclaw/src/**/*.ts\" --coverage.include=\"nemoclaw/src/**/*.cts\" --coverage.exclude=\"**/*.test.ts\" && tsx scripts/check-coverage-ratchet.mts coverage/plugin/coverage-summary.json ci/coverage-threshold-plugin.json \"Plugin coverage\"", "test:live-e2e": "npm run clean:cli && npm run build:cli && NEMOCLAW_RUN_LIVE_E2E=1 vitest run --project e2e-live", @@ -69,7 +70,6 @@ "test-conditionals:scan": "tsx scripts/find-test-conditionals.mts", "bump:version": "tsx scripts/bump-version.mts", "release:plan": "tsx scripts/release-plan.mts", - "release:e2e-evidence": "node --experimental-strip-types --no-warnings .agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts", "release:cut": "bash scripts/release-cut-tag.sh", "release:wait-latest": "bash scripts/release-wait-latest.sh", "release:notes-data": "tsx scripts/release-notes-data.mts", diff --git a/scripts/release-cut-tag.sh b/scripts/release-cut-tag.sh index e62ba611f0d..e82b713e243 100755 --- a/scripts/release-cut-tag.sh +++ b/scripts/release-cut-tag.sh @@ -7,6 +7,7 @@ set -euo pipefail PLAN_PATH="" CONFIRMATION="${RELEASE_CONFIRMATION:-}" PREFLIGHT_ONLY=false +SCHEDULED=false while [[ $# -gt 0 ]]; do case "$1" in @@ -22,14 +23,20 @@ while [[ $# -gt 0 ]]; do PREFLIGHT_ONLY=true shift ;; + --scheduled) + SCHEDULED=true + shift + ;; --help | -h) cat <<'USAGE' Usage: scripts/release-cut-tag.sh --plan PATH --preflight-only scripts/release-cut-tag.sh --plan PATH --confirm "CONFIRM RELEASE vX.Y.Z " + scripts/release-cut-tag.sh --plan PATH --scheduled -Preflight mode verifies that Git can create a signed annotated tag with the configured signer. -Cut mode creates and pushes only the signed annotated semver tag described by a release plan. +Preflight mode verifies that Git can create a signed annotated semver tag with the configured signer. +Confirmed mode cuts a manual plan or recovers a frozen scheduled plan. +Scheduled mode is accepted only in the canonical 4 AM workflow and never consults E2E state. USAGE exit 0 ;; @@ -47,9 +54,10 @@ fail() { [[ -n "$PLAN_PATH" ]] || fail "--plan is required" [[ -f "$PLAN_PATH" ]] || fail "Plan file not found: $PLAN_PATH" -if [[ "$PREFLIGHT_ONLY" != true ]]; then +if [[ "$PREFLIGHT_ONLY" != true && "$SCHEDULED" != true ]]; then [[ -n "$CONFIRMATION" ]] || fail "--confirm is required" fi +[[ "$PREFLIGHT_ONLY" != true || "$SCHEDULED" != true ]] || fail "--scheduled and --preflight-only are mutually exclusive" json_field() { node -e 'const fs=require("fs"); const data=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); const path=process.argv[2].split("."); let value=data; for (const key of path) value=value?.[key]; if (value == null) process.exit(1); process.stdout.write(String(value));' "$PLAN_PATH" "$1" @@ -61,32 +69,97 @@ cd "$repo_root" status="$(git status --short)" [[ -z "$status" ]] || fail "Release tagging requires a clean worktree" -node -e 'const fs=require("fs"); const crypto=require("crypto"); const data=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); const semver=/^v\d+\.\d+\.\d+$/; const sha=/^[0-9a-f]{40}$/; const hash=/^[0-9a-f]{64}$/; if (data.schemaVersion !== 1) throw new Error("schemaVersion must be 1"); if (data.mode !== "tag-only") throw new Error("mode must be tag-only"); if (!semver.test(data.previousTag)) throw new Error("previousTag must be semver"); if (!semver.test(data.nextTag)) throw new Error("nextTag must be semver"); if (!sha.test(data.originMainCommit)) throw new Error("originMainCommit must be a full SHA"); if (!hash.test(data.planHash)) throw new Error("planHash must be a sha256 hex string"); const {planHash, ...planWithoutHash}=data; const actual=crypto.createHash("sha256").update(JSON.stringify(planWithoutHash, null, 2)).digest("hex"); if (actual !== planHash) throw new Error("planHash mismatch: expected " + planHash + ", recomputed " + actual);' "$PLAN_PATH" +node -e ' +const fs=require("fs"); const crypto=require("crypto"); +const data=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); +const semver=/^v\d+\.\d+\.\d+$/; const sha=/^[0-9a-f]{40}$/; const hash=/^[0-9a-f]{64}$/; +if (data.schemaVersion !== 2) throw new Error("schemaVersion must be 2"); +if (data.mode !== "tag-only") throw new Error("mode must be tag-only"); +if (data.status !== "ready" && data.status !== "no-changes") throw new Error("invalid plan status"); +if (!semver.test(data.previousTag) || !semver.test(data.nextTag)) throw new Error("release tags must be semver"); +if (!sha.test(data.originMainAtPlanning) || !sha.test(data.candidateCommit)) throw new Error("plan commits must be full SHAs"); +if (!Number.isSafeInteger(data.untaggedCommitCount) || data.untaggedCommitCount < 0) throw new Error("untaggedCommitCount must be a nonnegative integer"); +if (data.status === "ready" && (typeof data.changelogEntry !== "string" || data.changelogEntry.length === 0)) throw new Error("ready plans require changelogEntry"); +if (data.status === "no-changes" && data.untaggedCommitCount !== 0) throw new Error("no-changes plans require zero untagged commits"); +if (data.authorization?.type !== "maintainer-confirmation" && data.authorization?.type !== "scheduled-workflow") throw new Error("invalid release authorization"); +if (data.authorization.type === "scheduled-workflow") { + if (data.authorization.repository !== "NVIDIA/NemoClaw") throw new Error("invalid scheduled repository"); + if (data.authorization.plannerWorkflow !== ".github/workflows/release-edition-close.yaml") throw new Error("invalid planner workflow"); + if (!/^\d{4}-\d{2}-\d{2}$/.test(data.authorization.editionDate)) throw new Error("invalid edition date"); + if (typeof data.authorization.cutoffAt !== "string" || Number.isNaN(Date.parse(data.authorization.cutoffAt))) throw new Error("invalid edition cutoff"); + const source=data.authorization.candidateSource; + if (source?.type !== "github-actions-push-run") throw new Error("invalid candidate source type"); + if (source.workflow !== ".github/workflows/post-merge-agent-review.yaml") throw new Error("invalid candidate source workflow"); + if (!/^\d+$/.test(source.runId)) throw new Error("invalid candidate source run ID"); + if (typeof source.recordedAt !== "string" || Number.isNaN(Date.parse(source.recordedAt))) throw new Error("invalid candidate source time"); + if (Date.parse(source.recordedAt) > Date.parse(data.authorization.cutoffAt)) throw new Error("candidate source is after edition cutoff"); +} +if (!hash.test(data.planHash)) throw new Error("planHash must be a sha256 hex string"); +const {planHash, ...planWithoutHash}=data; +const actual=crypto.createHash("sha256").update(JSON.stringify(planWithoutHash, null, 2)).digest("hex"); +if (actual !== planHash) throw new Error("planHash mismatch: expected " + planHash + ", recomputed " + actual); +' "$PLAN_PATH" schema_version="$(json_field schemaVersion)" -mode="$(json_field mode)" +plan_status="$(json_field status)" +authorization_type="$(json_field authorization.type)" +previous_tag="$(json_field previousTag)" tag="$(json_field nextTag)" -target="$(json_field originMainCommit)" +target="$(json_field candidateCommit)" expected_confirmation="$(json_field confirmationPhrase)" plan_hash="$(json_field planHash)" -[[ "$schema_version" == "1" ]] || fail "Unsupported plan schemaVersion: $schema_version" -[[ "$mode" == "tag-only" ]] || fail "Unsupported plan mode: $mode" -if [[ "$PREFLIGHT_ONLY" != true ]]; then +[[ "$schema_version" == "2" ]] || fail "Unsupported plan schemaVersion: $schema_version" +if [[ "$SCHEDULED" == true ]]; then + [[ "$authorization_type" == "scheduled-workflow" ]] || fail "Scheduled cuts require scheduled-workflow authorization" + [[ "${GITHUB_ACTIONS:-}" == "true" ]] || fail "Scheduled cuts require GitHub Actions" + [[ "${GITHUB_REPOSITORY:-}" == "NVIDIA/NemoClaw" ]] || fail "Scheduled cuts require NVIDIA/NemoClaw" + [[ "${GITHUB_EVENT_NAME:-}" == "schedule" ]] || fail "Scheduled cuts require a schedule event" + expected_workflow_ref="NVIDIA/NemoClaw/.github/workflows/release-edition-cut.yaml@refs/heads/main" + [[ "${GITHUB_WORKFLOW_REF:-}" == "$expected_workflow_ref" ]] || fail "Scheduled cuts require the canonical main-branch workflow" +elif [[ "$PREFLIGHT_ONLY" != true ]]; then [[ "$CONFIRMATION" == "$expected_confirmation" ]] || fail "Confirmation phrase does not match plan" fi + [[ "$tag" =~ ^v[0-9]+\.[0-9]+\.[0-9]+$ ]] || fail "Plan tag is not semver: $tag" [[ "$target" =~ ^[0-9a-f]{40}$ ]] || fail "Plan target commit is not a full SHA: $target" [[ "$plan_hash" =~ ^[0-9a-f]{64}$ ]] || fail "Plan hash is not a SHA-256 hex string: $plan_hash" git fetch origin main --tags --force - -current_origin_main="$(git rev-parse origin/main)" -[[ "$current_origin_main" == "$target" ]] || fail "origin/main moved from plan target $target to $current_origin_main; regenerate the plan" - git cat-file -e "${target}^{commit}" || fail "Target commit does not exist: $target" +git merge-base --is-ancestor "$previous_tag" "$target" || fail "$previous_tag is not an ancestor of target $target" git merge-base --is-ancestor "$target" origin/main || fail "Target commit is not reachable from origin/main: $target" +if [[ "$authorization_type" == "maintainer-confirmation" ]]; then + current_origin_main="$(git rev-parse origin/main)" + [[ "$current_origin_main" == "$target" ]] || fail "origin/main moved from manual plan target $target to $current_origin_main; regenerate the plan" +fi + +latest_remote_semver="$(git ls-remote --tags origin 'v*' | node -e ' +let input=""; process.stdin.setEncoding("utf8"); process.stdin.on("data", (chunk) => input += chunk); process.stdin.on("end", () => { + const tags=[...new Set(input.split("\n").map((line) => (line.trim().split(/\s+/)[1] || "").replace(/^refs\/tags\//, "").replace(/\^\{\}$/, "")).filter((tag) => /^v\d+\.\d+\.\d+$/.test(tag)))]; + tags.sort((a,b) => { const pa=a.slice(1).split(".").map(Number); const pb=b.slice(1).split(".").map(Number); for (let i=0;i<3;i+=1) if (pa[i] !== pb[i]) return pb[i]-pa[i]; return 0; }); + process.stdout.write(tags[0] || ""); +}); +')" +[[ "$latest_remote_semver" == "$previous_tag" ]] || fail "Latest remote semver changed from $previous_tag to ${latest_remote_semver:-none}; regenerate the plan" + +untagged_count="$(git rev-list --count "${previous_tag}..${target}")" +if [[ "$plan_status" == "no-changes" ]]; then + [[ "$untagged_count" == "0" ]] || fail "No-change plan target is ahead of $previous_tag" + result_path="$(dirname "$PLAN_PATH")/cut-result.json" + node -e 'const fs=require("fs"); const result={schemaVersion:2,status:"no-changes",planPath:process.argv[1],planHash:process.argv[2],tag:null,targetCommit:process.argv[3],latestTouched:false,lkgTouched:false,createdAt:new Date().toISOString()}; fs.writeFileSync(process.argv[4], JSON.stringify(result, null, 2) + "\n");' "$PLAN_PATH" "$plan_hash" "$target" "$result_path" + printf 'release-cut-tag: no untagged commits; no tag created for %s\n' "$target" + printf 'release-cut-tag: result written: %s\n' "$result_path" + exit 0 +fi + +[[ "$untagged_count" != "0" ]] || fail "Ready plan has no commits after $previous_tag" +tag_pattern="${tag//./\\.}" +changelog_matches="$(git grep -n -E "^## ${tag_pattern}$" "$target" -- ':(glob)docs/changelog/*.mdx' || true)" +changelog_count="$(printf '%s\n' "$changelog_matches" | awk 'NF { count += 1 } END { print count + 0 }')" +[[ "$changelog_count" == "1" ]] || fail "Expected exactly one direct changelog entry for $tag at $target, found $changelog_count" + if git show-ref --verify --quiet "refs/tags/$tag"; then fail "Local tag already exists: $tag" fi @@ -106,25 +179,25 @@ if [[ "$PREFLIGHT_ONLY" == true ]]; then } trap cleanup_preflight_tag EXIT - # Exercise Git's configured OpenPGP, SSH, or X.509 signer without publishing a ref. git tag -s "$preflight_tag" "$target" -m "NemoClaw release signing preflight" cleanup_preflight_tag trap - EXIT + result_path="$(dirname "$PLAN_PATH")/cut-result.json" + node -e 'const fs=require("fs"); const result={schemaVersion:2,status:"preflight",planPath:process.argv[1],planHash:process.argv[2],tag:process.argv[3],targetCommit:process.argv[4],latestTouched:false,lkgTouched:false,createdAt:new Date().toISOString()}; fs.writeFileSync(process.argv[5], JSON.stringify(result, null, 2) + "\n");' "$PLAN_PATH" "$plan_hash" "$tag" "$target" "$result_path" printf 'release-cut-tag: signing preflight passed for %s at %s\n' "$tag" "$target" exit 0 fi -# Release tags are immutable once pushed. Sign the tag on the release -# operator's workstation so the private signing key never enters CI. git tag -s "$tag" "$target" -m "$tag" -git push origin "refs/tags/$tag" +push_remote="${PUSH_REMOTE_URL:-origin}" +git push "$push_remote" "refs/tags/$tag" remote_peeled="$(git ls-remote --tags origin "refs/tags/$tag^{}" | awk '{print $1}')" [[ "$remote_peeled" == "$target" ]] || fail "Remote $tag peeled to $remote_peeled, expected $target" result_path="$(dirname "$PLAN_PATH")/cut-result.json" -node -e 'const fs=require("fs"); const result={schemaVersion:1,status:"ok",planPath:process.argv[1],planHash:process.argv[2],tag:process.argv[3],targetCommit:process.argv[4],remotePeeledCommit:process.argv[5],latestTouched:false,lkgTouched:false,createdAt:new Date().toISOString()}; fs.writeFileSync(process.argv[6], JSON.stringify(result, null, 2) + "\n");' "$PLAN_PATH" "$plan_hash" "$tag" "$target" "$remote_peeled" "$result_path" +node -e 'const fs=require("fs"); const result={schemaVersion:2,status:"tagged",planPath:process.argv[1],planHash:process.argv[2],tag:process.argv[3],targetCommit:process.argv[4],remotePeeledCommit:process.argv[5],latestTouched:false,lkgTouched:false,createdAt:new Date().toISOString()}; fs.writeFileSync(process.argv[6], JSON.stringify(result, null, 2) + "\n");' "$PLAN_PATH" "$plan_hash" "$tag" "$target" "$remote_peeled" "$result_path" printf 'release-cut-tag: pushed %s at %s\n' "$tag" "$target" printf 'release-cut-tag: result written: %s\n' "$result_path" diff --git a/scripts/release-notes-data.mts b/scripts/release-notes-data.mts index bbf5acaeb7d..34a83e2a633 100644 --- a/scripts/release-notes-data.mts +++ b/scripts/release-notes-data.mts @@ -12,11 +12,12 @@ type Options = { }; type ReleasePlan = { - schemaVersion: 1; + schemaVersion: 2; mode: "tag-only"; + status: "ready"; previousTag: string; nextTag: string; - originMainCommit: string; + candidateCommit: string; planHash: string; }; @@ -69,20 +70,23 @@ function validatePlan(value: unknown): ReleasePlan { const semver = /^v\d+\.\d+\.\d+$/; const sha = /^[0-9a-f]{40}$/; const hash = /^[0-9a-f]{64}$/; - if (plan.schemaVersion !== 1) { - throw new Error("Release plan schemaVersion must be 1"); + if (plan.schemaVersion !== 2) { + throw new Error("Release plan schemaVersion must be 2"); } if (plan.mode !== "tag-only") { throw new Error("Release plan mode must be tag-only"); } + if (plan.status !== "ready") { + throw new Error("Release notes require a ready release plan"); + } if (typeof plan.previousTag !== "string" || !semver.test(plan.previousTag)) { throw new Error("Release plan previousTag must be a semver tag"); } if (typeof plan.nextTag !== "string" || !semver.test(plan.nextTag)) { throw new Error("Release plan nextTag must be a semver tag"); } - if (typeof plan.originMainCommit !== "string" || !sha.test(plan.originMainCommit)) { - throw new Error("Release plan originMainCommit must be a full SHA"); + if (typeof plan.candidateCommit !== "string" || !sha.test(plan.candidateCommit)) { + throw new Error("Release plan candidateCommit must be a full SHA"); } if (typeof plan.planHash !== "string" || !hash.test(plan.planHash)) { throw new Error("Release plan planHash must be a sha256 hex string"); @@ -158,7 +162,7 @@ function main(): void { planHash: plan.planHash, previousTag: plan.previousTag, currentTag: plan.nextTag, - targetCommit: plan.originMainCommit, + targetCommit: plan.candidateCommit, compareRange, compare, prNumbers, diff --git a/scripts/release-plan.mts b/scripts/release-plan.mts index 4e62a3cfb19..7cedb0d83a0 100644 --- a/scripts/release-plan.mts +++ b/scripts/release-plan.mts @@ -1,8 +1,8 @@ // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. // SPDX-License-Identifier: Apache-2.0 -import { createHash } from "node:crypto"; import { execFileSync } from "node:child_process"; +import { createHash } from "node:crypto"; import { mkdirSync, writeFileSync } from "node:fs"; import path from "node:path"; import process from "node:process"; @@ -11,7 +11,11 @@ type Bump = "patch" | "minor" | "major"; type Options = { bump: Bump; + candidateRecordedAt?: string; + candidateRunId?: string; + candidateSha?: string; output?: string; + scheduledEdition?: string; }; type RemoteTag = { @@ -20,15 +24,36 @@ type RemoteTag = { peeledSha?: string; }; +type ReleaseAuthorization = + | { type: "maintainer-confirmation" } + | { + type: "scheduled-workflow"; + repository: "NVIDIA/NemoClaw"; + plannerWorkflow: ".github/workflows/release-edition-close.yaml"; + editionDate: string; + cutoffAt: string; + candidateSource: { + type: "github-actions-push-run"; + workflow: ".github/workflows/post-merge-agent-review.yaml"; + runId: string; + recordedAt: string; + }; + }; + type ReleasePlan = { - schemaVersion: 1; + schemaVersion: 2; mode: "tag-only"; + status: "ready" | "no-changes"; + authorization: ReleaseAuthorization; previousTag: string; nextTag: string; bump: Bump; originRemote: string; - originMainCommit: string; - originMainHeadline: string; + originMainAtPlanning: string; + candidateCommit: string; + candidateHeadline: string; + untaggedCommitCount: number; + changelogEntry: string | null; compareRange: string; latestBefore: RemoteTag | null; lkgBefore: RemoteTag | null; @@ -47,9 +72,7 @@ function run(command: string, args: string[], options: { allowFailure?: boolean stdio: ["ignore", "pipe", "pipe"], }); } catch (error) { - if (options.allowFailure) { - return ""; - } + if (options.allowFailure) return ""; const e = error as { stdout?: Buffer | string; stderr?: Buffer | string }; const stdout = e.stdout ? String(e.stdout).trim() : ""; const stderr = e.stderr ? String(e.stderr).trim() : ""; @@ -61,7 +84,11 @@ function run(command: string, args: string[], options: { allowFailure?: boolean function parseArgs(argv: string[]): Options { let bump: Bump = "patch"; + let candidateRecordedAt: string | undefined; + let candidateRunId: string | undefined; + let candidateSha: string | undefined; let output: string | undefined; + let scheduledEdition: string | undefined; for (let i = 0; i < argv.length; i += 1) { const arg = argv[i]; if (arg === "--bump") { @@ -80,6 +107,22 @@ function parseArgs(argv: string[]): Options { output = argv[++i]; } else if (arg.startsWith("--output=")) { output = arg.slice("--output=".length); + } else if (arg === "--scheduled-edition") { + scheduledEdition = argv[++i]; + } else if (arg.startsWith("--scheduled-edition=")) { + scheduledEdition = arg.slice("--scheduled-edition=".length); + } else if (arg === "--candidate-sha") { + candidateSha = argv[++i]; + } else if (arg.startsWith("--candidate-sha=")) { + candidateSha = arg.slice("--candidate-sha=".length); + } else if (arg === "--candidate-run-id") { + candidateRunId = argv[++i]; + } else if (arg.startsWith("--candidate-run-id=")) { + candidateRunId = arg.slice("--candidate-run-id=".length); + } else if (arg === "--candidate-recorded-at") { + candidateRecordedAt = argv[++i]; + } else if (arg.startsWith("--candidate-recorded-at=")) { + candidateRecordedAt = arg.slice("--candidate-recorded-at=".length); } else if (arg === "--help" || arg === "-h") { printHelp(); process.exit(0); @@ -87,20 +130,52 @@ function parseArgs(argv: string[]): Options { throw new Error(`Unknown argument: ${arg}`); } } - return { bump, output }; + if (scheduledEdition !== undefined && !/^\d{4}-\d{2}-\d{2}$/.test(scheduledEdition)) { + throw new Error(`Invalid --scheduled-edition value: ${scheduledEdition}`); + } + const candidateInputs = [candidateSha, candidateRunId, candidateRecordedAt]; + if (scheduledEdition && candidateInputs.some((value) => value === undefined)) { + throw new Error( + "Scheduled planning requires --candidate-sha, --candidate-run-id, and --candidate-recorded-at", + ); + } + if (!scheduledEdition && candidateInputs.some((value) => value !== undefined)) { + throw new Error("Candidate source inputs require --scheduled-edition"); + } + if (candidateSha !== undefined && !/^[0-9a-f]{40}$/.test(candidateSha)) { + throw new Error(`Invalid --candidate-sha value: ${candidateSha}`); + } + if (candidateRunId !== undefined && !/^\d+$/.test(candidateRunId)) { + throw new Error(`Invalid --candidate-run-id value: ${candidateRunId}`); + } + if (candidateRecordedAt !== undefined && Number.isNaN(Date.parse(candidateRecordedAt))) { + throw new Error(`Invalid --candidate-recorded-at value: ${candidateRecordedAt}`); + } + if (candidateRecordedAt !== undefined) { + candidateRecordedAt = new Date(candidateRecordedAt).toISOString(); + } + return { + bump, + candidateRecordedAt, + candidateRunId, + candidateSha, + output, + scheduledEdition, + }; } function printHelp(): void { - console.log( - `Usage: tsx scripts/release-plan.mts [--bump patch|minor|major] [--output PATH]\n\nCreates a deterministic tag-only release plan for origin/main.`, - ); + console.log(`Usage: tsx scripts/release-plan.mts [--bump patch|minor|major] [--output PATH] + tsx scripts/release-plan.mts --scheduled-edition YYYY-MM-DD \\ + --candidate-sha SHA --candidate-run-id ID --candidate-recorded-at ISO [--output PATH] + +Creates a consistency-hashed tag-only release plan. A scheduled plan binds the +latest GitHub-recorded main push at or before 4:00 PM America/Los_Angeles.`); } function semverParts(tag: string): [number, number, number] { const match = /^v(\d+)\.(\d+)\.(\d+)$/.exec(tag); - if (!match) { - throw new Error(`Invalid semver tag: ${tag}`); - } + if (!match) throw new Error(`Invalid semver tag: ${tag}`); return [Number(match[1]), Number(match[2]), Number(match[3])]; } @@ -108,21 +183,15 @@ function compareSemverDesc(a: string, b: string): number { const pa = semverParts(a); const pb = semverParts(b); for (let i = 0; i < 3; i += 1) { - if (pa[i] !== pb[i]) { - return pb[i] - pa[i]; - } + if (pa[i] !== pb[i]) return pb[i] - pa[i]; } return 0; } function bumpTag(tag: string, bump: Bump): string { const [major, minor, patch] = semverParts(tag); - if (bump === "major") { - return `v${major + 1}.0.0`; - } - if (bump === "minor") { - return `v${major}.${minor + 1}.0`; - } + if (bump === "major") return `v${major + 1}.0.0`; + if (bump === "minor") return `v${major}.${minor + 1}.0`; return `v${major}.${minor}.${patch + 1}`; } @@ -145,21 +214,75 @@ function readRemoteTag(tag: string): RemoteTag | null { .split("\n") .map((line) => line.trim()) .filter(Boolean); - if (lines.length === 0) { - return null; - } + if (lines.length === 0) return null; const result: RemoteTag = { tag, objectSha: "" }; for (const line of lines) { const [sha, ref] = line.split(/\s+/); - if (ref.endsWith("^{}")) { - result.peeledSha = sha; - } else { - result.objectSha = sha; - } + if (ref.endsWith("^{}")) result.peeledSha = sha; + else result.objectSha = sha; } return result.objectSha ? result : null; } +function zonedCutoff(editionDate: string): string { + const [year, month, day] = editionDate.split("-").map(Number); + const calendarDate = new Date(Date.UTC(year, month - 1, day)); + if (calendarDate.toISOString().slice(0, 10) !== editionDate) { + throw new Error(`Invalid edition date: ${editionDate}`); + } + const desiredWallTime = Date.UTC(year, month - 1, day, 16, 0, 0); + let instant = desiredWallTime; + const formatter = new Intl.DateTimeFormat("en-US", { + timeZone: "America/Los_Angeles", + year: "numeric", + month: "2-digit", + day: "2-digit", + hour: "2-digit", + minute: "2-digit", + second: "2-digit", + hourCycle: "h23", + }); + for (let attempt = 0; attempt < 3; attempt += 1) { + const parts = Object.fromEntries( + formatter + .formatToParts(new Date(instant)) + .filter((part) => part.type !== "literal") + .map((part) => [part.type, Number(part.value)]), + ); + const actualWallTime = Date.UTC( + parts.year, + parts.month - 1, + parts.day, + parts.hour, + parts.minute, + parts.second, + ); + instant += desiredWallTime - actualWallTime; + } + const cutoff = new Date(instant); + if (Number.isNaN(cutoff.valueOf())) throw new Error(`Invalid edition date: ${editionDate}`); + return cutoff.toISOString(); +} + +function findChangelogEntry(candidate: string, nextTag: string): string | null { + const headingPattern = `^## ${nextTag.replaceAll(".", "\\.")}$`; + const matches = run( + "git", + ["grep", "-n", "-E", headingPattern, candidate, "--", ":(glob)docs/changelog/*.mdx"], + { allowFailure: true }, + ) + .trim() + .split("\n") + .filter(Boolean); + if (matches.length === 0) return null; + if (matches.length !== 1) { + throw new Error( + `Expected exactly one direct changelog entry for ${nextTag}, found ${matches.length}`, + ); + } + return matches[0]; +} + function stablePlanHash(planWithoutHash: Omit): string { return createHash("sha256") .update(JSON.stringify(planWithoutHash, null, 2)) @@ -171,8 +294,7 @@ function main(): void { const repoRoot = run("git", ["rev-parse", "--show-toplevel"]).trim(); process.chdir(repoRoot); - const status = run("git", ["status", "--short"]); - if (status.trim()) { + if (run("git", ["status", "--short"]).trim()) { throw new Error("Release planning requires a clean worktree"); } @@ -185,51 +307,107 @@ function main(): void { } run("git", ["fetch", "origin", "main", "--tags", "--force"]); - const semverTags = readRemoteSemverTags(); - if (semverTags.length === 0) { - throw new Error("No remote semver tags found"); - } + if (semverTags.length === 0) throw new Error("No remote semver tags found"); const previousTag = semverTags[0]; const nextTag = bumpTag(previousTag, options.bump); const followingTag = bumpTag(nextTag, "patch"); - if (readRemoteTag(nextTag)) { - throw new Error(`Remote tag already exists: ${nextTag}`); + if (readRemoteTag(nextTag)) throw new Error(`Remote tag already exists: ${nextTag}`); + + const originMainAtPlanning = run("git", ["rev-parse", "origin/main"]).trim(); + let authorization: ReleaseAuthorization = { type: "maintainer-confirmation" }; + let candidateCommit = originMainAtPlanning; + if (options.scheduledEdition) { + const cutoffAt = zonedCutoff(options.scheduledEdition); + const candidateRecordedAt = options.candidateRecordedAt as string; + if (Date.parse(candidateRecordedAt) > Date.parse(cutoffAt)) { + throw new Error( + `Candidate source time ${candidateRecordedAt} is after edition cutoff ${cutoffAt}`, + ); + } + candidateCommit = options.candidateSha as string; + authorization = { + type: "scheduled-workflow", + repository: "NVIDIA/NemoClaw", + plannerWorkflow: ".github/workflows/release-edition-close.yaml", + editionDate: options.scheduledEdition, + cutoffAt, + candidateSource: { + type: "github-actions-push-run", + workflow: ".github/workflows/post-merge-agent-review.yaml", + runId: options.candidateRunId as string, + recordedAt: candidateRecordedAt, + }, + }; } - const originMainCommit = run("git", ["rev-parse", "origin/main"]).trim(); - const originMainHeadline = run("git", ["log", "--oneline", "-1", "origin/main"]).trim(); + run("git", ["cat-file", "-e", `${candidateCommit}^{commit}`]); + try { + run("git", ["merge-base", "--is-ancestor", candidateCommit, "origin/main"]); + } catch { + throw new Error(`Release candidate ${candidateCommit} is not reachable from origin/main`); + } + try { + run("git", ["merge-base", "--is-ancestor", previousTag, candidateCommit]); + } catch { + throw new Error(`${previousTag} is not an ancestor of release candidate ${candidateCommit}`); + } + const untaggedCommitCount = Number( + run("git", ["rev-list", "--count", `${previousTag}..${candidateCommit}`]).trim(), + ); + const status = untaggedCommitCount === 0 ? "no-changes" : "ready"; + const changelogEntry = status === "ready" ? findChangelogEntry(candidateCommit, nextTag) : null; + if (status === "ready" && changelogEntry === null) { + throw new Error( + `Candidate ${candidateCommit} is missing exactly one direct changelog entry for ${nextTag}`, + ); + } + + const candidateHeadline = run("git", ["log", "--oneline", "-1", candidateCommit]).trim(); const output = path.resolve( options.output ?? path.join(repoRoot, "..", `nemoclaw-release-${nextTag}`, "plan.json"), ); const planPath = output; - + const operations = + status === "no-changes" + ? [ + "record that the edition contains no commits after the latest semver tag", + "skip tag creation", + ] + : [ + `create signed annotated ${nextTag} tag at ${candidateCommit}`, + `push ${nextTag}`, + "invoke release-latest-tag directly after an automated token push", + `have release-latest-tag carry open ${nextTag} items forward to ${followingTag}`, + `have release-latest-tag delete released ${nextTag} label after carry-forward succeeds`, + "draft release notes from live compare data", + ]; const planWithoutHash: Omit = { - schemaVersion: 1, + schemaVersion: 2, mode: "tag-only", + status, + authorization, previousTag, nextTag, bump: options.bump, originRemote, - originMainCommit, - originMainHeadline, + originMainAtPlanning, + candidateCommit, + candidateHeadline, + untaggedCommitCount, + changelogEntry, compareRange: `${previousTag}...${nextTag}`, latestBefore: readRemoteTag("latest"), lkgBefore: readRemoteTag("lkg"), createdAt: new Date().toISOString(), planPath, - confirmationPhrase: `CONFIRM RELEASE ${nextTag} ${originMainCommit}`, - operations: [ - `create signed annotated ${nextTag} tag at ${originMainCommit}`, - `push ${nextTag}`, - "wait for release-latest-tag workflow to move latest", - `have release-latest-tag workflow carry open ${nextTag} items forward to ${followingTag}`, - `have release-latest-tag workflow delete released ${nextTag} label after carry-forward succeeds`, - "draft release notes from live compare data", - ], + confirmationPhrase: `CONFIRM RELEASE ${nextTag} ${candidateCommit}`, + operations, forbiddenOperations: [ - "push latest from the agent", + "consult E2E state as tag authorization", + "advance the candidate after the edition cutoff", + "push latest from the tag-cut script", "push or move lkg", "move existing remote semver tags", "delete tags", @@ -248,9 +426,14 @@ function main(): void { console.log(`Release plan written: ${planPath}`); console.log(`Plan hash: ${plan.planHash}`); + console.log(`Edition status: ${status}`); console.log(`Previous tag: ${previousTag}`); console.log(`Next tag: ${nextTag}`); - console.log(`Target commit: ${originMainHeadline}`); + console.log(`Planning-time origin/main: ${originMainAtPlanning}`); + console.log(`Frozen candidate: ${candidateHeadline}`); + if (authorization.type === "scheduled-workflow") { + console.log(`Edition cutoff: ${authorization.cutoffAt}`); + } console.log("Confirmation phrase:"); console.log(plan.confirmationPhrase); } diff --git a/scripts/release-wait-latest.sh b/scripts/release-wait-latest.sh index 4ecd682c93c..2dad240ab7c 100755 --- a/scripts/release-wait-latest.sh +++ b/scripts/release-wait-latest.sh @@ -45,14 +45,14 @@ fail() { [[ -n "$PLAN_PATH" ]] || fail "--plan is required" [[ -f "$PLAN_PATH" ]] || fail "Plan file not found: $PLAN_PATH" -node -e 'const fs=require("fs"); const data=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); const semver=/^v\d+\.\d+\.\d+$/; const sha=/^[0-9a-f]{40}$/; const hash=/^[0-9a-f]{64}$/; if (data.schemaVersion !== 1) throw new Error("schemaVersion must be 1"); if (data.mode !== "tag-only") throw new Error("mode must be tag-only"); if (!semver.test(data.previousTag)) throw new Error("previousTag must be semver"); if (!semver.test(data.nextTag)) throw new Error("nextTag must be semver"); if (!sha.test(data.originMainCommit)) throw new Error("originMainCommit must be a full SHA"); if (!hash.test(data.planHash)) throw new Error("planHash must be a sha256 hex string");' "$PLAN_PATH" +node -e 'const fs=require("fs"); const data=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); const semver=/^v\d+\.\d+\.\d+$/; const sha=/^[0-9a-f]{40}$/; const hash=/^[0-9a-f]{64}$/; if (data.schemaVersion !== 2) throw new Error("schemaVersion must be 2"); if (data.mode !== "tag-only") throw new Error("mode must be tag-only"); if (data.status !== "ready") throw new Error("latest verification requires a ready plan"); if (!semver.test(data.previousTag)) throw new Error("previousTag must be semver"); if (!semver.test(data.nextTag)) throw new Error("nextTag must be semver"); if (!sha.test(data.candidateCommit)) throw new Error("candidateCommit must be a full SHA"); if (!hash.test(data.planHash)) throw new Error("planHash must be a sha256 hex string");' "$PLAN_PATH" json_field() { node -e 'const fs=require("fs"); const data=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); const path=process.argv[2].split("."); let value=data; for (const key of path) value=value?.[key]; if (value == null) process.exit(1); process.stdout.write(String(value));' "$PLAN_PATH" "$1" } tag="$(json_field nextTag)" -target="$(json_field originMainCommit)" +target="$(json_field candidateCommit)" plan_hash="$(json_field planHash)" lkg_before="$(node -e 'const fs=require("fs"); const data=JSON.parse(fs.readFileSync(process.argv[1], "utf8")); process.stdout.write(data.lkgBefore?.peeledSha || data.lkgBefore?.objectSha || "")' "$PLAN_PATH")" diff --git a/test/deterministic-smoke-workflow.test.ts b/test/deterministic-smoke-workflow.test.ts new file mode 100644 index 00000000000..b061530557f --- /dev/null +++ b/test/deterministic-smoke-workflow.test.ts @@ -0,0 +1,46 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { readFileSync } from "node:fs"; +import { describe, expect, it } from "vitest"; + +import { type CompositeAction, readYaml, type WorkflowJob } from "./helpers/e2e-workflow-contract"; + +type Workflow = { + jobs: Record; +}; + +const buildTypecheck = readYaml(".github/actions/ci-build-typecheck/action.yaml"); +const pr = readYaml(".github/workflows/pr.yaml"); +const main = readYaml(".github/workflows/main.yaml"); +const packageJson = JSON.parse(readFileSync("package.json", "utf8")) as { + scripts: Record; +}; + +function requiredStep(stepName: string) { + const step = buildTypecheck.runs.steps.find((candidate) => candidate.name === stepName); + expect(step, `Missing build-typecheck step: ${stepName}`).toBeDefined(); + return step!; +} + +describe("deterministic smoke workflow floor", () => { + // source-shape-contract: security -- Post-merge live E2E must retain one hermetic built-artifact smoke floor inside the required aggregate + it("keeps the deterministic smoke floor in the required build and typecheck action", () => { + const smoke = requiredStep("Run deterministic smoke floor"); + const packageContracts = requiredStep("Verify compiled package contracts"); + const smokeIndex = buildTypecheck.runs.steps.indexOf(smoke); + const packageContractIndex = buildTypecheck.runs.steps.indexOf(packageContracts); + + expect(smoke.run).toBe("npm run test:smoke"); + expect(packageJson.scripts["test:smoke"]).toBe( + "npm run clean:cli && npm --prefix nemoclaw run clean && npm run build:cli && npm --prefix nemoclaw run build && vitest run --project package-contract test/package-contract/deterministic-smoke.test.ts", + ); + expect(packageJson.scripts["test:smoke"]).not.toMatch( + /live-e2e|workflow run|brev|docker|podman/iu, + ); + expect(smokeIndex).toBeLessThan(packageContractIndex); + expect(packageContracts.run).toBe("npx vitest run --project package-contract"); + expect(pr.jobs.checks.needs).toContain("build-typecheck"); + expect(main.jobs.checks.needs).toContain("build-typecheck"); + }); +}); diff --git a/test/maintainer-e2e-skill.test.ts b/test/maintainer-e2e-skill.test.ts index 3335391bc4e..6cd9d9c6916 100644 --- a/test/maintainer-e2e-skill.test.ts +++ b/test/maintainer-e2e-skill.test.ts @@ -196,11 +196,22 @@ describe("nemoclaw-maintainer-e2e workflow routing", () => { expect(skill).toContain("include_staging_brev_launchable=false"); expect(skill).toContain("Run the full E2E suite"); expect(skill).toContain("include_staging_brev_launchable=true"); - expect(skill).toContain("deploy pre-release full E2E"); - expect(skill).toContain("run pre-tag full E2E"); + expect(skill).toContain("diagnose post-merge E2E"); + expect(skill).toContain("Main-push inspection"); + expect(skill).toContain("--event push --branch main"); + expect(skill).toContain("More than one run is expected"); + expect(skill).toContain( + "may rerun failed jobs from a non-superseded main-push run up to two times", + ); + expect(skill).toContain("Do not silently substitute full mode"); + expect(skill).toContain("Dispatch full mode only for an explicit full"); + expect(skill).toContain( + "keep one agent session active from 4:00 PM through the 8:00 AM handoff", + ); + expect(skill).toContain("immediately choose the next actionable item"); + expect(skill).toContain("Continue across the 4:00 AM tag"); expect(skill).toContain("run release-candidate E2E"); expect(skill).toContain("must not authorize the Brev Launchable path"); - expect(skill).toContain("Pre-tag evidence still requires the full `workflow_dispatch` mode"); expect(skill).toContain( "an authorized environment reviewer must approve it before qualification starts", ); @@ -210,7 +221,7 @@ describe("nemoclaw-maintainer-e2e workflow routing", () => { expect(skill).not.toMatch(/variable (?:set|delete) NEMOCLAW_BREV_LAUNCHABLE_E2E_ENABLED/u); }); - it("binds dispatch, evidence, invalidation, and release handoff to one SHA (#7487)", () => { + it("binds dispatch and advisory diagnostic evidence to one SHA (#7487)", () => { expect(skill).toContain("git rev-parse origin/main"); expect(skill).toContain("correlation_id=${CORRELATION_ID}"); expect(skill).toContain("head_sha"); @@ -219,8 +230,8 @@ describe("nemoclaw-maintainer-e2e workflow routing", () => { expect(skill).toContain("cleanup.json"); expect(skill).toContain("dispatch.json"); expect(skill).toContain("validate-full-e2e-evidence.mts"); - expect(skill).toContain("provisional release evidence"); - expect(skill).toContain("If the release candidate SHA changes"); - expect(skill).toContain("nemoclaw-maintainer-cut-release-tag"); + expect(skill).toContain("advisory E2E evidence"); + expect(skill).toContain("Bind every result to the tested SHA"); + expect(skill).toContain("Never treat success as tag authorization"); }); }); diff --git a/test/maintainer-skills-policy.test.ts b/test/maintainer-skills-policy.test.ts index 61329218d4b..baeab2d12dd 100644 --- a/test/maintainer-skills-policy.test.ts +++ b/test/maintainer-skills-policy.test.ts @@ -132,24 +132,22 @@ describe("maintainer skills follow canonical workflow policy", () => { const priorities = read(".agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md"); const policy = read(".agents/skills/nemoclaw-maintainer-policies/references/release-train.md"); - expect(evening).toContain("automatically carry stragglers to the next patch"); - expect(evening).toContain("retire the released label"); + expect(evening).toContain("move to the next patch label"); + expect(evening).toContain("released-label retirement state"); expect(release).toContain("release-latest-tag"); - expect(release).toContain("signed annotated semver tag"); - expect(release).toContain("GitHub-Verified"); - expect(release).toContain("same tag object"); + expect(release).toContain("signed annotated tag object"); + expect(release).toContain("verifies the exact signed annotated tag object through GitHub"); expect(release).toContain("--preflight-only"); - expect(release).toContain("OpenPGP, SSH, or X.509 signer"); - expect(release).toContain("Do not run the retirement script directly"); - expect(release).toContain('--event push --commit "$RELEASE_SHA"'); - expect(release).toContain("Expected exactly one release-latest-tag push run"); + expect(release).toContain("dedicated SSH private signing key"); + expect(release).toContain("Do not retag, move `latest`, retire labels directly"); + expect(release).toContain("directly calls `release-latest-tag.yaml`"); expect(morning).toContain("post-tag housekeeping was interrupted"); - expect(priorities).toContain("Move open items to the next patch label"); - expect(priorities).toContain("delete the released label"); - expect(policy).toContain("automatically move every open straggler to the next patch label"); - expect(policy).toContain("delete the released version label"); - expect(policy).toContain("never renamed or reused"); - expect(policy).toContain("shared release-label coordination queue"); + expect(priorities).toContain("move its label to the next patch"); + expect(priorities).toContain("Delete the released label"); + expect(policy).toContain("moves every open straggler"); + expect(policy).toContain("deletes the released label"); + expect(policy).toContain("Never rename, recreate, or reuse them"); + expect(policy).toContain("shared release-label queue"); expect(fs.existsSync(path.join(root, "scripts/retire-release-label.mts"))).toBe(true); }); @@ -167,10 +165,8 @@ describe("maintainer skills follow canonical workflow policy", () => { quality_rules: { post_merge_untagged_release_labeling_allowed: boolean }; }; - expect(policy).toContain("After a PR merges to `main`"); - expect(policy).toContain("ahead of the latest release tag"); - expect(policy).toContain("only across the untagged interval"); - expect(policy).toContain("Tags and commit ancestry are the only durable"); + expect(policy).toContain("trusted post-merge workflow labels untagged merges"); + expect(policy).toContain("Tags and commit ancestry are the durable"); expect(policy).not.toContain("earliest containing release"); expect(policy).not.toContain("seven-day retention window"); expect(projectWorkflow).toContain("On open PRs"); @@ -185,69 +181,38 @@ describe("maintainer skills follow canonical workflow policy", () => { expect(taxonomy.quality_rules.post_merge_untagged_release_labeling_allowed).toBe(true); }); - it("requires E2E evidence for the release candidate commit or itemized maintainer exceptions", () => { + it("freezes the 4 PM candidate and keeps E2E advisory through the 4 AM cut", () => { const dailyFlow = read(".agents/skills/nemoclaw-maintainer-policies/references/daily-flow.md"); const evening = read(".agents/skills/nemoclaw-maintainer-evening/SKILL.md"); const priorities = read(".agents/skills/nemoclaw-maintainer-day/PR-REVIEW-PRIORITIES.md"); const release = read(".agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md"); const policy = read(".agents/skills/nemoclaw-maintainer-policies/references/release-train.md"); - expect(policy).toContain("full `origin/main` commit SHA"); - expect(policy).toContain("`.github/workflows/e2e.yaml` is the sole source of truth"); - expect(policy).toContain("Do not maintain a separate release-gating test list"); - expect(policy).toContain("at least one completed, successful execution"); - expect(policy).toContain("Successful evidence may accumulate across rerun attempts"); - expect(policy).toContain("Evidence from another workflow run does not satisfy the ledger"); - expect(policy).toContain("Require every declared `RELEASE_E2E_ACTIVATION_PATH`"); - expect(policy).toContain("A missing path is a preflight failure"); - expect(release).toContain("Each job that declares `RELEASE_E2E_ACTIVATION_PATH`"); - expect(release).toContain("A missing activation path is a preflight failure"); - expect(policy).toContain("each expanded matrix execution as a separate ledger entry"); - expect(policy).toContain("matrix `id`"); - expect(policy).toContain("A later failure does not erase an earlier successful execution"); - expect(policy).toContain( - "Skipped, unexecuted, queued, in-progress, cancelled, and failing results do not count as successful evidence", - ); - expect(policy).toContain("itemized maintainer exception"); - expect(policy).toContain("If the candidate SHA changes"); - expect(policy).toContain("This does not freeze `main` or prevent merges"); - expect(policy).toContain("Require one completed, successful full workflow run"); - expect(policy).toContain("discard the ledger and its exceptions"); - expect(policy).toContain("selector inputs"); - expect(release).toContain('"dispatchJson"'); - expect(release).toContain("the number of tests with successful evidence"); - expect(release).toContain("successful run or job URL and attempt"); - expect(release).toContain("npm run release:e2e-evidence"); - expect(release).toContain("filter=all"); - expect(release).toContain("actions/runs/$RUN_ID/artifacts"); - expect(release).toContain("sort_by(.created_at)"); - expect(release).not.toContain("RECEIPT_ATTEMPT"); - expect(release).toContain("rerun preflight and the full E2E workflow"); - expect(release).toContain("Immediately before asking, refresh `origin/main` once"); - const evidenceSummary = release.indexOf("Before showing the confirmation prompt"); - const confirmationPrompt = release.indexOf( - "Ask the maintainer to paste this phrase", - evidenceSummary, - ); - expect(evidenceSummary).toBeGreaterThanOrEqual(0); - expect(evidenceSummary).toBeLessThan(confirmationPrompt); - expect(evening).toContain( - "Each missing or skipped execution in that successful run requires its own itemized maintainer exception", - ); - expect(evening).toContain( - "Missing or invalid Launchable E2E evidence in that successful run requires a separate", - ); - expect(evening).toContain("Tag the confirmed release commit with `vX.Y.Z`"); - expect(evening).not.toContain("tag `main`"); - expect(dailyFlow).toContain("capture the candidate SHA and review every E2E test"); - expect(dailyFlow).toContain( - "`head_sha` and all associated evidence to match the candidate SHA", - ); - expect(dailyFlow).toContain("invalidate the prior run and evidence"); - expect(priorities).toContain("Record the release SHA and required E2E evidence"); + expect(policy).toContain("8:00 AM–4:00 PM"); + expect(policy).toContain("latest `post-merge-agent-review.yaml` push run"); + expect(policy).toContain("A merge after the cutoff belongs to the next edition"); + expect(policy).toContain("selects every workflow E2E"); + expect(policy).toContain("Tag the frozen candidate regardless of E2E state"); + expect(policy).toContain("Do not create an E2E waiver ledger"); + expect(policy).toContain("E2E never enters the tag authorization"); + expect(policy).toContain("Assign one overnight agent"); + expect(policy).toContain("Do not stop the loop when the 4:00 AM tag is cut"); + expect(policy).toContain("The deterministic merge floor remains fail-closed"); + expect(policy).toContain("The `build-typecheck` lane runs `npm run test:smoke`"); + expect(policy).toContain("the required deterministic `checks` aggregate"); + expect(release).toContain("Never regenerate or advance the frozen candidate"); + expect(release).toContain("E2E is advisory"); + expect(release).toContain("Assign one agent to the complete 4:00 PM–8:00 AM loop"); + expect(release).toContain("every immutable E2E run triggered by an edition `main` push"); + expect(release).toContain("Do not merge during the freeze"); + expect(evening).toContain("Do not merge fixes during the freeze"); + expect(evening).toContain("Assign one agent to remain active through the 8:00 AM handoff"); + expect(evening).toContain("E2E does not authorize or block the 4:00 AM tag"); + expect(dailyFlow).toContain("tag the frozen candidate at 4 AM regardless of its state"); + expect(priorities).toContain("Tag the frozen candidate regardless of E2E state"); }); - it("requires full-mode exact Brev Launchable evidence before release confirmation (#7487)", () => { + it("keeps full-mode E2E receipts available for advisory overnight diagnosis (#7487)", () => { const e2e = read(".agents/skills/nemoclaw-maintainer-e2e/SKILL.md"); const evening = read(".agents/skills/nemoclaw-maintainer-evening/SKILL.md"); const release = read(".agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md"); @@ -259,48 +224,16 @@ describe("maintainer skills follow canonical workflow policy", () => { expect(e2e).toContain("launchable-e2e.json"); expect(e2e).toContain("cleanup.json"); expect(e2e).toContain("dispatch.json"); - expect(e2e).toContain("If the release candidate SHA changes"); - expect(e2e).toContain("jobs?filter=all&per_page=100"); - expect(e2e).toContain("Reuse `run-$RUN_ID.json` and `jobs-$RUN_ID.json`"); - expect(release).toContain("reuse `run-$RUN_ID.json` and `jobs-$RUN_ID.json`"); - expect(release).toContain("load `nemoclaw-maintainer-e2e` and dispatch one full run"); - expect(release).toContain("Treat a skipped job as missing evidence"); - expect(release).toContain("include_staging_brev_launchable=true"); - expect(release).toContain("cleanup evidence that reports the qualified workspace as `ABSENT`"); - expect(release).toContain( - "a separate itemized maintainer exception for each missing or skipped execution", - ); - expect(release).toContain( - "a separate itemized maintainer exception for missing or invalid exact Brev Launchable E2E evidence", - ); - expect(release).toContain("when accepted full-mode exact Brev evidence exists"); - expect( - release.indexOf("load `nemoclaw-maintainer-e2e` and dispatch one full run"), - ).toBeLessThan(release.indexOf("Ask the maintainer to paste this phrase")); - expect(evening).toContain("load `nemoclaw-maintainer-e2e`"); - expect(evening).toContain( - "Run full mode unless one existing full run for the candidate SHA contains complete workflow E2E", - ); - expect(release).toContain( - "Run full mode unless one existing full run for the candidate SHA contains complete workflow E2E", - ); - expect(policy).toContain("A failed workflow run cannot supply the release ledger"); - expect(release).toContain("Reject a failed workflow run before presenting the ledger"); - expect(evening).not.toContain("readiness variable"); - expect(policy).toContain("Require one completed, successful full workflow run"); - expect(policy).toContain( - "Run `nemoclaw-maintainer-e2e` in full mode when the ledger lacks complete evidence", - ); - expect(policy).toContain("including `Exact staging Brev Launchable`"); - expect(policy).toContain("cleanup receipt"); - expect(policy).toContain("trusted dispatch receipt"); - expect(policy).toContain( - "Each missing or skipped execution in the accepted successful workflow run", - ); - expect(policy).toContain( - "Missing or invalid exact Brev Launchable E2E evidence in the accepted successful workflow run", - ); - expect(policy).toContain("No release-note-only delta exception is currently defined"); + expect(e2e).toContain("Bind every result to the tested SHA"); + expect(e2e).not.toContain("Release Coverage Dispatch Group"); + expect(e2e).not.toContain("manifest inputs"); + expect(release).toContain("Load `nemoclaw-maintainer-e2e`"); + expect(release).toContain("Classify failures"); + expect(release).toContain("Never inspect E2E state to authorize"); + expect(evening).toContain("Load `nemoclaw-maintainer-e2e`"); + expect(policy).toContain("all edition main-push results"); + expect(policy).toContain("selective reruns"); + expect(policy).toContain("Do not fail, wait, or branch on E2E state"); expect(skillsGuide).toContain("`nemoclaw-maintainer-e2e`"); }); @@ -329,18 +262,14 @@ describe("maintainer skills follow canonical workflow policy", () => { expect(updateDocs).toContain("Stop before PR creation"); expect(createPr).toContain('--label "area: docs"'); expect(createPr).not.toContain('--label "documentation"'); - expect(evening.indexOf("/nemoclaw-contributor-update-docs for ")).toBeLessThan( - evening.indexOf("Load `cut-release-tag`"), - ); - expect(evening).toContain("contains the exact `## ` heading"); - expect(release).toContain("git grep -n '^## vX\\.Y\\.Z$'"); - expect(release).toContain("Unless Step 1 records an explicit waiver"); - expect(release).toContain("show the recorded waiver reason"); - expect(release).toContain("A conventional Release Notes page or post-tag Announcement draft"); + expect(evening).toContain("Run `/nemoclaw-contributor-update-docs for ` early enough"); + expect(evening).toContain("exact `## ` heading"); + expect(release).toContain("git grep -n -E '^## vX\\.Y\\.Z$'"); + expect(release).toContain("No changelog waiver exists"); expect(releaseNotes).toContain("does not replace or create that canonical entry"); expect(policy).toContain("Run `/nemoclaw-contributor-update-docs for vX.Y.Z`"); - expect(policy).toContain("The pre-tag release-note docs PR must create or update"); - expect(priorities).toContain("the pre-tag changelog PR contains"); + expect(policy).toContain("No changelog waiver exists"); + expect(priorities).toContain("require the dated changelog entry"); expect(skillsGuide).toContain( "update their owning documentation under current repository policy", ); @@ -348,12 +277,12 @@ describe("maintainer skills follow canonical workflow policy", () => { expect(docsAgents).toContain("CONTRIBUTING.md#updating-the-changelog"); expect(docsAgents).not.toContain("Every pre-tag release-note docs PR must create or update"); expect(docsContributing).toContain("Create the planned release entry in the pre-tag"); - expect(policy).toContain("If any merge lands after `release:plan`, generate a fresh plan"); + expect(policy).toContain("A merge after the cutoff belongs to the next edition"); expect(releaseNotes).toContain( - "Keep the candidate SHA, E2E failure classifications, rerun ledger, and waiver rationale out of the public Announcement", + "Keep candidate internals, agent-review diagnostics, E2E classifications, rerun details, and failure rationale out of the public Announcement", ); expect(releaseNotes).toContain( - "Never include the candidate SHA, internal E2E failure classifications, rerun details, or waiver rationale in the public Announcement", + "Never include candidate internals, agent-review diagnostics, internal E2E classifications, rerun details, or failure rationale in the public Announcement", ); }); diff --git a/test/package-contract/deterministic-smoke.test.ts b/test/package-contract/deterministic-smoke.test.ts new file mode 100644 index 00000000000..a6f852aee62 --- /dev/null +++ b/test/package-contract/deterministic-smoke.test.ts @@ -0,0 +1,86 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { spawnSync } from "node:child_process"; +import { rmSync } from "node:fs"; +import path from "node:path"; +import { afterEach, describe, expect, it } from "vitest"; + +import { createPackageFixture } from "./helpers/package-fixture"; + +const REPOSITORY_ROOT = path.join(import.meta.dirname, "..", ".."); +const CLI_ENTRYPOINT = path.join(REPOSITORY_ROOT, "bin", "nemoclaw.js"); +const fixtureRoots: string[] = []; + +afterEach(() => { + fixtureRoots + .splice(0) + .forEach((fixtureRoot) => rmSync(fixtureRoot, { force: true, recursive: true })); +}); + +function runCli(...args: string[]): ReturnType { + const fixtureHome = createPackageFixture({ + prefix: "nemoclaw-deterministic-smoke-home-", + entries: [], + }); + fixtureRoots.push(fixtureHome); + return spawnSync(process.execPath, [CLI_ENTRYPOINT, ...args], { + cwd: REPOSITORY_ROOT, + encoding: "utf8", + env: { + ...process.env, + HOME: fixtureHome, + NEMOCLAW_NON_INTERACTIVE: "1", + NO_COLOR: "1", + }, + timeout: 30_000, + }); +} + +describe("deterministic PR smoke floor", () => { + it("starts the compiled CLI without credentials, containers, or external services", () => { + const version = runCli("--version"); + const help = runCli("--help"); + + expect(version.error).toBeUndefined(); + expect(version.signal).toBeNull(); + expect(version.status, String(version.stderr)).toBe(0); + expect(version.stdout).toMatch(/^nemoclaw v/u); + expect(help.error).toBeUndefined(); + expect(help.signal).toBeNull(); + expect(help.status, String(help.stderr)).toBe(0); + expect(help.stdout).toContain("Getting Started:"); + expect(help.stdout).toContain("nemoclaw onboard"); + }); + + it("packs the compiled CLI and plugin entrypoints from reviewed package metadata", { + timeout: 30_000, + }, () => { + const packageRoot = createPackageFixture({ + prefix: "nemoclaw-deterministic-smoke-package-", + entries: [ + "bin", + "dist", + "nemoclaw/dist", + "nemoclaw/openclaw.plugin.json", + "nemoclaw/package.json", + ], + }); + fixtureRoots.push(packageRoot); + const packed = spawnSync("npm", ["pack", "--dry-run", "--json", "--ignore-scripts"], { + cwd: packageRoot, + encoding: "utf8", + timeout: 30_000, + }); + + expect(packed.error).toBeUndefined(); + expect(packed.signal).toBeNull(); + expect(packed.status, packed.stderr).toBe(0); + const report = JSON.parse(packed.stdout) as Array<{ files?: Array<{ path?: string }> }>; + const files = new Set((report[0]?.files ?? []).map((entry) => entry.path)); + expect(files.has("bin/nemoclaw.js")).toBe(true); + expect(files.has("dist/nemoclaw.js")).toBe(true); + expect(files.has("nemoclaw/dist/index.js")).toBe(true); + expect(files.has("nemoclaw/openclaw.plugin.json")).toBe(true); + }); +}); diff --git a/test/release-e2e-evidence.test.ts b/test/release-e2e-evidence.test.ts deleted file mode 100644 index 8a12482c1d8..00000000000 --- a/test/release-e2e-evidence.test.ts +++ /dev/null @@ -1,298 +0,0 @@ -// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -// SPDX-License-Identifier: Apache-2.0 - -import { describe, expect, it } from "vitest"; - -import { - buildReleaseE2eLedger, - buildReleaseE2ePreflight, - type ReleaseE2eExecution, - type ReleaseE2ePreflight, - type ReleaseE2eRunEvidence, -} from "../.agents/skills/nemoclaw-maintainer-cut-release-tag/scripts/release-e2e-evidence.mts"; - -const candidateSha = "a".repeat(40); - -function preflight( - input: { candidatePathExists?: (candidateSha: string, candidatePath: string) => boolean } = {}, -) { - return buildReleaseE2ePreflight({ - candidateSha, - candidatePathExists: input.candidatePathExists ?? (() => true), - }); -} - -function runEvidence( - plan: ReleaseE2ePreflight, - group: ReleaseE2eExecution["group"], - options: { - attempt?: number; - conclusion?: (execution: ReleaseE2eExecution) => string; - only?: (execution: ReleaseE2eExecution) => boolean; - sha?: string; - status?: (execution: ReleaseE2eExecution) => string; - } = {}, -): ReleaseE2eRunEvidence { - const attempt = options.attempt ?? 1; - const runId = 1001; - const executions = plan.executions.filter( - (execution) => execution.group === group && (options.only?.(execution) ?? true), - ); - const selectors: string[] = []; - return { - dispatch: { - allowJetsonRunnerQueue: false, - candidateSha, - emptySelectors: true, - eventName: "workflow_dispatch", - includeStagingBrevLaunchable: plan.dispatches.completeRun.includeStagingBrevLaunchable, - jobs: selectors.join(","), - kind: "nemoclaw-e2e-dispatch-v1", - targets: "", - workflowRunAttempt: attempt, - workflowRunId: String(runId), - }, - jobs: { - jobs: executions.map((execution, index) => ({ - conclusion: options.conclusion?.(execution) ?? "success", - html_url: `https://github.com/NVIDIA/NemoClaw/actions/runs/${runId}/job/${index + 1}`, - name: execution.expectedName, - run_attempt: attempt, - run_id: runId, - status: options.status?.(execution) ?? "completed", - })), - }, - run: { - event: "workflow_dispatch", - head_branch: "main", - id: runId, - path: ".github/workflows/e2e.yaml", - status: "completed", - conclusion: "success", - run_attempt: attempt, - head_sha: options.sha ?? candidateSha, - html_url: `https://github.com/NVIDIA/NemoClaw/actions/runs/${runId}`, - }, - }; -} - -describe("release E2E evidence", () => { - it("derives one complete release E2E run from the workflow", () => { - const plan = preflight(); - - expect(plan.dispatches.completeRun).toEqual({ - includeStagingBrevLaunchable: true, - jobs: "", - mode: "full", - targets: "", - }); - expect(plan.launchableE2eJobId).toBe("staging-brev-launchable"); - expect(plan.exceptionsRequired).toEqual([]); - }); - - it("rejects a missing activation path for a default E2E", () => { - expect(() => - preflight({ - candidatePathExists: (_sha, candidatePath) => - candidatePath !== "ci/protected-managed-image-multiarch-activation-v1.json", - }), - ).toThrow( - "candidate commit is missing required E2E activation path ci/protected-managed-image-multiarch-activation-v1.json for managed-image-multiarch-startup", - ); - }); - - it("rejects an in-progress workflow with successful execution jobs", () => { - const plan = preflight(); - const evidence = runEvidence(plan, "default"); - (evidence.run as Record).status = "in_progress"; - - expect(() => buildReleaseE2eLedger(plan, [evidence])).toThrow( - 'runs[0].run.status must equal "completed"', - ); - }); - - it("rejects a failed workflow with successful execution jobs", () => { - const plan = preflight(); - const evidence = runEvidence(plan, "default"); - (evidence.run as Record).conclusion = "failure"; - - expect(() => buildReleaseE2eLedger(plan, [evidence])).toThrow( - 'runs[0].run.conclusion must equal "success"', - ); - }); - - it("fails when the candidate commit cannot be inspected for activation paths", () => { - expect(() => - buildReleaseE2ePreflight({ - candidateSha: "0".repeat(40), - }), - ).toThrow("could not inspect release E2E activation path"); - }); - - it("keeps every static and dynamic matrix row as a distinct execution", () => { - const plan = preflight(); - const ids = plan.executions.map((execution) => execution.id); - - expect(ids.filter((id) => id.startsWith("mcp-bridge-dev["))).toHaveLength(3); - expect(ids.filter((id) => id.startsWith("hermes-gpu-startup["))).toHaveLength(3); - expect(ids.filter((id) => id.startsWith("openshell-gateway-upgrade["))).toHaveLength(5); - expect(ids).toContain("live[id=ubuntu-repo-cloud-openclaw]"); - expect(ids).toContain("shared-e2e[id=vllm-docker-storage]"); - expect(new Set(ids).size).toBe(ids.length); - }); - - it("accumulates successful evidence across rerun attempts", () => { - const plan = preflight(); - const firstDefaultRun = runEvidence(plan, "default"); - const laterFailure = runEvidence(plan, "default", { - attempt: 2, - conclusion: () => "failure", - only: (execution) => execution.id === "snapshot-commands", - }); - (firstDefaultRun.run as Record).run_attempt = 2; - (firstDefaultRun.jobs as { jobs: unknown[] }).jobs.push( - ...(laterFailure.jobs as { jobs: unknown[] }).jobs, - ); - const ledger = buildReleaseE2eLedger(plan, [firstDefaultRun]); - - expect(ledger.successfulCount).toBe(ledger.requiredCount); - expect(ledger.missingCount).toBe(0); - expect(ledger.entries.find((entry) => entry.id === "snapshot-commands")).toMatchObject({ - attempts: [ - { attempt: 2, conclusion: "failure" }, - { attempt: 1, conclusion: "success" }, - ], - successfulEvidence: { attempt: 1 }, - status: "successful", - }); - }); - - it("rejects evidence assembled from multiple workflow runs", () => { - const plan = preflight(); - expect(() => - buildReleaseE2eLedger(plan, [runEvidence(plan, "default"), runEvidence(plan, "default")]), - ).toThrow("release E2E evidence requires exactly one workflow run, received 2"); - }); - - it("requires the full run to include staging Brev Launchable", () => { - const plan = preflight(); - const evidence = runEvidence(plan, "default"); - (evidence.dispatch as Record).includeStagingBrevLaunchable = false; - expect(() => buildReleaseE2eLedger(plan, [evidence])).toThrow( - "runs[0].dispatch.includeStagingBrevLaunchable must equal true", - ); - }); - - it("reports a failed matrix row without collapsing its successful siblings", () => { - const plan = preflight(); - const failedId = 'hermes-gpu-startup[scenario="fallback"]'; - const ledger = buildReleaseE2eLedger(plan, [ - runEvidence(plan, "default", { - conclusion: (execution) => (execution.id === failedId ? "failure" : "success"), - }), - ]); - - expect(ledger.missingCount).toBe(1); - expect(ledger.entries.find((entry) => entry.id === failedId)).toMatchObject({ - status: "missing", - attempts: [{ conclusion: "failure" }], - }); - expect( - ledger.entries.find( - (entry) => entry.id === 'hermes-gpu-startup[scenario="compatibility-only"]', - ), - ).toMatchObject({ status: "successful" }); - }); - - it("does not count an in-progress execution as successful", () => { - const plan = preflight(); - const pendingId = "snapshot-commands"; - const ledger = buildReleaseE2eLedger(plan, [ - runEvidence(plan, "default", { - status: (execution) => (execution.id === pendingId ? "in_progress" : "completed"), - }), - ]); - - expect(ledger.missingCount).toBe(1); - expect(ledger.entries.find((entry) => entry.id === pendingId)).toMatchObject({ - attempts: [{ conclusion: "success", status: "in_progress" }], - status: "missing", - }); - }); - - it("does not treat a skipped execution as successful evidence", () => { - const plan = preflight(); - const skippedId = "snapshot-commands"; - const ledger = buildReleaseE2eLedger(plan, [ - runEvidence(plan, "default", { - conclusion: (execution) => (execution.id === skippedId ? "skipped" : "success"), - }), - ]); - - expect(ledger.entries.find((entry) => entry.id === skippedId)).toMatchObject({ - attempts: [{ conclusion: "skipped", status: "completed" }], - status: "missing", - }); - }); - - it("ignores job evidence from another workflow run", () => { - const plan = preflight(); - const evidence = runEvidence(plan, "default"); - const ignoredId = plan.executions.find((execution) => execution.group === "default")!.id; - const jobs = evidence.jobs as { jobs: Array> }; - jobs.jobs[0]!.run_id = 999; - - const ledger = buildReleaseE2eLedger(plan, [evidence]); - - expect(ledger.entries.find((entry) => entry.id === ignoredId)).toMatchObject({ - attempts: [], - status: "missing", - }); - }); - - it("ignores job evidence newer than the enclosing workflow run attempt", () => { - const plan = preflight(); - const evidence = runEvidence(plan, "default"); - const ignoredId = plan.executions.find((execution) => execution.group === "default")!.id; - const jobs = evidence.jobs as { jobs: Array> }; - jobs.jobs[0]!.run_attempt = 2; - - const ledger = buildReleaseE2eLedger(plan, [evidence]); - - expect(ledger.entries.find((entry) => entry.id === ignoredId)).toMatchObject({ - attempts: [], - status: "missing", - }); - }); - - it("rejects malformed job evidence", () => { - const plan = preflight(); - const malformed = runEvidence(plan, "default"); - const jobs = malformed.jobs as { jobs: Array> }; - delete jobs.jobs[0]!.name; - - expect(() => buildReleaseE2eLedger(plan, [malformed])).toThrow( - "runs[0].job.name must be a non-empty string", - ); - }); - - it("rejects a selective dispatch receipt that claims empty selectors", () => { - const plan = preflight(); - const selective = runEvidence(plan, "default"); - const dispatch = selective.dispatch as Record; - dispatch.jobs = "snapshot-commands"; - dispatch.emptySelectors = true; - - expect(() => buildReleaseE2eLedger(plan, [selective])).toThrow( - 'runs[0].dispatch.jobs must equal ""', - ); - }); - - it("rejects evidence from another candidate SHA", () => { - const plan = preflight(); - - expect(() => - buildReleaseE2eLedger(plan, [runEvidence(plan, "default", { sha: "b".repeat(40) })]), - ).toThrow("runs[0].run.head_sha must equal"); - }); -}); diff --git a/test/release-edition-workflow.test.ts b/test/release-edition-workflow.test.ts new file mode 100644 index 00000000000..b416cdacb25 --- /dev/null +++ b/test/release-edition-workflow.test.ts @@ -0,0 +1,341 @@ +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 + +import { afterEach, describe, expect, it, vi } from "vitest"; + +import { readYaml, type WorkflowJob } from "./helpers/e2e-workflow-contract"; + +const AsyncFunction = Object.getPrototypeOf(async () => undefined).constructor as new ( + ...parameters: string[] +) => (...args: unknown[]) => Promise; + +type Workflow = { + on?: Record; + permissions?: Record; + concurrency?: Record; + jobs: Record>; +}; + +const close = readYaml(".github/workflows/release-edition-close.yaml"); +const cut = readYaml(".github/workflows/release-edition-cut.yaml"); +const postMerge = readYaml(".github/workflows/post-merge-agent-review.yaml"); +const advisor = readYaml(".github/workflows/pr-review-advisor.yaml"); + +afterEach(() => { + vi.useRealTimers(); + vi.unstubAllEnvs(); +}); + +async function evaluateScheduleGate( + workflow: Workflow, + now: string, + schedule: string, + eventName = "schedule", +): Promise { + vi.useFakeTimers(); + vi.setSystemTime(new Date(now)); + vi.stubEnv("EVENT_NAME", eventName); + vi.stubEnv("SCHEDULE", schedule); + const step = workflow.jobs["schedule-gate"].steps?.find( + (candidate) => candidate.name === "Select the active Los Angeles schedule", + ); + const script = String(step?.with?.script ?? ""); + const setOutput = vi.fn(); + await new AsyncFunction("core", script)({ info: vi.fn(), setOutput }); + return String(setOutput.mock.calls.find(([name]) => name === "active")?.[1]); +} + +describe("release edition workflows", () => { + // source-shape-contract: security -- The plan artifact freezes a trusted candidate without tag or label write authority + it("closes the edition after the exact Los Angeles cutoff with read-only permissions", () => { + expect(close.on?.schedule).toEqual([{ cron: "17 23 * * *" }, { cron: "17 0 * * *" }]); + expect(close.permissions).toEqual({ actions: "read", contents: "read" }); + expect(close.concurrency).toEqual({ + group: "release-edition-close", + "cancel-in-progress": false, + }); + const steps = close.jobs["freeze-candidate"].steps ?? []; + expect(close.jobs["freeze-candidate"].needs).toBe("schedule-gate"); + expect(close.jobs["freeze-candidate"].if).toContain( + "needs.schedule-gate.outputs.active == 'true'", + ); + const checkout = steps.find((step) => step.name === "Check out trusted main history"); + const freeze = steps.find((step) => step.name === "Freeze release candidate"); + const upload = steps.find((step) => step.name === "Upload immutable edition plan"); + + expect(checkout?.with).toMatchObject({ + ref: "main", + "fetch-depth": 0, + "persist-credentials": false, + }); + expect(freeze?.run).toContain('--scheduled-edition "$EDITION_DATE"'); + expect(freeze?.run).toContain('--candidate-sha "$CANDIDATE_SHA"'); + expect(freeze?.run).toContain('--candidate-run-id "$CANDIDATE_RUN_ID"'); + expect(freeze?.run).toContain('--candidate-recorded-at "$CANDIDATE_RECORDED_AT"'); + expect(freeze?.run).not.toMatch(/e2e/iu); + expect(upload?.with).toMatchObject({ + name: "release-edition-plan-${{ steps.edition.outputs.date }}", + "if-no-files-found": "error", + "retention-days": 3, + }); + }); + + // source-shape-contract: security -- Paired UTC schedules must select only the Los Angeles close boundary in both PST and PDT + it("admits only the UTC close schedule that is 4:17 PM in Los Angeles", async () => { + expect(await evaluateScheduleGate(close, "2026-07-15T23:20:00Z", "17 23 * * *")).toBe("true"); + expect(await evaluateScheduleGate(close, "2026-07-16T00:20:00Z", "17 0 * * *")).toBe("false"); + expect(await evaluateScheduleGate(close, "2026-01-15T00:20:00Z", "17 0 * * *")).toBe("true"); + expect(await evaluateScheduleGate(close, "2026-01-15T23:20:00Z", "17 23 * * *")).toBe("false"); + }); + + it("selects the latest GitHub-recorded main push at or before the cutoff", async () => { + const step = close.jobs["freeze-candidate"].steps?.find( + (candidate) => candidate.name === "Resolve latest server-recorded main push", + ); + const script = String(step?.with?.script ?? ""); + vi.stubEnv("CUTOFF_AT", "2026-08-06T23:00:00.000Z"); + const lateSha = "c".repeat(40); + const selectedSha = "b".repeat(40); + const listWorkflowRuns = vi.fn().mockResolvedValue({ + data: { + workflow_runs: [ + { + id: 44, + event: "push", + head_branch: "main", + head_sha: lateSha, + created_at: "2026-08-06T23:00:01Z", + html_url: "late", + }, + { + id: 43, + event: "push", + head_branch: "main", + head_sha: selectedSha, + created_at: "2026-08-06T22:59:59Z", + html_url: "selected", + }, + { + id: 42, + event: "push", + head_branch: "main", + head_sha: "a".repeat(40), + created_at: "2026-08-06T22:00:00Z", + html_url: "older", + }, + ], + }, + }); + const setOutput = vi.fn(); + const github = { rest: { actions: { listWorkflowRuns } } }; + const context = { repo: { owner: "NVIDIA", repo: "NemoClaw" } }; + const core = { info: vi.fn(), setOutput }; + + await new AsyncFunction("github", "context", "core", script as string)(github, context, core); + + expect(listWorkflowRuns).toHaveBeenCalledWith( + expect.objectContaining({ + workflow_id: "post-merge-agent-review.yaml", + branch: "main", + event: "push", + }), + ); + expect(setOutput).toHaveBeenCalledWith("sha", selectedSha); + expect(setOutput).toHaveBeenCalledWith("run_id", "43"); + expect(setOutput).toHaveBeenCalledWith("recorded_at", "2026-08-06T22:59:59Z"); + }); + + // source-shape-contract: security -- Only the canonical schedule may publish, while manual dispatch remains signing preflight + it("cuts at 4 AM without E2E input and delegates exact-object promotion", () => { + expect(cut.on?.schedule).toEqual([{ cron: "17 11 * * *" }, { cron: "17 12 * * *" }]); + expect(cut.on?.workflow_dispatch?.inputs?.edition_date).toMatchObject({ + required: true, + type: "string", + }); + expect(cut.permissions).toEqual({}); + const cutJob = cut.jobs.cut; + const steps = cutJob.steps ?? []; + const configure = steps.find((step) => step.name === "Configure dedicated release signer"); + const execute = steps.find( + (step) => step.name === "Cut scheduled tag or preflight manual dispatch", + ); + const cleanup = steps.find((step) => step.name === "Remove release signer"); + + expect(cutJob.environment).toBe("release-tag"); + expect(cutJob.needs).toBe("schedule-gate"); + expect(cutJob.if).toContain("needs.schedule-gate.outputs.active == 'true'"); + expect(cutJob.permissions).toEqual({ actions: "read", contents: "write" }); + expect(configure?.env).toMatchObject({ + SIGNING_KEY: "${{ secrets.NEMOCLAW_RELEASE_TAG_SIGNING_KEY }}", + SIGNER_EMAIL: "${{ vars.NEMOCLAW_RELEASE_TAG_SIGNER_EMAIL }}", + SIGNER_NAME: "${{ vars.NEMOCLAW_RELEASE_TAG_SIGNER_NAME }}", + }); + expect(execute?.run).toContain('scripts/release-cut-tag.sh --plan "$PLAN_PATH" --scheduled'); + expect(execute?.run).toContain( + 'scripts/release-cut-tag.sh --plan "$PLAN_PATH" --preflight-only', + ); + expect(execute?.run).not.toMatch(/e2e/iu); + expect(cleanup?.if).toBe("${{ always() }}"); + expect(cut.jobs.promote).toMatchObject({ + if: "${{ needs.cut.outputs.status == 'tagged' }}", + uses: "./.github/workflows/release-latest-tag.yaml", + permissions: { + contents: "write", + issues: "write", + "pull-requests": "write", + }, + with: { tag: "${{ needs.cut.outputs.tag }}" }, + }); + const handoff = cut.jobs["verify-and-handoff"]; + expect(handoff.if).toContain("needs.cut.outputs.status == 'no-changes'"); + const verify = handoff.steps?.find( + (step) => step.name === "Verify latest and create handoff data", + ); + expect(verify?.if).toBe("${{ needs.cut.outputs.status == 'tagged' }}"); + const upload = handoff.steps?.find((step) => step.name === "Upload verified handoff"); + expect(upload?.with?.name).toContain("no-changes-"); + }); + + // source-shape-contract: security -- Scheduled release authority must admit only the DST-correct 4:17 AM trigger while keeping manual preflight reachable + it("admits the correct UTC cut schedule in PST, PDT, and manual mode", async () => { + expect(await evaluateScheduleGate(cut, "2026-07-15T11:20:00Z", "17 11 * * *")).toBe("true"); + expect(await evaluateScheduleGate(cut, "2026-07-15T12:20:00Z", "17 12 * * *")).toBe("false"); + expect(await evaluateScheduleGate(cut, "2026-01-15T12:20:00Z", "17 12 * * *")).toBe("true"); + expect(await evaluateScheduleGate(cut, "2026-01-15T11:20:00Z", "17 11 * * *")).toBe("false"); + expect(await evaluateScheduleGate(cut, "2026-07-15T11:20:00Z", "", "workflow_dispatch")).toBe( + "true", + ); + }); + + it("selects only a successful scheduled close artifact with the exact edition name", async () => { + const step = cut.jobs.cut.steps?.find( + (candidate) => candidate.name === "Find successful edition-close artifact", + ); + const script = String(step?.with?.script ?? ""); + vi.stubEnv("ARTIFACT_NAME", "release-edition-plan-2026-08-05"); + const listWorkflowRuns = vi.fn().mockResolvedValue({ + data: { + workflow_runs: [ + { id: 40, conclusion: "success", head_branch: "feature", html_url: "wrong-branch" }, + { id: 41, conclusion: "failure", head_branch: "main", html_url: "failed" }, + { id: 42, conclusion: "success", head_branch: "main", html_url: "trusted" }, + ], + }, + }); + const listWorkflowRunArtifacts = vi.fn(); + const paginate = vi.fn(async (method: unknown, args: Record) => { + expect(method).toBe(listWorkflowRunArtifacts); + expect(args.run_id).toBe(42); + return [ + { + name: "release-edition-plan-2026-08-05", + expired: false, + }, + ]; + }); + const setOutput = vi.fn(); + const github = { + paginate, + rest: { actions: { listWorkflowRuns, listWorkflowRunArtifacts } }, + }; + const context = { repo: { owner: "NVIDIA", repo: "NemoClaw" } }; + const core = { info: vi.fn(), setOutput }; + + await new AsyncFunction("github", "context", "core", script as string)(github, context, core); + + expect(listWorkflowRuns).toHaveBeenCalledWith( + expect.objectContaining({ + workflow_id: "release-edition-close.yaml", + event: "schedule", + status: "completed", + branch: "main", + }), + ); + expect(setOutput).toHaveBeenCalledWith("run_id", "42"); + }); + + it("rejects ambiguous scheduled close artifacts for one edition", async () => { + const step = cut.jobs.cut.steps?.find( + (candidate) => candidate.name === "Find successful edition-close artifact", + ); + const script = String(step?.with?.script ?? ""); + vi.stubEnv("ARTIFACT_NAME", "release-edition-plan-2026-08-05"); + const listWorkflowRuns = vi.fn().mockResolvedValue({ + data: { + workflow_runs: [ + { id: 42, conclusion: "success", head_branch: "main", html_url: "first" }, + { id: 43, conclusion: "success", head_branch: "main", html_url: "second" }, + ], + }, + }); + const listWorkflowRunArtifacts = vi.fn(); + const paginate = vi + .fn() + .mockResolvedValue([{ name: "release-edition-plan-2026-08-05", expired: false }]); + const github = { + paginate, + rest: { actions: { listWorkflowRuns, listWorkflowRunArtifacts } }, + }; + const context = { repo: { owner: "NVIDIA", repo: "NemoClaw" } }; + const core = { info: vi.fn(), setOutput: vi.fn() }; + + let errorMessage = ""; + try { + await new AsyncFunction("github", "context", "core", script)(github, context, core); + } catch (error) { + errorMessage = error instanceof Error ? error.message : String(error); + } + expect(errorMessage).toContain("Expected exactly one successful scheduled source"); + expect(paginate).toHaveBeenCalledTimes(2); + }); +}); + +describe("post-merge agent review", () => { + // source-shape-contract: security -- The dispatcher has no code or secret access and binds analysis to the immutable push range + it("dispatches one asynchronous advisor run for the exact main SHA range", async () => { + expect(postMerge.on?.push?.branches).toEqual(["main"]); + expect(postMerge.permissions).toEqual({ actions: "write" }); + const job = postMerge.jobs["dispatch-exact-main-delta"]; + expect(job.steps).toHaveLength(1); + const step = job.steps?.[0]; + expect(step?.uses).toBe("actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3"); + const script = String(step?.with?.script ?? ""); + const base = "a".repeat(40); + const head = "b".repeat(40); + vi.stubEnv("BASE_SHA", base); + vi.stubEnv("HEAD_SHA", head); + const createWorkflowDispatch = vi.fn().mockResolvedValue(undefined); + const write = vi.fn().mockResolvedValue(undefined); + const summary = { + addHeading: vi.fn(), + addRaw: vi.fn(), + write, + }; + summary.addHeading.mockReturnValue(summary); + summary.addRaw.mockReturnValue(summary); + const github = { rest: { actions: { createWorkflowDispatch } } }; + const context = { + payload: { repository: { default_branch: "main" } }, + repo: { owner: "NVIDIA", repo: "NemoClaw" }, + }; + + await new AsyncFunction("github", "context", "core", script)(github, context, { + summary, + }); + + expect(createWorkflowDispatch).toHaveBeenCalledWith({ + owner: "NVIDIA", + repo: "NemoClaw", + workflow_id: "pr-review-advisor.yaml", + ref: "main", + inputs: { base_ref: base, head_ref: head, run_analysis: "true" }, + }); + expect(write).toHaveBeenCalled(); + }); + + // source-shape-contract: security -- Exact immutable-head concurrency preserves every asynchronous post-merge review turn + it("keeps independent merge reviews from cancelling each other", () => { + expect(advisor.concurrency?.group).toContain("${{ inputs.head_ref || '' }}"); + expect(advisor.concurrency?.["cancel-in-progress"]).toBe(true); + }); +}); diff --git a/test/release-latest-tag-workflow.test.ts b/test/release-latest-tag-workflow.test.ts index 8e28efc86dd..faf3a51994d 100644 --- a/test/release-latest-tag-workflow.test.ts +++ b/test/release-latest-tag-workflow.test.ts @@ -10,6 +10,7 @@ const AsyncFunction = Object.getPrototypeOf(async () => undefined).constructor a ) => (...args: unknown[]) => Promise; type ReleaseLatestWorkflow = { + on?: Record; jobs: Record; }; @@ -67,10 +68,15 @@ afterEach(() => { describe("release latest tag workflow", () => { // source-shape-contract: security -- Exact verified-object output wiring prevents latest promotion from bypassing GitHub signature verification it("binds latest promotion to the exact GitHub-verified tag object", () => { + expect(workflow.on?.workflow_call?.inputs?.tag).toMatchObject({ + required: true, + type: "string", + }); expect(verifyStep?.uses).toBe("actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3"); expect(moveStep?.env?.EXPECTED_RELEASE_TAG_OBJECT).toBe( "${{ steps.verify-release-tag.outputs.tag_object_sha }}", ); + expect(moveStep?.env?.RELEASE_TAG).toBe("${{ inputs.tag || github.ref_name }}"); }); it("accepts a GitHub-verified signed tag and emits its exact object SHA", async () => { diff --git a/test/release-latest-tag.test.ts b/test/release-latest-tag.test.ts index 48730441306..031a40a7dd2 100644 --- a/test/release-latest-tag.test.ts +++ b/test/release-latest-tag.test.ts @@ -110,6 +110,42 @@ function commit(fixture: Fixture, text: string): string { return run(fixture.work, ["git", "rev-parse", "HEAD"]).trim(); } +function commitRelease(fixture: Fixture, text: string): string { + const changelogDir = path.join(fixture.work, "docs", "changelog"); + fs.mkdirSync(changelogDir, { recursive: true }); + fs.writeFileSync( + path.join(changelogDir, "2026-08-05.mdx"), + "\n\n## v0.0.2\n\nRelease test.\n", + ); + fs.appendFileSync(path.join(fixture.work, "file.txt"), `${text}\n`); + run(fixture.work, ["git", "add", "file.txt", "docs/changelog/2026-08-05.mdx"]); + run(fixture.work, ["git", "commit", "-m", text]); + run(fixture.work, ["git", "push", "origin", "main"]); + return run(fixture.work, ["git", "rev-parse", "HEAD"]).trim(); +} + +function commitAt(fixture: Fixture, text: string, isoDate: string): string { + fs.appendFileSync(path.join(fixture.work, "file.txt"), `${text}\n`); + run(fixture.work, ["git", "add", "."]); + const commitResult = runScript(fixture.work, ["git", "commit", "-m", text], { + GIT_AUTHOR_DATE: isoDate, + GIT_COMMITTER_DATE: isoDate, + }); + expect(commitResult.status).toBe(0); + run(fixture.work, ["git", "push", "origin", "main"]); + return run(fixture.work, ["git", "rev-parse", "HEAD"]).trim(); +} + +function commitReleaseAt(fixture: Fixture, text: string, isoDate: string): string { + const changelogDir = path.join(fixture.work, "docs", "changelog"); + fs.mkdirSync(changelogDir, { recursive: true }); + fs.writeFileSync( + path.join(changelogDir, "2026-08-06.mdx"), + "\n\n## v0.0.2\n\nFrozen release test.\n", + ); + return commitAt(fixture, text, isoDate); +} + function pushTag(fixture: Fixture, tag: string, target = "HEAD", annotated = true): void { const args = annotated ? ["git", "-c", "tag.gpgSign=false", "tag", "-a", tag, target, "-m", tag] @@ -215,15 +251,18 @@ function createPlan( { NEMOCLAW_RELEASE_ALLOW_NON_CANONICAL: "1" }, ); - expect(result.status).toBe(0); + expect(result.status, String(result.stderr)).toBe(0); const plan = readJson(planPath); expect(plan.previousTag).toBe("v0.0.1"); expect(plan.nextTag).toBe("v0.0.2"); - expect(plan.originMainCommit).toBe(releaseCommit); + expect(plan.schemaVersion).toBe(2); + expect(plan.status).toBe("ready"); + expect(plan.authorization).toEqual({ type: "maintainer-confirmation" }); + expect(plan.candidateCommit).toBe(releaseCommit); expect(plan.operations).toContain(`create signed annotated v0.0.2 tag at ${releaseCommit}`); - const carryForward = "have release-latest-tag workflow carry open v0.0.2 items forward to v0.0.3"; + const carryForward = "have release-latest-tag carry open v0.0.2 items forward to v0.0.3"; const deleteReleased = - "have release-latest-tag workflow delete released v0.0.2 label after carry-forward succeeds"; + "have release-latest-tag delete released v0.0.2 label after carry-forward succeeds"; const carryForwardIndex = plan.operations.indexOf(carryForward); const deleteReleasedIndex = plan.operations.indexOf(deleteReleased); expect(carryForwardIndex).toBeGreaterThanOrEqual(0); @@ -458,7 +497,7 @@ describe("release-latest-tag.sh", () => { const fixture = createFixture(); pushTag(fixture, "lkg", fixture.firstCommit); pushTag(fixture, "v0.0.1", fixture.firstCommit); - const releaseCommit = commit(fixture, "planned release commit"); + const releaseCommit = commitRelease(fixture, "planned release commit"); const planPath = path.join(fixture.root, "release", "plan.json"); const { plan } = createPlan(fixture, planPath, releaseCommit); @@ -511,12 +550,229 @@ describe("release-latest-tag.sh", () => { lkgPeeledCommitBefore: fixture.firstCommit, lkgPeeledCommitAfter: fixture.firstCommit, }); + }, 30_000); + + it("freezes the GitHub-recorded main push before the Los Angeles cutoff", () => { + const fixture = createFixture(); + pushTag(fixture, "v0.0.1", fixture.firstCommit); + const frozenCommit = commitReleaseAt( + fixture, + "candidate before cutoff", + "2026-08-06T15:00:00-07:00", + ); + const lateCommit = commitAt(fixture, "late next-edition merge", "2026-08-06T16:30:00-07:00"); + const planPath = path.join(fixture.root, "release", "plan.json"); + + const planResult = runScript( + fixture.work, + [ + tsxPath, + planScriptPath, + "--scheduled-edition", + "2026-08-06", + "--candidate-sha", + frozenCommit, + "--candidate-run-id", + "43", + "--candidate-recorded-at", + "2026-08-06T22:59:59Z", + "--output", + planPath, + ], + { NEMOCLAW_RELEASE_ALLOW_NON_CANONICAL: "1" }, + ); + + expect(planResult.status).toBe(0); + const plan = readJson(planPath); + expect(plan).toMatchObject({ + status: "ready", + candidateCommit: frozenCommit, + originMainAtPlanning: lateCommit, + authorization: { + type: "scheduled-workflow", + editionDate: "2026-08-06", + cutoffAt: "2026-08-06T23:00:00.000Z", + candidateSource: { + type: "github-actions-push-run", + workflow: ".github/workflows/post-merge-agent-review.yaml", + runId: "43", + recordedAt: "2026-08-06T22:59:59.000Z", + }, + }, + }); + + const cutResult = runScript( + fixture.work, + ["bash", cutScriptPath, "--plan", planPath, "--scheduled"], + { + GITHUB_ACTIONS: "true", + GITHUB_EVENT_NAME: "schedule", + GITHUB_REPOSITORY: "NVIDIA/NemoClaw", + GITHUB_WORKFLOW_REF: + "NVIDIA/NemoClaw/.github/workflows/release-edition-cut.yaml@refs/heads/main", + }, + ); + + expect(cutResult.status).toBe(0); + expect(remoteCommit(fixture, "refs/tags/v0.0.2")).toBe(frozenCommit); + expect(remoteCommit(fixture, "refs/heads/main")).toBe(lateCommit); + }, 30_000); + + it("rejects a scheduled candidate recorded after the Los Angeles cutoff", () => { + const fixture = createFixture(); + pushTag(fixture, "v0.0.1", fixture.firstCommit); + const candidate = commitReleaseAt( + fixture, + "candidate before cutoff", + "2026-08-06T15:00:00-07:00", + ); + const planPath = path.join(fixture.root, "release", "plan.json"); + + const planResult = runScript( + fixture.work, + [ + tsxPath, + planScriptPath, + "--scheduled-edition", + "2026-08-06", + "--candidate-sha", + candidate, + "--candidate-run-id", + "43", + "--candidate-recorded-at", + "2026-08-06T23:00:01Z", + "--output", + planPath, + ], + { NEMOCLAW_RELEASE_ALLOW_NON_CANONICAL: "1" }, + ); + + expect(planResult.status).not.toBe(0); + expect(planResult.stderr).toContain( + "Candidate source time 2026-08-06T23:00:01.000Z is after edition cutoff 2026-08-06T23:00:00.000Z", + ); + expect(fs.existsSync(planPath)).toBe(false); + }); + + it("rejects scheduled authority outside the canonical schedule event", () => { + const fixture = createFixture(); + pushTag(fixture, "v0.0.1", fixture.firstCommit); + const candidate = commitReleaseAt( + fixture, + "candidate before cutoff", + "2026-08-06T15:00:00-07:00", + ); + const planPath = path.join(fixture.root, "release", "plan.json"); + expect( + runScript( + fixture.work, + [ + tsxPath, + planScriptPath, + "--scheduled-edition", + "2026-08-06", + "--candidate-sha", + candidate, + "--candidate-run-id", + "43", + "--candidate-recorded-at", + "2026-08-06T22:59:59Z", + "--output", + planPath, + ], + { NEMOCLAW_RELEASE_ALLOW_NON_CANONICAL: "1" }, + ).status, + ).toBe(0); + + const cutResult = runScript( + fixture.work, + ["bash", cutScriptPath, "--plan", planPath, "--scheduled"], + { + GITHUB_ACTIONS: "true", + GITHUB_EVENT_NAME: "workflow_dispatch", + GITHUB_REPOSITORY: "NVIDIA/NemoClaw", + GITHUB_WORKFLOW_REF: + "NVIDIA/NemoClaw/.github/workflows/release-edition-cut.yaml@refs/heads/main", + }, + ); + + expect(cutResult.status).not.toBe(0); + expect(cutResult.stderr).toContain("Scheduled cuts require a schedule event"); + expect(localTagObject(fixture, "v0.0.2")).toBe(""); + }); + + it("rejects a scheduled candidate that is not reachable from origin main", () => { + const fixture = createFixture(); + pushTag(fixture, "v0.0.1", fixture.firstCommit); + commit(fixture, "current main commit"); + run(fixture.work, ["git", "switch", "-c", "foreign", fixture.firstCommit]); + const foreignCandidate = commitRelease(fixture, "foreign candidate"); + const planPath = path.join(fixture.root, "release", "plan.json"); + + const result = runScript( + fixture.work, + [ + tsxPath, + planScriptPath, + "--scheduled-edition", + "2026-08-06", + "--candidate-sha", + foreignCandidate, + "--candidate-run-id", + "43", + "--candidate-recorded-at", + "2026-08-06T22:59:59Z", + "--output", + planPath, + ], + { NEMOCLAW_RELEASE_ALLOW_NON_CANONICAL: "1" }, + ); + + expect(result.status).not.toBe(0); + expect(result.stderr).toContain("is not reachable from origin/main"); + }); + + it("records a no-change edition without creating a tag", () => { + const fixture = createFixture(); + pushTag(fixture, "v0.0.1", fixture.firstCommit); + const planPath = path.join(fixture.root, "release", "plan.json"); + const planResult = runScript(fixture.work, [tsxPath, planScriptPath, "--output", planPath], { + NEMOCLAW_RELEASE_ALLOW_NON_CANONICAL: "1", + }); + expect(planResult.status).toBe(0); + const plan = readJson(planPath); + expect(plan).toMatchObject({ status: "no-changes", untaggedCommitCount: 0 }); + + const cutResult = cutFromPlan(fixture, planPath, plan.confirmationPhrase); + + expect(cutResult.status).toBe(0); + expect(localTagObject(fixture, "v0.0.2")).toBe(""); + expect(readJson(path.join(fixture.root, "release", "cut-result.json"))).toMatchObject({ + status: "no-changes", + tag: null, + targetCommit: fixture.firstCommit, + }); + }); + + it("fails closed when a non-empty candidate lacks its changelog entry", () => { + const fixture = createFixture(); + pushTag(fixture, "v0.0.1", fixture.firstCommit); + commit(fixture, "release without changelog"); + const planPath = path.join(fixture.root, "release", "plan.json"); + + const planResult = runScript(fixture.work, [tsxPath, planScriptPath, "--output", planPath], { + NEMOCLAW_RELEASE_ALLOW_NON_CANONICAL: "1", + }); + + expect(planResult.status).not.toBe(0); + expect(planResult.stderr).toContain("missing exactly one direct changelog entry for v0.0.2"); + expect(fs.existsSync(planPath)).toBe(false); }); it("rejects signing preflight when the configured signer is unavailable", () => { const fixture = createFixture(); pushTag(fixture, "v0.0.1", fixture.firstCommit); - const releaseCommit = commit(fixture, "planned release commit"); + const releaseCommit = commitRelease(fixture, "planned release commit"); const planPath = path.join(fixture.root, "release", "plan.json"); createPlan(fixture, planPath, releaseCommit); run(fixture.work, ["git", "config", "gpg.format", "openpgp"]); @@ -536,12 +792,12 @@ describe("release-latest-tag.sh", () => { expect( run(fixture.work, ["git", "tag", "--list", "nemoclaw-release-signing-preflight-*"]), ).toBe(""); - }); + }, 30_000); it("rejects a distinct latest tag object even when it peels to the release commit", () => { const fixture = createFixture(); pushTag(fixture, "v0.0.1", fixture.firstCommit); - const releaseCommit = commit(fixture, "planned release commit"); + const releaseCommit = commitRelease(fixture, "planned release commit"); const planPath = path.join(fixture.root, "release", "plan.json"); const { plan } = createPlan(fixture, planPath, releaseCommit); expect(cutFromPlan(fixture, planPath, plan.confirmationPhrase).status).toBe(0); @@ -552,12 +808,12 @@ describe("release-latest-tag.sh", () => { expect(waitResult.status).not.toBe(0); expect(waitResult.stderr).toContain("latest tag object"); expect(waitResult.stderr).toContain("does not match v0.0.2 object"); - }); + }, 30_000); it("rejects a tampered release plan before cutting the tag", () => { const fixture = createFixture(); pushTag(fixture, "v0.0.1", fixture.firstCommit); - const releaseCommit = commit(fixture, "planned release commit"); + const releaseCommit = commitRelease(fixture, "planned release commit"); const planPath = path.join(fixture.root, "release", "plan.json"); const { plan } = createPlan(fixture, planPath, releaseCommit); const tampered = { ...plan, forbiddenOperations: [] }; @@ -567,13 +823,13 @@ describe("release-latest-tag.sh", () => { expect(cutResult.status).not.toBe(0); expect(cutResult.stderr).toContain("planHash mismatch"); - }); + }, 30_000); it("verifies unchanged lightweight lkg tags", () => { const fixture = createFixture(); pushTag(fixture, "lkg", fixture.firstCommit, false); pushTag(fixture, "v0.0.1", fixture.firstCommit); - const releaseCommit = commit(fixture, "planned release commit"); + const releaseCommit = commitRelease(fixture, "planned release commit"); const planPath = path.join(fixture.root, "release", "plan.json"); const { plan } = createPlan(fixture, planPath, releaseCommit); expect(plan.lkgBefore).toMatchObject({ @@ -593,12 +849,12 @@ describe("release-latest-tag.sh", () => { lkgPeeledCommitBefore: fixture.firstCommit, lkgPeeledCommitAfter: fixture.firstCommit, }); - }); + }, 30_000); it("detects lkg creation after a plan captured lkg as absent", () => { const fixture = createFixture(); pushTag(fixture, "v0.0.1", fixture.firstCommit); - const releaseCommit = commit(fixture, "planned release commit"); + const releaseCommit = commitRelease(fixture, "planned release commit"); const planPath = path.join(fixture.root, "release", "plan.json"); const { plan } = createPlan(fixture, planPath, releaseCommit); expect(plan.lkgBefore).toBeNull(); @@ -610,7 +866,7 @@ describe("release-latest-tag.sh", () => { expect(waitResult.status).not.toBe(0); expect(waitResult.stderr).toContain("lkg was created after the release plan was generated"); - }); + }, 30_000); it("extracts only squash-merge PR numbers from release notes compare commits", () => { const fixture = createFixture(); @@ -640,11 +896,12 @@ exit 2 planPath, `${JSON.stringify( { - schemaVersion: 1, + schemaVersion: 2, mode: "tag-only", + status: "ready", previousTag: "v0.0.1", nextTag: "v0.0.2", - originMainCommit: "0123456789abcdef0123456789abcdef01234567", + candidateCommit: "0123456789abcdef0123456789abcdef01234567", planHash: "a".repeat(64), }, null, @@ -711,11 +968,12 @@ exit 2 planPath, `${JSON.stringify( { - schemaVersion: 1, + schemaVersion: 2, mode: "tag-only", + status: "ready", previousTag: "v0.0.1", nextTag: "v0.0.2", - originMainCommit: "0123456789abcdef0123456789abcdef01234567", + candidateCommit: "0123456789abcdef0123456789abcdef01234567", planHash: "a".repeat(64), }, null,