Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .agents/skills/nemoclaw-maintainer-cut-release-tag/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,8 @@ The preflight derives every required execution and these dispatch groups from th
- `parallelExplicit`: explicit-only selectors that require neither the Launchable E2E job nor runner confirmation; and
- `conditional`: Jetson or another lane that must not queue until its authoritative runner inventory is confirmed online.

An explicit-only job may declare `RELEASE_E2E_ACTIVATION_PATH` in its workflow environment. The preflight includes that job and all of its expanded executions only when the exact relative path exists at the candidate SHA. When the path is absent, the job is a dormant lane: do not dispatch it and do not treat it as missing release evidence.

First feed applicable existing runs for the candidate SHA into the ledger.
Dispatch only groups that still lack green evidence.
When no applicable exact Brev evidence exists, load `nemoclaw-maintainer-e2e` and dispatch full mode for that SHA.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ const REPO_ROOT = path.join(path.dirname(fileURLToPath(import.meta.url)), "..",
const DEFAULT_WORKFLOW_PATH = path.join(REPO_ROOT, ".github", "workflows", "e2e.yaml");
const SHA_PATTERN = /^[a-f0-9]{40}$/u;
const SELECTOR_PATTERN = /^[A-Za-z0-9_-]+$/u;
const SAFE_REPO_PATH_PATTERN = /^(?!\/)(?!.*(?:^|\/)\.\.(?:\/|$))[^\\]+$/u;
const MATRIX_EXPRESSION_PATTERN = /\$\{\{\s*matrix\.([A-Za-z0-9_-]+)\s*\}\}/gu;

function record(value: unknown, label: string): JsonRecord {
Expand Down Expand Up @@ -271,8 +272,43 @@ function requiresConfirmedJetsonRunner(job: JsonRecord): boolean {
return typeof runsOn === "string" && runsOn.includes("inputs.allow_jetson_runner_queue");
}

function releaseActivationPath(job: JsonRecord, jobId: string): string | undefined {
const rawEnvironment = job.env;
if (rawEnvironment === undefined) return undefined;
const environment = record(rawEnvironment, `workflow.jobs.${jobId}.env`);
const activationPath = environment.RELEASE_E2E_ACTIVATION_PATH;
if (activationPath === undefined) return undefined;
if (
typeof activationPath !== "string" ||
activationPath.length === 0 ||
!SAFE_REPO_PATH_PATTERN.test(activationPath)
) {
throw new Error(
`${jobId}.env.RELEASE_E2E_ACTIVATION_PATH must be a nonempty relative repository path without backslashes or parent-directory segments`,
);
}
return activationPath;
}

function candidatePathExists(candidateSha: string, candidatePath: string): boolean {
try {
const output = execFileSync("git", ["ls-tree", "--name-only", candidateSha, "--", candidatePath], {
cwd: REPO_ROOT,
encoding: "utf8",
stdio: ["ignore", "pipe", "pipe"],
});
return output.trim() === candidatePath;
} catch (error) {
throw new Error(
`could not inspect release E2E activation path ${candidatePath} at candidate ${candidateSha}`,
{ cause: error },
);
}
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.

export function buildReleaseE2ePreflight(input: {
candidateSha: string;
candidatePathExists?: (candidateSha: string, candidatePath: string) => boolean;
jetsonRunnerOnline?: RunnerStatus;
plan?: E2eWorkflowPlan;
workflowPath?: string;
Expand All @@ -285,7 +321,15 @@ export function buildReleaseE2ePreflight(input: {
const inventory = readFreeStandingJobsInventory(workflowPath);
const plan = input.plan ?? buildE2eWorkflowPlan();
const explicitJobs = new Set(inventory.explicitOnlyJobs);
const launchableE2eJobs = inventory.explicitOnlyJobs.filter((jobId) =>
const pathExists = input.candidatePathExists ?? candidatePathExists;
const releaseExplicitJobs = inventory.explicitOnlyJobs.filter((jobId) => {
const activationPath = releaseActivationPath(
record(jobs[jobId], `workflow.jobs.${jobId}`),
jobId,
);
return activationPath === undefined || pathExists(input.candidateSha, activationPath);
});
const launchableE2eJobs = releaseExplicitJobs.filter((jobId) =>
isLaunchableE2eJob(record(jobs[jobId], `workflow.jobs.${jobId}`)),
);
if (launchableE2eJobs.length !== 1) {
Expand All @@ -294,12 +338,12 @@ export function buildReleaseE2ePreflight(input: {
);
}
const launchableE2eJobId = launchableE2eJobs[0]!;
const conditionalJobs = inventory.explicitOnlyJobs.filter(
const conditionalJobs = releaseExplicitJobs.filter(
(jobId) =>
jobId !== launchableE2eJobId &&
requiresConfirmedJetsonRunner(record(jobs[jobId], `workflow.jobs.${jobId}`)),
);
const parallelExplicitJobs = inventory.explicitOnlyJobs.filter(
const parallelExplicitJobs = releaseExplicitJobs.filter(
(jobId) => jobId !== launchableE2eJobId && !conditionalJobs.includes(jobId),
);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -54,13 +54,14 @@ Before asking for the release confirmation phrase, build and show an evidence le
- Preflight the candidate workflow, conditional runner readiness, and existing candidate evidence before dispatching new work.
- Dispatch independent default-suite and unconditional explicit-only work concurrently. Dispatch a conditional hardware lane only after its authoritative runner inventory confirms that it is online; otherwise record its required itemized exception without queueing it.
- Derive the denominator and dispatch selectors from the candidate workflow. Do not copy them into a second release test list.
- An explicit-only job that declares `RELEASE_E2E_ACTIVATION_PATH` enters the release denominator only when that exact relative path exists at the candidate SHA. Until then, treat it as a declared dormant lane: do not dispatch it and do not count it as missing evidence.
- For every accepted run, require the workflow-produced trusted dispatch receipt to bind the candidate SHA, run ID, attempt, and actual selector inputs. Derive default-suite or selective coverage only from that receipt, never from a release manifest claim.
- Run `nemoclaw-maintainer-e2e` in full mode if no applicable exact Brev Launchable evidence exists for the candidate SHA.
- For full-mode exact Brev evidence, require one workflow run for the candidate SHA that includes the default-enabled suite and a successful `Exact staging Brev Launchable` job.
- For that evidence, require the trusted dispatch receipt to bind the run and attempt to empty selectors and `include_staging_brev_launchable=true`.
- Require its Launchable E2E receipt to identify the candidate SHA in the repository and provision records.
- Require its cleanup receipt to identify the qualified workspace and report `ABSENT`.
- Every E2E test execution declared by the workflow must have at least one completed, successful execution for the candidate SHA. This includes tests that require explicit selection and every expanded matrix execution.
- Every release-eligible E2E test execution declared by the workflow must have at least one completed, successful execution for the candidate SHA. This includes tests that require explicit selection, including activation-gated jobs whose path exists at that SHA, and every expanded matrix execution.
- Treat each expanded matrix execution as a separate ledger entry. Use its matrix `id`, or all distinguishing matrix dimensions when no single ID exists, in the test identifier so results for distinct expansions are never collapsed under the parent job.
- Green evidence may accumulate across multiple workflow runs, selective runs, reruns, and attempts. A later failure does not erase an earlier successful execution for the same test and SHA.
- Skipped, unexecuted, queued, in-progress, cancelled, and failing results are not green evidence.
Expand Down
Loading
Loading