diff --git a/CHANGELOG.md b/CHANGELOG.md index 22abb3231..1aa8594be 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,7 @@ ## Unreleased - 검증된 active-orphan 워크플로 하나를 운영자가 호출할 수 있는 `operations:workflow-registry-disable` 경로를 추가한다. 저장소와 워크플로 ID를 `NOEMA_MAINTAINER_TOKEN_PATH` 위임 토큰 파일 읽기 전에 검사하고, 신선한 전체 레지스트리 감사·즉시 live refresh·프로세스 로컬 plan·보호된 main/워크플로 재검증·사후 전체 감사 봉투(`schema_version` 1, `PASS`/`FAIL`, `remaining_failure_codes`, `remaining_active_orphan_ids`)를 통과한 뒤에만 영수증을 유지한다. 성공 종료와 `post_audit_status: FAIL`은 해당 ID만 `disabled_manually`가 되었고 레지스트리는 아직 더럽을 수 있음을 뜻하므로, 운영자는 영수증의 `remaining_active_orphan_ids`로 다음 단일 호출을 이어간다. 배치 비활성화·자가 수리 워크플로·거버넌스 완화는 추가하지 않으며 호출 계약은 doctoring에 기록한다. - 읽기 전용 `operations:runner-assignment` audit를 추가해 exact workflow run/source head에 대한 runner assignment를 완전 pagination으로 진단하고, 신선한 unassigned queue는 bounded grace 이후 실패-폐쇄한다. 이 증빙은 runner assignment와 required Check/CI, formal review, merge, release, deployment authority를 분리하며 assigned runner 이후 workflow failure를 성공으로 승격하지 않는다. +- production `operations:runner-assignment` audit는 `NOEMA_MAINTAINER_TOKEN_PATH`의 owner-only capability file만 읽고, ambient `GH_TOKEN`만 있으면 실패-폐쇄한다. `gh` spawn/stderr 진단은 활성 토큰을 exact-match로 `[REDACTED]` 치환하며, 빈 secret에 대해서는 원문 진단을 보존한다. assignment authority는 양의 `runner_id` 또는 비어 있지 않은 `runner_name`만 인정하며 queued `started_at`은 assignment evidence가 아니다. 운영자는 `printf '%s'`로 capability file을 만들고(`echo`/`printf '%s\\n'`는 trailing newline 때문에 실패-폐쇄), Actions workflow-run/job read만 가진 짧은 토큰을 준비한 뒤 PASS를 required Check·formal review·merge 권한으로 해석하지 마십시오. - coordinated vulnerability disclosure 정책과 evidence-preserving vulnerability handling lifecycle, read-only private-vulnerability-reporting setting audit를 추가한다. 이 source 변경은 live private reporting 활성화·notification staffing·end-to-end advisory exercise·release/deployment authority를 증명하지 않는다. - 개발 의존성 체인의 transitive `nanoid` lockfile resolution을 `3.3.17`에서 `3.3.18`로 최소 갱신하여 GHSA-2v37-7h3g-55p8 / CVE-2026-67213 보안 게이트를 복구한다. PostCSS의 선언 범위 `^3.3.16`과 다른 package metadata는 변경하지 않으며 audit waiver·ignore·severity 완화 없이 `npm ci`/`npm audit --audit-level=high`가 exact head에서 재검증되도록 유지한다. - lockfile 재생성 도구 체인을 Node.js 24.19.0/npm 11.17.0으로 정확히 고정하고, `strict-allow-scripts=true` 아래 승인된 install-script identity만 실행하며 schema v3 exact-base lockfile change control로 package metadata drift를 실패-폐쇄한다. exact package before/after digest에 더해 top-level metadata digest와 대규모 package-set bulk evidence를 결합하며, 선행 `nanoid@3.3.18` 보안 수정과 explicit `npm ci --legacy-peer-deps=false --install-links=false` 계약을 보존한다. package-manager/toolchain·install-script authority·vulnerability audit·review/merge authority는 별도 증거 계층으로 유지한다. diff --git a/docs/doctoring/actions-runner-assignment-audit.md b/docs/doctoring/actions-runner-assignment-audit.md index 3920cc778..fa01a3329 100644 --- a/docs/doctoring/actions-runner-assignment-audit.md +++ b/docs/doctoring/actions-runner-assignment-audit.md @@ -13,12 +13,12 @@ The operator supplies an explicit bounded set of GitHub Actions workflow-run IDs Required invariants are: 1. the repository is exactly `ContextualWisdomLab/noema`; -2. `GH_TOKEN` exists only as transport authority for read-only API calls and is never retained in the report; +2. the production CLI reads its short-lived GitHub transport authority from an owner-only delegated token capability file named by `NOEMA_MAINTAINER_TOKEN_PATH`; the bearer value is never read from ambient `GH_TOKEN` and is never retained in the report; 3. the expected source head is one canonical lowercase 40-character commit SHA; 4. one to twenty unique positive run IDs are selected explicitly; 5. selected runs must be `pull_request` runs bound to that exact source head; 6. job pages are fully paginated with `per_page=100` and `filter=all`, bounded to at most 2,000 retained jobs; -7. runner assignment is observed only from job evidence such as `started_at`, a positive `runner_id`, or a non-empty `runner_name`; +7. runner assignment is observed only from a positive `runner_id` or a non-empty `runner_name`; `started_at` is not runner-assignment authority because GitHub may populate it while a job is still queued without runner identity; 8. a `waiting`, `pending`, or `requested` job remains non-passing `PENDING` because those states do not by themselves isolate runner allocation; 9. a queued job in a run where another job has already received a runner remains non-passing `PENDING`, because the queued job may be waiting on an explicit `jobs..needs` dependency rather than runner capacity; 10. the bounded grace may produce `runner_assignment_stalled` only when the workflow run itself remains `queued`, the job remains `queued`, and no job in that selected run has assignment evidence; @@ -26,7 +26,7 @@ Required invariants are: The default runner-allocation grace is five minutes and may be bounded by `NOEMA_ACTIONS_AUDIT_QUEUE_GRACE_MILLISECONDS`; the evaluator rejects values above thirty minutes rather than allowing a true isolated queue condition to remain indefinitely pending. -This classifier is intentionally conservative because the GitHub workflow-job REST representation does not expose a durable repository-consumable timestamp meaning “this job became eligible for runner allocation.” A workflow run's `created_at` is therefore not a trustworthy age for every downstream job. The evaluator uses run age only after the selected evidence isolates the top-level queued runner-allocation boundary described above. +This classifier is intentionally conservative because the GitHub workflow-job REST representation does not expose a durable repository-consumable timestamp meaning “this job became eligible for runner allocation.” A workflow run's `created_at` is therefore not a trustworthy age for every downstream job, and a job's `started_at` is not runner-assignment authority. The evaluator uses run age only after the selected evidence isolates the top-level queued runner-allocation boundary described above. ### Pre-run waits are not runner stalls @@ -36,10 +36,19 @@ Those states remain operationally non-passing, but they are not evidence that Gi ## Operator contract +The production command consumes an owner-only delegated token capability file. The path itself is non-secret; the token bytes must be materialized by an authorized caller and removed promptly after the audit. + Example: ```bash -export GH_TOKEN='' +umask 077 +token_dir="$(mktemp -d)" +token_path="$token_dir/runner-audit-token" +printf '%s' '' >"$token_path" +chmod 0600 "$token_path" +trap 'rm -rf "$token_dir"' EXIT + +export NOEMA_MAINTAINER_TOKEN_PATH="$token_path" export NOEMA_ACTIONS_AUDIT_REPOSITORY='ContextualWisdomLab/noema' export NOEMA_ACTIONS_AUDIT_HEAD_SHA='' export NOEMA_ACTIONS_AUDIT_RUN_IDS='31343034891,31343034896,31343034900' @@ -47,6 +56,12 @@ export NOEMA_ACTIONS_AUDIT_RUN_IDS='31343034891,31343034896,31343034900' npm run operations:runner-assignment ``` +Do not use `echo` or `printf '%s\n'` to write the capability file. Those commands append a newline, and the reader then fails closed because tokens must not contain control characters. Use `printf '%s'` exactly as shown, then remove the file in the `trap` so the bearer does not remain on disk. + +The short-lived token needs GitHub Actions **read** for workflow runs and workflow jobs on `ContextualWisdomLab/noema` only. It must not carry contents write, pull-request write, administration, workflow, or workflow-dispatch scopes. A broader token is the wrong credential for this diagnostic. + +An ambient `GH_TOKEN` is intentionally not a production credential source for this command. The descriptor-safe shared capability reader rejects symlinks, non-regular files, group/world-readable files, wrong-owner files where UID inspection is available, oversized content, invalid UTF-8, control-bearing tokens, and file identity/version changes during the bounded read. + The command writes the fixed report path: ```text @@ -55,7 +70,7 @@ artifacts/operations/actions-runner-assignment-audit.json The report records repository, expected head, selected run IDs, observation time, queue grace, deterministic checks/failures, and explicit false authority flags for required-check success, review, merge, release, and deployment. Temporary report bytes are created owner-only and atomically renamed onto the fixed report path. -`PASS` exits zero. `PENDING` and `FAIL` both exit nonzero. A malformed source identity, cross-repository request, missing credential, malformed API JSON, GitHub CLI failure, pagination-shape failure, excessive evidence, or head mismatch fails closed. +`PASS` exits zero. `PENDING` and `FAIL` both exit nonzero. A malformed source identity, cross-repository request, missing/unsafe capability file, malformed API JSON, GitHub CLI failure, pagination-shape failure, excessive evidence, or head mismatch fails closed. ## RCA interpretation @@ -69,9 +84,9 @@ This separation matters for issue #30 because historical Noema runs exhibited qu ## Security and privacy -The collector uses GitHub Actions REST **read** endpoints only. It does not rerun, cancel, dispatch, approve, merge, modify refs, or change settings. The report does not contain `GH_TOKEN`, repository secrets, workflow logs, source contents, personal data beyond ordinary GitHub workflow/job metadata needed for the operational decision, or model output. +The collector uses GitHub Actions REST **read** endpoints only. It does not rerun, cancel, dispatch, approve, merge, modify refs, or change settings. The report does not contain the delegated token, repository secrets, workflow logs, source contents, personal data beyond ordinary GitHub workflow/job metadata needed for the operational decision, or model output. -The `gh` child process also receives a purpose-built minimal environment rather than ambient process state: only `PATH`, the read-only `GH_TOKEN`, pinned `GH_HOST=github.com`, and `NO_COLOR=1` cross the process boundary. `GITHUB_TOKEN`, `NVIDIA_NIM_API_KEY`, Maintainer/Reviewer App private material, `HOME`, and ambient proxy variables are excluded. This prevents a read-only diagnostic subprocess from accidentally inheriting stronger publication/model credentials or redirecting credential-bearing requests through an unreviewed proxy path. +The production CLI reads bearer authority only through `NOEMA_MAINTAINER_TOKEN_PATH`. The `gh` child process then receives a purpose-built minimal environment: only `PATH`, the in-memory read-only `GH_TOKEN` required by the GitHub CLI, pinned `GH_HOST=github.com`, and `NO_COLOR=1` cross that child-process boundary. `GITHUB_TOKEN`, ambient `GH_TOKEN`, `NVIDIA_NIM_API_KEY`, Maintainer/Reviewer App private material, `HOME`, and ambient proxy variables are excluded. This prevents the read-only diagnostic subprocess from inheriting stronger publication/model credentials or redirecting credential-bearing requests through an unreviewed proxy path. The audit is diagnostic evidence. A passing assignment audit cannot satisfy branch protection, required checks, formal review, security scanning, release provenance, production deployment, or acquisition evidence. @@ -80,6 +95,8 @@ The audit is diagnostic evidence. A passing assignment audit cannot satisfy bran The repository-owned slice is acceptable when: - realistic tests reproduce an isolated runner stall, a fresh queue, deployment/environment waiting, downstream dependency waiting, assigned-but-failed jobs, head mismatch, malformed evidence, pagination, and bounded selection; +- queued `started_at` timestamps without runner identity remain unassigned, while a positive `runner_id` or a non-empty `runner_name` is assignment evidence; +- the production entrypoint succeeds with an owner-only delegated token capability file and fails closed when only ambient `GH_TOKEN` is present; - environment-protected and dependency-blocked jobs remain nonzero `PENDING` and are not mislabeled as runner-allocation stalls; - the pure evaluator and bounded source collector are GREEN; - the operator adapter performs only the two documented read families and fully paginates jobs; @@ -97,6 +114,8 @@ GitHub. (2026). *REST API endpoints for workflow jobs*. GitHub Docs. https://doc GitHub. (2026). *REST API endpoints for workflow runs*. GitHub Docs. https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2026-03-10 +GitHub. (2026). *Permissions required for fine-grained personal access tokens*. GitHub Docs. https://docs.github.com/en/rest/authentication/permissions-required-for-fine-grained-personal-access-tokens + GitHub. (2026). *Deployments and environments*. GitHub Docs. https://docs.github.com/en/actions/concepts/workflows-and-actions/deployment-environments GitHub. (2026). *Workflow syntax for GitHub Actions*. GitHub Docs. https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idneeds diff --git a/scripts/actions-runner-assignment-audit.mjs b/scripts/actions-runner-assignment-audit.mjs index 1be13757d..92c254a7a 100644 --- a/scripts/actions-runner-assignment-audit.mjs +++ b/scripts/actions-runner-assignment-audit.mjs @@ -21,6 +21,7 @@ import { collectRunnerAssignmentEvidence, parseSelectedRunIds, } from "./lib/actions-runner-assignment-source.mjs"; +import { readDelegatedGithubToken } from "./lib/delegated-github-token.mjs"; import { hasDuplicateJsonObjectKeys } from "./normalize-commercial-readiness-evidence.mjs"; const AUDITED_REPOSITORY = "ContextualWisdomLab/noema"; @@ -30,6 +31,7 @@ const GH_API_MAX_BUFFER_BYTES = 2 * 1024 * 1024; const REPORT_PATH = "artifacts/operations/actions-runner-assignment-audit.json"; const canonicalShaPattern = /^[0-9a-f]{40}$/; const fatalUtf8Decoder = new TextDecoder("utf-8", { fatal: true }); +const auditCredentialPresenceMarker = "delegated-capability-present"; const defaultGhRuntime = { spawn_sync: spawnSync, @@ -51,6 +53,25 @@ function boundedErrorText(value) { return text.replace(/[\u0000-\u001f\u007f]/g, " ").trim().slice(0, 1000); } +/** + * Replace every exact occurrence of an active secret with a fixed marker. + * + * An empty or non-string secret is left untouched. `String.prototype.split("")` + * would otherwise insert the marker between every character and leak a + * transformed diagnostic that is no longer the original failure text. + * + * @param {unknown} value Untrusted diagnostic text. + * @param {unknown} secret Active credential that must not be retained. + * @returns {string} Diagnostic text with exact secret occurrences removed. + */ +export function redactExactSecret(value, secret) { + const text = typeof value === "string" ? value : String(value ?? ""); + if (typeof secret !== "string" || secret.length === 0) { + return text; + } + return text.split(secret).join("[REDACTED]"); +} + /** * Build the least-privilege environment inherited by the `gh` subprocess. * @@ -159,19 +180,25 @@ export function ghApi(path, options = {}, runtime = defaultGhRuntime) { } args.push(path); + const subprocessEnvironment = createGhSubprocessEnvironment(runtime.environment ?? process.env); const result = runtime.spawn_sync("gh", args, { timeout: GH_API_TIMEOUT_MILLISECONDS, maxBuffer: GH_API_MAX_BUFFER_BYTES, - env: createGhSubprocessEnvironment(runtime.environment ?? process.env), + env: subprocessEnvironment, stdio: ["ignore", "pipe", "pipe"], }); if (result.error) { - throw new Error(`GitHub Actions evidence read failed: ${boundedErrorText(result.error.message)}`); + throw new Error( + `GitHub Actions evidence read failed: ${boundedErrorText(redactExactSecret(result.error.message, subprocessEnvironment.GH_TOKEN))}`, + ); } if (result.status !== 0) { + const diagnostic = boundedErrorText( + redactExactSecret(result.stderr, subprocessEnvironment.GH_TOKEN), + ); throw new Error( - `GitHub Actions evidence read failed with gh exit ${result.status}: ${boundedErrorText(result.stderr)}`, + `GitHub Actions evidence read failed with gh exit ${result.status}:${diagnostic ? ` ${diagnostic}` : ""}`, ); } @@ -344,14 +371,43 @@ export async function runActionsRunnerAssignmentAudit(input) { /** * Execute the CLI with injectable boundaries while preserving production defaults. * + * Production reads the delegated bearer credential only from the repository's + * descriptor-safe capability-file boundary. Tests may inject a pre-authenticated + * reader and explicit environment without requiring filesystem credential access. + * * @param {object} options Runtime overrides used only by tests/operators. * @returns {Promise<{exit_code: number, report: object}>} Audit result. */ export async function main(options = {}) { + const sourceEnvironment = options.env ?? process.env; + let auditEnvironment = sourceEnvironment; + let githubApi = options.gh_api; + + if (githubApi === undefined) { + const tokenPath = String(sourceEnvironment.NOEMA_MAINTAINER_TOKEN_PATH ?? "").trim(); + const delegatedToken = readDelegatedGithubToken(tokenPath); + const subprocessEnvironment = { + PATH: sourceEnvironment.PATH, + GH_TOKEN: delegatedToken, + }; + auditEnvironment = { + GH_TOKEN: auditCredentialPresenceMarker, + NOEMA_ACTIONS_AUDIT_REPOSITORY: sourceEnvironment.NOEMA_ACTIONS_AUDIT_REPOSITORY, + NOEMA_ACTIONS_AUDIT_HEAD_SHA: sourceEnvironment.NOEMA_ACTIONS_AUDIT_HEAD_SHA, + NOEMA_ACTIONS_AUDIT_RUN_IDS: sourceEnvironment.NOEMA_ACTIONS_AUDIT_RUN_IDS, + NOEMA_ACTIONS_AUDIT_QUEUE_GRACE_MILLISECONDS: + sourceEnvironment.NOEMA_ACTIONS_AUDIT_QUEUE_GRACE_MILLISECONDS, + }; + githubApi = (path, apiOptions) => ghApi(path, apiOptions, { + spawn_sync: spawnSync, + environment: subprocessEnvironment, + }); + } + const result = await runActionsRunnerAssignmentAudit({ - env: options.env ?? process.env, + env: auditEnvironment, observed_at: options.observed_at ?? new Date().toISOString(), - gh_api: options.gh_api ?? ghApi, + gh_api: githubApi, write_report: options.write_report ?? writeReportAtomically, }); const writeOutput = options.write_output ?? ((value) => process.stdout.write(value)); diff --git a/scripts/lib/actions-runner-assignment-audit.mjs b/scripts/lib/actions-runner-assignment-audit.mjs index 8641835b5..fb3692fde 100644 --- a/scripts/lib/actions-runner-assignment-audit.mjs +++ b/scripts/lib/actions-runner-assignment-audit.mjs @@ -32,10 +32,9 @@ function boundedName(value) { } function assignmentObserved(job) { - const startedAt = parseTimestamp(job?.started_at); const runnerId = job?.runner_id; const runnerName = typeof job?.runner_name === "string" ? job.runner_name.trim() : ""; - return startedAt !== null || positiveSafeInteger(runnerId) || runnerName.length > 0; + return positiveSafeInteger(runnerId) || runnerName.length > 0; } function invalidEvidence(detail) { @@ -51,9 +50,11 @@ function invalidEvidence(detail) { * * The evaluator answers only whether each selected pull-request workflow job * obtained a runner. A later test, security, or workflow conclusion remains a - * separate evidence class. Freshly queued jobs remain non-passing `PENDING`. - * A grace-window stall is emitted only when both the workflow run and the job - * remain queued with no assignment observed anywhere in the selected run; + * separate evidence class. GitHub may populate `started_at` while a queued job + * still has runner_id=0 and no runner_name, so timestamps are not assignment + * authority. Freshly queued jobs remain non-passing `PENDING`. A grace-window + * stall is emitted only when both the workflow run and the job remain queued + * with no runner identity observed anywhere in the selected run; * protection/dependency waits stay non-passing without being mislabeled as a * runner-allocation failure. * @@ -190,7 +191,7 @@ export function evaluateRunnerAssignmentEvidence(evidence) { check( "runner_assignment_observed", true, - "GitHub job evidence shows that a runner was assigned; the later job conclusion remains separate.", + "GitHub job evidence contains a positive runner id or non-empty runner name; the later job conclusion remains separate.", jobContext, ), ); @@ -202,7 +203,7 @@ export function evaluateRunnerAssignmentEvidence(evidence) { failures.push( failure( "runner_assignment_not_observed", - "The job reached a non-queue state without trustworthy started_at or runner identity evidence.", + "The job reached a non-queue state without trustworthy runner identity evidence.", { ...jobContext, job_status: jobStatus }, ), ); diff --git a/test/actions-runner-assignment-audit.test.ts b/test/actions-runner-assignment-audit.test.ts index 45ca88e2a..2f2cebd1b 100644 --- a/test/actions-runner-assignment-audit.test.ts +++ b/test/actions-runner-assignment-audit.test.ts @@ -52,6 +52,27 @@ describe("GitHub Actions runner-assignment evidence", () => { expect(failureCodes(result)).toContain("runner_assignment_stalled"); }); + it("does not mistake GitHub's queued started_at timestamp for runner assignment", () => { + const result = evaluate([workflowRun({ + jobs: [{ + id: 201, + name: "verify", + status: "queued", + conclusion: null, + started_at: "2026-08-09T23:50:00.000Z", + completed_at: null, + runner_id: 0, + runner_name: "", + }], + })]); + + expect(result.status).toBe("FAIL"); + expect(failureCodes(result)).toContain("runner_assignment_stalled"); + expect(result.checks).not.toContainEqual( + expect.objectContaining({ code: "runner_assignment_observed", job_id: 201 }), + ); + }); + it("keeps a recently queued unassigned job pending rather than calling it healthy", () => { const result = evaluate([workflowRun({ created_at: "2026-08-09T23:58:00.000Z" })]); expect(result.status).toBe("PENDING"); diff --git a/test/actions-runner-assignment-cli.test.ts b/test/actions-runner-assignment-cli.test.ts index 028cb6bf1..083ed4a0d 100644 --- a/test/actions-runner-assignment-cli.test.ts +++ b/test/actions-runner-assignment-cli.test.ts @@ -63,10 +63,18 @@ function auditEnvironment(overrides: Record = {}) { }; } -function createGhShim(directory: string) { +function createGhShim(directory: string, expectedToken: string) { const executable = join(directory, "gh"); + const expectedTokenPath = join(directory, "expected-gh-token"); + writeFileSync(expectedTokenPath, expectedToken, { encoding: "utf8", mode: 0o600 }); + const tokenGuard = `expected_token=$(cat -- "${expectedTokenPath}") +if [ "$GH_TOKEN" != "$expected_token" ]; then + printf '%s' 'unexpected delegated GH_TOKEN' >&2 + exit 91 +fi +`; writeFileSync(executable, `#!/bin/sh -case "$*" in +${tokenGuard}case "$*" in *"/jobs?filter=all&per_page=100"*) printf '%s' '[{"jobs":[{"id":1001,"name":"verify","status":"completed","conclusion":"failure","started_at":"2026-08-09T23:52:00.000Z","completed_at":"2026-08-09T23:53:00.000Z","runner_id":77,"runner_name":"GitHub Actions 77"}]}]' ;; @@ -356,14 +364,20 @@ describe("runner-assignment operator audit", () => { expect(setExitCode).toHaveBeenCalledWith(0); }); - it("runs the default CLI dependencies against a real local gh shim without credential widening", async () => { + it("runs the default CLI dependencies against a real local gh shim and capability file", async () => { const directory = mkdtempSync(join(tmpdir(), "noema-runner-audit-")); const previousPath = process.env.PATH ?? ""; try { - createGhShim(directory); + createGhShim(directory, "read-only-capability-token"); process.chdir(directory); - Object.assign(process.env, auditEnvironment()); + const { GH_TOKEN: _ambientToken, ...operatorEnvironment } = auditEnvironment(); + Object.assign(process.env, operatorEnvironment); + process.env.GH_TOKEN = "ambient-runner-audit-token-decoy"; process.env.PATH = `${directory}:${previousPath}`; + const tokenPath = join(directory, "runner-audit-token"); + writeFileSync(tokenPath, "read-only-capability-token", { encoding: "utf8", mode: 0o600 }); + chmodSync(tokenPath, 0o600); + process.env.NOEMA_MAINTAINER_TOKEN_PATH = tokenPath; const result = await main(); expect(result.exit_code).toBe(0); const reportPath = resolve(directory, "artifacts/operations/actions-runner-assignment-audit.json"); @@ -372,6 +386,9 @@ describe("runner-assignment operator audit", () => { status: "PASS", expected_head_sha: expectedHead, }); + const reportText = readFileSync(reportPath, "utf8"); + expect(reportText).not.toContain("read-only-capability-token"); + expect(reportText).not.toContain("ambient-runner-audit-token-decoy"); } finally { process.chdir(originalCwd); rmSync(directory, { recursive: true, force: true }); diff --git a/test/actions-runner-assignment-docs.test.ts b/test/actions-runner-assignment-docs.test.ts index 83076fbf1..fe803a409 100644 --- a/test/actions-runner-assignment-docs.test.ts +++ b/test/actions-runner-assignment-docs.test.ts @@ -20,6 +20,7 @@ describe("runner-assignment operator documentation contract", () => { "ContextualWisdomLab/noema", "NOEMA_ACTIONS_AUDIT_HEAD_SHA", "NOEMA_ACTIONS_AUDIT_RUN_IDS", + "NOEMA_MAINTAINER_TOKEN_PATH", "filter=all", "PENDING", "runner_assignment_stalled", @@ -39,9 +40,24 @@ describe("runner-assignment operator documentation contract", () => { expect(doctoring).toContain(phrase); } + expect(doctoring).toContain("`started_at` is not runner-assignment authority"); + expect(doctoring).toContain("positive `runner_id` or a non-empty `runner_name`"); + expect(doctoring).not.toContain("such as `started_at`, a positive `runner_id`"); + expect(doctoring).not.toContain("export GH_TOKEN="); + expect(doctoring).toContain("owner-only delegated token capability file"); + expect(doctoring).toContain("Do not use `echo` or `printf '%s\\n'`"); + expect(doctoring).toContain("Use `printf '%s'` exactly as shown"); + expect(doctoring).toContain("workflow runs and workflow jobs"); + expect(doctoring).toContain("Permissions required for fine-grained personal access tokens"); + expect(changelog).toContain("`operations:runner-assignment` audit"); expect(changelog).toContain("runner assignment"); expect(changelog).toContain("required Check"); expect(changelog).toContain("formal review"); + expect(changelog).toContain("NOEMA_MAINTAINER_TOKEN_PATH"); + expect(changelog).toContain("[REDACTED]"); + expect(changelog).toContain("`started_at`"); + expect(changelog).toContain("`runner_id`"); + expect(changelog).toContain("`runner_name`"); }); }); diff --git a/test/actions-runner-assignment-error-coverage.test.ts b/test/actions-runner-assignment-error-coverage.test.ts new file mode 100644 index 000000000..00cbba9c0 --- /dev/null +++ b/test/actions-runner-assignment-error-coverage.test.ts @@ -0,0 +1,19 @@ +import { describe, expect, it, vi } from "vitest"; + +import { startCli } from "../scripts/actions-runner-assignment-audit.mjs"; + +describe("runner-assignment CLI defensive error normalization", () => { + it("fails closed when an injected execution boundary rejects without an Error object", async () => { + const writeError = vi.fn(); + const setExitCode = vi.fn(); + + await expect(startCli({ + execute: async () => { throw null; }, + write_error: writeError, + set_exit_code: setExitCode, + })).resolves.toBeUndefined(); + + expect(writeError).toHaveBeenCalledWith("runner-assignment audit failed: \n"); + expect(setExitCode).toHaveBeenCalledWith(2); + }); +}); diff --git a/test/actions-runner-assignment-production-branches.test.ts b/test/actions-runner-assignment-production-branches.test.ts index f88a6cd09..b1c28a302 100644 --- a/test/actions-runner-assignment-production-branches.test.ts +++ b/test/actions-runner-assignment-production-branches.test.ts @@ -6,11 +6,14 @@ import { const originalExitCode = process.exitCode; const originalGithubToken = process.env.GH_TOKEN; +const originalMaintainerTokenPath = process.env.NOEMA_MAINTAINER_TOKEN_PATH; afterEach(() => { process.exitCode = originalExitCode; if (originalGithubToken === undefined) delete process.env.GH_TOKEN; else process.env.GH_TOKEN = originalGithubToken; + if (originalMaintainerTokenPath === undefined) delete process.env.NOEMA_MAINTAINER_TOKEN_PATH; + else process.env.NOEMA_MAINTAINER_TOKEN_PATH = originalMaintainerTokenPath; }); describe("runner-assignment production boundary coverage", () => { @@ -59,13 +62,15 @@ describe("runner-assignment production boundary coverage", () => { }); }); - it("uses the production audit entrypoint and fails closed before GitHub I/O without a token", async () => { + it("uses the production audit entrypoint and fails closed before GitHub I/O without a delegated capability path", async () => { const previousExitCode = process.exitCode; const previousToken = process.env.GH_TOKEN; + const previousTokenPath = process.env.NOEMA_MAINTAINER_TOKEN_PATH; const errors: string[] = []; const exitCodes: number[] = []; try { delete process.env.GH_TOKEN; + delete process.env.NOEMA_MAINTAINER_TOKEN_PATH; const result = await startCli({ write_error: (value: string) => errors.push(value), set_exit_code: (code: number) => exitCodes.push(code), @@ -73,12 +78,14 @@ describe("runner-assignment production boundary coverage", () => { expect(result).toBeUndefined(); expect(errors).toHaveLength(1); - expect(errors[0]).toMatch(/GH_TOKEN is required for read-only GitHub Actions evidence collection/i); + expect(errors[0]).toMatch(/Maintainer token file path is required/i); expect(exitCodes).toEqual([2]); } finally { process.exitCode = previousExitCode; if (previousToken === undefined) delete process.env.GH_TOKEN; else process.env.GH_TOKEN = previousToken; + if (previousTokenPath === undefined) delete process.env.NOEMA_MAINTAINER_TOKEN_PATH; + else process.env.NOEMA_MAINTAINER_TOKEN_PATH = previousTokenPath; } }); diff --git a/test/actions-runner-assignment-secret-redaction.test.ts b/test/actions-runner-assignment-secret-redaction.test.ts new file mode 100644 index 000000000..85523dfd4 --- /dev/null +++ b/test/actions-runner-assignment-secret-redaction.test.ts @@ -0,0 +1,78 @@ +import { describe, expect, it } from "vitest"; +import { ghApi, redactExactSecret } from "../scripts/actions-runner-assignment-audit.mjs"; + +const apiPath = "repos/ContextualWisdomLab/noema/actions/runs/100"; +const activeToken = "github_pat_noema_runner_audit_secret_123"; +const runtimeEnvironment = { PATH: "/usr/bin", GH_TOKEN: activeToken }; + +describe("runner-assignment GitHub credential redaction", () => { + it("redacts the active credential from spawn errors before they can reach retained output", () => { + let thrown: unknown; + try { + ghApi(apiPath, {}, { + spawn_sync: () => ({ + error: new Error(`spawn failed while using ${activeToken}`), + status: null, + stdout: Buffer.alloc(0), + stderr: Buffer.alloc(0), + }), + environment: runtimeEnvironment, + }); + } catch (error) { + thrown = error; + } + + expect(thrown).toBeInstanceOf(Error); + expect((thrown as Error).message).toContain("[REDACTED]"); + expect((thrown as Error).message).not.toContain(activeToken); + }); + + it("redacts the active credential from nonzero gh stderr before it can reach retained output", () => { + let thrown: unknown; + try { + ghApi(apiPath, {}, { + spawn_sync: () => ({ + error: undefined, + status: 7, + stdout: Buffer.alloc(0), + stderr: Buffer.from(`request failed for token ${activeToken}\n`, "utf8"), + }), + environment: runtimeEnvironment, + }); + } catch (error) { + thrown = error; + } + + expect(thrown).toBeInstanceOf(Error); + expect((thrown as Error).message).toContain("[REDACTED]"); + expect((thrown as Error).message).not.toContain(activeToken); + }); + + it("keeps a nonzero gh failure bounded when the runtime supplies no stderr bytes", () => { + let thrown: unknown; + try { + ghApi(apiPath, {}, { + spawn_sync: () => ({ + error: undefined, + status: 7, + stdout: Buffer.alloc(0), + stderr: undefined, + }), + environment: runtimeEnvironment, + }); + } catch (error) { + thrown = error; + } + + expect(thrown).toBeInstanceOf(Error); + expect((thrown as Error).message).toBe("GitHub Actions evidence read failed with gh exit 7:"); + expect((thrown as Error).message).not.toContain(activeToken); + }); + + it("leaves diagnostic text unchanged when the active secret is empty or missing", () => { + const diagnostic = "spawn failed while using a placeholder"; + expect(redactExactSecret(diagnostic, "")).toBe(diagnostic); + expect(redactExactSecret(diagnostic, undefined)).toBe(diagnostic); + expect(redactExactSecret(diagnostic, null)).toBe(diagnostic); + }); +}); diff --git a/test/actions-runner-assignment-token-capability.test.ts b/test/actions-runner-assignment-token-capability.test.ts new file mode 100644 index 000000000..59fcf7951 --- /dev/null +++ b/test/actions-runner-assignment-token-capability.test.ts @@ -0,0 +1,162 @@ +import { + chmodSync, + mkdtempSync, + readFileSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { afterEach, describe, expect, it, vi } from "vitest"; +import { main } from "../scripts/actions-runner-assignment-audit.mjs"; + +const expectedHead = "0123456789abcdef0123456789abcdef01234567"; +const originalCwd = process.cwd(); +const originalEnvironment = { ...process.env }; +const originalExitCode = process.exitCode; +const temporaryDirectories: string[] = []; + +function restoreEnvironment() { + for (const key of Object.keys(process.env)) { + if (!(key in originalEnvironment)) delete process.env[key]; + } + Object.assign(process.env, originalEnvironment); +} + +function temporaryDirectory() { + const directory = mkdtempSync(join(tmpdir(), "noema-runner-token-capability-")); + temporaryDirectories.push(directory); + return directory; +} + +function createGhShim(directory: string, expectedToken = "short-lived-runner-audit-token") { + const executable = join(directory, "gh"); + const expectedTokenPath = join(directory, "expected-gh-token"); + writeFileSync(expectedTokenPath, expectedToken, { encoding: "utf8", mode: 0o600 }); + writeFileSync(executable, `#!/bin/sh +expected_token=$(cat -- "${expectedTokenPath}") +if [ "$GH_TOKEN" != "$expected_token" ]; then + printf '%s' 'unexpected delegated GH_TOKEN' >&2 + exit 91 +fi +case "$*" in + *"/jobs?filter=all&per_page=100"*) + printf '%s' '[{"jobs":[{"id":1001,"name":"verify","status":"completed","conclusion":"failure","started_at":"2026-08-09T23:52:00.000Z","completed_at":"2026-08-09T23:53:00.000Z","runner_id":77,"runner_name":"GitHub Actions 77"}]}]' + ;; + *) + printf '%s' '{"id":100,"name":"ci","event":"pull_request","head_sha":"${expectedHead}","status":"completed","conclusion":"failure","created_at":"2026-08-09T23:50:00.000Z"}' + ;; +esac +`, "utf8"); + chmodSync(executable, 0o700); +} + +function createTokenFile(directory: string) { + const tokenPath = join(directory, "runner-audit-token"); + writeFileSync(tokenPath, "short-lived-runner-audit-token", { encoding: "utf8", mode: 0o600 }); + chmodSync(tokenPath, 0o600); + return tokenPath; +} + +function configureAuditEnvironment(directory: string, tokenPath?: string) { + const previousPath = originalEnvironment.PATH ?? "/usr/bin:/bin"; + process.env.PATH = `${directory}:${previousPath}`; + process.env.NOEMA_ACTIONS_AUDIT_REPOSITORY = "ContextualWisdomLab/noema"; + process.env.NOEMA_ACTIONS_AUDIT_HEAD_SHA = expectedHead; + process.env.NOEMA_ACTIONS_AUDIT_RUN_IDS = "100"; + if (tokenPath) process.env.NOEMA_MAINTAINER_TOKEN_PATH = tokenPath; +} + +afterEach(() => { + process.chdir(originalCwd); + restoreEnvironment(); + process.exitCode = originalExitCode; + for (const directory of temporaryDirectories.splice(0)) { + rmSync(directory, { recursive: true, force: true }); + } + vi.restoreAllMocks(); +}); + +describe("runner-assignment delegated GitHub token capability", () => { + it("runs the production CLI from an owner-only capability file instead of ambient GH_TOKEN", async () => { + const directory = temporaryDirectory(); + createGhShim(directory); + const tokenPath = createTokenFile(directory); + process.chdir(directory); + configureAuditEnvironment(directory, tokenPath); + process.env.GH_TOKEN = "ambient-runner-audit-token-decoy"; + + const result = await main({ + observed_at: "2026-08-10T00:00:00.000Z", + write_output: vi.fn(), + set_exit_code: vi.fn(), + }); + + expect(result.exit_code).toBe(0); + const reportPath = resolve(directory, "artifacts/operations/actions-runner-assignment-audit.json"); + expect(JSON.parse(readFileSync(reportPath, "utf8"))).toMatchObject({ + status: "PASS", + expected_head_sha: expectedHead, + }); + expect(readFileSync(reportPath, "utf8")).not.toContain("short-lived-runner-audit-token"); + expect(readFileSync(reportPath, "utf8")).not.toContain("ambient-runner-audit-token-decoy"); + }); + + it("does not enumerate unrelated parent secrets while constructing audit inputs", async () => { + const directory = temporaryDirectory(); + createGhShim(directory); + const tokenPath = createTokenFile(directory); + process.chdir(directory); + const hostileEnvironment: Record = { + PATH: `${directory}:${originalEnvironment.PATH ?? "/usr/bin:/bin"}`, + NOEMA_ACTIONS_AUDIT_REPOSITORY: "ContextualWisdomLab/noema", + NOEMA_ACTIONS_AUDIT_HEAD_SHA: expectedHead, + NOEMA_ACTIONS_AUDIT_RUN_IDS: "100", + NOEMA_MAINTAINER_TOKEN_PATH: tokenPath, + }; + Object.defineProperty(hostileEnvironment, "NVIDIA_NIM_API_KEY", { + enumerable: true, + get() { + throw new Error("unrelated secret was enumerated"); + }, + }); + + await expect(main({ + env: hostileEnvironment, + observed_at: "2026-08-10T00:00:00.000Z", + write_output: vi.fn(), + set_exit_code: vi.fn(), + })).resolves.toMatchObject({ exit_code: 0 }); + }); + + it("fails closed when echo-style trailing newline contaminates the capability file", async () => { + const directory = temporaryDirectory(); + createGhShim(directory); + const tokenPath = join(directory, "runner-audit-token"); + writeFileSync(tokenPath, "short-lived-runner-audit-token\n", { encoding: "utf8", mode: 0o600 }); + chmodSync(tokenPath, 0o600); + process.chdir(directory); + configureAuditEnvironment(directory, tokenPath); + + await expect(main({ + observed_at: "2026-08-10T00:00:00.000Z", + write_output: vi.fn(), + set_exit_code: vi.fn(), + })).rejects.toThrow("Maintainer token must not contain control characters."); + }); + + it("fails closed instead of accepting an ambient GH_TOKEN when the capability path is absent", async () => { + const directory = temporaryDirectory(); + createGhShim(directory); + process.chdir(directory); + configureAuditEnvironment(directory); + process.env.GH_TOKEN = "ambient-runner-audit-token-must-not-be-used"; + delete process.env.NOEMA_MAINTAINER_TOKEN_PATH; + + await expect(main({ + observed_at: "2026-08-10T00:00:00.000Z", + write_output: vi.fn(), + set_exit_code: vi.fn(), + })).rejects.toThrow("Maintainer token file path is required."); + }); +});