Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
e7ecfea
test(security): require runner audit token redaction
seonghobae Aug 16, 2026
bdb4864
fix(security): redact runner audit credential errors
seonghobae Aug 16, 2026
a39f9fd
fix(security): keep token redaction coverage exact
seonghobae Aug 16, 2026
f0d4aed
test(security): cover missing gh stderr redaction path
seonghobae Aug 16, 2026
fb04907
test(operations): reproduce queued job timestamp false assignment
seonghobae Aug 16, 2026
9f2557d
fix(operations): require runner identity for assignment evidence
seonghobae Aug 16, 2026
2cd4103
test(docs): reject timestamp runner-assignment authority
seonghobae Aug 16, 2026
b4e0f99
docs(operations): align runner assignment authority
seonghobae Aug 16, 2026
5752552
test(security): require runner audit token capability
seonghobae Aug 16, 2026
fa0cefd
fix(security): broker runner audit token capability
seonghobae Aug 16, 2026
627b0cb
test(docs): require runner token capability contract
seonghobae Aug 16, 2026
c58fed9
docs(security): document runner token capability ingress
seonghobae Aug 16, 2026
61c6892
test(security): align default runner audit credential ingress
seonghobae Aug 16, 2026
bbbfe8e
fix(operations): normalize runner audit failure boundaries
seonghobae Aug 16, 2026
0634ba5
test(operations): align production runner audit capability boundary
seonghobae Aug 16, 2026
fe09e34
merge: integrate protected main after #396
seonghobae Aug 16, 2026
5f861fb
fix(operations): preserve malformed JSON evidence contract
seonghobae Aug 16, 2026
b0a7a38
merge: restack runner audit on protected main a634066
seonghobae Aug 16, 2026
c866654
fix(coverage): remove unreachable GitHub API default branch
seonghobae Aug 16, 2026
f387edf
test(security): prove delegated runner audit token wins
seonghobae Aug 16, 2026
c4a0ffb
test(security): reject ambient runner audit credential selection
seonghobae Aug 16, 2026
f4a4bc9
test: cover non-error runner audit rejection
seonghobae Aug 16, 2026
64c6213
docs(operations): record runner audit credential and assignment contract
cursoragent Aug 16, 2026
fddec97
fix(operations): harden runner-audit first-run credential contract
cursoragent Aug 16, 2026
5dca5d6
merge main into runner-audit credential contract
seonghobae Aug 16, 2026
a27a563
docs: preserve canonical changelog newline
seonghobae Aug 16, 2026
b52a2f6
test(operations): remove unreachable runner shim branch
seonghobae Aug 16, 2026
f5d97ba
test(security): reject ambient secret enumeration
seonghobae Aug 16, 2026
7927a5b
fix(security): minimize runner audit secret scope
seonghobae Aug 16, 2026
44d4589
fix(security): preserve runner audit contracts while minimizing secre…
seonghobae Aug 16, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
## Unreleased
- 검증된 active-orphan 워크플로 하나를 운영자가 호출할 수 있는 `operations:workflow-registry-disable` 경로를 추가한다. 저장소와 워크플로 ID를 `NOEMA_MAINTAINER_TOKEN_PATH` 위임 토큰 파일 읽기 전에 검사하고, 신선한 전체 레지스트리 감사·즉시 live refresh·프로세스 로컬 plan·보호된 main/워크플로 재검증·사후 전체 감사 봉투(`schema_version` 1, `PASS`/`FAIL`, `remaining_failure_codes`, `remaining_active_orphan_ids`)를 통과한 뒤에만 영수증을 유지한다. 성공 종료와 `post_audit_status: FAIL`은 해당 ID만 `disabled_manually`가 되었고 레지스트리는 아직 더럽을 수 있음을 뜻하므로, 운영자는 영수증의 `remaining_active_orphan_ids`로 다음 단일 호출을 이어간다. 배치 비활성화·자가 수리 워크플로·거버넌스 완화는 추가하지 않으며 호출 계약은 doctoring에 기록한다.
- 읽기 전용 `operations:runner-assignment` audit를 추가해 exact workflow run/source head에 대한 runner assignment를 완전 pagination으로 진단하고, 신선한 unassigned queue는 bounded grace 이후 실패-폐쇄한다. 이 증빙은 runner assignment와 required Check/CI, formal review, merge, release, deployment authority를 분리하며 assigned runner 이후 workflow failure를 성공으로 승격하지 않는다.
- production `operations:runner-assignment` audit는 `NOEMA_MAINTAINER_TOKEN_PATH`의 owner-only capability file만 읽고, ambient `GH_TOKEN`만 있으면 실패-폐쇄한다. `gh` spawn/stderr 진단은 활성 토큰을 exact-match로 `[REDACTED]` 치환하며, 빈 secret에 대해서는 원문 진단을 보존한다. assignment authority는 양의 `runner_id` 또는 비어 있지 않은 `runner_name`만 인정하며 queued `started_at`은 assignment evidence가 아니다. 운영자는 `printf '%s'`로 capability file을 만들고(`echo`/`printf '%s\\n'`는 trailing newline 때문에 실패-폐쇄), Actions workflow-run/job read만 가진 짧은 토큰을 준비한 뒤 PASS를 required Check·formal review·merge 권한으로 해석하지 마십시오.
- coordinated vulnerability disclosure 정책과 evidence-preserving vulnerability handling lifecycle, read-only private-vulnerability-reporting setting audit를 추가한다. 이 source 변경은 live private reporting 활성화·notification staffing·end-to-end advisory exercise·release/deployment authority를 증명하지 않는다.
- 개발 의존성 체인의 transitive `nanoid` lockfile resolution을 `3.3.17`에서 `3.3.18`로 최소 갱신하여 GHSA-2v37-7h3g-55p8 / CVE-2026-67213 보안 게이트를 복구한다. PostCSS의 선언 범위 `^3.3.16`과 다른 package metadata는 변경하지 않으며 audit waiver·ignore·severity 완화 없이 `npm ci`/`npm audit --audit-level=high`가 exact head에서 재검증되도록 유지한다.
- lockfile 재생성 도구 체인을 Node.js 24.19.0/npm 11.17.0으로 정확히 고정하고, `strict-allow-scripts=true` 아래 승인된 install-script identity만 실행하며 schema v3 exact-base lockfile change control로 package metadata drift를 실패-폐쇄한다. exact package before/after digest에 더해 top-level metadata digest와 대규모 package-set bulk evidence를 결합하며, 선행 `nanoid@3.3.18` 보안 수정과 explicit `npm ci --legacy-peer-deps=false --install-links=false` 계약을 보존한다. package-manager/toolchain·install-script authority·vulnerability audit·review/merge authority는 별도 증거 계층으로 유지한다.
Expand Down
33 changes: 26 additions & 7 deletions docs/doctoring/actions-runner-assignment-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,20 +13,20 @@ The operator supplies an explicit bounded set of GitHub Actions workflow-run IDs
Required invariants are:

1. the repository is exactly `ContextualWisdomLab/noema`;
2. `GH_TOKEN` exists only as transport authority for read-only API calls and is never retained in the report;
2. the production CLI reads its short-lived GitHub transport authority from an owner-only delegated token capability file named by `NOEMA_MAINTAINER_TOKEN_PATH`; the bearer value is never read from ambient `GH_TOKEN` and is never retained in the report;
3. the expected source head is one canonical lowercase 40-character commit SHA;
4. one to twenty unique positive run IDs are selected explicitly;
5. selected runs must be `pull_request` runs bound to that exact source head;
6. job pages are fully paginated with `per_page=100` and `filter=all`, bounded to at most 2,000 retained jobs;
7. runner assignment is observed only from job evidence such as `started_at`, a positive `runner_id`, or a non-empty `runner_name`;
7. runner assignment is observed only from a positive `runner_id` or a non-empty `runner_name`; `started_at` is not runner-assignment authority because GitHub may populate it while a job is still queued without runner identity;
8. a `waiting`, `pending`, or `requested` job remains non-passing `PENDING` because those states do not by themselves isolate runner allocation;
9. a queued job in a run where another job has already received a runner remains non-passing `PENDING`, because the queued job may be waiting on an explicit `jobs.<job_id>.needs` dependency rather than runner capacity;
10. the bounded grace may produce `runner_assignment_stalled` only when the workflow run itself remains `queued`, the job remains `queued`, and no job in that selected run has assignment evidence;
11. an assigned job may produce runner-assignment `PASS` even if its later workflow/test conclusion is `failure`, because those are separate evidence classes.

The default runner-allocation grace is five minutes and may be bounded by `NOEMA_ACTIONS_AUDIT_QUEUE_GRACE_MILLISECONDS`; the evaluator rejects values above thirty minutes rather than allowing a true isolated queue condition to remain indefinitely pending.

This classifier is intentionally conservative because the GitHub workflow-job REST representation does not expose a durable repository-consumable timestamp meaning “this job became eligible for runner allocation.” A workflow run's `created_at` is therefore not a trustworthy age for every downstream job. The evaluator uses run age only after the selected evidence isolates the top-level queued runner-allocation boundary described above.
This classifier is intentionally conservative because the GitHub workflow-job REST representation does not expose a durable repository-consumable timestamp meaning “this job became eligible for runner allocation.” A workflow run's `created_at` is therefore not a trustworthy age for every downstream job, and a job's `started_at` is not runner-assignment authority. The evaluator uses run age only after the selected evidence isolates the top-level queued runner-allocation boundary described above.

### Pre-run waits are not runner stalls

Expand All @@ -36,17 +36,32 @@ Those states remain operationally non-passing, but they are not evidence that Gi

## Operator contract

The production command consumes an owner-only delegated token capability file. The path itself is non-secret; the token bytes must be materialized by an authorized caller and removed promptly after the audit.

Example:

```bash
export GH_TOKEN='<read-only token materialized outside repository files>'
umask 077
token_dir="$(mktemp -d)"
token_path="$token_dir/runner-audit-token"
printf '%s' '<short-lived read-only token>' >"$token_path"
chmod 0600 "$token_path"
trap 'rm -rf "$token_dir"' EXIT

export NOEMA_MAINTAINER_TOKEN_PATH="$token_path"
export NOEMA_ACTIONS_AUDIT_REPOSITORY='ContextualWisdomLab/noema'
export NOEMA_ACTIONS_AUDIT_HEAD_SHA='<exact current PR source head>'
export NOEMA_ACTIONS_AUDIT_RUN_IDS='31343034891,31343034896,31343034900'

npm run operations:runner-assignment
```

Do not use `echo` or `printf '%s\n'` to write the capability file. Those commands append a newline, and the reader then fails closed because tokens must not contain control characters. Use `printf '%s'` exactly as shown, then remove the file in the `trap` so the bearer does not remain on disk.

The short-lived token needs GitHub Actions **read** for workflow runs and workflow jobs on `ContextualWisdomLab/noema` only. It must not carry contents write, pull-request write, administration, workflow, or workflow-dispatch scopes. A broader token is the wrong credential for this diagnostic.

An ambient `GH_TOKEN` is intentionally not a production credential source for this command. The descriptor-safe shared capability reader rejects symlinks, non-regular files, group/world-readable files, wrong-owner files where UID inspection is available, oversized content, invalid UTF-8, control-bearing tokens, and file identity/version changes during the bounded read.

The command writes the fixed report path:

```text
Expand All @@ -55,7 +70,7 @@ artifacts/operations/actions-runner-assignment-audit.json

The report records repository, expected head, selected run IDs, observation time, queue grace, deterministic checks/failures, and explicit false authority flags for required-check success, review, merge, release, and deployment. Temporary report bytes are created owner-only and atomically renamed onto the fixed report path.

`PASS` exits zero. `PENDING` and `FAIL` both exit nonzero. A malformed source identity, cross-repository request, missing credential, malformed API JSON, GitHub CLI failure, pagination-shape failure, excessive evidence, or head mismatch fails closed.
`PASS` exits zero. `PENDING` and `FAIL` both exit nonzero. A malformed source identity, cross-repository request, missing/unsafe capability file, malformed API JSON, GitHub CLI failure, pagination-shape failure, excessive evidence, or head mismatch fails closed.

## RCA interpretation

Expand All @@ -69,9 +84,9 @@ This separation matters for issue #30 because historical Noema runs exhibited qu

## Security and privacy

The collector uses GitHub Actions REST **read** endpoints only. It does not rerun, cancel, dispatch, approve, merge, modify refs, or change settings. The report does not contain `GH_TOKEN`, repository secrets, workflow logs, source contents, personal data beyond ordinary GitHub workflow/job metadata needed for the operational decision, or model output.
The collector uses GitHub Actions REST **read** endpoints only. It does not rerun, cancel, dispatch, approve, merge, modify refs, or change settings. The report does not contain the delegated token, repository secrets, workflow logs, source contents, personal data beyond ordinary GitHub workflow/job metadata needed for the operational decision, or model output.

The `gh` child process also receives a purpose-built minimal environment rather than ambient process state: only `PATH`, the read-only `GH_TOKEN`, pinned `GH_HOST=github.com`, and `NO_COLOR=1` cross the process boundary. `GITHUB_TOKEN`, `NVIDIA_NIM_API_KEY`, Maintainer/Reviewer App private material, `HOME`, and ambient proxy variables are excluded. This prevents a read-only diagnostic subprocess from accidentally inheriting stronger publication/model credentials or redirecting credential-bearing requests through an unreviewed proxy path.
The production CLI reads bearer authority only through `NOEMA_MAINTAINER_TOKEN_PATH`. The `gh` child process then receives a purpose-built minimal environment: only `PATH`, the in-memory read-only `GH_TOKEN` required by the GitHub CLI, pinned `GH_HOST=github.com`, and `NO_COLOR=1` cross that child-process boundary. `GITHUB_TOKEN`, ambient `GH_TOKEN`, `NVIDIA_NIM_API_KEY`, Maintainer/Reviewer App private material, `HOME`, and ambient proxy variables are excluded. This prevents the read-only diagnostic subprocess from inheriting stronger publication/model credentials or redirecting credential-bearing requests through an unreviewed proxy path.

The audit is diagnostic evidence. A passing assignment audit cannot satisfy branch protection, required checks, formal review, security scanning, release provenance, production deployment, or acquisition evidence.

Expand All @@ -80,6 +95,8 @@ The audit is diagnostic evidence. A passing assignment audit cannot satisfy bran
The repository-owned slice is acceptable when:

- realistic tests reproduce an isolated runner stall, a fresh queue, deployment/environment waiting, downstream dependency waiting, assigned-but-failed jobs, head mismatch, malformed evidence, pagination, and bounded selection;
- queued `started_at` timestamps without runner identity remain unassigned, while a positive `runner_id` or a non-empty `runner_name` is assignment evidence;
- the production entrypoint succeeds with an owner-only delegated token capability file and fails closed when only ambient `GH_TOKEN` is present;
- environment-protected and dependency-blocked jobs remain nonzero `PENDING` and are not mislabeled as runner-allocation stalls;
- the pure evaluator and bounded source collector are GREEN;
- the operator adapter performs only the two documented read families and fully paginates jobs;
Expand All @@ -97,6 +114,8 @@ GitHub. (2026). *REST API endpoints for workflow jobs*. GitHub Docs. https://doc

GitHub. (2026). *REST API endpoints for workflow runs*. GitHub Docs. https://docs.github.com/en/rest/actions/workflow-runs?apiVersion=2026-03-10

GitHub. (2026). *Permissions required for fine-grained personal access tokens*. GitHub Docs. https://docs.github.com/en/rest/authentication/permissions-required-for-fine-grained-personal-access-tokens

GitHub. (2026). *Deployments and environments*. GitHub Docs. https://docs.github.com/en/actions/concepts/workflows-and-actions/deployment-environments

GitHub. (2026). *Workflow syntax for GitHub Actions*. GitHub Docs. https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idneeds
66 changes: 61 additions & 5 deletions scripts/actions-runner-assignment-audit.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import {
collectRunnerAssignmentEvidence,
parseSelectedRunIds,
} from "./lib/actions-runner-assignment-source.mjs";
import { readDelegatedGithubToken } from "./lib/delegated-github-token.mjs";
import { hasDuplicateJsonObjectKeys } from "./normalize-commercial-readiness-evidence.mjs";

const AUDITED_REPOSITORY = "ContextualWisdomLab/noema";
Expand All @@ -30,6 +31,7 @@ const GH_API_MAX_BUFFER_BYTES = 2 * 1024 * 1024;
const REPORT_PATH = "artifacts/operations/actions-runner-assignment-audit.json";
const canonicalShaPattern = /^[0-9a-f]{40}$/;
const fatalUtf8Decoder = new TextDecoder("utf-8", { fatal: true });
const auditCredentialPresenceMarker = "delegated-capability-present";

const defaultGhRuntime = {
spawn_sync: spawnSync,
Expand All @@ -51,6 +53,25 @@ function boundedErrorText(value) {
return text.replace(/[\u0000-\u001f\u007f]/g, " ").trim().slice(0, 1000);
}

/**
* Replace every exact occurrence of an active secret with a fixed marker.
*
* An empty or non-string secret is left untouched. `String.prototype.split("")`
* would otherwise insert the marker between every character and leak a
* transformed diagnostic that is no longer the original failure text.
*
* @param {unknown} value Untrusted diagnostic text.
* @param {unknown} secret Active credential that must not be retained.
* @returns {string} Diagnostic text with exact secret occurrences removed.
*/
export function redactExactSecret(value, secret) {
const text = typeof value === "string" ? value : String(value ?? "");
if (typeof secret !== "string" || secret.length === 0) {
return text;
}
return text.split(secret).join("[REDACTED]");
}

/**
* Build the least-privilege environment inherited by the `gh` subprocess.
*
Expand Down Expand Up @@ -159,19 +180,25 @@ export function ghApi(path, options = {}, runtime = defaultGhRuntime) {
}
args.push(path);

const subprocessEnvironment = createGhSubprocessEnvironment(runtime.environment ?? process.env);
const result = runtime.spawn_sync("gh", args, {
timeout: GH_API_TIMEOUT_MILLISECONDS,
maxBuffer: GH_API_MAX_BUFFER_BYTES,
env: createGhSubprocessEnvironment(runtime.environment ?? process.env),
env: subprocessEnvironment,
stdio: ["ignore", "pipe", "pipe"],
});

if (result.error) {
throw new Error(`GitHub Actions evidence read failed: ${boundedErrorText(result.error.message)}`);
throw new Error(
`GitHub Actions evidence read failed: ${boundedErrorText(redactExactSecret(result.error.message, subprocessEnvironment.GH_TOKEN))}`,
);
}
if (result.status !== 0) {
const diagnostic = boundedErrorText(
redactExactSecret(result.stderr, subprocessEnvironment.GH_TOKEN),
);
throw new Error(
`GitHub Actions evidence read failed with gh exit ${result.status}: ${boundedErrorText(result.stderr)}`,
`GitHub Actions evidence read failed with gh exit ${result.status}:${diagnostic ? ` ${diagnostic}` : ""}`,
);
}

Expand Down Expand Up @@ -344,14 +371,43 @@ export async function runActionsRunnerAssignmentAudit(input) {
/**
* Execute the CLI with injectable boundaries while preserving production defaults.
*
* Production reads the delegated bearer credential only from the repository's
* descriptor-safe capability-file boundary. Tests may inject a pre-authenticated
* reader and explicit environment without requiring filesystem credential access.
*
* @param {object} options Runtime overrides used only by tests/operators.
* @returns {Promise<{exit_code: number, report: object}>} Audit result.
*/
export async function main(options = {}) {
const sourceEnvironment = options.env ?? process.env;
let auditEnvironment = sourceEnvironment;
let githubApi = options.gh_api;

if (githubApi === undefined) {
const tokenPath = String(sourceEnvironment.NOEMA_MAINTAINER_TOKEN_PATH ?? "").trim();
const delegatedToken = readDelegatedGithubToken(tokenPath);
const subprocessEnvironment = {
PATH: sourceEnvironment.PATH,
GH_TOKEN: delegatedToken,
};
auditEnvironment = {
GH_TOKEN: auditCredentialPresenceMarker,
NOEMA_ACTIONS_AUDIT_REPOSITORY: sourceEnvironment.NOEMA_ACTIONS_AUDIT_REPOSITORY,
NOEMA_ACTIONS_AUDIT_HEAD_SHA: sourceEnvironment.NOEMA_ACTIONS_AUDIT_HEAD_SHA,
NOEMA_ACTIONS_AUDIT_RUN_IDS: sourceEnvironment.NOEMA_ACTIONS_AUDIT_RUN_IDS,
NOEMA_ACTIONS_AUDIT_QUEUE_GRACE_MILLISECONDS:
sourceEnvironment.NOEMA_ACTIONS_AUDIT_QUEUE_GRACE_MILLISECONDS,
};
githubApi = (path, apiOptions) => ghApi(path, apiOptions, {
spawn_sync: spawnSync,
environment: subprocessEnvironment,
});
}

const result = await runActionsRunnerAssignmentAudit({
env: options.env ?? process.env,
env: auditEnvironment,
observed_at: options.observed_at ?? new Date().toISOString(),
gh_api: options.gh_api ?? ghApi,
gh_api: githubApi,
write_report: options.write_report ?? writeReportAtomically,
});
const writeOutput = options.write_output ?? ((value) => process.stdout.write(value));
Expand Down
Loading
Loading