Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .github/workflows/acquisition-readiness-scan.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,6 @@ jobs:
fi
test "$(git rev-parse HEAD)" = "${{ github.sha }}"

- name: build data-room manifest
run: npm run acquisition:manifest

- name: run acquisition audit
env:
NOEMA_AUDIT_REPORT_ONLY: ${{ github.event_name == 'schedule' && '1' || '0' }}
Expand Down
8 changes: 1 addition & 7 deletions .github/workflows/hourly-commercial-readiness.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,12 +203,6 @@ jobs:
NOEMA_SMOKE_EVIDENCE_PATH: noema-smoke-evidence.json
run: npm run readiness:audit

- name: refresh acquisition manifest when the queue is empty
if: steps.loop.outputs.remaining_open_pull_request_count == '0'
env:
NOEMA_AUDIT_REPORT_ONLY: "1"
run: npm run acquisition:manifest

- name: refresh acquisition-readiness evidence when the queue is empty
if: steps.loop.outputs.remaining_open_pull_request_count == '0'
env:
Expand Down Expand Up @@ -246,4 +240,4 @@ jobs:
exchange-30d.ndjson.provenance.json
noema-smoke-evidence.json
if-no-files-found: error
retention-days: 90
retention-days: 90
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
## Unreleased
- External scheduler evidence audits now retain source authority through final report publication: reports are owner-only, no-follow, exclusive one-shot receipts, so a concurrent rename cannot move the accepted source inode onto the report pathname and have it replaced. Source/report path and inode alias checks, single-link retained-source validation, and Unicode control sanitization remain fail closed.
- production runtime credential envelope parsing을 fail-closed로 강화한다. GitHub App PKCS#1 key의 canonical PKCS#8 변환은 유지하되, bare carriage return처럼 비정규 body bytes가 포함된 PKCS#8 PEM은 readiness/import 단계의 암묵적 정규화에 넘기지 않고 즉시 거부해 malformed secret이 ready 상태로 승인되지 않게 한다.
- acquisition tracked-byte 인증이 descriptor에서 읽은 bytes를 Git blob framing으로 Node 표준 crypto에서 직접 해시해, 파일마다 `git hash-object` subprocess를 만들던 대형 checkout 병목을 제거한다. exact tree inventory는 Git 2.36 전용 `ls-tree --format` 대신 호환되는 기본 NUL 형식을 사용하며, object ID, SHA-1/SHA-256 저장소, no-follow·descriptor identity·byte limit 실패-폐쇄 계약은 유지한다. dependency-license inventory가 실제로 소비한 `package-lock.json` bytes도 pinned source commit의 Git blob과 직접 대조해 transient file swap을 차단한다. 실패한 audit stage 뒤에도 source를 다시 인증한 다음 원래 child status로 종료하므로 failure evidence가 stale revision으로 남지 않으며, release·publication·deployment evidence producer와 acquisition consumer는 canonical SHA-1/SHA-256 commit identity를 동일하게 지원한다.
- `acquisition:audit`가 POSIX shell 문법 없이 Node 오케스트레이터로 exact HEAD 기반 단일 기본 output directory를 manifest·integrity·readiness·deployment 단계에 전달해 Windows에서도 새 manifest를 같은 실행에서 소비하며, 기존 `NOEMA_ACQUISITION_AUDIT_OUTPUT_DIR`·`NOEMA_DATA_ROOM_OUTPUT_DIR` 경로 override는 유지한다.
- Governance and Maintainer App GitHub CLI subprocesses now keep CLI config and XDG state inside the validated capability file's private parent directory, preventing a missing ambient home from writing `.local` state into the checkout.
- Maintainer App readiness now requires the retained governance audit's `protected_main_sha` to equal the freshly collected default-branch head, preventing evidence from different protected-main revisions from being combined into one passing report. The governance collector authenticates every tracked checkout byte against exact HEAD before and after live collection, so modified audit source cannot emit PASS evidence attributed to protected main. Governance and readiness report paths also retain their existing non-symlink private-output authority. The product/technical gap baseline is refreshed to the same protected-main and live issue/run/release observation, and describes the hourly loop through `contextual-orchestrator` rather than retired direct-provider execution.
- Exact-tree source authentication uses the long-supported NUL-delimited `git ls-tree` output instead of Git 2.36-only `--format`, preserving fail-closed byte verification on older supported Git installations.
Expand Down
6 changes: 3 additions & 3 deletions docs/acquisition-data-room-integrity.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ Manifest와 integrity 단계가 별도 경로를 추측하지 않도록 `NOEMA_D
- `schemaVersion: 1`
- `repository: ContextualWisdomLab/noema`
- `objective: NOEMA-GOAL-ACQUISITION-2B-2026-07-02`
- `source.commitSha`: manifest를 생성한 checkout의 정확한 40자리 Git commit
- `source.commitSha`: manifest를 생성한 checkout의 정확한 40자리 SHA-1 또는 64자리 SHA-256 Git commit
- 선택된 release가 있을 때 `release.tag`와 그 tag가 실제로 가리키는 `release.commitSha`

`NOEMA_DATA_ROOM_SOURCE_COMMIT`을 지정하면 현재 checkout `HEAD`와 정확히 같아야 한다. `NOEMA_RELEASE_UNDER_DILIGENCE_TAG`를 지정하면 immutable SemVer tag가 로컬 Git object database에서 exact commit으로 해석되어야 한다. 불일치하거나 해석할 수 없는 identity는 fail-closed이다.
Expand All @@ -30,7 +30,7 @@ Manifest와 integrity 단계가 별도 경로를 추측하지 않도록 `NOEMA_D

`source.commitSha`는 단순한 `git rev-parse HEAD` 기록이 아니다. Manifest generator와 integrity audit는 catalog/verifier를 읽기 전에 `scripts/lib/acquisition-git-preflight.mjs`로 tracked checkout을 인증한다.

1. `HEAD^{commit}`을 local Git object database에서 exact 40-character SHA로 해석한다.
1. `HEAD^{commit}`을 local Git object database에서 exact 40-character SHA-1 또는 64-character SHA-256 commit으로 해석한다.
2. system/global Git configuration, hooks, filesystem monitor, untracked cache, replacement objects, lazy fetch, terminal prompt를 비활성화하고 필요한 process-discovery 환경만 전달한다. 격리된 config에서도 CI checkout의 dubious-ownership 보호를 우회하지 않고 정확히 현재 command `cwd`만 command-scope `safe.directory`로 허용한다. 같은 exact `cwd`를 `GIT_WORK_TREE`에도 고정하여 repository-local `core.worktree`가 Git의 tracked-byte 비교를 다른 디렉터리로 redirect하지 못하게 한다.
3. `git ls-files -v -z --cached --`의 전체 NUL-delimited 결과를 최대 2 MiB로 bounded read하고, `S`로 표시되는 `skip-worktree` 또는 lowercase tag로 표시되는 `assume-unchanged` entry가 하나라도 있으면 tracked-byte 비교 전에 실패한다. 이 index hint들은 정상적인 working-tree 검사를 생략하게 할 수 있으므로 acquisition checkout에서는 허용하지 않는다.
4. `git diff --cached --quiet --no-ext-diff --no-textconv --ignore-submodules=none <exact-head> --`로 index의 staged content/mode가 exact commit과 같은지 확인한다. `--cached` 비교는 on-disk worktree를 고려하지 않으므로 repository-configured clean filter가 이 단계의 source identity를 바꿀 수 없다.
Expand Down Expand Up @@ -80,7 +80,7 @@ Receipt는 다음을 모두 충족해야 한다.
{
"schemaVersion": 1,
"repository": "ContextualWisdomLab/noema",
"source": { "commitSha": "<exact 40-character commit>" },
"source": { "commitSha": "<exact 40-character SHA-1 or 64-character SHA-256 commit>" },
"sourceUrl": "<exact catalog URL>",
"collectedAt": "<canonical UTC timestamp>",
"collector": "<producer or collector identity>",
Expand Down
2 changes: 1 addition & 1 deletion docs/hourly-commercial-readiness-loop.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Credential-bearing loop가 실행되면 `commercial-readiness-loop-report` artif
}
```

PR 처리 후 남은 열린 PR이 0개이면 기존 `readiness:audit`, `acquisition:manifest`, `acquisition:audit`를 `NOEMA_AUDIT_REPORT_ONLY=1`로 실행하고 `no-pr-commercial-readiness-evidence` artifact를 남깁니다.
PR 처리 후 남은 열린 PR이 0개이면 기존 `readiness:audit`manifest 생성을 포함한 `acquisition:audit`를 `NOEMA_AUDIT_REPORT_ONLY=1`로 실행하고 `no-pr-commercial-readiness-evidence` artifact를 남깁니다.

## 실패-폐쇄 경계

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@
"patch-validator:image:verify-receipts": "node scripts/verify-patch-validator-image.mjs",
"acquisition:deployment-evidence": "node scripts/acquisition-deployment-evidence-audit.mjs",
"acquisition:integrity": "node scripts/acquisition-data-room-integrity-audit.mjs",
"acquisition:audit": "npm run release:dependency-license-inventory && npm run acquisition:integrity && node scripts/acquisition-readiness-audit.mjs && npm run acquisition:deployment-evidence",
"acquisition:audit": "node scripts/acquisition-audit.mjs",
"acquisition:manifest": "node scripts/acquisition-data-room-manifest.mjs",
"readiness:audit": "node scripts/saleable-readiness-audit.mjs"
},
Expand Down
85 changes: 85 additions & 0 deletions scripts/acquisition-audit.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
#!/usr/bin/env node
import { spawnSync } from "node:child_process";
import { join } from "node:path";
import { pathToFileURL } from "node:url";
import { verifyAcquisitionTrackedCheckout } from "./lib/acquisition-git-preflight.mjs";

const stages = [
["npm", "release:dependency-license-inventory"],
["npm", "acquisition:manifest"],
["npm", "acquisition:integrity"],
["node", "scripts/acquisition-readiness-audit.mjs"],
["npm", "acquisition:deployment-evidence"],
];

function checkedSpawn(command, args, options) {
const result = spawnSync(command, args, options);
if (result.error) throw result.error;
return result.status ?? 1;
}

function resolveHeadRevision(cwd) {
const git = spawnSync("git", ["rev-parse", "HEAD"], { cwd, encoding: "utf8" });
if (git.error) throw git.error;
if (git.status !== 0) throw new Error("git rev-parse HEAD failed");
return git.stdout.trim();
}

function canonicalRevision(value) {
if (!/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/i.test(value)) {
throw new Error("one full commit SHA is required");
}
return value.toLowerCase();
}

export function runAcquisitionAudit({
cwd = process.cwd(),
env = process.env,
spawn = checkedSpawn,
revision,
resolveRevision,
} = {}) {
const requireLiveSource = revision === undefined;
const liveRevision = resolveRevision ?? (() => resolveHeadRevision(cwd));
const expectedRevision = canonicalRevision(
requireLiveSource ? liveRevision() : revision,
);

const outputDirectory = env.NOEMA_ACQUISITION_AUDIT_OUTPUT_DIR
|| env.NOEMA_DATA_ROOM_OUTPUT_DIR
|| join(cwd, "artifacts", "acquisition-readiness", expectedRevision);
const stageEnv = {
...env,
NOEMA_ACQUISITION_AUDIT_OUTPUT_DIR: outputDirectory,
NOEMA_DATA_ROOM_OUTPUT_DIR: outputDirectory,
NOEMA_DATA_ROOM_SOURCE_COMMIT: expectedRevision,
};
const npmExecPath = env.npm_execpath;
if (!npmExecPath) throw new Error("npm_execpath is required");

const assertLiveSource = () => {
if (!requireLiveSource) return;
if (canonicalRevision(liveRevision()) !== expectedRevision) {
throw new Error("acquisition audit source revision changed during execution");
}
if (resolveRevision === undefined) {
verifyAcquisitionTrackedCheckout({
cwd,
expectedCommitSha: expectedRevision,
});
}
};

for (const [runtime, name] of stages) {
assertLiveSource();
const args = runtime === "npm" ? [npmExecPath, "run", name] : [name];
const status = spawn(process.execPath, args, { cwd, env: stageEnv, stdio: "inherit" });
assertLiveSource();
if (Number.isInteger(status) && status !== 0) return status;
}
return 0;
}

if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) {
process.exitCode = runAcquisitionAudit();
}
2 changes: 1 addition & 1 deletion scripts/acquisition-data-room-integrity-audit.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
writeAcquisitionPrivateFile,
} from "./lib/acquisition-private-output.mjs";

const fullShaPattern = /^[0-9a-f]{40}$/;
const fullShaPattern = /^(?:[0-9a-f]{40}|[0-9a-f]{64})$/;
const releaseTagPattern = /^v(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)(?:-(?:(?:0|[1-9]\d*|[0-9A-Za-z-]*[A-Za-z-][0-9A-Za-z-]*)(?:\.(?:0|[1-9]\d*|[0-9A-Za-z-]*[A-Za-z-][0-9A-Za-z-]*))*))?$/;
const now = new Date().toISOString();
const configuredOutputDir = process.env.NOEMA_DATA_ROOM_OUTPUT_DIR
Expand Down
2 changes: 1 addition & 1 deletion scripts/acquisition-data-room-manifest-secure.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import {
writeAcquisitionPrivateFile,
} from "./lib/acquisition-private-output.mjs";

const fullShaPattern = /^[0-9a-f]{40}$/;
const fullShaPattern = /^(?:[0-9a-f]{40}|[0-9a-f]{64})$/;
const releaseTagPattern = /^v(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)\.(?:0|[1-9]\d*)(?:-(?:(?:0|[1-9]\d*|[0-9A-Za-z-]*[A-Za-z-][0-9A-Za-z-]*)(?:\.(?:0|[1-9]\d*|[0-9A-Za-z-]*[A-Za-z-][0-9A-Za-z-]*))*))?$/;
const now = new Date().toISOString();
const configuredOutputDir = process.env.NOEMA_DATA_ROOM_OUTPUT_DIR
Expand Down
4 changes: 2 additions & 2 deletions scripts/acquisition-readiness-audit.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -486,7 +486,7 @@ function validateLicensingIpEvidence(value) {
if (releaseUnderDiligenceTag && release.tag !== releaseUnderDiligenceTag) {
failures.push("release_rights.tag must match the release under diligence");
}
if (!/^[0-9a-f]{40}$/i.test(String(release.commit_sha ?? ""))) {
if (!/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/i.test(String(release.commit_sha ?? ""))) {
failures.push("release_rights.commit_sha must be a full Git SHA");
}
validateDigestBoundArtifact(release.sbom, "release_rights.sbom", failures);
Expand Down Expand Up @@ -542,7 +542,7 @@ function validateReleasePublicationReceipt(value, expectedTag) {
if (value.source?.tag !== expectedTag) {
failures.push(`source.tag must match release under diligence ${expectedTag}`);
}
if (!/^[0-9a-f]{40}$/i.test(String(value.source?.commitSha ?? ""))) {
if (!/^(?:[0-9a-f]{40}|[0-9a-f]{64})$/i.test(String(value.source?.commitSha ?? ""))) {
failures.push("source.commitSha must be a full SHA");
}
if (value.source?.tag !== `v${value.source?.version ?? ""}`) {
Expand Down
17 changes: 15 additions & 2 deletions scripts/dependency-license-inventory.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import {
import { dirname, normalize, resolve } from "node:path";
import { pathToFileURL } from "node:url";
import { writeAcquisitionPrivateFile } from "./lib/acquisition-private-output.mjs";
import { verifyAcquisitionTrackedFileBytes } from "./lib/acquisition-git-preflight.mjs";
import { readStableRegularFile } from "./lib/stable-file-evidence.mjs";
import { hasDuplicateJsonObjectKeys } from "./normalize-commercial-readiness-evidence.mjs";

Expand Down Expand Up @@ -430,8 +431,20 @@ export function buildDependencyLicenseInventory(
export function generateDependencyLicenseInventory({
lockPath = DEFAULT_LOCK_PATH,
outputPath = DEFAULT_OUTPUT_PATH,
expectedCommitSha = process.env.NOEMA_DATA_ROOM_SOURCE_COMMIT || "",
cwd = process.cwd(),
readLock = readEvidenceFile,
authenticateLock = verifyAcquisitionTrackedFileBytes,
} = {}) {
const lockBytes = readEvidenceFile(lockPath);
const lockBytes = readLock(lockPath);
if (expectedCommitSha) {
authenticateLock({
cwd,
exactHead: expectedCommitSha,
path: lockPath,
bytes: Buffer.from(lockBytes, "utf8"),
});
}
const inventory = buildDependencyLicenseInventory(lockBytes, { sourcePath: lockPath });
assertCanonicalEvidencePath(outputPath, "output");
assertPathParents(outputPath, "output");
Expand Down Expand Up @@ -477,4 +490,4 @@ export function runIfDirect(metaUrl, argv, execute) {
return true;
}

runIfDirect(import.meta.url, process.argv, startCli);
runIfDirect(import.meta.url, process.argv, startCli);
4 changes: 2 additions & 2 deletions scripts/deployment-evidence.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ const EXPECTED_REPOSITORY = "ContextualWisdomLab/noema";
const EXPECTED_WORKER = "noema";
const MAX_INPUT_BYTES = 16 * 1024 * 1024;
const MAX_WRANGLER_RECORDS = 1_000;
const shaPattern = /^[0-9a-f]{40}$/;
const shaPattern = /^(?:[0-9a-f]{40}|[0-9a-f]{64})$/;
const digestPattern = /^[0-9a-f]{64}$/;
const opaqueIdPattern = /^[A-Za-z0-9][A-Za-z0-9._:-]{0,199}$/;
const uuidPattern = /^[0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}$/i;
Expand Down Expand Up @@ -190,7 +190,7 @@ export function buildDeploymentEvidence(input) {
fail(`release tag must be semantic version tag v<version>, received ${releaseTag}`);
}
if (commitSha !== commitShaSource || !shaPattern.test(commitSha)) {
fail("deployment commit SHA must be a canonical 40-character lowercase hexadecimal SHA");
fail("deployment commit SHA must be a canonical lowercase full hexadecimal Git identity");
}
if (!new Set(["production", "staging"]).has(environment)) {
fail(`deployment environment must be production or staging, received ${environment}`);
Expand Down
Empty file modified scripts/external-scheduler-evidence-audit.mjs
100755 → 100644
Empty file.
5 changes: 5 additions & 0 deletions scripts/lib/acquisition-data-room-catalog.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,11 @@ function finalEvidenceFile(id, category, path, validatedBy, statusMeaning) {
*/
export const DATA_ROOM_CATALOG = Object.freeze([
...BASE_DATA_ROOM_CATALOG,
requiredFile(
"acquisition-audit-script",
"automation",
"scripts/acquisition-audit.mjs",
),
requiredFile(
"orchestrator-gateway-contract",
"product",
Expand Down
4 changes: 2 additions & 2 deletions scripts/lib/acquisition-data-room-integrity.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ export const MAX_DATA_ROOM_JSON_BYTES = 2 * 1024 * 1024;
export const MAX_DATA_ROOM_EVIDENCE_BYTES = 32 * 1024 * 1024;
const MAX_ENTRY_COUNT = 256;
const MAX_RELATIVE_PATH_BYTES = 1024;
const fullShaPattern = /^[0-9a-f]{40}$/i;
const fullShaPattern = /^(?:[0-9a-f]{40}|[0-9a-f]{64})$/i;
const sha256Pattern = /^[0-9a-f]{64}$/i;
const canonicalTimestampPattern = /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/;
const unsafeControlPattern = /[\u0000-\u001f\u007f]/;
Expand Down Expand Up @@ -625,7 +625,7 @@ export function materializeDataRoomManifest({
fileSystem = defaultFileSystem,
} = {}) {
if (!fullShaPattern.test(String(commitSha ?? ""))) {
throw new TypeError("commitSha must be the exact 40-character audited Git commit");
throw new TypeError("commitSha must be an exact audited Git commit ID");
}
const entries = catalog.map((expected) => {
if (expected.kind === "file") {
Expand Down
2 changes: 1 addition & 1 deletion scripts/lib/acquisition-deployment-evidence.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ const EXPECTED_SIGNER_WORKFLOW = `${EXPECTED_REPOSITORY}/.github/workflows/cd.ym
const EXPECTED_PREDICATE_TYPE =
"https://contextualwisdomlab.org/attestations/noema-deployment/v1";
const EXPECTED_OIDC_ISSUER = "https://token.actions.githubusercontent.com";
const shaPattern = /^[0-9a-f]{40}$/;
const shaPattern = /^(?:[0-9a-f]{40}|[0-9a-f]{64})$/;
const digestPattern = /^[0-9a-f]{64}$/;
const tagPattern = /^v\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?$/;
const isoCalendarPrefixPattern = /^(\d{4})-(\d{2})-(\d{2})T/;
Expand Down
Loading
Loading