Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 23 additions & 34 deletions .github/workflows/hourly-product-development.yml
Original file line number Diff line number Diff line change
Expand Up @@ -149,15 +149,19 @@ jobs:
contextual-orchestrator for every new product-runtime LLM path. Do not alter
the existing reviewer-agent credential names, trust boundary, or provider route.

Work test-first: add one realistic executable regression or product contract,
run it, and record the expected failure before implementation. Then make the
smallest coherent change that closes the selected gap. Tests must exercise
realistic Noema traffic, operational, security, provenance, or buyer workflows,
including adversarial and failure cases where applicable. Maintain 100% production
statement and branch coverage plus 100% production function coverage. When reviewer
Python changes, maintain 100% line and branch coverage and 100% docstring coverage.
Public modules, classes, functions, methods, properties, inputs, outputs, errors,
and trust boundaries must be understandable to a beginner without reverse-engineering.
The credential-bearing proposer has no shell execution authority. Do not claim that
you executed tests or shell commands. Work test-first at the proposal boundary: add
one realistic executable regression or product contract and describe the expected
pre-implementation RED condition before implementation. Then make the smallest
coherent source change that closes the selected gap. A separate uncredentialed verifier
will execute the proposal on a fresh runner. Treat execution results as pending trusted
verifier evidence until that verifier succeeds. Tests must ultimately exercise realistic
Noema traffic, operational, security, provenance, or buyer workflows, including adversarial
and failure cases where applicable. Maintain 100% production statement and branch coverage
plus 100% production function coverage. When reviewer Python changes, maintain 100% line
and branch coverage and 100% docstring coverage. Public modules, classes, functions,
methods, properties, inputs, outputs, errors, and trust boundaries must be understandable
to a beginner without reverse-engineering.

Where behavior is ambiguous, use current authoritative international standards,
primary official documentation, or peer-reviewed research. Record sources and
Expand All @@ -168,8 +172,11 @@ jobs:
Preserve standalone use and modular MSA integration. If a database boundary is
introduced, every new object must use a descriptive two-word-or-longer snake_case
name. Do not add skipped, ignored, quarantined, or weakened release-gate tests.
Run focused tests and npm run release:verify. Update CHANGELOG.md and all affected
product, architecture, security, operations, API, support, and buyer documentation.
The separate uncredentialed verifier must execute `npm run release:verify`
on the immutable proposal.
Do not execute that command or claim its result in the credential-bearing proposer.
Update CHANGELOG.md and all affected product, architecture, security,
operations, API, support, and buyer documentation.

Follow Semantic Versioning. Change the package version only when the integrated
repository is genuinely release-ready and all immutable release, deployment,
Expand All @@ -184,9 +191,9 @@ jobs:

Leave the working tree with one bounded increment and write PR_MESSAGE.md at the
repository root. Put the pull-request title on the first line, then a body containing
the product gap, design, RED-to-GREEN evidence, complete verification commands and
results, APA 7 sources where applicable, version decision, and explicit residual risk.
A separate uncredentialed verifier will execute the proposal on a fresh runner.
the product gap, design, expected RED condition, implementation rationale, requested
verifier commands, APA 7 sources where applicable, version decision, and explicit residual
risk. A separate uncredentialed verifier will execute the proposal on a fresh runner.
A third credential-bearing publisher will reconstruct the same immutable artifact
without executing proposed code. Existing review → repair → exact-head Checks → merge
governance retains every final decision.
Expand Down Expand Up @@ -270,24 +277,7 @@ jobs:
"question": "deny",
"webfetch": "deny",
"websearch": "deny",
"bash": {
"*": "allow",
"curl *": "deny",
"wget *": "deny",
"ssh *": "deny",
"scp *": "deny",
"nc *": "deny",
"ncat *": "deny",
"socat *": "deny",
"git clone *": "deny",
"git fetch *": "deny",
"git pull *": "deny",
"git commit *": "deny",
"git push *": "deny",
"git tag *": "deny",
"git remote *": "deny",
"gh *": "deny"
}
"bash": "deny"
},
"provider": {
"nvidia-nim": {
Expand Down Expand Up @@ -391,7 +381,7 @@ jobs:
exit 1
fi

- name: Verify, bound, and export the uncredentialed proposal
- name: Bound and export proposal without executing it
id: proposal
if: steps.gate.outputs.dispatch == 'true' && env.DRY_RUN != 'true'
shell: bash
Expand All @@ -406,7 +396,6 @@ jobs:
exit 0
fi

npm run release:verify
git add -A
git diff --cached --check

Expand Down
84 changes: 84 additions & 0 deletions test/hourly-product-development-nim-shell-isolation.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
import { readFileSync } from "node:fs";
import { describe, expect, it } from "vitest";

const workflow = readFileSync(
".github/workflows/hourly-product-development.yml",
"utf8",
);

function sliceBetween(startMarker: string, endMarker: string): string {
const start = workflow.indexOf(startMarker);
const end = workflow.indexOf(endMarker, start + startMarker.length);
expect(start).toBeGreaterThan(-1);
expect(end).toBeGreaterThan(start);
return workflow.slice(start, end);
}

describe("NVIDIA NIM proposer shell isolation", () => {
it("denies shell execution in the credential-bearing OpenCode policy", () => {
const config = sliceBetween(
" - name: Configure OpenCode for NVIDIA NIM only",
" - name: Run bounded NVIDIA NIM model fallback",
);

expect(config).toContain('"bash": "deny"');
expect(config).not.toContain('"bash": {');
expect(config).not.toContain('"*": "allow",\n "curl *"');
});

it("requires exact verifier instructions without granting proposer execution authority", () => {
const prompt = sliceBetween(
" - name: Prepare bounded commercial-quality task",
" - name: Record dry-run task contract",
);

expect(prompt).toContain(
"The credential-bearing proposer has no shell execution authority.",
);
expect(prompt).toContain(
"A separate uncredentialed verifier will execute the proposal on a fresh runner.",
);
expect(prompt).toContain(
"The separate uncredentialed verifier must execute `npm run release:verify`",
);
expect(prompt).toContain(
"Do not execute that command or claim its result in the credential-bearing proposer.",
);
expect(prompt).not.toContain("run it, and record the expected failure");
expect(prompt).not.toContain("Run focused tests and npm run release:verify");
expect(prompt).not.toContain("complete verification commands and\n results");
});

it("never executes proposed repository code on the credential-bearing proposer runner", () => {
const proposer = sliceBetween(
" propose_product_increment:",
" package_product_increment:",
);
const releaseVerifyCommands = workflow.match(
/^\s+npm run release:verify\s*$/gm,
) ?? [];

expect(proposer).toContain(
" - name: Bound and export proposal without executing it",
);
expect(proposer).not.toContain(
" - name: Verify, bound, and export the uncredentialed proposal",
);
expect(proposer).not.toMatch(/^\s+npm run /m);
expect(releaseVerifyCommands).toHaveLength(1);
});

it("retains executable verification in the fresh uncredentialed job", () => {
const verifier = sliceBetween(
" package_product_increment:",
" publish_product_increment:",
);

expect(verifier).toContain(
"Re-run complete release verification on the fresh runner",
);
expect(verifier).toContain("npm run release:verify");
expect(verifier).not.toContain("NVIDIA_API_KEY");
expect(verifier).not.toContain("NOEMA_MAINTAINER_APP_PRIVATE_KEY");
});
Comment thread
coderabbitai[bot] marked this conversation as resolved.
});
7 changes: 2 additions & 5 deletions test/hourly-product-development-workflow.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -207,11 +207,8 @@ describe("hourly NVIDIA NIM OpenCode product-development workflow", () => {
expect(workflow).toContain('"task": "deny"');
expect(workflow).toContain('"webfetch": "deny"');
expect(workflow).toContain('"websearch": "deny"');
expect(workflow).toContain('"git commit *": "deny"');
expect(workflow).toContain('"git push *": "deny"');
expect(workflow).toContain('"git tag *": "deny"');
expect(workflow).toContain('"git remote *": "deny"');
expect(workflow).toContain('"gh *": "deny"');
expect(workflow).toContain('"bash": "deny"');
expect(workflow).not.toContain('"bash": {');
});

it("fits every candidate, termination grace, cleanup, and final diagnostic inside the proposal-job budget", () => {
Expand Down
Loading