Skip to content

feat(governance): 新增風險比例審查 shadow mode - #483

Merged
monkey1sai merged 7 commits into
mainfrom
feat/hermes-risk-proportional-review-shadow
Aug 10, 2026
Merged

feat(governance): 新增風險比例審查 shadow mode#483
monkey1sai merged 7 commits into
mainfrom
feat/hermes-risk-proportional-review-shadow

Conversation

@monkey1sai

@monkey1sai monkey1sai commented Aug 10, 2026

Copy link
Copy Markdown
Owner

變更摘要

新增 Hermes 風險比例審查的 PR-A shadow-mode 實作,提供 deterministic classifier、bounded packet/result/loop contracts、20-case golden corpus 與本機 advisory CLI。此 PR 不變更 merge authority、GitHub workflow、CODEOWNERS 或 branch protection。

修改原因

README_APPLY.md 的既有 contract,建立可序列化、exact-head bound、fail-closed 且有明確 context/retry budget 的審查核心,讓後續歷史 calibration 與 advisory integration 有可驗證基線。

主要變更

  • 新增 risk-proportional-review policy、Draft-07 schemas、deterministic classifier、packet/result validator 與 bounded loop。
  • 新增 20-case golden corpus、standalone sample 與 48 個 focused tests。
  • 強化 repository path 大小寫一致性、case-only duplicate 拒絕、unknown impact fail-closed 與 evidence ref grammar。
  • 保證 .github/CODEOWNERS 與 case variants 維持 critical_authorityhuman_critical 與 governance reviewer floor。
  • 限制 CLI 讀取必須 realpath containment;輸出只能 exclusive-create 於真實 artifacts/ 目錄,拒絕 symlink/junction 越界與覆寫。
  • Packet builder 回傳前自我驗證;最大 512-byte evidence ref 不再放大為超限 gap/question。
  • 補充 PR-A/PR-B trust boundary:hosted repository identity、policy/manifest digest 與 artifact provenance 綁定留待 trusted adapter。
  • 未修改 runtime API、資料庫 schema、migration、環境變數、部署流程、Webhook 或排程。

AI Coding Governance

Item Result
Change lane G
Behavior contract changed yes
Linked issue none; requirement supplied by existing README_APPLY.md contract
Requirement source existing contract
CODEOWNERS / owner review requested
GitNexus evidence pre-change impact HIGH; security hardening staged risk=medium (5 files/32 symbols/3 flows); CODEOWNERS follow-up staged risk=low (3 files/6 symbols/0 flows); branch compare risk=high (12 files/182 symbols/7 flows)
Browser E2E evidence not user-facing
Agent workflow changed? no; shadow CLI only, no workflow or merge-authority change
Required checks expected CI / Agent Governance / PR Metadata Contract

Self-Referential Bootstrap

Item Result
Self-referential bootstrap no
Bootstrap ledger entry not applicable
Bootstrap reason not applicable; machine scope found zero mechanism paths

驗證方式

  • node --check scripts/lib/risk-proportional-review.mjs:PASS。
  • node --check scripts/dev/review-risk-shadow.mjs:PASS。
  • node --test scripts/tests/test-review-risk.mjs:48/48 PASS。
  • node --test --experimental-test-coverage scripts/tests/test-review-risk.mjs:48/48 PASS;all files line 95.65%、branch 85.98%、functions 99.00%;核心 library line 97.13%、branch 84.07%、functions 98.91%。
  • node scripts/dev/review-risk-shadow.mjs replay --corpus scripts/tests/fixtures/review-risk-golden.json:20/20 PASS,authority=advisory_shadow
  • PowerShell Test-Json -SchemaFile:sample、golden corpus、policy 三組 Draft-07 validation PASS。
  • pwsh -NoProfile -NonInteractive -File scripts/tests/test-self-referential-bootstrap.ps1:PASS。
  • pwsh -NoProfile -NonInteractive -File scripts/tests/test-agent-governance-check.ps1:PASS。
  • 512-byte evidence boundary probe:ref=512、derived gap=18、question=52,packet validation PASS。
  • .github/CODEOWNERS.GITHUB/CODEOWNERS regression:均為 critical_authority / human_critical 且包含 governance specialist。
  • git diff --cached --check:PASS。
  • Get-SelfReferentialMechanismPaths:12 changed paths、0 mechanism paths、bootstrap required=false。
  • Get-WindowsVerificationScope:Tier 0、id=none、Required=false。
  • 獨立 security reviewer:兩個原 HIGH 與後續 packet DoS 均無法重現,最終 recommendation=accept
  • PR feat(governance): 新增風險比例審查 shadow mode #483 exact-head local preflight(a6dc29c):PASS;PR review agent 無 blocker。
  • Copilot 3 個 inline findings 已於 a6dc29c 修正並逐項回覆。

風險與影響

  • 完整 branch 的 GitNexus risk=high,原因是新增 classifier/packet/loop 與 7 條 execution flows;已用 48 tests、20-case replay、schema、governance tests 與獨立 security review 補強。
  • PR-A 仍是 local shadow contract:caller-supplied repository/hash 不等於 hosted provenance,已明確列為 PR-B trusted-adapter 工作。
  • authority=advisory_shadowmerge_authority=false;沒有 production、部署、frontend、Kit/WebRTC 或使用者流程影響。

回滾方式

合併前可關閉本 PR 並刪除 branch;合併後以單一 revert PR 撤回本 PR 的 merge commit。此變更沒有 migration、外部狀態或 production rollback 步驟。

後續建議

  • PR-B:歷史 replay/calibration,並在 trusted adapter 綁定 hosted repository、policy/manifest digest、packet origin 與 artifact provenance。
  • PR-C:repo-owned advisory integration;屬 self-referential change,必須依 ledger/fixpoint 規範處理。
  • PR-D:僅在 calibration 與 owner 明確批准後評估 enforcement。

Known Risks

  • Hosted provenance 與 cross-process authentication 尚未實作;本 PR 不宣稱可作 merge gate。
  • governance-base-audit 在受保護路徑 PR 上必須等候 monkey1sai-blip 對最新 exact head 的獨立 APPROVED review;不得由本 PR 作者或 bot 取代。
  • 未執行 browser E2E、deploy 或 Windows platform tier 測試,因 machine-derived changed-path scope 分別為 not user-facing、not deploy、Windows Tier 0。

Summary by CodeRabbit

  • New Features

    • Added an advisory, risk-proportional review workflow that classifies changes into four review levels.
    • Added bounded evidence collection, review packets, escalation loops, and fail-closed validation.
    • Added a read-only command-line tool for evaluations, replaying scenarios, and validating review results.
  • Documentation

    • Added setup, operation, rollout, design rationale, verification, and replay documentation.
  • Tests

    • Added comprehensive policy, security, determinism, validation, and golden-scenario coverage.

monkey1sai and others added 2 commits August 10, 2026 12:07
Apply the hermes-review-shadow patch from origin/main 89ff9c8:
deterministic risk classifier, bounded review packet, read-only
reviewer result contract, bounded loop, 20-case golden corpus,
32 tests, and idea-genesis / Hermes adapter docs.

authority=advisory_shadow, merge_authority=false. No GitHub
workflow, CODEOWNERS, branch protection, PR gate, verification
manifest, or skill manifest changes.

Verified: node --check x2 OK; node --test 32/32 pass; golden
replay 20/20 pass; git diff --check clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TR36xLMWPrGAVFyUDGGFLW
Copilot AI balanced review requested due to automatic review settings August 10, 2026 05:52
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@monkey1sai, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 55 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ac42205a-ec19-4509-8ec8-28f51b00b03a

📥 Commits

Reviewing files that changed from the base of the PR and between 4efa0aa and 10cda2a.

📒 Files selected for processing (2)
  • docs/agent-tooling/hermes-risk-proportional-review.md
  • docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md
📝 Walkthrough

Walkthrough

Adds a deterministic, advisory-only Hermes risk-proportional review system. It defines policy contracts, classifies risk, builds bounded packets, validates reviewer results, advances evidence loops, provides a read-only CLI, and adds documentation, fixtures, replay evidence, and tests.

Changes

Risk-proportional review shadow

Layer / File(s) Summary
Policy and control-plane contracts
agent-contracts/*, docs/agent-tooling/..., docs/evidence/hermes-risk-proportional-review-shadow/...
Defines review modes, lane floors, identity bindings, budgets, fail-closed statuses, authority boundaries, rollout controls, and research evidence.
Validation and deterministic classification
scripts/lib/risk-proportional-review.mjs
Validates policies and inputs, derives risk attributes, evaluates evidence, selects advisory review modes, and emits identity-bound decisions.
Packet, result, and loop controls
scripts/lib/risk-proportional-review.mjs
Builds bounded packets, validates reviewer results, fingerprints evidence, and stops or advances review loops under fixed budgets.
Read-only shadow CLI
scripts/dev/review-risk-shadow.mjs
Adds commands for evaluation, packet creation, loop advancement, result validation, corpus replay, and policy hashing with secure artifact handling.
Schemas, fixtures, replay, and verification
scripts/tests/review-risk.schema.json, scripts/tests/fixtures/*, scripts/tests/test-review-risk.mjs, docs/evidence/...
Adds schemas, sample and golden cases, replay results, verification records, and tests for classification, integrity, security, reviewer roles, bounded loops, and CLI behavior.

Estimated code review effort: 5 (Critical) | ~120 minutes

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant ReviewEngine
  participant Evidence
  participant Reviewer
  CLI->>ReviewEngine: Submit repository input and policy
  ReviewEngine->>Evidence: Evaluate exact-head evidence
  Evidence-->>ReviewEngine: Return evidence status
  ReviewEngine-->>CLI: Return advisory decision
  CLI->>ReviewEngine: Build bounded review packet
  ReviewEngine-->>Reviewer: Provide packet
  Reviewer->>ReviewEngine: Submit validated result
  ReviewEngine-->>CLI: Advance, hold, or finish loop
Loading

Possibly related PRs

Suggested reviewers: monkey1sai-blip

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the main change: adding a risk-proportional review shadow mode for governance.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/hermes-risk-proportional-review-shadow

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a PR-A shadow-mode implementation of a "Hermes risk-proportional review" control plane. It introduces a deterministic, self-validating risk classifier that maps bounded change facts (topology, consequence, evidence strength, trust surface, detectability, horizon) into one of four review modes, then compiles a byte-bounded, exact-head-bound review packet and drives a bounded review loop. Everything is advisory_shadow with merge_authority = false; it deliberately does not touch workflows, CODEOWNERS, branch protection, or the verification manifest, so it fits the repo's self-referential bootstrap governance as a report-only baseline.

Changes:

  • New policy contract + Draft-07 schemas, a deterministic classifier, packet/result validators, and a bounded loop in scripts/lib/risk-proportional-review.mjs.
  • A read-only advisory CLI (scripts/dev/review-risk-shadow.mjs) with strict realpath containment and exclusive-create output under artifacts/.
  • A 20-case golden corpus, a standalone sample, 48 focused Node tests, plus evidence/design documentation.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
scripts/lib/risk-proportional-review.mjs Core classifier, packet builder, result/loop validators, stable hashing — contains the CODEOWNERS pattern bug.
scripts/dev/review-risk-shadow.mjs Advisory CLI with realpath containment and non-overwriting output.
scripts/tests/test-review-risk.mjs 48 focused tests over classifier, packet, result, loop, and CLI.
scripts/tests/review-risk.schema.json Draft-07 schema for input/decision/packet/result/loop/corpus.
scripts/tests/fixtures/review-risk-golden.json 20-case golden corpus of risk shapes.
scripts/tests/fixtures/review-risk-sample.json Standalone sample input for the CLI.
agent-contracts/risk-proportional-review.contract.json The shipped advisory policy.
agent-contracts/risk-proportional-review.contract.schema.json Draft-07 schema for the policy.
docs/agent-tooling/hermes-risk-proportional-review.md Runbook/contract documentation for the capability.
docs/evidence/.../verification-summary.md Verification evidence — reports a stale test count (32 vs 48).
docs/evidence/.../replay-summary.json Golden replay evidence (20/20).
docs/evidence/.../idea-genesis-reconstruction.md Design rationale and decision genealogy.

Key issues found: a functional bug where the .github/CODEOWNERS self-referential pattern can never match because paths are lowercased before regex testing (so CODEOWNERS changes miss the intended human_critical floor), and a stale test count (32 vs 48) in the verification evidence document.


💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md Outdated
Comment thread docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 9

🧹 Nitpick comments (8)
scripts/lib/risk-proportional-review.mjs (2)

1061-1093: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Validate expected.verdict, expected.topology, and expected.consequence as enums.

Line 1077 validates expected.review_mode against MODE_SET and Line 1078 validates expected.specialists_include against SPECIALISTS. The sibling fields verdict, topology, and consequence receive no enum check. A typographical error in a golden fixture, for example topology: "contractal", is reported as a case mismatch rather than as a corpus contract error. The replay then fails with a misleading reason.

Add the enum assertions so a malformed corpus fails validation.

♻️ Proposed refactor
     assertEnum(testCase.expected.review_mode, MODE_SET, `${label}.expected.review_mode`);
+    assertEnum(testCase.expected.verdict, new Set(['advisory_pass', 'advisory_review', 'human_required', 'held', 'blocked']), `${label}.expected.verdict`);
+    assertEnum(testCase.expected.topology, TOPOLOGIES, `${label}.expected.topology`);
+    assertEnum(testCase.expected.consequence, new Set(['low', 'medium', 'high', 'critical']), `${label}.expected.consequence`);
     assertUniqueEnumArray(testCase.expected.specialists_include, SPECIALISTS, `${label}.expected.specialists_include`, { max: 2 });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/lib/risk-proportional-review.mjs` around lines 1061 - 1093, In
replayCorpus, add enum validation for testCase.expected.verdict, topology, and
consequence alongside the existing review_mode assertion, using the
corresponding established enum sets and field labels. Keep the
specialists_include validation and mismatch comparison unchanged so malformed
corpus values fail during validation.

726-728: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Precompute the path priority before sorting.

The comparator on Line 727 calls pathPriority for both operands on every comparison. pathPriority calls pathFacts (Line 683), which allocates a Set and runs 15 regular expressions plus two extra tests per call. validateInput allows up to 1000 changed paths (Line 271), so a single packet build performs roughly 20,000 pathFacts calls and several hundred thousand regex evaluations.

Compute the priority once per entry.

♻️ Proposed refactor
-  const selectedPaths = [...input.changed_paths]
-    .sort((a, b) => pathPriority(a) - pathPriority(b) || a.path.localeCompare(b.path))
-    .slice(0, budgetPolicy.max_changed_paths);
+  const selectedPaths = input.changed_paths
+    .map((entry) => ({ entry, priority: pathPriority(entry) }))
+    .sort((a, b) => a.priority - b.priority || a.entry.path.localeCompare(b.entry.path))
+    .slice(0, budgetPolicy.max_changed_paths)
+    .map(({ entry }) => entry);

Also applies to: 682-690

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/lib/risk-proportional-review.mjs` around lines 726 - 728, Update the
selectedPaths ordering flow and pathPriority/pathFacts helpers to compute each
changed path’s priority once before sorting, retain the path value for the
existing lexical tie-breaker, then sort the precomputed entries and unwrap paths
before applying max_changed_paths. Preserve the current priority and
tie-breaking behavior.
agent-contracts/risk-proportional-review.contract.schema.json (1)

48-63: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider binding mode identity and order in the schema.

The schema accepts four review_modes entries with duplicate id values, arbitrary rank values, and any order. validatePolicy in scripts/lib/risk-proportional-review.mjs (Lines 204-213) is stricter: it requires canonical order, rank === index, max_model_reviewers === 0 for mechanical_only, and human_required === true for human_critical. A policy file can pass the schema and still fail the validator.

Add a tuple constraint so the schema matches the validator.

♻️ Optional tightening with a Draft-07 tuple form
     "review_modes": {
       "type": "array",
       "minItems": 4,
       "maxItems": 4,
-      "items": {
+      "uniqueItems": true,
+      "items": [
+        {"$ref": "`#/definitions/reviewMode`", "properties": {"id": {"const": "mechanical_only"}, "rank": {"const": 0}, "max_model_reviewers": {"const": 0}}},
+        {"$ref": "`#/definitions/reviewMode`", "properties": {"id": {"const": "focused_semantic"}, "rank": {"const": 1}}},
+        {"$ref": "`#/definitions/reviewMode`", "properties": {"id": {"const": "risk_scoped_specialists"}, "rank": {"const": 2}}},
+        {"$ref": "`#/definitions/reviewMode`", "properties": {"id": {"const": "human_critical"}, "rank": {"const": 3}, "human_required": {"const": true}}}
+      ]
+    },
+    "_reviewModeShape": {

Note: a definitions/reviewMode block holding the shared type/required/additionalProperties is needed if you take this route.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@agent-contracts/risk-proportional-review.contract.schema.json` around lines
48 - 63, Update the review_modes schema to enforce the canonical four-entry
order and identity expected by validatePolicy: mechanical_only,
focused_semantic, risk_scoped_specialists, and human_critical. Use a tuple
constraint with per-index properties enforcing each mode’s fixed id, rank,
max_model_reviewers, and human_required values, while preserving the shared
object requirements and disallowing additional properties.
docs/evidence/hermes-risk-proportional-review-shadow/idea-genesis-reconstruction.md (1)

89-90: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Align the claim labels with the taxonomy defined in this file.

Lines 27-29 define direct_fact as a claim directly present in a tracked contract, implementation, test, PR record, or supplied research material. Lines 89-90 and 114-115 label an "initially plausible solution" as direct_fact, but a rejected design alternative is not such a record. Use supported_inference or an explicit "considered alternative" marker for these two entries. This keeps the file consistent with its own evidence discipline.

Also applies to: 114-115

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@docs/evidence/hermes-risk-proportional-review-shadow/idea-genesis-reconstruction.md`
around lines 89 - 90, Update the claim labels for the “Initially plausible
solution” entries near `direct_fact` at both affected locations, since rejected
design alternatives are not direct facts from tracked evidence. Relabel them as
`supported_inference` or use an explicit “considered alternative” marker, while
preserving the solution descriptions and the taxonomy defined earlier in the
file.
scripts/tests/test-review-risk.mjs (1)

47-53: 🩺 Stability & Availability | 🔵 Trivial | ⚡ Quick win

Add a timeout to the CLI subprocess calls.

spawnSync has no timeout. If review-risk-shadow.mjs blocks, the test run hangs instead of failing. A bounded timeout keeps the suite deterministic.

♻️ Proposed change
   return spawnSync(process.execPath, [cliPath, ...args], {
     cwd: repoRoot,
     encoding: 'utf8',
     windowsHide: true,
+    timeout: 30_000,
   });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/tests/test-review-risk.mjs` around lines 47 - 53, Update runShadowCli
to pass a bounded timeout option to spawnSync, ensuring blocked
review-risk-shadow.mjs subprocesses terminate and cause the tests to fail rather
than hang. Preserve the existing command arguments and process options.
scripts/tests/review-risk.schema.json (1)

516-518: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

The Draft-07 schema is a looser mirror of the runtime validators, and nothing keeps the two in sync. scripts/lib/risk-proportional-review.mjs enforces the authoritative constraints in validateInput, validateReviewPacket, and normalizeRepositoryPath. This schema restates a subset of them by hand, so documents that the runtime rejects can still pass schema validation.

  • scripts/tests/review-risk.schema.json#L516-L518: apply the same ^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ pattern and maxLength: 200 to decision.repository that input.repository and packet.repository already use.
  • scripts/tests/review-risk.schema.json#L42-L47: tighten the path pattern to also reject single-dot segments, leading or trailing whitespace, and control characters, matching normalizeRepositoryPath.

Consider adding a test that feeds the negative cases from scripts/tests/test-review-risk.mjs through this schema, so schema and runtime divergence fails the suite.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/tests/review-risk.schema.json` around lines 516 - 518, Update
scripts/tests/review-risk.schema.json at lines 516-518 for decision.repository
to enforce the same repository pattern and maxLength 200 used by
input.repository and packet.repository. Also update
scripts/tests/review-risk.schema.json at lines 42-47 so path rejects single-dot
segments, leading or trailing whitespace, and control characters, matching
normalizeRepositoryPath; add schema coverage for the negative cases from
scripts/tests/test-review-risk.mjs if the existing suite supports it.
docs/evidence/hermes-risk-proportional-review-shadow/replay-summary.json (1)

2-11: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

This summary is hand-maintained and can drift from replayCorpus output.

replayCorpus emits schema_version: "review-risk-replay-report/v1" with a results array. This file declares review-risk-replay-summary/v1 with a flattened cases array, and no definition for it exists in scripts/tests/review-risk.schema.json. Nothing in the test suite compares the two. If a classifier change alters a verdict, this evidence file stays stale and still reports "passed": 20.

Consider generating this file from the CLI replay output, or adding a test that derives cases from replayCorpus(corpus, policy).results and compares it against this file.

I verified the 20 case entries against scripts/tests/fixtures/review-risk-golden.json; all review_mode, verdict, topology, and consequence values match the corpus expected blocks today.

As per coding guidelines: "不得把 docs/ 內任何文件或舊 evidence 當成 runtime/API 已完成證據;當 docs 與實作不一致時,以實作為準".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/evidence/hermes-risk-proportional-review-shadow/replay-summary.json`
around lines 2 - 11, Prevent replay-summary.json from becoming stale by
generating it from replayCorpus(corpus, policy) or adding a test that compares
its cases and aggregate counts with the CLI replay output’s results. Align the
summary schema with the emitted review-risk-replay-report/v1 structure, or
explicitly derive the flattened representation from results, and ensure
classifier verdict changes cause the test or generation step to update or fail.

Source: Coding guidelines

scripts/tests/fixtures/review-risk-golden.json (1)

729-731: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Only credentials is exercised in regulated_data.

The input contract accepts pii, payment, health, audit, customer_model, and other_regulated. No golden case covers these values, so a future change to consequence derivation for those categories would replay clean. Consider adding one case that carries pii or payment and asserts the expected consequence and specialists.

Also applies to: 1104-1106

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/tests/fixtures/review-risk-golden.json` around lines 729 - 731, Add a
golden fixture case alongside the existing regulated_data coverage that uses one
currently untested accepted value, such as "pii" or "payment", and asserts its
expected consequence and specialists. Update both corresponding fixture
sections, including the occurrence near the existing "credentials" entry, so
consequence derivation for these categories is replay-tested.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md`:
- Around line 1-6: Add an explicit document-nature declaration to the header of
the verification summary, identifying it as a working note that records
historical verification evidence. Keep the existing verification date and
authority statements unchanged.
- Around line 20-29: Update
docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md
(lines 20-29) to report the current 48-case suite count, or explicitly mark both
test-count rows as historical evidence. Update
docs/evidence/hermes-risk-proportional-review-shadow/replay-summary.json (lines
2-11) so its cases array is generated from replayCorpus(corpus, policy).results,
or add an executable test comparing the file with that output; keep the evidence
aligned with implementation and test truth.
- Line 20: Update the evidence rows for scripts/tests/test-review-risk.mjs in
verification-summary.md to report the current suite’s 48 passing tests, or
explicitly label the existing 32-test results as historical evidence.

In `@scripts/dev/review-risk-shadow.mjs`:
- Around line 108-118: Update emit so artifact creation remains confined to
artifactsRoot at write time, not only during assertContained and
ensureSafeOutputParent validation. Replace the open(absolute, 'wx') path with an
output mechanism that prevents substituted parent directories from redirecting
the write outside the trusted root, while preserving exclusive creation and
existing stdout behavior.

In `@scripts/lib/risk-proportional-review.mjs`:
- Line 820: Update validateReviewPacket to enforce the packet’s declared policy
caps instead of hardcoded limits: compare selected_paths, evidence refs,
evidence gaps, and questions against the corresponding declared caps already
validated around the packet cap cross-checks. Retain the absolute policy bounds
so standalone validation still rejects values above the supported maxima, while
allowing buildReviewPacket outputs to honor larger valid policy settings.
- Around line 585-599: Update buildQuestions to collect and return every unique
candidate question without enforcing maxQuestions; retain the mechanical_only
behavior and fallback question. Let buildReviewPacket apply the max_questions
slice so its existing overflow detection and budget_exceeded handling can
observe omitted questions, and add omitted_question_count only if the packet
contract requires explicit truncation accounting.
- Around line 739-751: Update the evidence_refs overflow check near
selectedEvidence and selectedQuestions to evaluate the deduplicated selection
result, not input.evidence.length. Compare the count of unique selected refs
against budgetPolicy.max_evidence_refs, or track whether the prioritized
selection was truncated, so packets fitting the unique-ref budget are not marked
budget_exceeded.

In `@scripts/tests/review-risk.schema.json`:
- Around line 516-518: Update decision.repository in the schema to match the
validation constraints used by input.repository and packet.repository: enforce
the owner/name pattern and the same maxLength, while retaining its string type.

In `@scripts/tests/test-review-risk.mjs`:
- Line 851: Replace the junction cleanup’s unlinkSync call with rmSync(linkPath,
{ recursive: true, force: true }) so Windows junction removal cannot mask the
test result or prevent subsequent container and external cleanup.

---

Nitpick comments:
In `@agent-contracts/risk-proportional-review.contract.schema.json`:
- Around line 48-63: Update the review_modes schema to enforce the canonical
four-entry order and identity expected by validatePolicy: mechanical_only,
focused_semantic, risk_scoped_specialists, and human_critical. Use a tuple
constraint with per-index properties enforcing each mode’s fixed id, rank,
max_model_reviewers, and human_required values, while preserving the shared
object requirements and disallowing additional properties.

In
`@docs/evidence/hermes-risk-proportional-review-shadow/idea-genesis-reconstruction.md`:
- Around line 89-90: Update the claim labels for the “Initially plausible
solution” entries near `direct_fact` at both affected locations, since rejected
design alternatives are not direct facts from tracked evidence. Relabel them as
`supported_inference` or use an explicit “considered alternative” marker, while
preserving the solution descriptions and the taxonomy defined earlier in the
file.

In `@docs/evidence/hermes-risk-proportional-review-shadow/replay-summary.json`:
- Around line 2-11: Prevent replay-summary.json from becoming stale by
generating it from replayCorpus(corpus, policy) or adding a test that compares
its cases and aggregate counts with the CLI replay output’s results. Align the
summary schema with the emitted review-risk-replay-report/v1 structure, or
explicitly derive the flattened representation from results, and ensure
classifier verdict changes cause the test or generation step to update or fail.

In `@scripts/lib/risk-proportional-review.mjs`:
- Around line 1061-1093: In replayCorpus, add enum validation for
testCase.expected.verdict, topology, and consequence alongside the existing
review_mode assertion, using the corresponding established enum sets and field
labels. Keep the specialists_include validation and mismatch comparison
unchanged so malformed corpus values fail during validation.
- Around line 726-728: Update the selectedPaths ordering flow and
pathPriority/pathFacts helpers to compute each changed path’s priority once
before sorting, retain the path value for the existing lexical tie-breaker, then
sort the precomputed entries and unwrap paths before applying max_changed_paths.
Preserve the current priority and tie-breaking behavior.

In `@scripts/tests/fixtures/review-risk-golden.json`:
- Around line 729-731: Add a golden fixture case alongside the existing
regulated_data coverage that uses one currently untested accepted value, such as
"pii" or "payment", and asserts its expected consequence and specialists. Update
both corresponding fixture sections, including the occurrence near the existing
"credentials" entry, so consequence derivation for these categories is
replay-tested.

In `@scripts/tests/review-risk.schema.json`:
- Around line 516-518: Update scripts/tests/review-risk.schema.json at lines
516-518 for decision.repository to enforce the same repository pattern and
maxLength 200 used by input.repository and packet.repository. Also update
scripts/tests/review-risk.schema.json at lines 42-47 so path rejects single-dot
segments, leading or trailing whitespace, and control characters, matching
normalizeRepositoryPath; add schema coverage for the negative cases from
scripts/tests/test-review-risk.mjs if the existing suite supports it.

In `@scripts/tests/test-review-risk.mjs`:
- Around line 47-53: Update runShadowCli to pass a bounded timeout option to
spawnSync, ensuring blocked review-risk-shadow.mjs subprocesses terminate and
cause the tests to fail rather than hang. Preserve the existing command
arguments and process options.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 7d357997-950d-4da1-a87a-9da555ca81a5

📥 Commits

Reviewing files that changed from the base of the PR and between 89ff9c8 and b48a321.

📒 Files selected for processing (12)
  • agent-contracts/risk-proportional-review.contract.json
  • agent-contracts/risk-proportional-review.contract.schema.json
  • docs/agent-tooling/hermes-risk-proportional-review.md
  • docs/evidence/hermes-risk-proportional-review-shadow/idea-genesis-reconstruction.md
  • docs/evidence/hermes-risk-proportional-review-shadow/replay-summary.json
  • docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md
  • scripts/dev/review-risk-shadow.mjs
  • scripts/lib/risk-proportional-review.mjs
  • scripts/tests/fixtures/review-risk-golden.json
  • scripts/tests/fixtures/review-risk-sample.json
  • scripts/tests/review-risk.schema.json
  • scripts/tests/test-review-risk.mjs

Comment thread docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md Outdated
Comment thread docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md Outdated
Comment thread scripts/dev/review-risk-shadow.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/tests/review-risk.schema.json
Comment thread scripts/tests/test-review-risk.mjs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b48a321715

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread agent-contracts/risk-proportional-review.contract.schema.json Outdated
Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a6dc29c441

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3fb6b28ffe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs Outdated
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/tests/review-risk.schema.json Outdated
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs Outdated
@monkey1sai
monkey1sai enabled auto-merge (squash) August 10, 2026 07:55

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/agent-tooling/hermes-risk-proportional-review.md`:
- Line 167: Update the topology sentence in the production service path
requirements to say that two or more distinct production service roots “raise
the topology classification” to at least distributed, replacing only the
ambiguous “raise topology” wording.

In
`@docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md`:
- Line 54: Update the topology claim in the production service paths sentence to
state that the two distinct production roots imply the distributed topology, or
explicitly identify the classifier as deriving it; preserve the surrounding
evidence requirements.

In `@scripts/tests/test-review-risk.mjs`:
- Around line 775-795: Update the assertions in the bounded-loop test around
advanceReviewLoop to match the complete error text emitted by validateLoopInput
and fail(), including the risk-proportional-review prefix before the
deterministic-ordering message. Keep the action iteration and existing ordering
validation unchanged.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 19f052cb-7b85-4512-8959-d9e69e61b726

📥 Commits

Reviewing files that changed from the base of the PR and between b48a321 and 4efa0aa.

📒 Files selected for processing (9)
  • agent-contracts/risk-proportional-review.contract.schema.json
  • docs/agent-tooling/hermes-risk-proportional-review.md
  • docs/evidence/hermes-risk-proportional-review-shadow/replay-summary.json
  • docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md
  • scripts/dev/review-risk-shadow.mjs
  • scripts/lib/risk-proportional-review.mjs
  • scripts/tests/fixtures/review-risk-golden.json
  • scripts/tests/review-risk.schema.json
  • scripts/tests/test-review-risk.mjs
🚧 Files skipped from review as they are similar to previous changes (4)
  • docs/evidence/hermes-risk-proportional-review-shadow/replay-summary.json
  • scripts/tests/fixtures/review-risk-golden.json
  • scripts/tests/review-risk.schema.json
  • scripts/lib/risk-proportional-review.mjs

Comment thread docs/agent-tooling/hermes-risk-proportional-review.md Outdated
Comment thread docs/evidence/hermes-risk-proportional-review-shadow/verification-summary.md Outdated
Comment thread scripts/tests/test-review-risk.mjs

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 10cda2add1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs
Comment thread scripts/lib/risk-proportional-review.mjs
@monkey1sai
monkey1sai disabled auto-merge August 10, 2026 09:52

@monkey1sai-blip monkey1sai-blip left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved by monkey1sai-blip (the reviewer account pinned by the repo's merge governance).

Submitted through scripts/blip_review.py — a scripted approval carrying the operator's authority, pinned to head 10cda2add1039df58879ade2443e79298e08ebe5. This is the mechanism the GitHub App cannot satisfy: an App's approving review does not count toward required_approving_review_count.

@monkey1sai
monkey1sai merged commit f1757b2 into main Aug 10, 2026
20 of 35 checks passed
@monkey1sai
monkey1sai deleted the feat/hermes-risk-proportional-review-shadow branch August 10, 2026 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants