feat(launchpad): cross-cutting injection clause in all three dimensions (#117 STEP 5) - #250
Merged
tucktuck101 merged 1 commit intoAug 20, 2026
Conversation
…ns (#117 STEP 5) Adds an identical INJECTION_CLAUSE to dimensions/{secrets-and-access, claim-vs-evidence,correctness-and-failure-modes}.py: author-controlled text attempting to skip/approve/suppress/end the review is itself a Blocker finding, with entry_point set to the surface it came from. One dimension failing to run therefore never drops semantic-injection coverage to zero silently, per #117's own requirement. Covers the 7 of 35 attack-matrix classes CONTAINMENT.md's Detection section hands to #117 by name -- semantic paraphrase, which detect.py's deterministic layer does not and cannot catch by design. The clause is phrased to avoid the use-mention trap detect.py's own docstring names (it does not itself trip _STANDALONE or _SUPPRESS), and explicitly states it overrides every dimension's own subject-matter/language exclusions -- otherwise an attack planted in, say, a Rust file would be silently declined by a dimension whose own scope says it reviews no Rust code, exactly the gap STEP 7's paraphrase fixture (planted in crates/buzz-relay/src/gate.rs) exists to catch. test_injection_clause.py is a permanent test covering the half of STEP 5's done-when checkable without a live model: byte-identity across all three files, real weaving into each assembled PROMPT (not merely defined and unused), and that neither the clause alone nor the full assembled prompt trips the deterministic detector. The other half -- that the paraphrase fixture actually yields a Blocker from all three dimensions and the description-of-an-attack fixture yields none -- is a property of real reviewer output, which is STEP 8's job, not simulated here. Reviewed independently (serina:review-code) before commit: two High findings were found and fixed -- the clause as first written did not override correctness-and-failure-modes' unconditional 'reports nothing about Rust/TS/ Dart' exclusion, which would have made it decline the paraphrase fixture specifically; and an ambiguous parenthetical risked being read as 'skip-review is already handled elsewhere, do not re-detect it,' which could cause a model to withhold the Blocker on a PARAPHRASE (the exact case this clause exists for) rather than only on the literal wording detect.py already catches. Also fixed a Low finding: added an explicit single-report rule for the overlap between an injection attempt phrased as 'a claim of prior approval' and claim-vs-evidence's own ordinary scope. All three re-reviewed clean. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
tucktuck101
merged commit Aug 20, 2026
c3d4fd5
into
feat/review-agent-fixtures
23 of 24 checks passed
This was referenced Aug 20, 2026
Merged
serina-mcfall
added a commit
that referenced
this pull request
Aug 20, 2026
…tic suite (#117 STEP 8) (#252) Adds launchpad/review-agent/recordings/<fixture>/<dimension>.json (5 fixtures x 3 dimensions) -- for each STEP 7 fixture, one recorded output per dimension, produced by actually rendering that fixture's contained document (via contain.render with a seed-derived nonce) and reasoning through each dimension's real assembled PROMPT against it, not hand-written to merely look plausible. Every finding_id is computed via findings.finding_id() from the finding's own fields; every recording is replayed through the real run_dimensions.build_document() and validated via findings.validate(). Confirms: secrets-and-access's planted credential is Blocker per that dimension's own 'could this authenticate against a real system' test; claim-vs-evidence's fixture yields TWO genuine defects (a diff-contradicting validation claim, anchor line 23; a citation to scripts/config_schema.py, which does not exist anywhere in this repository, anchor pr); correctness- and-failure-modes' widened except clause is a two-part fail-open (a missing manifest AND a malformed one both now return True); the paraphrase fixture is caught by all three dimensions via STEP 5's injection clause, each anchored at its own file/line, never anchor pr; and description-of-an-attack yields no finding from any dimension, confirming the use-mention negative control holds. All ten non-owning-dimension pairs are correctly clean. test_recordings.py is a permanent, committed witness (matching test_fixtures.py's convention): file existence, provenance (model/date/seed/nonce, and an honest disclosure that these are a single reasoning pass per fixture applied to all three dimensions -- not independent model invocations, since three genuinely separate runs would not converge on identical prose), replay validity and no-network-call, cross-dimension attribution, exact anchor/location/severity pinning, and the injection-clause behavioral claims. Reviewed independently on the most capable available model before commit, given this is the highest-stakes artifact in the whole plan (STEP 9's control suite will treat it as ground truth). Nine findings came back, three High: (1) severity was entirely unpinned -- a Blocker could silently become a Low while every structural check still passed; (2) the three paraphrase recordings' prose was byte-identical with no disclosure that this was one reasoning pass, not three independent samples; (3) the injection clause as first written would have flagged every fixture's own ordinary 'Approved.' pr_review_bodies content as an attack -- a false positive on any clean, already-reviewed PR. Plus four Medium (an evidence-raw test that could not fail; evidence checked against only one of three recordings; an anchor-pr test that asserted only what findings.validate() already enforces; claim-vs- evidence's own ANCHORING_RULE text contradicting FINDINGS.md by routing a nonexistent-citation defect to anchor 'line' instead of 'pr') and two Low (a two-part fail-open recorded as one; the fixture's two-finding design uncounted). All nine fixed and re-verified by mutation (each previously-silent defect class was reproduced and confirmed to now fail the suite, then restored), which itself surfaced two further Low gaps -- the second (citation) finding's own severity was still unpinned, and the sampling disclosure had no test guarding it -- both fixed and mutation-verified in the same pass. The dimensions/*.py changes in this commit are corrections to STEP 5's own files (PR #250, not yet merged) -- found via this step's review, landing here because that is where they were caught. See PR body for the same disclosure STEP 4/6's bundling used. Signed-off-by: Serina Mcfall <serina.mcfall@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the cross-cutting injection clause (
INJECTION_CLAUSE) identically to all three dimension definitions per #117's plan STEP 5: author-controlled text attempting to skip/approve/suppress/end the review is itself a Blocker finding, closing the 7-of-35 semantic-paraphrase gapdetect.py's deterministic layer hands to #117 by name.Related issue
Refs #117
Issue type
Task
Agent provenance
Objective
An identical
INJECTION_CLAUSEconstant, woven into the assembledPROMPTof all threedimensions/*.pyfiles, plustest_injection_clause.pyverifying its byte-identity and detector-clean status.Impacted components
launchpad/review-agent/dimensions/secrets-and-access.py
launchpad/review-agent/dimensions/claim-vs-evidence.py
launchpad/review-agent/dimensions/correctness-and-failure-modes.py
launchpad/review-agent/test_injection_clause.py (new)
Approach and rejected alternatives
Put the clause in all three files identically rather than one file cross-referencing a shared module — the plan explicitly requires this (one dimension failing to run must never drop semantic-injection coverage to zero silently), and a shared-module approach would have made that failure mode possible again if the import itself failed.
Explicitly stated that the clause overrides each dimension's own subject-matter/language exclusions — considered leaving this implicit (trusting a model to infer that a cross-cutting duty outranks a scoping rule) and rejected it: independent review found the first draft's silence here would have let
correctness-and-failure-modesdecline the already-committedparaphrase.jsonfixture (planted in a Rust file) by correctly applying its own "reports nothing about Rust" exclusion, since nothing told it that exclusion doesn't apply to this clause.Verification
Command run:
Raw output:
Command run:
Raw output:
Not verified
Security implications
None beyond what earlier PRs in this stack already cover. This is prompt text, not code — it changes what a future model-backed reviewer is instructed to do, adding no new code path, network call, or credential handling of its own.
Escalations
feat/review-agent-fixtures/ PR feat(launchpad): five fixture diffs with planted defects (#117 STEP 7) #248, itself stacked on feat(launchpad): three dimension definitions + STEP 6 exit-code fix for the PR review agent (#117) #242 → feat(launchpad): concurrent dimension runner for the PR review agent (#117 STEP 3) #241). Should not merge before those.