Repository navigation
feat: implement issue #1105 — [Phase 2] Rewire the security-audit + single-review tier prompts to consume pre-fed context #1163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
07c4383
feat: implement issue #1105 — [Phase 2] Rewire the security-audit + s…
donpetry-bot a95c789
fix(reviews): address review comments [skip ci-relay]
donpetry-bot 536757e
chore: dev-lead update (review-changes) [skip ci-relay]
donpetry-bot 88e09e6
fix(reviews): address review comments [skip ci-relay]
donpetry-bot File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,140 @@ | ||
| #!/usr/bin/env bats | ||
| # Structural guard for the tier-3 audit + single-review pre-fed-context rewire | ||
| # (epic #1101, Story 5 / #1105). | ||
| # | ||
| # Story 2 (#1103) persists the FULL diff + superset metadata ONCE to SHA-bound | ||
| # files exposed as PR_CONTEXT_DIFF_FILE / PR_CONTEXT_METADATA_FILE (gated | ||
| # DEFAULT-OFF behind PREFETCH_CONTEXT_ENABLED). This story rewires | ||
| # prompts/security-audit.md and prompts/single-review.md to CONSUME those files | ||
| # instead of re-running `gh pr view` / `gh pr diff` when they are present and | ||
| # fresh, while staying byte-identical when the flag is off (files absent), and | ||
| # keeping their dynamic `gh api` / MCP / LSP steps intact. | ||
| # | ||
| # These checks are deterministic and offline — the live single/audit smoke run | ||
| # (AC #5) needs the real cascade engine. They guard against the pre-fed section | ||
| # being silently removed or hollowed out by a future edit/template sync, the same | ||
| # silent-revert regression class the repo defends against elsewhere (#655, #823). | ||
|
|
||
| setup() { | ||
| ROOT="$(cd "$BATS_TEST_DIRNAME/.." && pwd)" | ||
| AUDIT="$ROOT/prompts/security-audit.md" | ||
| SINGLE="$ROOT/prompts/single-review.md" | ||
| ACTION="$ROOT/prompts/cascade-action.md" | ||
| } | ||
|
|
||
| # Extract the body of the "Pre-fed PR context" section (from its heading up to | ||
| # the next `## ` heading) so the pre-fed-specific assertions cannot accidentally | ||
| # match text elsewhere in the prompt. | ||
| _prefed_section() { | ||
| [ -f "${1:-}" ] || return 1 | ||
| awk ' | ||
| /^##[[:space:]]+Pre-fed PR context/ { insec=1; next } | ||
| insec && /^##[[:space:]]/ { insec=0 } | ||
| insec { print } | ||
| ' "$1" | ||
| } | ||
|
|
||
| # Extract the body of an arbitrary named section (heading word(s) passed as $1, | ||
| # file as $2) so fallback-step assertions cannot accidentally match prose in the | ||
| # Pre-fed PR context section that also mentions `gh pr view` / `gh pr diff`. | ||
| _section() { | ||
| local heading="$1" file="$2" | ||
| [ -f "${file:-}" ] || return 1 | ||
| awk -v pat="^##[[:space:]]+${heading}" ' | ||
| $0 ~ pat { insec=1; next } | ||
| insec && /^##[[:space:]]/ { insec=0 } | ||
| insec { print } | ||
| ' "$file" | ||
| } | ||
|
|
||
| # --- security-audit.md ------------------------------------------------------ | ||
|
|
||
| @test "audit prompt has a Pre-fed PR context section" { | ||
| grep -Eq '^##[[:space:]]+Pre-fed PR context' "$AUDIT" | ||
| } | ||
|
|
||
| @test "audit pre-fed section references both context env vars" { | ||
| section="$(_prefed_section "$AUDIT")" | ||
| grep -q 'PR_CONTEXT_METADATA_FILE' <<<"$section" | ||
| grep -q 'PR_CONTEXT_DIFF_FILE' <<<"$section" | ||
| } | ||
|
|
||
| @test "audit pre-fed section says to read the files INSTEAD of gh, else fall back" { | ||
| # The 'if set, read instead of gh; else run gh as before' contract keeps the | ||
| # flag-off path byte-identical. | ||
| section="$(_prefed_section "$AUDIT")" | ||
| grep -Eiq 'instead of' <<<"$section" | ||
| grep -Eiq 'fall back|else|otherwise' <<<"$section" | ||
| } | ||
|
|
||
| @test "audit pre-fed section verifies the PR_HEAD_SHA freshness stamp" { | ||
| section="$(_prefed_section "$AUDIT")" | ||
| grep -q 'PR_HEAD_SHA' <<<"$section" | ||
| grep -Eiq 'pr_head_sha|stamp|match|fresh' <<<"$section" | ||
| } | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
|
|
||
| @test "audit prompt retains its dynamic gh api standards fetch" { | ||
| # The org-standards fetch (CONTRIBUTING/AGENTS/CODEOWNERS via gh api) is | ||
| # DYNAMIC and must survive the rewire — it is not part of the pre-fed context. | ||
| grep -q 'gh api' "$AUDIT" | ||
| } | ||
|
|
||
| @test "audit prompt still keeps gh pr view / gh pr diff as the fallback fetch" { | ||
| steps="$(_section 'Steps' "$AUDIT")" | ||
| grep -q 'gh pr view' <<<"$steps" | ||
| grep -q 'gh pr diff' <<<"$steps" | ||
| } | ||
|
|
||
| # --- single-review.md ------------------------------------------------------- | ||
|
|
||
| @test "single-review prompt has a Pre-fed PR context section" { | ||
| grep -Eq '^##[[:space:]]+Pre-fed PR context' "$SINGLE" | ||
| } | ||
|
|
||
| @test "single-review pre-fed section references both context env vars" { | ||
| section="$(_prefed_section "$SINGLE")" | ||
| grep -q 'PR_CONTEXT_METADATA_FILE' <<<"$section" | ||
| grep -q 'PR_CONTEXT_DIFF_FILE' <<<"$section" | ||
| } | ||
|
|
||
| @test "single-review pre-fed section says to read the files INSTEAD of gh, else fall back" { | ||
| section="$(_prefed_section "$SINGLE")" | ||
| grep -Eiq 'instead of' <<<"$section" | ||
| grep -Eiq 'fall back|else|otherwise' <<<"$section" | ||
| } | ||
|
|
||
| @test "single-review pre-fed section verifies the PR_HEAD_SHA freshness stamp" { | ||
| section="$(_prefed_section "$SINGLE")" | ||
| grep -q 'PR_HEAD_SHA' <<<"$section" | ||
| grep -Eiq 'pr_head_sha|stamp|match|fresh' <<<"$section" | ||
| } | ||
|
|
||
| @test "single-review prompt keeps its dynamic MCP secret scan + incremental compare" { | ||
| # These are beyond the pre-fed metadata/diff and must remain dynamic. | ||
| grep -q 'run_secret_scanning' "$SINGLE" | ||
| grep -q 'compare/' "$SINGLE" | ||
| } | ||
|
|
||
| @test "single-review prompt still keeps gh pr view / gh pr diff as the fallback fetch" { | ||
| ctx="$(_section 'Context-gathering' "$SINGLE")" | ||
| grep -q 'gh pr view' <<<"$ctx" | ||
| grep -q 'gh pr diff' <<<"$ctx" | ||
| } | ||
|
|
||
| @test "single-review prompt preserves the draft / head-sha-changed skip checks" { | ||
| # The step-1 skip guards must not be lost when rewiring the fetch — they gate | ||
| # off the metadata regardless of whether it was pre-fed or fetched live. | ||
| grep -q 'isDraft' "$SINGLE" | ||
| grep -q 'head-sha-changed' "$SINGLE" | ||
| } | ||
|
|
||
| # --- cascade-action.md (AC #4: confirmed out of scope) ---------------------- | ||
|
|
||
| @test "cascade-action prompt is documented as not fetching the diff (out of scope)" { | ||
| # It synthesizes prior verdicts from $FINAL_RESULT and never fetches the diff, | ||
| # so it needs no pre-fed-context rewire — documented explicitly per AC #4. | ||
| grep -Eiq 'does not fetch|no.*fetch|out of scope|synthes' "$ACTION" | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
| # Verify implementation: cascade-action must contain no live fetch commands. | ||
| ! grep -q 'gh pr view' "$ACTION" | ||
| ! grep -q 'gh pr diff' "$ACTION" | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.