Skip to content
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/aw/actions-lock.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,10 @@
"version": "v7.0.1",
"sha": "043fb46d1a93c77aae656e7c1c64a875d1fc6a0a"
},
"github/gh-aw-actions/setup@v0.82.14": {
"github/gh-aw-actions/setup@v0.83.4": {
"repo": "github/gh-aw-actions/setup",
"version": "v0.82.14",
"sha": "b6d1443e05b8716267fa19425b99aa4f12006b4a"
"version": "v0.83.4",
"sha": "e89c65e17eb281bbd5ff2ff9e9199a03e96654c7"
}
}
}
406 changes: 406 additions & 0 deletions .github/scripts/CiScanMutation.Tests.ps1

Large diffs are not rendered by default.

42 changes: 19 additions & 23 deletions .github/scripts/CiScanReconcile.Core.Tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -570,7 +570,7 @@ Describe 'Set-CiScanStateMarker' {
# calls it. The READ side is live -- `Get-CiScanStateMarker` runs at
# Invoke-CiScanReconcile.ps1:1428 -- so the state marker is consumed but never
# produced. Consequence, measured below in 'never becomes a close candidate
# without observation state': an issue with no marker stops at
# without observation state': an issue with no state marker stops at
# `awaiting-canonical-data`, so `candidate` is unreachable in production today
# regardless of mode. That is a second safety property independent of report-only,
# and reviewers should know the N-consecutive-absence criterion has never executed
Expand Down Expand Up @@ -2204,17 +2204,11 @@ true -- the static invariants they power are the strongest tests in this file.
}
}

Describe 'The fingerprint marker never survives compilation into the agent prompt' {
# WHY THIS EXISTS: the reconciler's entire eligibility model assumes issues can be
# keyed by a canonical fingerprint marker. Today none can be, and the reason is not
# the one previously recorded. The marker template IS in each scanner's source `.md`
# and is ABSENT from every compiled `.lock.yml`, so the agent is never shown it and
# cannot emit it. Output-side sanitization is not needed to explain the absence.
#
# If this test fails, gh-aw has changed and the template now reaches the prompt.
# That is good news and a REQUIRED REVIEW: markers will start appearing, issues
# become keyable, and `candidate` stops being unreachable. Re-read Gate 3 in
# CiScanReconcile.Core.ps1 and the enforcement rollout before updating this test.
Describe 'Fingerprint marker ownership stays in the trusted publisher' {
# gh-aw strips literal HTML comments from prompt text, so correctness must not depend
# on the agent seeing or emitting a marker template. Both scanner twins instead run
# the trusted validator before their publisher, then the compiled publisher requires
# the exact marker derived from each validated manifest fingerprint.
BeforeAll {
$script:WfDir = Join-Path (Split-Path -Parent $PSScriptRoot) 'workflows'
$script:Tmpl = 'ci-scan-fingerprint: {FINGERPRINT}'
Expand All @@ -2232,26 +2226,28 @@ Describe 'The fingerprint marker never survives compilation into the agent promp
}
}

It 'finds at least one scanner source/lock pair to judge' {
# Anti-vacuity floor: without this, a renamed workflow or a partial checkout
# would make every assertion below iterate an empty set and pass green.
It 'finds both scanner source/lock pairs to judge' {
# Exact anti-vacuity guard: a renamed or omitted twin must fail rather than
# silently reducing this invariant to the workflow that remains.
@($script:Pairs).Count |
Should -BeGreaterThan 0 -Because 'the assertions below are vacuous with no pair to read'
Should -Be 2 -Because 'publisher ownership must be pinned for both scanner twins'
}

It 'still carries the marker template in the scanner source' {
# Control for the assertion that follows: proves the needle is findable at all,
# so a zero count in the lock means "stripped", not "wrong search string".
It 'does not ask the agent to emit the literal marker template' {
foreach ($p in $script:Pairs) {
([regex]::Matches($p.Md, [regex]::Escape($script:Tmpl))).Count |
Should -BeGreaterThan 0 -Because "$($p.Stem).md must still instruct the agent to emit a marker"
Should -Be 0 -Because "$($p.Stem).md must keep marker ownership out of the prompt"
([regex]::Matches($p.Lock, [regex]::Escape($script:Tmpl))).Count |
Should -Be 0 -Because "$($p.Stem).lock.yml must not depend on a stripped prompt template"
}
}

It 'loses that template in the compiled lock, which is why no issue can be keyed' {
It 'compiles trusted validation before publisher-side exact-marker checks' {
foreach ($p in $script:Pairs) {
([regex]::Matches($p.Lock, [regex]::Escape($script:Tmpl))).Count |
Should -Be 0 -Because "$($p.Stem).lock.yml reaching the agent with the template would make issues keyable and change Gate 3's premise"
$validator = $p.Lock.IndexOf('run: .github/scripts/Validate-CiScanManifest.ps1')
$publisherMarker = $p.Lock.IndexOf('const exactMarker = `<!-- ci-scan-fingerprint: ${issue.Fingerprint} -->`;')
$validator | Should -BeGreaterThan 0 -Because "$($p.Stem).lock.yml must invoke the trusted validator"
$publisherMarker | Should -BeGreaterThan $validator -Because "$($p.Stem).lock.yml must check the trusted marker after validation"
}
}
}
21 changes: 9 additions & 12 deletions .github/scripts/CiScanReconcile.Core.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -847,8 +847,9 @@ function Set-CiScanStateMarker {
NOTE: there is no such caller yet. This function has no production invocation --
only the read side (`Get-CiScanStateMarker`, live at Invoke-CiScanReconcile.ps1)
currently runs, so the ci-scan-state marker is consumed but never produced. The
practical effect is that no open issue carries a marker, and a markerless issue
stops at `awaiting-canonical-data` / `no-observation-state-recorded`, so the
practical effect is that no open issue carries a state marker. Even an issue with
the publisher-owned fingerprint marker stops at `awaiting-canonical-data` /
`no-observation-state-recorded`, so the
N-consecutive-absence criterion has never executed end to end and `candidate` is
unreachable in production regardless of mode. That is a safety property
independent of report-only, and it is deliberate for now: wiring a writer is what
Expand Down Expand Up @@ -1285,16 +1286,12 @@ function Get-CiScanIssueVerdict {
}

# --- Gate 3: canonical data required -------------------------------------
# The entire current backlog stops here, and will keep stopping here. The marker
# template lives in the scanner's source `.md` (twice) but survives into NO compiled
# `.lock.yml` -- both twins, before and after #36848. Reproduce:
# git show <ref>:.github/workflows/ci-status-net11.md | grep -c 'ci-scan-fingerprint: {FINGERPRINT}' -> 2
# git show <ref>:.github/workflows/ci-status-net11.lock.yml | grep -c 'ci-scan-fingerprint: {FINGERPRINT}' -> 0
# So the agent is never SHOWN the template and cannot emit it. That alone explains
# every markerless issue; output-side sanitization is not needed to explain the data
# and is untested here -- unobservable while nothing is emitted to sanitize. Do not
# "fix" this by bypassing safe outputs: the loss is upstream of them.
# Closing an issue we cannot key is exactly the failure mode this design prevents.
# The legacy backlog stops here and remains ineligible. New scanner payloads are
# marker-free while agent-authored, then the trusted validator derives and injects
# the canonical fingerprint marker from the validated manifest before the publisher
# re-validates the exact body. That makes new issues keyable without trusting prompt
# emission, but it deliberately does not retrofit old markerless issues. Closing an
# issue we cannot key is exactly the failure mode this gate prevents.
if ($null -eq $fp) {
$verdict.Decision = 'awaiting-canonical-data'
$verdict.Reason = 'no-canonical-fingerprint-marker'
Expand Down
Loading
Loading