Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/aw/actions-lock.json
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,10 @@
"version": "v7.0.1",
"sha": "043fb46d1a93c77aae656e7c1c64a875d1fc6a0a"
},
"github/gh-aw-actions/setup@v0.82.14": {
"github/gh-aw-actions/setup@v0.83.4": {
"repo": "github/gh-aw-actions/setup",
"version": "v0.82.14",
"sha": "b6d1443e05b8716267fa19425b99aa4f12006b4a"
"version": "v0.83.4",
"sha": "e89c65e17eb281bbd5ff2ff9e9199a03e96654c7"
}
}
}
444 changes: 444 additions & 0 deletions .github/scripts/CiScanMutation.Tests.ps1

Large diffs are not rendered by default.

42 changes: 19 additions & 23 deletions .github/scripts/CiScanReconcile.Core.Tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -570,7 +570,7 @@ Describe 'Set-CiScanStateMarker' {
# calls it. The READ side is live -- `Get-CiScanStateMarker` runs at
# Invoke-CiScanReconcile.ps1:1428 -- so the state marker is consumed but never
# produced. Consequence, measured below in 'never becomes a close candidate
# without observation state': an issue with no marker stops at
# without observation state': an issue with no state marker stops at
# `awaiting-canonical-data`, so `candidate` is unreachable in production today
# regardless of mode. That is a second safety property independent of report-only,
# and reviewers should know the N-consecutive-absence criterion has never executed
Expand Down Expand Up @@ -2204,17 +2204,11 @@ true -- the static invariants they power are the strongest tests in this file.
}
}

Describe 'The fingerprint marker never survives compilation into the agent prompt' {
# WHY THIS EXISTS: the reconciler's entire eligibility model assumes issues can be
# keyed by a canonical fingerprint marker. Today none can be, and the reason is not
# the one previously recorded. The marker template IS in each scanner's source `.md`
# and is ABSENT from every compiled `.lock.yml`, so the agent is never shown it and
# cannot emit it. Output-side sanitization is not needed to explain the absence.
#
# If this test fails, gh-aw has changed and the template now reaches the prompt.
# That is good news and a REQUIRED REVIEW: markers will start appearing, issues
# become keyable, and `candidate` stops being unreachable. Re-read Gate 3 in
# CiScanReconcile.Core.ps1 and the enforcement rollout before updating this test.
Describe 'Fingerprint marker ownership stays in the trusted publisher' {
# gh-aw strips literal HTML comments from prompt text, so correctness must not depend
# on the agent seeing or emitting a marker template. Both scanner twins instead run
# the trusted validator before their publisher, then the compiled publisher requires
# the exact marker derived from each validated manifest fingerprint.
BeforeAll {
$script:WfDir = Join-Path (Split-Path -Parent $PSScriptRoot) 'workflows'
$script:Tmpl = 'ci-scan-fingerprint: {FINGERPRINT}'
Expand All @@ -2232,26 +2226,28 @@ Describe 'The fingerprint marker never survives compilation into the agent promp
}
}

It 'finds at least one scanner source/lock pair to judge' {
# Anti-vacuity floor: without this, a renamed workflow or a partial checkout
# would make every assertion below iterate an empty set and pass green.
It 'finds both scanner source/lock pairs to judge' {
# Exact anti-vacuity guard: a renamed or omitted twin must fail rather than
# silently reducing this invariant to the workflow that remains.
@($script:Pairs).Count |
Should -BeGreaterThan 0 -Because 'the assertions below are vacuous with no pair to read'
Should -Be 2 -Because 'publisher ownership must be pinned for both scanner twins'
}

It 'still carries the marker template in the scanner source' {
# Control for the assertion that follows: proves the needle is findable at all,
# so a zero count in the lock means "stripped", not "wrong search string".
It 'does not ask the agent to emit the literal marker template' {
foreach ($p in $script:Pairs) {
([regex]::Matches($p.Md, [regex]::Escape($script:Tmpl))).Count |
Should -BeGreaterThan 0 -Because "$($p.Stem).md must still instruct the agent to emit a marker"
Should -Be 0 -Because "$($p.Stem).md must keep marker ownership out of the prompt"
([regex]::Matches($p.Lock, [regex]::Escape($script:Tmpl))).Count |
Should -Be 0 -Because "$($p.Stem).lock.yml must not depend on a stripped prompt template"
}
}

It 'loses that template in the compiled lock, which is why no issue can be keyed' {
It 'compiles trusted validation before publisher-side exact-marker checks' {
foreach ($p in $script:Pairs) {
([regex]::Matches($p.Lock, [regex]::Escape($script:Tmpl))).Count |
Should -Be 0 -Because "$($p.Stem).lock.yml reaching the agent with the template would make issues keyable and change Gate 3's premise"
$validator = $p.Lock.IndexOf('run: .github/scripts/Validate-CiScanManifest.ps1')
$publisherMarker = $p.Lock.IndexOf('const exactMarker = `<!-- ci-scan-fingerprint: ${issue.Fingerprint} -->`;')
$validator | Should -BeGreaterThan 0 -Because "$($p.Stem).lock.yml must invoke the trusted validator"
$publisherMarker | Should -BeGreaterThan $validator -Because "$($p.Stem).lock.yml must check the trusted marker after validation"
}
}
}
21 changes: 9 additions & 12 deletions .github/scripts/CiScanReconcile.Core.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -847,8 +847,9 @@ function Set-CiScanStateMarker {
NOTE: there is no such caller yet. This function has no production invocation --
only the read side (`Get-CiScanStateMarker`, live at Invoke-CiScanReconcile.ps1)
currently runs, so the ci-scan-state marker is consumed but never produced. The
practical effect is that no open issue carries a marker, and a markerless issue
stops at `awaiting-canonical-data` / `no-observation-state-recorded`, so the
practical effect is that no open issue carries a state marker. Even an issue with
the publisher-owned fingerprint marker stops at `awaiting-canonical-data` /
`no-observation-state-recorded`, so the
N-consecutive-absence criterion has never executed end to end and `candidate` is
unreachable in production regardless of mode. That is a safety property
independent of report-only, and it is deliberate for now: wiring a writer is what
Expand Down Expand Up @@ -1285,16 +1286,12 @@ function Get-CiScanIssueVerdict {
}

# --- Gate 3: canonical data required -------------------------------------
# The entire current backlog stops here, and will keep stopping here. The marker
# template lives in the scanner's source `.md` (twice) but survives into NO compiled
# `.lock.yml` -- both twins, before and after #36848. Reproduce:
# git show <ref>:.github/workflows/ci-status-net11.md | grep -c 'ci-scan-fingerprint: {FINGERPRINT}' -> 2
# git show <ref>:.github/workflows/ci-status-net11.lock.yml | grep -c 'ci-scan-fingerprint: {FINGERPRINT}' -> 0
# So the agent is never SHOWN the template and cannot emit it. That alone explains
# every markerless issue; output-side sanitization is not needed to explain the data
# and is untested here -- unobservable while nothing is emitted to sanitize. Do not
# "fix" this by bypassing safe outputs: the loss is upstream of them.
# Closing an issue we cannot key is exactly the failure mode this design prevents.
# The legacy backlog stops here and remains ineligible. New scanner payloads are
# marker-free while agent-authored, then the trusted validator derives and injects
# the canonical fingerprint marker from the validated manifest before the publisher
# re-validates the exact body. That makes new issues keyable without trusting prompt
# emission, but it deliberately does not retrofit old markerless issues. Closing an
# issue we cannot key is exactly the failure mode this gate prevents.
if ($null -eq $fp) {
$verdict.Decision = 'awaiting-canonical-data'
$verdict.Reason = 'no-canonical-fingerprint-marker'
Expand Down
Loading
Loading