fix: harden test-deployment rebuild and verification profiles #402
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| # scripts/tests/test-verify-all.ps1 | ||
| # Verifies the canonical aggregate verifier's developer and pruned-deployment profiles. | ||
|
Comment on lines
+1
to
+2
|
||
|
|
||
| Set-StrictMode -Version Latest | ||
| $ErrorActionPreference = 'Stop' | ||
|
|
||
| $repoRoot = (Resolve-Path (Join-Path $PSScriptRoot '..\..')).Path | ||
| $verifyScript = Join-Path $repoRoot 'scripts\verify-all.ps1' | ||
| . (Join-Path $PSScriptRoot 'test-helpers.ps1') | ||
|
|
||
| function Invoke-VerificationPlan { | ||
| param( | ||
| [Parameter(Mandatory = $true)][string] $Profile, | ||
| [Parameter(Mandatory = $true)][string] $RepoRoot | ||
| ) | ||
|
|
||
| $output = @(& (Get-Command pwsh -ErrorAction Stop).Source -NoProfile -NonInteractive -ExecutionPolicy Bypass -File $verifyScript ` | ||
| -Profile $Profile -PlanOnly -RepoRoot $RepoRoot 2>&1) | ||
| return [pscustomobject]@{ | ||
| ExitCode = $LASTEXITCODE | ||
| Output = ($output -join "`n") | ||
| } | ||
| } | ||
|
|
||
| $sandbox = New-TestSandbox -Prefix 'verify-all-profile' | ||
| try { | ||
| $developerPlan = Invoke-VerificationPlan -Profile 'Developer' -RepoRoot $repoRoot | ||
| Assert-Equal 0 $developerPlan.ExitCode 'default developer profile plan is available' | ||
| Assert-True ($developerPlan.Output -match '\[PLAN\] profile=developer') 'default profile identifies itself as developer' | ||
| foreach ($target in @( | ||
| 'tests \(contracts\+fakes\)', | ||
| 'bim-review-coordinator', | ||
| 'web-viewer-sample', | ||
| 'bim-streaming-server' | ||
| )) { | ||
| Assert-True ($developerPlan.Output -match "\[EXECUTE\] $target") "developer profile retains complete target '$target'" | ||
| } | ||
| Assert-True ($developerPlan.Output -notmatch '\[OMIT\]') 'developer profile does not omit its normal contract targets' | ||
|
|
||
| $deploymentRoot = Join-Path $sandbox 'pruned-deployment' | ||
| foreach ($directory in @( | ||
| 'scripts', | ||
| 'docs\plans', | ||
| 'bim-review-coordinator', | ||
| 'web-viewer-sample', | ||
| 'bim-streaming-server\scripts\tests' | ||
| )) { | ||
| New-Item -ItemType Directory -Path (Join-Path $deploymentRoot $directory) -Force | Out-Null | ||
| } | ||
| 'deploy entrypoint' | Set-Content -LiteralPath (Join-Path $deploymentRoot 'scripts\deploy.ps1') -Encoding ascii | ||
| 'retained production design token' | Set-Content -LiteralPath (Join-Path $deploymentRoot 'docs\plans\ai-bim-governance.css') -Encoding ascii | ||
| 'streaming contract entrypoint' | Set-Content -LiteralPath (Join-Path $deploymentRoot 'bim-streaming-server\scripts\tests\test-stage-loading-contract.ps1') -Encoding ascii | ||
|
|
||
| $deploymentPlan = Invoke-VerificationPlan -Profile 'Deployment' -RepoRoot $deploymentRoot | ||
| Assert-Equal 0 $deploymentPlan.ExitCode 'deployment profile accepts the intentionally pruned fixture inventory' | ||
| Assert-True ($deploymentPlan.Output -match '\[PLAN\] profile=deployment') 'deployment profile identifies itself explicitly' | ||
| foreach ($target in @( | ||
| 'deployment required artifacts', | ||
| 'coordinator health', | ||
| 'governance health', | ||
| 'conversion health', | ||
| 'kit manager health', | ||
| 'viewer endpoint' | ||
| )) { | ||
| Assert-True ($deploymentPlan.Output -match "\[EXECUTE\] $target") "deployment profile executes retained target '$target'" | ||
| } | ||
| foreach ($target in @( | ||
| 'tests \(contracts\+fakes\)', | ||
| 'bim-review-coordinator \(full verify\)', | ||
| 'web-viewer-sample \(full verify\)', | ||
| 'bim-streaming-server stage-loading contract' | ||
| )) { | ||
| Assert-True ($deploymentPlan.Output -match "\[OMIT\] $target") "deployment profile explicitly records authoring-only omission '$target'" | ||
| } | ||
|
|
||
| $verifyShell = Get-Content -LiteralPath (Join-Path $repoRoot 'scripts\verify-all.sh') -Raw | ||
| Assert-True ($verifyShell -match '--profile') 'POSIX verifier mirror accepts an explicit deployment profile' | ||
| Assert-True ($verifyShell -match '--plan-only') 'POSIX verifier mirror publishes the same profile inventory without executing it' | ||
|
Comment on lines
+76
to
+78
Contributor
There was a problem hiding this comment. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== locate files =="
git ls-files | rg '(^|/)test-verify-all\.ps1$|verify-all\.sh$|verify-all\.ps1$|AGENTS\.md$' || true
echo "== script sizes =="
for f in scripts/tests/test-verify-all.ps1 scripts/verify-all.sh scripts/verify-all.ps1 scripts/lib/StructLog.psm1; do
[ -f "$f" ] && wc -l "$f"
done
echo "== inspect test file around assertions =="
if [ -f scripts/tests/test-verify-all.ps1 ]; then
cat -n scripts/tests/test-verify-all.ps1 | sed -n '1,140p'
fi
echo "== inspect shell verifier outline/content =="
if [ -f scripts/verify-all.sh ]; then
wc -l scripts/verify-all.sh
cat -n scripts/verify-all.sh | sed -n '1,260p'
fi
echo "== inspect powershell verifier outline/content around relevant sections =="
if [ -f scripts/verify-all.ps1 ]; then
wc -l scripts/verify-all.ps1
cat -n scripts/verify-all.ps1 | sed -n '1,260p'
fi
echo "== search for test-verify-all usages and fixtures =="
rg -n "test-verify-all|verify-all|--profile|--plan-only|deployment profile|profile inventory|Plan output|missing|artifact" scripts AGENTS.md .github 2>/dev/null || trueRepository: monkey1sai/AI-BIM-governance Length of output: 50385 🏁 Script executed: #!/bin/bash
set -euo pipefail
echo "== git status/stat =="
git status --short
git diff --stat
echo "== shell syntax parse if available =="
bash -n scripts/verify-all.sh 2>&1 || true
pwsh -Command "& {if (Test-Path scripts/verify-all.ps1) { $f='scripts/verify-all.ps1'; $tokens=$errors=$null; [System.Management.Automation.Parser]::ParseInput((Get-Content -Raw $f), [ref]$tokens, [ref]$errors); $errors?.Write(); $tokens } else { 'verify-all.ps1 missing' } }" 2>&1 || true
echo "== deterministic flag-token semantic probe =="
python3 - <<'PY'
from pathlib import Path
import re
p=Path('scripts/tests/test-verify-all.ps1')
if p.exists():
text=p.read_text()
print("has profile token assertion:", bool(re.search(r'--profile', text)))
print("has plan-only token assertion:", bool(re.search(r'--plan-only', text)))
print("calls verify shell exec:", bool(re.search(r'Invoke-Expression|Start-Process|powershell|pwsh|verify-all', text)))
print("contains parity output assertions:", bool(re.search(r'parity|output|assert.*profile|profile inventory', text, re.I)))
PYRepository: monkey1sai/AI-BIM-governance Length of output: 261 Exercise the Bash profile contract instead of scanning tokens. These assertions pass if the flags only occur as strings; they do not verify Bash parsing, the Deployment inventory generated from the pruned fixture, omitted targets, or missing-artifact failure. Run 🤖 Prompt for AI AgentsSource: Coding guidelines |
||
|
|
||
| Remove-Item -LiteralPath (Join-Path $deploymentRoot 'docs\plans\ai-bim-governance.css') -Force | ||
| $missingArtifactPlan = Invoke-VerificationPlan -Profile 'Deployment' -RepoRoot $deploymentRoot | ||
| Assert-True ($missingArtifactPlan.ExitCode -ne 0) 'deployment profile fails when a production-required retained artifact is missing' | ||
| Assert-True ($missingArtifactPlan.Output -match 'deployment required artifact missing') 'deployment profile reports the missing production-required artifact' | ||
|
|
||
| Write-TestPass 'verify-all profiles' | ||
| } catch { | ||
| Write-TestFail 'verify-all profiles' $_.Exception.Message | ||
| throw | ||
| } finally { | ||
| Remove-TestSandbox -Path $sandbox | ||
| } | ||
|
|
||
| exit 0 | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With this condition, the new aggregate verifier tests only run when
rebuild_test_deployis true, but the classifier pattern above does not includescripts/verify-all.ps1orscripts/verify-all.sh—only deploy/rebuild helpers and test files. A PR that changes the verifier itself will therefore get only static analysis and skip the regression tests added here; gate this step on a classifier that also matches the verifier scripts.Useful? React with 👍 / 👎.