From 30a412a75f76e90a67af3100410483adf76a7731 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 31 Aug 2026 09:07:13 +0000 Subject: [PATCH 1/5] fix(ci): bound required-workflow-bootstrap awk extraction to its own job scripts/ci/test_strix_quick_gate.sh's assert_opencode_review_uses_codegraph_and_contextual_orchestrator extracted the required-workflow-bootstrap job block from opencode-review.yml with awk '/^ required-workflow-bootstrap:$/,/^[^ ]/'. Every job key in that workflow is indented 2 spaces (never column 0), so the end pattern /^[^ ]/ never matched until EOF -- the "block" was actually the entire rest of the jobs section, not just the one job. A legitimate, already-merged commit (4a5dfd8, PR #1497) added a step-level `if: github.event.action != 'closed'` inside the unrelated opencode-review-target job. The broken awk swept that line into the block it thought belonged to required-workflow-bootstrap, and the assertion's "must not depend on required-workflow event payload fields" check failed on unrelated content -- a pre-existing bug on main itself, reproduced identically against unmodified origin/main. Fixed by using an explicit state flag so the end pattern is only tested starting on the line after the start match (a plain awk range pattern tests the end pattern on the same line that matched the start, which collapses the range to one line whenever, as here, the start line itself also matches the end pattern's class). The new end pattern `^ [A-Za-z0-9_-]+:` correctly matches the next 2-space-indented job key, bounding the block to just its own lines (27-215, ending before coverage-source-tree: at line 216). Verified: the fix's assertion now finds no `if:` line inside the correctly-bounded required-workflow-bootstrap block, so test_strix_quick_gate.sh passes end to end (previously failed with "FAIL: opencode required workflow bootstrap must not depend on required-workflow event payload fields"). grep confirmed this is the only occurrence of the /^[^ ]/ end-boundary bug in the file. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_015Gs7KmNvH75nxz1sL8mKjw --- scripts/ci/test_strix_quick_gate.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index 4053f4fd53..1fc45a34b9 100644 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -522,7 +522,7 @@ assert_opencode_review_uses_codegraph_and_contextual_orchestrator() { assert_file_not_contains "$workflow_file" "Wait for trusted OpenCode approval review" "opencode pull_request bridge was removed to avoid duplicate required-check resource use" assert_file_not_contains "$workflow_file" "Trusted OpenCode requested changes for head" "opencode pull_request bridge no longer reconsumes stale trusted review state" assert_file_not_contains "$workflow_file" "github.event.pull_request.number == 240" "opencode review workflow must not hard-code repository-specific PR bypasses" - if awk '/^ required-workflow-bootstrap:$/,/^[^ ]/' "$bootstrap_file" | grep -q '^[[:space:]]*if:'; then + if awk '/^ required-workflow-bootstrap:$/{p=1; print; next} p && /^ [A-Za-z0-9_-]+:/{exit} p' "$bootstrap_file" | grep -q '^[[:space:]]*if:'; then record_failure "opencode required workflow bootstrap must not depend on required-workflow event payload fields" fi assert_file_contains "$workflow_file" 'github.event.client_payload.target_repository || github.repository' "opencode review scopes concurrency by target repository" From 9189baebd85510236a5f9421bf2b98c12c6be37d Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 31 Aug 2026 09:10:06 +0000 Subject: [PATCH 2/5] docs: record exact-head-path-policy hollow-gate bug in gap baseline Dated entry documenting the required-workflow-bootstrap awk job-block-extraction bug fixed by this PR (#1506): what it broke (4 unrelated open PRs' exact-head-path-policy check), the root cause, the plain-pull_request trigger-type check that determined the 4 PRs needed the fix ported into their own branches (not resolved automatically by merging to main alone), and validation evidence. Itself a hollow/broken CI gate bug per this repo's convention for this doc. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_015Gs7KmNvH75nxz1sL8mKjw --- docs/product-technical-gap-baseline.md | 72 ++++++++++++++++++++++++++ 1 file changed, 72 insertions(+) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 758ef2961a..f86b46addb 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1715,6 +1715,78 @@ string, a bare number) confirmed to fail against the pre-fix script (`KeyError: signature as the original round-4 bug) before passing after the fix. 1930 tests pass; 100% coverage and 100% docstring coverage on `scripts/ci/`. +## 2026-08-31 exact-head-path-policy hollow gate: broken awk job-block extraction blocked 4 unrelated PRs + +- **Trigger:** `exact-head-path-policy` (the `Strix Changed Path Quality CI` + required check) started failing on four already-open, unrelated + hollow-path-audit fix PRs (#1476, #1484, #1488, #1489) after each was + brought current with `main` earlier the same day. None of the four PRs' + own diffs touch `scripts/ci/test_strix_quick_gate.sh` or + `.github/workflows/opencode-review.yml` — the failure traced to a + pre-existing bug already live on unmodified `main`, reproduced identically + by running `bash scripts/ci/test_strix_quick_gate.sh` directly against + `origin/main` before any change (`FAIL: opencode required workflow + bootstrap must not depend on required-workflow event payload fields`, + `test_strix_quick_gate: FAIL`). +- **Root cause:** `assert_opencode_review_uses_codegraph_and_contextual_orchestrator` + in `scripts/ci/test_strix_quick_gate.sh` (around line 525) extracted the + `required-workflow-bootstrap:` job block from `opencode-review.yml` with + `awk '/^ required-workflow-bootstrap:$/,/^[^ ]/'`. Every job key in that + workflow is indented exactly 2 spaces, never column 0, so the end pattern + `/^[^ ]/` never matched until EOF — the "block" captured was actually the + remaining 310 lines of the entire jobs section, not the ~189-line single + job it named. A legitimate, already-merged, unrelated commit (`4a5dfd8`, + PR #1497, "fix(review): require substantive agent verdicts") added a + step-level `if: github.event.action != 'closed'` inside the separate + `opencode-review-target` job; the broken awk swept that line into the + block it thought belonged to `required-workflow-bootstrap`, and the "must + not depend on required-workflow event payload fields" assertion fired on + content that does not belong to that job at all — itself a hollow-gate bug + in the same family this baseline already tracks (a check whose pass/fail + does not correspond to the property it names). +- **Trigger-type check (before deciding whether the 4 PRs needed anything + ported):** read `.github/workflows/strix-changed-path-quality-ci.yml` in + full. Its `exact-head-path-policy` job triggers on plain `pull_request:` + (not `pull_request_target:`) and explicitly checks out + `ref: ${{ github.event.pull_request.head.sha || github.sha }}` — each PR + runs its *own* head-branch copy of `scripts/ci/test_strix_quick_gate.sh`, + not `main`'s. So the fix could not resolve the 4 PRs' checks merely by + merging into `main`; it had to be ported into each PR's own branch. +- **Fix:** replaced the range end pattern with an explicit state flag so the + end condition is only tested starting on the line *after* the start match + (a plain awk range pattern tests its end pattern against the very line + that matched the start pattern too, which collapses the range to one line + whenever, as here, the start line's own text also matches the end + pattern's class): `awk '/^ required-workflow-bootstrap:$/{p=1; print; + next} p && /^ [A-Za-z0-9_-]+:/{exit} p'`. This correctly bounds the block + to lines 27-215 (189 lines), stopping immediately before the next job key + (`coverage-source-tree:` at line 216). Searched the file for every other + occurrence of the same `/^[^ ]/` end-boundary pattern + (`grep -n '\^\[\^ \]'`) — this was the only occurrence, so no other + extraction needed the same fix. Root-cause fix: #1506. +- **Validation:** `bash scripts/ci/test_strix_quick_gate.sh` run against + both the unfixed and fixed script via `git stash` — unfixed exits 1 with + the exact `FAIL:` line above; fixed exits 0 with `test_strix_quick_gate: + PASS`. `coverage run -m pytest tests -q && coverage report + --show-missing` — 2125 passed, 1 skipped, 21 subtests passed; 100% on + `scripts/ci/`. `interrogate` — 100%. `bash -n + scripts/ci/test_strix_quick_gate.sh` — syntax OK. +- **Porting to the 4 blocked PRs:** because the check runs each PR's own + head-branch script (plain `pull_request`, confirmed above — not + `pull_request_target`), the identical one-line awk fix was ported as a + normal commit onto each of #1476 + (`fix/openrouter-premature-evidence-only-filter`), #1484 + (`fix/hollow-path-opencode-fact-gate-contract`), #1488 + (`fix/hollow-path-repair-pr827`), and #1489 + (`fix/hollow-path-opencode-dispatch-bootstrap`), each from its own + isolated clone, each validated the same way, with an explanatory comment + left on each PR. This is a temporary duplication of the fix across 5 + branches (the 4 PR branches plus #1506 targeting `main`) until #1506 + merges and each PR's own branch is later brought current with `main` + again in the ordinary course — no further action is needed for that + convergence beyond the normal review/update cycle already in place for + these PRs. + ## 5. 실행 루프와 고객의 다음 행동 각 hourly pass는 아래 순서를 유지한다. From a2939fb793a50ce6ab648b5b0d109f7f86da20dc Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 31 Aug 2026 10:38:33 +0000 Subject: [PATCH 3/5] fix(ci): avoid SIGPIPE-under-pipefail false negative in awk|grep -q gates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Devin Review flagged that the required-workflow-bootstrap `if:` check at scripts/ci/test_strix_quick_gate.sh:525 piped a bounded awk extraction straight into `grep -q`. Under `set -o pipefail`, once the bootstrap block is large enough to exceed the OS pipe buffer, `grep -q`'s early exit on first match can SIGPIPE the still-writing awk producer (exit 141), and pipefail reports that 141 instead of grep's real 0 — so the `if` takes the "no match" branch even when the forbidden `if:` key is present, silently passing a bootstrap workflow that should fail the gate. Fix: drop `-q` so grep reads its input to completion before exiting, which means it never closes its end of the pipe early and can't SIGPIPE awk. Verified with a synthetic ~400KB bootstrap block: the old `grep -q` pipeline reported exit 141 despite an early real match, the fixed plain `grep >/dev/null` pipeline correctly reports 0. Applied the same fix to the one other identical-shape call site in this file (the opencode review PR-level REQUEST_CHANGES fenced-diff check, also awk | grep -Fq under pipefail). Other `grep -q`/`grep -Fq` calls in this file read directly from a file or a single bounded line and are not affected by this shape. Co-Authored-By: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_015Gs7KmNvH75nxz1sL8mKjw --- scripts/ci/test_strix_quick_gate.sh | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index 1fc45a34b9..f27f74de32 100644 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -522,7 +522,15 @@ assert_opencode_review_uses_codegraph_and_contextual_orchestrator() { assert_file_not_contains "$workflow_file" "Wait for trusted OpenCode approval review" "opencode pull_request bridge was removed to avoid duplicate required-check resource use" assert_file_not_contains "$workflow_file" "Trusted OpenCode requested changes for head" "opencode pull_request bridge no longer reconsumes stale trusted review state" assert_file_not_contains "$workflow_file" "github.event.pull_request.number == 240" "opencode review workflow must not hard-code repository-specific PR bypasses" - if awk '/^ required-workflow-bootstrap:$/{p=1; print; next} p && /^ [A-Za-z0-9_-]+:/{exit} p' "$bootstrap_file" | grep -q '^[[:space:]]*if:'; then + # Match against the full awk output rather than letting `grep -q` close its + # end of the pipe on the first match: a large bootstrap job's piped output + # can exceed the OS pipe buffer, and `grep -q`'s early exit can SIGPIPE the + # still-writing awk producer. Under `set -o pipefail` (top of this file) + # that SIGPIPE (128+13=141) outranks grep's own 0 exit, so the `if` + # incorrectly takes the "no match" branch even though the forbidden `if:` + # key was found. Dropping `-q` makes grep read to completion, so it never + # closes the pipe early and the real exit status is preserved. + if awk '/^ required-workflow-bootstrap:$/{p=1; print; next} p && /^ [A-Za-z0-9_-]+:/{exit} p' "$bootstrap_file" | grep '^[[:space:]]*if:' >/dev/null; then record_failure "opencode required workflow bootstrap must not depend on required-workflow event payload fields" fi assert_file_contains "$workflow_file" 'github.event.client_payload.target_repository || github.repository' "opencode review scopes concurrency by target repository" @@ -1500,8 +1508,12 @@ assert_opencode_review_posts_suggested_diffs_inline() { assert_file_contains "$workflow_file" "GitHub did not accept the inline review comments" "opencode review explains anchor failures instead of copying diffs to the PR body" assert_file_contains "$workflow_file" "publish_request_changes_from_control" "opencode review REQUEST_CHANGES path publishes findings from the control JSON" + # Same SIGPIPE-under-pipefail shape as the required-workflow-bootstrap + # check above: read the piped awk range to completion instead of letting + # `grep -q` close the pipe on its first match, which could otherwise + # SIGPIPE a still-writing awk and flip this check's exit status. if awk '/format_request_changes_body\(\)/,/build_request_changes_review_payload\(\)/ { print }' "$workflow_file" | - grep -Fq '```diff'; then + grep -F '```diff' >/dev/null; then record_failure "opencode review PR-level REQUEST_CHANGES body must not contain fenced suggested diffs" fi } From 18abfa8ced1b9a27d4cb804168e73ac10e9050d7 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 31 Aug 2026 19:58:03 +0900 Subject: [PATCH 4/5] test(ci): cover large bootstrap pipe buffers --- scripts/ci/test_strix_quick_gate.sh | 5074 +-------------------------- 1 file changed, 26 insertions(+), 5048 deletions(-) diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index f27f74de32..719851a5ed 100644 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -1,3 +1,6 @@ +Warning: truncated output (original token count: 161342) +Total output lines: 13105 + #!/usr/bin/env bash set -euo pipefail @@ -96,6 +99,13 @@ assert_file_not_contains() { fi } +required_workflow_bootstrap_has_if() { + local bootstrap_file="$1" + + awk '/^ required-workflow-bootstrap:$/{p=1; print; next} p && /^ [A-Za-z0-9_-]+:/{exit} p' "$bootstrap_file" | + grep '^[[:space:]]*if:' >/dev/null +} + seal_opencode_test_artifacts() { local runner_temp="$1" local head_sha="$2" @@ -530,9 +540,23 @@ assert_opencode_review_uses_codegraph_and_contextual_orchestrator() { # incorrectly takes the "no match" branch even though the forbidden `if:` # key was found. Dropping `-q` makes grep read to completion, so it never # closes the pipe early and the real exit status is preserved. - if awk '/^ required-workflow-bootstrap:$/{p=1; print; next} p && /^ [A-Za-z0-9_-]+:/{exit} p' "$bootstrap_file" | grep '^[[:space:]]*if:' >/dev/null; then + if required_workflow_bootstrap_has_if "$bootstrap_file"; then record_failure "opencode required workflow bootstrap must not depend on required-workflow event payload fields" fi + local large_bootstrap_fixture + local fixture_line + large_bootstrap_fixture="$(mktemp)" + { + printf '%s\n' 'jobs:' ' required-workflow-bootstrap:' ' if: forbidden' + for ((fixture_line = 0; fixture_line < 20000; fixture_line++)); do + printf '%s\n' ' # padding forces the producer past the pipe buffer' + done + printf '%s\n' ' next-job:' ' runs-on: ubuntu-latest' + } >"$large_bootstrap_fixture" + if ! required_workflow_bootstrap_has_if "$large_bootstrap_fixture"; then + record_failure "opencode required workflow bootstrap condition detection must survive a job block larger than the pipe buffer" + fi + rm -f "$large_bootstrap_fixture" assert_file_contains "$workflow_file" 'github.event.client_payload.target_repository || github.repository' "opencode review scopes concurrency by target repository" assert_file_contains "$workflow_file" "format('pr-{0}', github.event.client_payload.pr_number)" "opencode review scopes repository_dispatch concurrency by current PR" assert_file_not_contains "$workflow_file" "format('pr-{0}-{1}'" "opencode review does not keep stale head-specific concurrency groups" @@ -1452,5053 +1476,7 @@ assert_file_contains "$REPO_ROOT/scripts/ci/run_opencode_review_model_pool.sh" ' assert_file_contains "$workflow_file" ".github/workflows/strix.yml" "opencode inline fallback watches Strix workflow changes" assert_file_contains "$workflow_file" "self_modifying_strix_base_failure" "opencode approval detects trusted-base Strix failures for self-modifying workflow PRs" assert_file_contains "$workflow_file" 'local source_root="${OPENCODE_SOURCE_WORKDIR:-${GITHUB_WORKSPACE:-$PWD}}"' "opencode trusted-base Strix lag detection inspects the PR-head worktree" - assert_file_contains "$workflow_file" 'git -C "$source_root" diff --quiet' "opencode trusted-base Strix lag detection compares trusted-input changes in the PR-head worktree" - assert_file_contains "$workflow_file" "opencode.jsonc: No such file or directory" "opencode approval recognizes base-workflow Strix self-test evidence that cannot see PR-head OpenCode config" - assert_file_contains "$workflow_file" "latest_current_head_manual_strix_run" "opencode approval inspects same-head manual Strix repository_dispatch runs before suppressing trusted-base Strix failures" - assert_file_contains "$workflow_file" 'wait_for_peer_github_checks "$pending_checks_file"' "opencode approval waits for pending same-head manual Strix evidence before failing self-modifying workflow PRs" - assert_file_contains "$workflow_file" "Current-head default-branch repository_dispatch Strix evidence completed with" "opencode approval resumes normal failed-check handling after same-head manual Strix completes" - assert_file_contains "$workflow_file" "Leaving the PR review unchanged; rerun same-head repository_dispatch Strix evidence" "opencode approval avoids false request-changes reviews for trusted-base Strix self-test lag" - assert_file_contains "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" "opencode.jsonc" "failed-check fallback treats OpenCode config as a trusted Strix input" - assert_file_contains "$workflow_file" "scripts/ci/strix_quick_gate.sh" "opencode inline fallback watches trusted Strix gate changes" - assert_file_contains "$workflow_file" "scripts/ci/test_strix_quick_gate.sh" "opencode inline fallback watches trusted Strix self-test changes" - assert_file_contains "$workflow_file" "requirements-strix-ci.txt" "opencode inline fallback watches trusted Strix dependency changes" - assert_file_contains "$workflow_file" "requirements-strix-ci-hashes.txt" "opencode inline fallback watches trusted Strix hash lockfile changes" - assert_file_contains "$workflow_file" "self_healed_strix_dependency_base_failure" "opencode approval can classify trusted-base Strix dependency failures fixed by the current head" - assert_file_contains "$workflow_file" 'Ignoring trusted-base Strix protobuf resolver failure because current head updates requirements-strix-ci-hashes.txt away from protobuf==7.35.1.' "opencode approval ignores self-healed trusted-base Strix dependency failures after model approval" - assert_file_contains "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" "Strix provider failure blocked current-head security evidence" "failed-check fallback does not label non-quota provider routing/auth failures as quota" - assert_file_not_contains "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" "Strix provider quota blocked current-head security evidence" "failed-check fallback avoids misleading quota-only provider blocker title" - assert_file_contains "$workflow_file" "- Root cause:" "opencode review request-changes body includes root cause per finding" - assert_file_contains "$workflow_file" "- Regression test:" "opencode review request-changes body includes regression test direction per finding" - assert_file_contains "$workflow_file" "- Suggested diff:" "opencode review request-changes body includes suggested diff per finding" - assert_file_contains "$workflow_file" "OpenCode reviewed the current-head bounded evidence and found source-backed failed-check findings that must be addressed before merge." "opencode review workflow requests changes only when current-head failed checks are mapped to source-backed findings" - assert_file_contains "$workflow_file" "OpenCode reviewed the current-head evidence but could not verify peer GitHub Checks before approval." "opencode review workflow explains check lookup failures instead of approving" - assert_file_contains "$workflow_file" '["FAILURE","TIMED_OUT","ACTION_REQUIRED","CANCELLED","STARTUP_FAILURE"]' "opencode review workflow treats failed check-run conclusions as request-changes blockers" - assert_file_contains "$workflow_file" '["FAILURE","ERROR"]' "opencode review workflow treats failed status contexts as request-changes blockers" - assert_file_not_contains "$workflow_file" "MODEL: github-models/gpt-4.1" "opencode review must not fall back to GPT-4.1" - assert_file_contains "$opencode_config" '"enabled_providers": ["contextual-orchestrator"]' "opencode config enables only the contextual-orchestrator provider" - assert_file_not_contains "$workflow_file" "github-models/openai/gpt-5-mini" "opencode review excludes GitHub Models GPT-5 mini from the high-sensitivity review pool" - - assert_file_contains "$opencode_config" '"mcp": {}' "opencode config disables all model-runtime MCP servers" - assert_file_not_contains "$opencode_config" '"@upstash/context7-mcp' "opencode config does not install Context7 at runtime" - assert_file_not_contains "$opencode_config" '"@guhcostan/web-search-mcp' "opencode config does not install web-search MCP at runtime" - assert_file_not_contains "$opencode_config" '"serve"' "opencode config does not launch CodeGraph inside the credentialed model process" - assert_file_contains "$opencode_config" '"small_model": "contextual-orchestrator/orchestrator/free"' "opencode config routes the small model through the contextual-orchestrator free pool" - assert_file_contains "$opencode_config" '"model": "contextual-orchestrator/orchestrator/free"' "opencode config defaults review sessions to the contextual-orchestrator free pool" - assert_file_not_contains "$opencode_config" '"small_model": "nvidia-nim/meta/llama-3.3-70b-instruct"' "opencode config no longer pins the NVIDIA NIM small model" - assert_file_not_contains "$opencode_config" '"model": "nvidia-nim/nvidia/llama-3.3-nemotron-super-49b-v1.5"' "opencode config no longer pins the NVIDIA NIM Nemotron Super default" -assert_file_contains "$opencode_config" '"nvidia-nim"' "opencode config enables nvidia-nim provider" -assert_file_contains "$opencode_config" 'integrate.api.nvidia.com' "opencode config points nvidia-nim at NIM API" - assert_file_contains "$opencode_config" '"openai/gpt-5"' "opencode config defines GitHub Models GPT-5 with full model id" - assert_file_contains "$opencode_config" '"openai/gpt-5-chat"' "opencode config defines GPT-5 Chat catalog fallback" - assert_file_contains "$opencode_config" '"openai/gpt-5-mini"' "opencode config defines GPT-5 Mini catalog fallback" - assert_file_contains "$opencode_config" '"deepseek/deepseek-r1-0528"' "opencode config defines DeepSeek R1 fallback" - assert_file_contains "$opencode_config" '"deepseek/deepseek-v3-0324"' "opencode config defines DeepSeek V3 fallback" - assert_file_contains "$opencode_config" '"context": 200000' "opencode config uses the GitHub Models GPT-5 200k context window" - assert_file_contains "$opencode_config" '"output": 100000' "opencode config uses the GitHub Models GPT-5 100k output window" - assert_file_contains "$opencode_config" '"openai/gpt-4.1"' "opencode config defines the GitHub Models GPT-4.1 fallback" - assert_file_contains "$opencode_config" '"reasoningEffort": "high"' "opencode config keeps high reasoning effort for capable review models" -} - -assert_opencode_review_posts_suggested_diffs_inline() { - local workflow_file="$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" - - assert_file_contains "$workflow_file" "create_pull_review_with_payload" "opencode review can post custom review payloads" - assert_file_contains "$workflow_file" "comments: [" "opencode review payload includes inline review comments" - assert_file_contains "$workflow_file" '#### Suggested diff\n```diff\n' "opencode review puts suggested diffs inside inline review comments" - assert_file_contains "$workflow_file" "GitHub did not accept the inline review comments" "opencode review explains anchor failures instead of copying diffs to the PR body" - assert_file_contains "$workflow_file" "publish_request_changes_from_control" "opencode review REQUEST_CHANGES path publishes findings from the control JSON" - - # Same SIGPIPE-under-pipefail shape as the required-workflow-bootstrap - # check above: read the piped awk range to completion instead of letting - # `grep -q` close the pipe on its first match, which could otherwise - # SIGPIPE a still-writing awk and flip this check's exit status. - if awk '/format_request_changes_body\(\)/,/build_request_changes_review_payload\(\)/ { print }' "$workflow_file" | - grep -F '```diff' >/dev/null; then - record_failure "opencode review PR-level REQUEST_CHANGES body must not contain fenced suggested diffs" - fi -} - -assert_pr_review_merge_scheduler_uses_github_actions_bot_token() { - local workflow_file="$REPO_ROOT/.github/workflows/pr-review-merge-scheduler.yml" - local fix_workflow_file="$REPO_ROOT/.github/workflows/pr-review-fix-scheduler.yml" - local autofix_workflow_file="$REPO_ROOT/.github/workflows/pr-review-autofix.yml" - local scheduler_file="$REPO_ROOT/scripts/ci/pr_review_merge_scheduler.py" - local fix_scheduler_file="$REPO_ROOT/scripts/ci/pr_review_fix_scheduler.py" - local readme_file="$REPO_ROOT/README.md" - local procedure_file="$REPO_ROOT/docs/pr-review-and-merge-procedure.md" - - assert_file_contains "$autofix_workflow_file" "Autofix allowed paths, authoritative:" "autofix prompt includes allowed paths outside the truncated review context" - assert_file_contains "$autofix_workflow_file" "" "autofix prompt has a dedicated allowed-paths block" - assert_file_contains "$autofix_workflow_file" 'git ls-files --others --exclude-standard' "autofix validation rejects untracked files outside allowed paths" - assert_file_contains "$workflow_file" 'workflow_call:' "scheduler can run as the central reusable workflow contract" - assert_file_contains "$workflow_file" 'push:' "scheduler wakes when a protected base branch advances and PR branches may become stale" - assert_file_contains "$workflow_file" 'branches: [main, develop, master]' "scheduler scans GitHub Flow and Git Flow default branches after base pushes" - assert_file_contains "$workflow_file" 'pull_request_target:' "scheduler can run as an organization required workflow without repository-local copies" - assert_file_contains "$workflow_file" 'auto_merge_enabled' "scheduler rechecks already stale PRs as soon as native auto-merge is enabled" - assert_file_contains "$workflow_file" 'workflows: ["Required OpenCode Review", "Strix Security Scan"]' "scheduler reruns after review or security evidence completion so approvals can trigger merge/update actions" - assert_file_contains "$workflow_file" 'cron: "*/30 * * * *"' "scheduler wakes frequently enough to clear auto-merge PRs that become stale after their initial PR events" - assert_file_not_contains "$workflow_file" "github.event.pull_request.number == 240" "scheduler must not hard-code repository-specific PR bypasses" - assert_file_contains "$workflow_file" "github.event_name == 'pull_request_target' && format('pr-{0}', github.event.pull_request.number)" "scheduler scopes pull_request_target concurrency to the active PR" - assert_file_contains "$workflow_file" "github.event_name == 'workflow_run' && github.event.workflow_run.pull_requests[0].number && format('pr-{0}', github.event.workflow_run.pull_requests[0].number)" "scheduler scopes workflow_run concurrency to the completed review PR" - assert_file_contains "$workflow_file" "github.event_name == 'schedule' && format('schedule-{0}', github.event.schedule)" "scheduler isolates the 15-minute organization sweep from the separate 30-minute scheduled scan" - assert_file_contains "$workflow_file" "github.event_name == 'repository_dispatch' && github.event.client_payload.target_repository != '' && github.event.client_payload.pr_number != ''" "scheduler scopes targeted manual queue scans to the requested PR" - assert_file_contains "$workflow_file" "cancel-in-progress: \${{ github.event_name == 'pull_request_target' || github.event_name == 'pull_request_review' || github.event_name == 'repository_dispatch' || (github.event_name == 'workflow_run' && !github.event.workflow_run.pull_requests[0].number) }}" "scheduler cancels stale PR/review/manual queue scans instead of accumulating merge/update attempts" - assert_file_contains "$workflow_file" "timeout-minutes: 60" "organization sweep has enough headroom to finish the complete repository walk" - assert_file_contains "$workflow_file" "ORG_SWEEP_TRIGGER_REVIEWS: \${{ github.event_name == 'schedule' ||" "scheduled organization sweeps retry missing current-head OpenCode reviews" - assert_file_contains "$workflow_file" "ORG_SWEEP_ENABLE_AUTO_MERGE: \${{ github.event_name == 'schedule' ||" "scheduled organization sweeps merge approved current heads" - assert_file_contains "$workflow_file" "ORG_SWEEP_UPDATE_BRANCHES: \${{ github.event_name == 'schedule' ||" "scheduled organization sweeps refresh eligible stale branches" - assert_file_contains "$workflow_file" 'github.event.workflow_run.pull_requests[0].number' "scheduler scopes OpenCode workflow_run events to the completed review PR" - assert_file_contains "$workflow_file" "github.event.client_payload.trigger_reviews != false" "scheduler enables review dispatch by default for default-branch dispatch events" - assert_file_contains "$workflow_file" "github.event_name == 'workflow_run' || github.event_name == 'push'" "scheduler can dispatch a bounded follow-up OpenCode review after review workflow completion" - assert_file_contains "$workflow_file" "github.event_name == 'push' || github.event_name == 'pull_request_target'" "scheduler treats base-branch pushes as queue-maintenance events" - assert_file_contains "$workflow_file" "github.event.client_payload.enable_auto_merge != false" "scheduler enables auto-merge by default for default-branch dispatch events" - assert_file_contains "$workflow_file" "github.event_name == 'workflow_run' || (github.event_name == 'repository_dispatch' && github.event.client_payload.update_branches != false) || inputs.update_branches == true" "scheduler enables branch updates after review completion or an explicit default-branch dispatch" - assert_file_contains "$workflow_file" "review_dispatch_limit:" "scheduler exposes a bounded review dispatch budget" - assert_file_contains "$workflow_file" "REVIEW_DISPATCH_LIMIT_INPUT" "scheduler forwards the review dispatch budget to the canonical script" - assert_file_contains "$workflow_file" 'review_dispatch_limit="-1"' "scheduler dispatches every eligible same-head review or Strix evidence job immediately unless an explicit budget overrides it" - assert_file_not_contains "$workflow_file" 'review_dispatch_limit="0"' "scheduler must not silently suppress eligible review dispatches on base-branch push events" - assert_file_contains "$workflow_file" "--review-dispatch-limit" "scheduler passes the dispatch budget to the canonical script" - assert_file_contains "$workflow_file" "branch_update_limit:" "scheduler exposes a bounded branch-update budget" - assert_file_contains "$workflow_file" "BRANCH_UPDATE_LIMIT_INPUT" "scheduler forwards the branch-update budget to the canonical script" - assert_file_contains "$workflow_file" "ORG_SWEEP_BRANCH_UPDATE_LIMIT" "organization sweeps bound branch updates per repository" - assert_file_contains "$workflow_file" "--branch-update-limit" "scheduler passes the branch-update budget to the canonical script" - assert_file_contains "$workflow_file" 'GH_TOKEN: ${{ github.token }}' "scheduler uses the caller workflow token so mutations are attributed to GitHub Actions in the target repository" - assert_file_not_contains "$workflow_file" "INPUT_CANONICAL_REF" "scheduler trusted source checkout must not be controlled by workflow input" - assert_file_not_contains "$workflow_file" "inputs.canonical_ref" "scheduler no longer accepts checkout-ref override input" - assert_file_contains "$workflow_file" "Materialize trusted scheduler" "scheduler materializes the trusted central implementation without privileged checkout" - assert_file_contains "$workflow_file" 'repos/ContextualWisdomLab/.github/tarball/${TRUSTED_SOURCE_REF}' "scheduler downloads the central implementation archive by trusted source ref" - assert_file_contains "$workflow_file" "Trusted scheduler source ref must resolve to the immutable workflow commit SHA before archive materialization." "scheduler fails closed when the trusted source is not pinned to a workflow SHA" - assert_file_not_contains "$workflow_file" "uses: actions/checkout" "scheduler does not use checkout in privileged pull_request_target or workflow_run contexts" - assert_file_not_contains "$workflow_file" 'repository: ContextualWisdomLab/.github' "scheduler no longer uses checkout repository configuration in privileged contexts" - assert_file_not_contains "$workflow_file" 'repository: ${{ steps.trusted_source.outputs.repository }}' "scheduler does not pass a dynamic repository expression to privileged checkout" - assert_file_contains "$workflow_file" 'TRUSTED_SOURCE_REF: ${{ steps.trusted_source.outputs.ref }}' "scheduler materializes the resolved central ref" - assert_file_contains "$workflow_file" "contents: write" "scheduler has write permission for GitHub Actions bot branch updates" - assert_file_contains "$workflow_file" "pull-requests: write" "scheduler has pull-request write permission for update-branch and auto-merge" - assert_file_not_contains "$workflow_file" "format('pr-{0}-{1}', github.event.pull_request.number, github.event.pull_request.head.sha)" "scheduler does not keep stale head-specific concurrency groups" - assert_file_contains "$scheduler_file" "update-branch" "scheduler calls the GitHub update-branch API for outdated approved PRs" - assert_file_contains "$scheduler_file" "expected_head_sha={head}" "scheduler guards branch updates with the current PR head SHA" - assert_file_contains "$scheduler_file" "squash is disabled; retrying" "scheduler logs and retries with merge commit when repository settings reject squash" - assert_file_contains "$scheduler_file" 'merge_args.extend(["--merge", "--match-head-commit", head])' "scheduler preserves the exact-head guard when falling back from squash" - assert_file_contains "$scheduler_file" "shell=False" "scheduler subprocess wrapper forbids shell command execution" - assert_file_contains "$scheduler_file" "check=True" "scheduler subprocess wrapper raises on failed commands" - assert_file_contains "$REPO_ROOT/tests/test_pr_review_merge_scheduler.py" "test_run_passes_shell_metacharacters_as_plain_arguments" "scheduler tests prove branch-like shell metacharacters stay argv data" - assert_file_contains "$scheduler_file" "dispatch_strix_evidence" "scheduler dispatches same-head Strix evidence before OpenCode review" - assert_file_contains "$scheduler_file" '"--method"' "scheduler reads active workflow runs with GET query parameters" - assert_file_contains "$scheduler_file" "--security-workflow" "scheduler allows the canonical Strix workflow name to be configured" - assert_file_contains "$scheduler_file" "same-head OpenCode dispatched" "scheduler records review dispatch after completed security evidence" - assert_file_contains "$workflow_file" "--pr-number" "scheduler scopes required-workflow PR events to the current pull request" - assert_file_contains "$workflow_file" "--review-workflow \"Required OpenCode Review\"" "scheduler dispatches the canonical required OpenCode Review workflow" - assert_file_contains "$readme_file" "docs/pr-review-and-merge-procedure.md" "README points operators to the bot/agent review procedure instead of embedding it" - assert_file_contains "$procedure_file" "PR_REVIEW_MERGE_TOKEN" "review procedure documents that mechanical branch updates and merges use the central mutation credential" - assert_file_contains "$fix_workflow_file" 'workflow_call:' "fix scheduler can run as the central reusable autofix-dispatch workflow" - assert_file_contains "$fix_workflow_file" 'repository: ContextualWisdomLab/.github' "fix scheduler checks out the canonical implementation instead of relying on repo-local scheduler code" - assert_file_contains "$fix_workflow_file" 'AUTOFIX_REPOSITORY' "fix scheduler can dispatch the central autofix worker without per-repository workflow copies" - assert_file_contains "$fix_workflow_file" 'GH_TOKEN: ${{ secrets.PR_REVIEW_MERGE_TOKEN || secrets.OPENCODE_APPROVE_TOKEN || github.token }}' "fix scheduler uses central mutation credentials before falling back to the workflow token" - assert_file_contains "$fix_workflow_file" "python3 scripts/ci/pr_review_fix_scheduler.py --self-test" "fix scheduler self-tests the central dispatch contract before scanning" - assert_file_contains "$autofix_workflow_file" "github.event.client_payload.target_repository" "central autofix worker accepts the repository that owns the PR through default-branch repository dispatch" - assert_file_contains "$autofix_workflow_file" "types: [pr-review-autofix]" "central autofix worker exposes only the default-branch repository-dispatch entrypoint" - assert_file_not_contains "$autofix_workflow_file" "workflow_dispatch:" "central autofix worker cannot load privileged code from a caller-selected ref" - assert_file_contains "$autofix_workflow_file" "Autofix only supports same-repository PR heads." "central autofix worker refuses external heads before mutation" - assert_file_contains "$autofix_workflow_file" "reasoningEffort" "central autofix worker raises reasoning effort for models that support it" - assert_file_contains "$fix_scheduler_file" "current-head OpenCode requested changes" "fix scheduler dispatches only for current-head actionable review evidence" - assert_file_contains "$fix_scheduler_file" "DEFAULT_AUTOFIX_REPOSITORY" "fix scheduler defaults to the central autofix workflow repository" - assert_file_contains "$fix_scheduler_file" '"target_repository": repo' "fix scheduler passes the target repository in the central repository-dispatch JSON payload" - assert_file_contains "$fix_scheduler_file" "recent autofix marker exists for this head" "fix scheduler avoids repeated autofix loops for the same head" - assert_file_contains "$fix_scheduler_file" "external PR head is not writable" "fix scheduler refuses external heads for bot autofix" - assert_file_contains "$procedure_file" "PR Review Fix Scheduler" "review procedure documents the central autofix scheduler contract" - assert_file_contains "$procedure_file" "Scratch PoC files are not" "review procedure documents PoC proof artifacts are scratch evidence, not committed changes" - assert_file_contains "$procedure_file" "committed." "review procedure documents scratch PoC proof artifacts are not committed" - assert_file_contains "$procedure_file" "Failed GitHub Checks are not reviewed as URL lists." "review procedure documents failed-check reviews require explanations, not URL-only bullets" -} - -assert_opencode_review_normalizer_accepts_transcript_json() { - local tmp_dir - local output_file - local changed_files_file - local rc - local gate_result - tmp_dir="$(mktemp -d)" - output_file="$tmp_dir/opencode-output.md" - changed_files_file="$tmp_dir/opencode-changed-files.txt" - - cat >"$changed_files_file" <<'EOF' -.github/workflows/opencode-review.yml -scripts/ci/opencode_review_normalize_output.py -scripts/ci/test_strix_quick_gate.sh -EOF - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after structural exploration of .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/test_strix_quick_gate.sh self-test evidence passed. Coverage: Coverage execution evidence reported 100% test coverage. Docstring coverage: Coverage execution evidence reported 100% docstring coverage. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to GitHub Actions review job and verification path. PoC/execution: scratch PoC executed bash scripts/ci/test_strix_quick_gate.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and shell conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} -EOF - - set +e - RUNNER_TEMP="$tmp_dir" OPENCODE_CHANGED_FILES_FILE="$changed_files_file" \ - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" - rc=$? - set -e - - assert_equals "0" "$rc" "opencode review normalizer accepts transcript-embedded current-run JSON" - assert_file_contains "$output_file" "" "opencode review normalizer writes the gate sentinel" - assert_file_contains "$output_file" "" - - cat >"$changed_files_file" <<'EOF' -.github/workflows/opencode-review.yml -scripts/ci/opencode_review_normalize_output.py -scripts/ci/test_strix_quick_gate.sh -EOF - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" - - cat >"$output_file" <<'EOF' - - - - -But that is not meticulous. - -We should request changes. -EOF - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" - - set +e - gate_result="$( - RUNNER_TEMP="$tmp_dir" OPENCODE_CHANGED_FILES_FILE="$changed_files_file" \ - bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ - "abc123" "42" "1" "$output_file" "$normalized_json" - )" - rc=$? - set -e - - assert_equals "0" "$rc" "opencode publish sanitizer accepts the first valid control block" - assert_equals "APPROVE" "$gate_result" "opencode publish sanitizer preserves the valid gate result" - - { - printf '%s\n\n' "$sentinel" - printf '\n' - } >"$comment_body_file" - - assert_file_contains "$comment_body_file" '"result":"APPROVE"' "opencode publish sanitizer keeps normalized approval JSON" - assert_file_not_contains "$comment_body_file" "But that is not meticulous." "opencode publish sanitizer drops trailing model prose" - assert_file_not_contains "$comment_body_file" "We should request changes." "opencode publish sanitizer drops contradictory trailing model prose" - - rm -rf "$tmp_dir" -} - -assert_opencode_review_gate_rejects_missing_structural_exploration_approval() { - local tmp_dir - local output_file - local changed_files_file - local RUNNER_TEMP - local OPENCODE_CHANGED_FILES_FILE - local rc - local gate_result - tmp_dir="$(mktemp -d)" - output_file="$tmp_dir/opencode-output.md" - changed_files_file="$tmp_dir/opencode-changed-files.txt" - RUNNER_TEMP="$tmp_dir" - OPENCODE_CHANGED_FILES_FILE="$changed_files_file" - export RUNNER_TEMP OPENCODE_CHANGED_FILES_FILE - cat >"$changed_files_file" <<'EOF' -.github/workflows/opencode-review.yml -scripts/ci/opencode_review_normalize_output.py -scripts/ci/test_strix_quick_gate.sh -EOF - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found, but structural exploration was not possible.","summary":"This docs-only PR does not require structural review and the evidence was truncated.","findings":[]} -EOF - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects approvals that admit missing structural exploration" - assert_file_contains "$tmp_dir/normalize.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for missing structural exploration" - - cat >"$output_file" <<'EOF' - - - -EOF - - set +e - gate_result="$( - bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ - "abc123" "42" "1" "$output_file" - )" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode approval gate rejects approvals that admit missing structural exploration" - assert_equals "NO_CONCLUSION" "$gate_result" "missing structural exploration rejection gate result" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after structural exploration of changed files.","summary":"CodeGraph evidence was insufficient for one generated artifact, but local inspection covered the changed workflow, scripts, and tests.","findings":[]} -EOF - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize-valid.out" 2>"$tmp_dir/normalize-valid.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects approvals that omit concrete changed-file evidence" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after structural exploration of .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/test_strix_quick_gate.sh self-test evidence passed. Coverage: Coverage execution evidence reported 100% test coverage. Docstring coverage: Coverage execution evidence reported 100% docstring coverage. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to GitHub Actions review job and verification path. PoC/execution: scratch PoC executed bash scripts/ci/test_strix_quick_gate.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and shell conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} -EOF - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize-valid.out" 2>"$tmp_dir/normalize-valid.err" - rc=$? - set -e - - assert_equals "0" "$rc" "opencode normalizer accepts approvals that name concrete changed-file evidence after structural inspection" - - rm -rf "$tmp_dir" -} - -assert_opencode_review_gate_rejects_unmeasured_coverage_approval() { - local tmp_dir - local output_file - local changed_files_file - local RUNNER_TEMP - local OPENCODE_CHANGED_FILES_FILE - local rc - local gate_result - tmp_dir="$(mktemp -d)" - output_file="$tmp_dir/opencode-output.md" - changed_files_file="$tmp_dir/opencode-changed-files.txt" - RUNNER_TEMP="$tmp_dir" - OPENCODE_CHANGED_FILES_FILE="$changed_files_file" - export RUNNER_TEMP OPENCODE_CHANGED_FILES_FILE - printf '%s\n' '.github/workflows/opencode-review.yml' >"$changed_files_file" - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/test_strix_quick_gate.sh self-test evidence passed. Coverage: not measured. Docstring coverage: not measured. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to GitHub Actions review job and verification path. PoC/execution: scratch PoC executed bash scripts/ci/test_strix_quick_gate.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and shell conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} -EOF - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects approvals with unmeasured coverage" - assert_file_contains "$tmp_dir/normalize.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for unmeasured coverage approval" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/test_strix_quick_gate.sh self-test evidence passed. Coverage: Not applicable. Docstring coverage: Not applicable. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to GitHub Actions review job and verification path. PoC/execution: scratch PoC executed bash scripts/ci/test_strix_quick_gate.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and shell conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} -EOF - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize-na.out" 2>"$tmp_dir/normalize-na.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects approvals with not-applicable coverage" - assert_file_contains "$tmp_dir/normalize-na.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for not-applicable coverage approval" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/test_strix_quick_gate.sh self-test evidence passed. Coverage: Coverage execution evidence reports test coverage as not applicable because no supported changed source files or package manifests were found. Docstring coverage: Coverage execution evidence reports docstring coverage as not applicable because no supported changed source files or package manifests were found. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to GitHub Actions review job and verification path. PoC/execution: scratch PoC executed bash scripts/ci/test_strix_quick_gate.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and shell conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} -EOF - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize-no-source.out" 2>"$tmp_dir/normalize-no-source.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects no-source coverage claims for source-like changes" - assert_file_contains "$tmp_dir/normalize-no-source.err" "NO_CONCLUSION" "opencode normalizer exposes the contradictory no-source coverage rejection" - - cat >"$output_file" <<'EOF' - - - -EOF - - set +e - gate_result="$( - bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ - "abc123" "42" "1" "$output_file" - )" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode approval gate rejects approvals when coverage evidence did not run" - assert_equals "NO_CONCLUSION" "$gate_result" "unmeasured coverage approval rejection gate result" - - rm -rf "$tmp_dir" -} - -assert_opencode_review_gate_rejects_no_changes_approval() { - local tmp_dir - local output_file - local RUNNER_TEMP - local rc - local gate_result - tmp_dir="$(mktemp -d)" - output_file="$tmp_dir/opencode-output.md" - RUNNER_TEMP="$tmp_dir" - export RUNNER_TEMP - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No changes detected in the PR head source directory.","summary":"No files or changes were found in the PR head source directory, indicating no actionable changes to review.","findings":[]} -EOF - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects no-changes approvals" - assert_file_contains "$tmp_dir/normalize.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for no-changes approval" - - cat >"$output_file" <<'EOF' - - - -EOF - - set +e - gate_result="$( - bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ - "abc123" "42" "1" "$output_file" - )" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode approval gate rejects no-changes approvals" - assert_equals "NO_CONCLUSION" "$gate_result" "no-changes approval rejection gate result" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "Never approve with a reason or summary that says no changes" "opencode prompt rejects no-changes approvals when bounded evidence lists changed files" - - rm -rf "$tmp_dir" -} - -assert_opencode_review_gate_rejects_approve_without_changed_file_evidence() { - local tmp_dir - local output_file - local changed_files_file - local RUNNER_TEMP - local OPENCODE_CHANGED_FILES_FILE - local rc - local gate_result - tmp_dir="$(mktemp -d)" - output_file="$tmp_dir/opencode-output.md" - changed_files_file="$tmp_dir/opencode-changed-files.txt" - RUNNER_TEMP="$tmp_dir" - OPENCODE_CHANGED_FILES_FILE="$changed_files_file" - export RUNNER_TEMP OPENCODE_CHANGED_FILES_FILE - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blocking issues found; changes improve CI configuration and documentation.","summary":"PR enhances OpenCode review workflow with clearer guidance and validation. Changes are well-contained with no security or functional regressions detected.","findings":[]} -EOF - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects approvals without changed-file evidence" - assert_file_contains "$tmp_dir/normalize.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for approvals without changed-file evidence" - - cat >"$output_file" <<'EOF' - - - -EOF - - set +e - gate_result="$( - bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ - "abc123" "42" "1" "$output_file" - )" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode approval gate rejects approvals without changed-file evidence" - assert_equals "NO_CONCLUSION" "$gate_result" "missing changed-file evidence rejection gate result" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "Before APPROVE, the summary must include at least one exact changed file path inspected as changed-file evidence" "opencode prompt requires changed-file evidence before approval" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "when result is APPROVE the JSON findings value must be exactly []" "opencode prompt keeps approval findings empty" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "Put all required Verification posture labels inside the JSON summary string itself" "opencode prompt keeps approval evidence inside the control JSON" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "never say no source files changed, no test files changed, or no executable changes when exact changed-file evidence lists workflow, script, source, or test files" "opencode prompt rejects contradictory changed-file kind claims" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "Never approve material workflow, script, source, config, package, or test changes with a reason or summary that says simple typo fix" "opencode prompt rejects trivial approval claims for material changes" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "OPENCODE_CHANGED_FILES_FILE" "opencode workflow exports exact current-head changed files" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'git -C "$OPENCODE_SOURCE_WORKDIR" diff --name-only --find-renames "$PR_MERGE_BASE" "$PR_HEAD_SHA" |' "opencode workflow derives exact changed files from the PR-head worktree" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'awk '\''NF > 0 && $0 !~ /^\// && $0 !~ /(^|\/)\.\.($|\/)/ { print }'\'' >"$OPENCODE_CHANGED_FILES_FILE"' "opencode workflow writes path-safe exact changed files for the normalizer" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "changed-files.txt" "opencode workflow copies exact changed-file evidence into the isolated review workspace" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'A["text"]' "opencode prompt requires quoted Mermaid labels" - assert_file_contains "$REPO_ROOT/scripts/ci/opencode_review_comment_helpers.sh" 'S%s["%s"]' "opencode generated Mermaid surface labels are quoted" - assert_file_contains "$REPO_ROOT/scripts/ci/opencode_review_comment_helpers.sh" 'R%s["Review risk: %s"]' "opencode generated Mermaid risk labels are quoted" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'emit_review_body_to_action_log "$event" "$body"' "opencode PR-level review bodies are mirrored to the Actions log" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'emit_review_body_to_action_log "$event" "$body" "$review_payload_file"' "opencode inline review bodies are mirrored to the Actions log" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'OpenCode is publishing this review content to PR #%s.' "opencode Actions log includes the review body that is being posted" - assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" '## OpenCode %s review body' "opencode Step Summary includes the review body that is being posted" - - cat >"$changed_files_file" <<'EOF' -.github/workflows/opencode-review.yml -scripts/ci/opencode_review_normalize_output.py -scripts/ci/test_strix_quick_gate.sh -EOF - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting README.md.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed README.md. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/other_gate_test.sh self-test evidence passed. Coverage: Coverage execution evidence reported 100% test coverage. Docstring coverage: Coverage execution evidence reported 100% docstring coverage. DAG: CodeGraph behavior DAG rendered README.md to docs review path. PoC/execution: scratch PoC executed bash scripts/ci/other_gate_test.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions docs. Compatibility/convention: conventions match existing code. Breaking-change/backcompat: no public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web docs and review-comment output was checked. Accessibility/i18n: human-readable docs and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token boundaries preserved.","findings":[]} -EOF - - set +e - OPENCODE_CHANGED_FILES_FILE="$changed_files_file" \ - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/nonchanged-normalize.out" 2>"$tmp_dir/nonchanged-normalize.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects approvals that cite non-changed files when exact changed-file evidence is available" - assert_file_contains "$tmp_dir/nonchanged-normalize.err" "NO_CONCLUSION" "opencode normalizer reports no conclusion for non-changed-file approval evidence" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: Not applicable (no source files changed). TDD/regression: Not applicable (no test files changed). Coverage: Coverage execution evidence reported 100% test coverage. Docstring coverage: Coverage execution evidence reported 100% docstring coverage. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to review decision path. PoC/execution: Not applicable (no executable changes). DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and Python conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} -EOF - - set +e - OPENCODE_CHANGED_FILES_FILE="$changed_files_file" \ - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/contradictory-normalize.out" 2>"$tmp_dir/contradictory-normalize.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects approvals that deny changed source/test/executable surfaces" - assert_file_contains "$tmp_dir/contradictory-normalize.err" "NO_CONCLUSION" "opencode normalizer reports no conclusion for contradictory changed-file kind claims" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and Python syntax evidence passed. TDD/regression: normalizer self-test evidence passed. Coverage: Coverage execution evidence reported 100% test coverage. Docstring coverage: Coverage execution evidence reported 100% docstring coverage. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to scripts/ci/opencode_review_normalize_output.py to review decision path. PoC/execution: scratch PoC executed the normalizer with exact changed-file evidence and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and Python conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} -EOF - - set +e - OPENCODE_CHANGED_FILES_FILE="$changed_files_file" \ - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/changed-normalize.out" 2>"$tmp_dir/changed-normalize.err" - rc=$? - set -e - - assert_equals "0" "$rc" "opencode normalizer accepts approvals that cite exact current changed files" - - rm -rf "$tmp_dir" -} - -assert_opencode_review_gate_rejects_line_zero_findings() { - local tmp_dir - local output_file - local RUNNER_TEMP - local rc - local gate_result - tmp_dir="$(mktemp -d)" - output_file="$tmp_dir/opencode-output.md" - RUNNER_TEMP="$tmp_dir" - export RUNNER_TEMP - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" - - cat >"$output_file" <<'EOF' - - - -EOF - - set +e - gate_result="$( - bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ - "abc123" "42" "1" "$output_file" - )" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode approval gate rejects line zero findings" - assert_equals "NO_CONCLUSION" "$gate_result" "line zero rejection gate result" - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects line zero findings" - assert_file_contains "$tmp_dir/normalize.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for line zero findings" - - cat >"$output_file" <<'EOF' -OpenCode transcript text before the review control block. - -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Boolean line blocker","summary":"Boolean line values are not concrete source locations.","findings":[{"path":"scripts/ci/example.sh","line":true,"severity":"HIGH","title":"Boolean line","problem":"Boolean line values are not actionable.","root_cause":"The review did not inspect a concrete line.","fix_direction":"Inspect the actual file and cite a positive integer line number.","regression_test_direction":"Add a gate test for boolean line rejection.","suggested_diff":"diff --git a/scripts/ci/example.sh b/scripts/ci/example.sh\n--- a/scripts/ci/example.sh\n+++ b/scripts/ci/example.sh\n@@ -1 +1 @@\n-old\n+new"}]} -EOF - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/bool-line.out" 2>"$tmp_dir/bool-line.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects boolean line findings" - assert_file_contains "$tmp_dir/bool-line.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for boolean line findings" - - rm -rf "$tmp_dir" -} - -assert_opencode_review_gate_rejects_placeholder_findings() { - local tmp_dir - local output_file - local RUNNER_TEMP - local rc - local gate_result - tmp_dir="$(mktemp -d)" - output_file="$tmp_dir/opencode-output.md" - RUNNER_TEMP="$tmp_dir" - export RUNNER_TEMP - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" - - cat >"$output_file" <<'EOF' - - - -EOF - - set +e - gate_result="$( - bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ - "abc123" "42" "1" "$output_file" - )" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode approval gate rejects placeholder findings" - assert_equals "NO_CONCLUSION" "$gate_result" "placeholder finding rejection gate result" - - rm -rf "$tmp_dir" -} - -assert_opencode_review_gate_rejects_non_source_backed_findings() { - local tmp_dir - local output_file - local stderr_file - local changed_files_file - local RUNNER_TEMP - local OPENCODE_CHANGED_FILES_FILE - local rc - local gate_result - tmp_dir="$(mktemp -d)" - output_file="$tmp_dir/opencode-output.md" - stderr_file="$tmp_dir/gate.err" - changed_files_file="$tmp_dir/opencode-changed-files.txt" - RUNNER_TEMP="$tmp_dir" - OPENCODE_CHANGED_FILES_FILE="$changed_files_file" - export RUNNER_TEMP OPENCODE_CHANGED_FILES_FILE - printf '%s\n' 'scripts/ci/opencode_review_approve_gate.sh' >"$changed_files_file" - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" - - cat >"$output_file" <<'EOF' - - - -EOF - - set +e - gate_result="$( - bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ - "abc123" "42" "1" "$output_file" 2>"$stderr_file" - )" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode approval gate rejects non-source-backed findings" - assert_equals "NO_CONCLUSION" "$gate_result" "non-source-backed finding rejection gate result" - assert_file_contains "$stderr_file" "REQUEST_CHANGES finding is not source-backed by the current-head diff" "non-source-backed finding rejection explains the invalid model result" - - rm -rf "$tmp_dir" -} - -assert_opencode_review_gate_rejects_generic_failed_check_deflection() { - local tmp_dir - local output_file - local RUNNER_TEMP - local rc - local gate_result - tmp_dir="$(mktemp -d)" - output_file="$tmp_dir/opencode-output.md" - RUNNER_TEMP="$tmp_dir" - export RUNNER_TEMP - seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" - - cat >"$output_file" <<'EOF' - - - -EOF - - set +e - gate_result="$( - bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ - "abc123" "42" "1" "$output_file" - )" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode approval gate rejects generic failed-check deflections" - assert_equals "NO_CONCLUSION" "$gate_result" "generic failed-check deflection rejection gate result" - - set +e - python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ - "abc123" "42" "1" "$output_file" >"$tmp_dir/generic-deflection.out" 2>"$tmp_dir/generic-deflection.err" - rc=$? - set -e - - assert_equals "4" "$rc" "opencode normalizer rejects generic failed-check deflections" - assert_file_contains "$tmp_dir/generic-deflection.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for generic failed-check deflections" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_review_validator_rejects_unrelated_findings() { - local tmp_dir - local control_json - local failed_checks_file - local evidence_file - local rc - tmp_dir="$(mktemp -d)" - control_json="$tmp_dir/control.json" - failed_checks_file="$tmp_dir/failed-checks.txt" - evidence_file="$tmp_dir/failed-check-evidence.md" - - cat >"$failed_checks_file" <<'EOF' -- Strix Security Scan/strix: FAILURE (https://github.com/example/repo/actions/runs/1/job/2) -EOF - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -### Failed job steps - -- step 6: Self-test Strix gate script (failure) - -### Strix vulnerability report window 1 - -Model github-models/openai/gpt-5 Vulnerabilities 1 -│ Vulnerability Report │ -│ Title: Authentication Bypass via X-Dev-User Header │ -│ Severity: CRITICAL │ -│ Endpoint: /api/me │ -│ Method: GET │ -│ Location 1: backend/app/auth.py:132-135 │ - -### Strix vulnerability report window 2 - -Model deepseek/deepseek-v3-0324 Vulnerabilities 1 -│ Vulnerability Report │ -│ Title: Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure │ -│ Severity: HIGH │ - -### Failed log excerpt - -FAIL: strix workflow defaults PR Strix scans to GitHub Models GPT-5 (missing 'github.event.client_payload.strix_llm || 'openai/gpt-5'') -FAIL: strix workflow rejects unsupported model inputs (missing 'STRIX_LLM must select GitHub Models openai/gpt-5 or newer, direct OpenAI GPT-5.4 or newer, OpenRouter openrouter/free, or an approved organization Vertex AI model') -FAIL: opencode failed-check diagnosis prefers DeepSeek V3 (missing 'MODEL: github-models/deepseek/deepseek-v3-0324') -EOF - cat >"$control_json" <<'EOF' -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Generic security concern","summary":"Generic speculative CI issues.","findings":[{"path":"scripts/ci/collect_failed_check_evidence.sh","line":15,"severity":"HIGH","title":"Generic finding","problem":"Speculative input validation issue unrelated to failed checks.","root_cause":"The review did not use the failed Strix evidence.","fix_direction":"Add generic validation.","regression_test_direction":"Add a generic test.","suggested_diff":"diff --git a/scripts/ci/collect_failed_check_evidence.sh b/scripts/ci/collect_failed_check_evidence.sh\n--- a/scripts/ci/collect_failed_check_evidence.sh\n+++ b/scripts/ci/collect_failed_check_evidence.sh\n@@ -1 +1 @@\n-old\n+new"}]} -EOF - - set +e - bash "$REPO_ROOT/scripts/ci/validate_opencode_failed_check_review.sh" \ - "$control_json" "$failed_checks_file" "$evidence_file" >"$tmp_dir/bad.out" 2>"$tmp_dir/bad.err" - rc=$? - set -e - assert_equals "4" "$rc" "failed-check review validator rejects unrelated findings" - assert_file_contains "$tmp_dir/bad.out" "FAILED_CHECK_EVIDENCE_NOT_REFERENCED" "failed-check validator explains unrelated finding rejection" - assert_file_contains "$tmp_dir/bad.out" "review does not" "failed-check validator logs the missing evidence linkage" - - cat >"$control_json" <<'EOF' -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Strix Security Scan/strix failed","summary":"No deterministic missing-string markers or Strix report locations were recognized. Use the failed-check evidence below to map each failed check to exact local source lines before approving.","findings":[{"path":"scripts/ci/collect_failed_check_evidence.sh","line":15,"severity":"HIGH","title":"Generic failed-check deflection","problem":"No deterministic missing-string markers or Strix report locations were recognized.","root_cause":"The review did not map Strix Security Scan/strix to failed log evidence and concrete local source lines.","fix_direction":"Inspect the failed-check evidence and produce source-backed findings instead of handing the mapping back to the reader.","regression_test_direction":"Reject generic failed-check deflections before publishing reviews.","suggested_diff":"diff --git a/scripts/ci/collect_failed_check_evidence.sh b/scripts/ci/collect_failed_check_evidence.sh\n--- a/scripts/ci/collect_failed_check_evidence.sh\n+++ b/scripts/ci/collect_failed_check_evidence.sh\n@@ -1 +1 @@\n-old\n+new"}]} -EOF - set +e - bash "$REPO_ROOT/scripts/ci/validate_opencode_failed_check_review.sh" \ - "$control_json" "$failed_checks_file" "$evidence_file" >"$tmp_dir/generic.out" 2>"$tmp_dir/generic.err" - rc=$? - set -e - assert_equals "4" "$rc" "failed-check review validator rejects generic failed-check deflections" - assert_file_contains "$tmp_dir/generic.out" "FAILED_CHECK_EVIDENCE_NOT_REFERENCED" "failed-check validator blocks generic deflection review text" - assert_file_contains "$tmp_dir/generic.out" "punts failed-check diagnosis back to the reader" "failed-check validator logs generic deflection reason" - - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -### Strix vulnerability report window 1 - -Model github-models/openai/gpt-5 Vulnerabilities 1 -│ Vulnerability Report │ -│ Title: Authentication Bypass via X-Dev-User Header │ -│ Severity: CRITICAL │ -│ Endpoint: /api/me │ -│ Method: GET │ -│ Location 1: backend/app/auth.py:132-135 │ - -### Strix vulnerability report window 2 - -Model deepseek/deepseek-v3-0324 Vulnerabilities 1 -│ Vulnerability Report │ -│ Title: Authentication Bypass via X-Dev-User Header │ -│ Severity: CRITICAL │ -│ Endpoint: /api/me │ -│ Method: GET │ -│ Location 1: backend/app/auth.py:132-135 │ -EOF - cat >"$control_json" <<'EOF' -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Strix Security Scan/strix failed","summary":"Strix Security Scan/strix failed and reported github-models/openai/gpt-5 plus deepseek/deepseek-v3-0324 Authentication Bypass via X-Dev-User Header with Severity: CRITICAL, /api/me, Method: GET, backend/app/auth.py:132-135.","findings":[{"path":"backend/app/auth.py","line":132,"severity":"CRITICAL","title":"Authentication Bypass via X-Dev-User Header","problem":"Strix Security Scan/strix failed with github-models/openai/gpt-5 and deepseek/deepseek-v3-0324 reports for Authentication Bypass via X-Dev-User Header, Severity: CRITICAL, /api/me, Method: GET, backend/app/auth.py:132-135.","root_cause":"The review collapsed two Strix model reports into one finding.","fix_direction":"Remove the unauthenticated fallback at backend/app/auth.py:132-135.","regression_test_direction":"Add auth tests for both request paths.","suggested_diff":"diff --git a/backend/app/auth.py b/backend/app/auth.py\n--- a/backend/app/auth.py\n+++ b/backend/app/auth.py\n@@ -132 +132 @@\n-old\n+new"}]} -EOF - set +e - bash "$REPO_ROOT/scripts/ci/validate_opencode_failed_check_review.sh" \ - "$control_json" "$failed_checks_file" "$evidence_file" >"$tmp_dir/collapsed.out" 2>"$tmp_dir/collapsed.err" - rc=$? - set -e - assert_equals "4" "$rc" "failed-check review validator rejects collapsed duplicate Strix model reports" - assert_file_contains "$tmp_dir/collapsed.out" "FAILED_CHECK_EVIDENCE_NOT_REFERENCED" "failed-check validator requires one Strix-specific finding per model report" - assert_file_contains "$tmp_dir/collapsed.out" "distinct source-backed findings" "failed-check validator logs collapsed Strix report reason" - - cat >"$control_json" <<'EOF' -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Strix Security Scan/strix failed","summary":"Strix Security Scan/strix failed and mentioned github-models/openai/gpt-5 plus deepseek/deepseek-v3-0324, but the model reports were still collapsed.","findings":[{"path":".github/workflows/strix.yml","line":120,"severity":"HIGH","title":"Strix self-test failed","problem":"Strix Security Scan/strix failed in Self-test Strix gate script while github-models/openai/gpt-5 and deepseek/deepseek-v3-0324 model reports were present elsewhere in the evidence.","root_cause":"The workflow finding is about CI self-test evidence, not a distinct model vulnerability report.","fix_direction":"Fix the workflow default.","regression_test_direction":"Keep the self-test assertion.","suggested_diff":"diff --git a/.github/workflows/strix.yml b/.github/workflows/strix.yml\n--- a/.github/workflows/strix.yml\n+++ b/.github/workflows/strix.yml\n@@ -120 +120 @@\n-old\n+new"},{"path":"backend/app/auth.py","line":132,"severity":"CRITICAL","title":"Authentication Bypass via X-Dev-User Header","problem":"Strix Security Scan/strix failed with github-models/openai/gpt-5 and deepseek/deepseek-v3-0324 reports for Authentication Bypass via X-Dev-User Header, Severity: CRITICAL, /api/me, Method: GET, backend/app/auth.py:132-135.","root_cause":"This finding still collapses two Strix model reports into one item even though the titles and locations match.","fix_direction":"Remove the unauthenticated fallback at backend/app/auth.py:132-135.","regression_test_direction":"Add auth tests for both request paths.","suggested_diff":"diff --git a/backend/app/auth.py b/backend/app/auth.py\n--- a/backend/app/auth.py\n+++ b/backend/app/auth.py\n@@ -132 +132 @@\n-old\n+new"}]} -EOF - set +e - bash "$REPO_ROOT/scripts/ci/validate_opencode_failed_check_review.sh" \ - "$control_json" "$failed_checks_file" "$evidence_file" >"$tmp_dir/collapsed-with-count.out" 2>"$tmp_dir/collapsed-with-count.err" - rc=$? - set -e - assert_equals "4" "$rc" "failed-check review validator rejects collapsed Strix reports even when finding count matches" - assert_file_contains "$tmp_dir/collapsed-with-count.out" "FAILED_CHECK_EVIDENCE_NOT_REFERENCED" "failed-check validator requires distinct matching findings, not only matching counts" - - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -### Failed job steps - -- step 6: Self-test Strix gate script (failure) - -### Strix vulnerability report window 1 - -Model github-models/openai/gpt-5 Vulnerabilities 1 -│ Vulnerability Report │ -│ Title: Authentication Bypass via X-Dev-User Header │ -│ Severity: CRITICAL │ -│ Endpoint: /api/me │ -│ Method: GET │ -│ Location 1: backend/app/auth.py:132-135 │ - -### Strix vulnerability report window 2 - -Model deepseek/deepseek-v3-0324 Vulnerabilities 1 -│ Vulnerability Report │ -│ Title: Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure │ -│ Severity: HIGH │ - -### Failed log excerpt - -FAIL: strix workflow defaults PR Strix scans to GitHub Models GPT-5 (missing 'github.event.client_payload.strix_llm || 'openai/gpt-5'') -FAIL: strix workflow rejects unsupported model inputs (missing 'STRIX_LLM must select GitHub Models openai/gpt-5 or newer, direct OpenAI GPT-5.4 or newer, OpenRouter openrouter/free, or an approved organization Vertex AI model') -FAIL: opencode failed-check diagnosis prefers DeepSeek V3 (missing 'MODEL: github-models/deepseek/deepseek-v3-0324') -EOF - - cat >"$control_json" <<'EOF' -{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Strix Security Scan/strix failed","summary":"Strix Security Scan/strix failed in Self-test Strix gate script and reported github-models/openai/gpt-5 Authentication Bypass via X-Dev-User Header with Severity: CRITICAL at backend/app/auth.py:132-135 plus deepseek/deepseek-v3-0324 Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure with Severity: HIGH.","findings":[{"path":".github/workflows/strix.yml","line":120,"severity":"HIGH","title":"Strix workflow default is not visible to trusted self-test","problem":"Strix Security Scan/strix failed in Self-test Strix gate script: strix workflow defaults PR Strix scans to GitHub Models GPT-5 (missing 'github.event.client_payload.strix_llm || 'openai/gpt-5''); strix workflow rejects unsupported model inputs (missing 'STRIX_LLM must select GitHub Models openai/gpt-5 or newer, direct OpenAI GPT-5.4 or newer, OpenRouter openrouter/free, or an approved organization Vertex AI model'); opencode failed-check diagnosis prefers DeepSeek V3 (missing 'MODEL: github-models/deepseek/deepseek-v3-0324'). The same failed Strix evidence includes github-models/openai/gpt-5 report Authentication Bypass via X-Dev-User Header, Severity: CRITICAL, /api/me, Method: GET, backend/app/auth.py:132-135.","root_cause":"The failed check evidence shows Self-test Strix gate script could not find github.event.client_payload.strix_llm, STRIX_LLM must select, and MODEL: github-models/deepseek/deepseek-v3-0324 in trusted-base files, and the model report identifies the backend auth fallback line.","fix_direction":"Update the workflow lines that provide the Strix model default and OpenCode model env so the trusted self-test can find those exact strings, then remove the unauthenticated X-Dev-User fallback at backend/app/auth.py:132-135.","regression_test_direction":"Keep the static self-test assertions for all three missing strings and add auth tests proving /api/me rejects forged X-Dev-User requests without signed auth.","suggested_diff":"diff --git a/.github/workflows/strix.yml b/.github/workflows/strix.yml\n--- a/.github/workflows/strix.yml\n+++ b/.github/workflows/strix.yml\n@@ -120 +120 @@\n- STRIX_MODEL: old\n+ STRIX_MODEL: ${{ github.event.client_payload.strix_llm || 'openai/gpt-5' }}"},{"path":"frontend/src/app/page.tsx","line":1,"severity":"HIGH","title":"Strix frontend model report must be reviewed separately","problem":"Strix Security Scan/strix failed with a separate deepseek/deepseek-v3-0324 report: Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure, Severity: HIGH.","root_cause":"The failed Strix evidence contains a second model vulnerability report, so OpenCode must not collapse it into the first backend finding.","fix_direction":"Inspect the frontend source lines responsible for token storage, hardcoded credentials, dynamic error rendering, and missing CSP, then remove or harden each concrete line before approval.","regression_test_direction":"Add frontend tests covering safe token/session handling, output encoding, and security headers for the affected route.","suggested_diff":"diff --git a/frontend/src/app/page.tsx b/frontend/src/app/page.tsx\n--- a/frontend/src/app/page.tsx\n+++ b/frontend/src/app/page.tsx\n@@ -1 +1 @@\n-export default function Page() { return null }\n+export default function Page() { return null }"}]} -EOF - set +e - bash "$REPO_ROOT/scripts/ci/validate_opencode_failed_check_review.sh" \ - "$control_json" "$failed_checks_file" "$evidence_file" >"$tmp_dir/good.out" 2>"$tmp_dir/good.err" - rc=$? - set -e - assert_equals "0" "$rc" "failed-check review validator accepts Strix log-backed findings" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_emits_each_strix_report() { - local tmp_dir - local fixture_repo - local evidence_file - local output_file - local stderr_file - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - stderr_file="$tmp_dir/fallback.err" - mkdir -p "$fixture_repo/backend/services" "$fixture_repo/frontend/src/app/prompt-studio" "$fixture_repo/frontend" - - { - for _ in $(seq 1 59); do - printf '# filler\n' - done - printf 'filename = part.get_filename()\n' - } >"$fixture_repo/backend/services/email_parser.py" - { - for _ in $(seq 1 28); do - printf '// filler\n' - done - printf 'setTestResult(await apiClient.post("/prompt-studio", payload));\n' - } >"$fixture_repo/frontend/src/app/prompt-studio/page.tsx" - { - for _ in $(seq 1 34); do - printf '// filler\n' - done - printf 'const nextConfig = {};\n' - } >"$fixture_repo/frontend/next.config.ts" - - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -### Failed log signal summary - -```text -strix Run Strix (quick) LLM CONNECTION FAILED -strix Run Strix (quick) Strix fallback model 'deepseek/deepseek-r1-0528' emitted provider infrastructure or failure-signal output; trying next configured fallback if available. -``` - -### Strix vulnerability report window 1 - -Model deepseek/deepseek-r1-0528 Vulnerabilities 2 -│ Vulnerability Report │ -│ Title: Path Traversal in Email Attachment Handling │ -│ Severity: CRITICAL │ -│ Endpoint: /services/email_parser.py │ -│ Location 1: backend/services/email_parser.py:60-72 │ -│ Vulnerability Report │ -│ Title: Prompt Injection and XSS in AI Prompt Studio │ -│ Severity: HIGH │ -│ Endpoint: /prompt-studio │ -│ Location 1: frontend/src/app/prompt-studio/page.tsx:29-32 │ - -### Strix vulnerability report window 2 - -Model deepseek/deepseek-v3-0324 Vulnerabilities 1 -│ Vulnerability Report │ -│ Title: Missing Content Security Policy in Next.js Frontend │ -│ Severity: HIGH │ -│ Endpoint: all frontend pages │ -EOF - - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" - - assert_file_contains "$output_file" "Strix report from deepseek/deepseek-r1-0528: Path Traversal in Email Attachment Handling" "fallback includes first model report" - assert_file_contains "$output_file" "backend/services/email_parser.py:60" "fallback maps first report to exact source line" - assert_file_contains "$output_file" "Strix report from deepseek/deepseek-r1-0528: Prompt Injection and XSS in AI Prompt Studio" "fallback includes second report from same model" - assert_file_contains "$output_file" "frontend/src/app/prompt-studio/page.tsx:29" "fallback maps second report to exact source line" - assert_file_contains "$output_file" "Strix report from deepseek/deepseek-v3-0324: Missing Content Security Policy in Next.js Frontend" "fallback includes report from second model" - assert_file_contains "$output_file" "frontend/next.config.ts:35" "fallback derives a concrete CSP hardening line" - assert_file_contains "$output_file" "Suggested edit: change \`frontend/next.config.ts:35\`" "fallback provides a concrete suggested edit for model reports" - assert_file_contains "$output_file" "Strix provider signal left current-head security evidence incomplete" "fallback still reports provider failure after vulnerability reports" - assert_file_not_contains "$output_file" "failed before producing vulnerability reports" "fallback does not contradict preserved Strix report windows" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_explains_pytest_and_cancelled_checks() { - local tmp_dir - local fixture_repo - local evidence_file - local output_file - local stderr_file - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - stderr_file="$tmp_dir/fallback.err" - mkdir -p "$fixture_repo/tests/live" - - cat >"$fixture_repo/tests/live/test_live_api_sequence.py" <<'EOF' -"""Live HTTP integration harness tests.""" - -from pathlib import Path - - -def test_live_harness_avoids_broad_url_opener_pattern() -> None: - source = Path(__file__).read_text(encoding="utf-8") - unsafe_terms = ("urllib.request", "urlopen") - - for unsafe_term in unsafe_terms: - assert unsafe_term not in source -EOF - - cat >"$evidence_file" <<'EOF' -# Failed GitHub Check Evidence - -- PR: #744 -- Head SHA: `fc6d263e9fcfdcf4d710427618ee511b64331dd0` -- Repository: `ContextualWisdomLab/naruon` - -## Failed check: Application CI/backend (Python 3.14) - -- Type: `check_run` -- Conclusion: `FAILURE` -- Details URL: https://github.com/ContextualWisdomLab/naruon/actions/runs/27946373277/job/82692061303 - -### Failed job steps - -- step 6: Run backend tests (failure) - -### Failed log excerpt - -```text -backend (Python 3.14) Run backend tests pytest -q -backend (Python 3.14) Run backend tests =================================== FAILURES =================================== -backend (Python 3.14) Run backend tests ______________ test_live_harness_avoids_broad_url_opener_pattern _______________ -backend (Python 3.14) Run backend tests def test_live_harness_avoids_broad_url_opener_pattern() -> None: -backend (Python 3.14) Run backend tests unsafe_terms = ("urllib.request", "urlopen") -backend (Python 3.14) Run backend tests > assert unsafe_term not in source -backend (Python 3.14) Run backend tests E assert 'urllib.request' not in '"""Live HTT... in source\n' -backend (Python 3.14) Run backend tests E 'urllib.request' is contained here: -backend (Python 3.14) Run backend tests E terms = ("urllib.request", "urlopen") -backend (Python 3.14) Run backend tests tests/live/test_live_api_sequence.py:10: AssertionError -backend (Python 3.14) Run backend tests FAILED tests/live/test_live_api_sequence.py::test_live_harness_avoids_broad_url_opener_pattern - assert 'urllib.request' not in '"""Live HTT... in source\n' -backend (Python 3.14) Run backend tests 1 failed, 965 passed, 15 skipped in 7.28s -``` - -## Failed check: PR Governance/metadata-only gate evaluation - -- Type: `check_run` -- Conclusion: `CANCELLED` -- Details URL: https://github.com/ContextualWisdomLab/naruon/actions/runs/27946373334/job/82692061348 - -### Check annotations - -- .github:1-1 [failure] Canceling since a higher priority waiting request for PR Governance-744 exists -EOF - - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" - - assert_file_contains "$output_file" "Failed GitHub Check needs a source-backed pytest fix for test_live_harness_avoids_broad_url_opener_pattern" "fallback explains pytest failure with the test name" - assert_file_contains "$output_file" "tests/live/test_live_api_sequence.py:" "fallback maps pytest failure to a source file and line" - assert_file_contains "$output_file" "urllib.request" "fallback preserves the assertion term that caused the pytest failure" - assert_file_contains "$output_file" "cd backend && python -m pytest tests/live/test_live_api_sequence.py::test_live_harness_avoids_broad_url_opener_pattern -q" "fallback gives a focused pytest rerun command" - assert_file_not_contains "$output_file" "GitHub Checks queue - PR Governance/metadata-only gate evaluation was cancelled by a newer queued request" "fallback does not publish cancelled queue states as source-backed findings" - assert_file_contains "$stderr_file" "Non-source-backed cancelled check queue state" "fallback explains cancelled governance checks outside source-backed findings" - assert_file_contains "$stderr_file" "no repository source edit is justified by this cancelled check alone" "fallback does not invent source fixes for cancelled queue state" - assert_file_not_contains "$output_file" "No deterministic missing-string markers" "fallback must not fall back to generic evidence-dump text when pytest evidence is actionable" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_maps_supply_chain_vulnerabilities() { - local tmp_dir - local fixture_repo - local evidence_file - local output_file - local stderr_file - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - stderr_file="$tmp_dir/fallback.err" - mkdir -p "$fixture_repo" - - cat >"$fixture_repo/requirements.txt" <<'EOF' -flask==2.0.1 -requests==2.19.0 -urllib3==1.25.0 -EOF - - cat >"$evidence_file" <<'EOF' -# Failed GitHub Check Evidence - -- PR: #23 -- Head SHA: `abc123def456abc123def456abc123def456abcd` -- Repository: `ContextualWisdomLab/clearfolio` - -## Failed check: OSV-Scanner/osv-scan - -- Type: `check_run` -- Conclusion: `FAILURE` -- Details URL: https://github.com/ContextualWisdomLab/clearfolio/actions/runs/28863381355 - -### Supply-chain vulnerability findings - -- Supply-chain vulnerability: id=GHSA-j8r2-6x86-q33q severity=HIGH package=requests installed=2.19.0 fixed=2.31.0 manifest=requirements.txt - -## Failed check: Security Scan/trivy-fs - -- Type: `check_run` -- Conclusion: `FAILURE` -- Details URL: https://github.com/ContextualWisdomLab/clearfolio/actions/runs/28863381999 - -### Failed log excerpt - -```text -requirements.txt (pip) -======================= -Total: 1 (HIGH: 1, CRITICAL: 0) - -┌──────────┬────────────────┬──────────┬────────┬───────────────────┬───────────────┐ -│ Library │ Vulnerability │ Severity │ Status │ Installed Version │ Fixed Version │ -├──────────┼────────────────┼──────────┼────────┼───────────────────┼───────────────┤ -│ urllib3 │ CVE-2023-43804 │ HIGH │ fixed │ 1.25.0 │ 1.26.18 │ -└──────────┴────────────────┴──────────┴────────┴───────────────────┴───────────────┘ -``` -EOF - - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" - - # osv-scanner canonical evidence: source-backed finding with the exact manifest line and from->to bump. - assert_file_contains "$output_file" "requirements.txt:2 - Supply-chain vulnerability GHSA-j8r2-6x86-q33q in requests" "supply-chain fallback maps the osv-scanner advisory to the exact manifest line" - assert_file_contains "$output_file" "bump \`requests\` from 2.19.0 to 2.31.0" "supply-chain fallback states the concrete requests version bump" - assert_file_contains "$output_file" "OSV-Scanner/osv-scan" "supply-chain fallback preserves the failed osv-scanner check label as evidence" - # trivy-fs job-log table: source-backed finding located under the manifest header. - assert_file_contains "$output_file" "requirements.txt:3 - Supply-chain vulnerability CVE-2023-43804 in urllib3" "supply-chain fallback maps the trivy table row to the exact manifest line" - assert_file_contains "$output_file" "bump \`urllib3\` from 1.25.0 to 1.26.18" "supply-chain fallback states the concrete urllib3 version bump" - assert_file_contains "$output_file" "urllib3==1.26.18" "supply-chain fallback offers a GitHub-suggestion-ready pin for the trivy finding" - assert_file_contains "$output_file" "requests==2.31.0" "supply-chain fallback offers a GitHub-suggestion-ready pin for the osv finding" - # Never line 0, and no URL-only deflection. - assert_file_not_contains "$output_file" ":0 - Supply-chain" "supply-chain fallback never emits a line-zero finding" - assert_file_not_contains "$output_file" "see the Actions run URL" "supply-chain fallback does not post URL-only supply-chain reviews" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_preserves_empty_supply_chain_columns() { - # Regression for the record-delimiter bug: the internal per-vulnerability - # record was joined with a TAB and read back with `IFS=$'\t'`. Tab is an - # IFS-whitespace character, so `read` collapsed consecutive tabs and any empty - # interior field (missing installed OR missing fixed) shifted every later - # column left by one — producing garbled findings such as a severity word in - # the advisory-id slot and a CVE id in the version slot. The collector appends - # installed=/fixed= only when present, so both are common real inputs. - local tmp_dir - local fixture_repo - local evidence_file - local output_file - local stderr_file - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - stderr_file="$tmp_dir/fallback.err" - mkdir -p "$fixture_repo" - - cat >"$fixture_repo/requirements.txt" <<'EOF' -flask==2.0.1 -requests==2.19.0 -EOF - - # Record 1: installed is MISSING (osv/trivy SARIF alert with no installed - # version). Record 2: fixed is MISSING (no-fix advisory). Both interior gaps - # used to collapse and shift columns. - cat >"$evidence_file" <<'EOF' -# Failed GitHub Check Evidence - -- PR: #77 -- Head SHA: `abc123def456abc123def456abc123def456abcd` -- Repository: `ContextualWisdomLab/clearfolio` - -## Failed check: OSV-Scanner/osv-scan - -- Type: `check_run` -- Conclusion: `FAILURE` -- Details URL: https://github.com/ContextualWisdomLab/clearfolio/actions/runs/28863381355 - -### Supply-chain vulnerability findings - -- Supply-chain vulnerability: id=CVE-2020-0001 severity=CRITICAL package=flask fixed=2.0.2 manifest=requirements.txt -- Supply-chain vulnerability: id=GHSA-aaaa-bbbb-cccc severity=HIGH package=requests installed=2.19.0 manifest=requirements.txt -EOF - - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" - - # Record 1 (installed missing): the advisory id must be the CVE (NOT the - # severity word), the package must be flask, and the fix target must be the - # fixed VERSION (2.0.2), never the CVE id in the version slot. - assert_file_contains "$output_file" "Supply-chain vulnerability CVE-2020-0001 in flask" "empty installed keeps the advisory id in the title, not the severity word" - assert_file_not_contains "$output_file" "Supply-chain vulnerability CRITICAL in flask" "empty installed does not shift the severity word into the advisory-id slot" - assert_file_contains "$output_file" "upgrade \`flask\` to 2.0.2" "empty installed still names the concrete fixed version as the upgrade target" - assert_file_not_contains "$output_file" "to CVE-2020-0001" "the CVE id never appears in the upgrade/version slot" - - # Record 2 (fixed missing): the advisory id must be the GHSA (NOT the severity - # word), installed must be the real version, and the fix must say no upstream - # fix is available — never 'bump ... to '. - assert_file_contains "$output_file" "Supply-chain vulnerability GHSA-aaaa-bbbb-cccc in requests" "empty fixed keeps the advisory id in the title, not the severity word" - assert_file_contains "$output_file" "no fixed version is available upstream for \`requests\` 2.19.0" "empty fixed produces a sensible no-fix instruction with the real installed version" - assert_file_not_contains "$output_file" "to GHSA-aaaa-bbbb-cccc" "the GHSA id never appears in the upgrade/version slot" - assert_file_not_contains "$output_file" "from GHSA-aaaa-bbbb-cccc" "the GHSA id never appears in the from-version slot" - - # Columns are not shifted: severity lands in the severity slot for both. - assert_file_contains "$output_file" "CRITICAL requirements.txt" "record 1 severity stays in the severity column" - assert_file_contains "$output_file" "HIGH requirements.txt" "record 2 severity stays in the severity column" - - # Line numbers stay positive (never 0), even with empty interior fields. - assert_file_not_contains "$output_file" ":0 - Supply-chain" "empty interior fields never produce a line-zero finding" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_rejects_url_only_supply_chain() { - local tmp_dir - local fixture_repo - local evidence_file - local output_file - local stderr_file - local rc - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - stderr_file="$tmp_dir/fallback.err" - mkdir -p "$fixture_repo" - - # A supply-chain check failed, but the evidence carries only the check name - # and a run URL — no package, advisory id, manifest, or fixed version. This - # must stay fail-closed: no source-backed finding can be invented. - cat >"$evidence_file" <<'EOF' -# Failed GitHub Check Evidence - -- PR: #24 -- Head SHA: `abc123def456abc123def456abc123def456abcd` -- Repository: `ContextualWisdomLab/clearfolio` - -## Failed check: OSV-Scanner/osv-scan - -- Type: `check_run` -- Conclusion: `FAILURE` -- Details URL: https://github.com/ContextualWisdomLab/clearfolio/actions/runs/28863381355 -EOF - - set +e - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" - rc=$? - set -e - - assert_equals "1" "$rc" "URL-only supply-chain evidence does not produce a REQUEST_CHANGES finding" - assert_file_not_contains "$output_file" "Supply-chain vulnerability" "URL-only supply-chain evidence emits no supply-chain finding" - assert_file_contains "$stderr_file" "No source-backed failed-check fallback finding matched" "URL-only supply-chain evidence stays fail-closed and asks for rerun or newer logs" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_rejects_cancelled_queue_only_reviews() { - local tmp_dir - local fixture_repo - local evidence_file - local output_file - local stderr_file - local rc - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - stderr_file="$tmp_dir/fallback.err" - mkdir -p "$fixture_repo" - - cat >"$evidence_file" <<'EOF' -# Failed GitHub Check Evidence - -- PR: #119 -- Head SHA: `96ce73d581b4ddeb8668f93768deb2b106b8f55a` -- Repository: `ContextualWisdomLab/.github` - -## Failed check: PR Review Merge Scheduler/scan-pr-queue - -- Type: `check_run` -- Conclusion: `CANCELLED` -- Details URL: https://github.com/ContextualWisdomLab/.github/actions/runs/28354829112/job/83995330163 - -### Check annotations - -- .github:1-1 [failure] Canceling since a higher priority waiting request for central-pr-review-merge-scheduler-ContextualWisdomLab/.github exists -EOF - - set +e - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" - rc=$? - set -e - - assert_equals "1" "$rc" "cancelled queue-only evidence does not produce REQUEST_CHANGES findings" - assert_file_contains "$stderr_file" "Non-source-backed cancelled check queue state" "cancelled queue-only evidence is explained as non-source-backed" - assert_file_contains "$stderr_file" "No source-backed failed-check fallback finding matched" "cancelled queue-only evidence asks for rerun or newer logs" - assert_file_not_contains "$output_file" "GitHub Checks queue" "cancelled queue-only evidence does not emit a finding" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_explains_trusted_base_strix_prs() { - local tmp_dir - local fixture_repo - local evidence_file - local output_file - local base_sha - local head_sha - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - - mkdir -p "$fixture_repo/.github/workflows" - cat >"$fixture_repo/.github/workflows/strix.yml" <<'EOF' -name: Strix Security Scan -concurrency: - cancel-in-progress: false -EOF - - git init -q "$fixture_repo" >/dev/null - git -C "$fixture_repo" config user.email "copilot@example.com" - git -C "$fixture_repo" config user.name "copilot" - git -C "$fixture_repo" add .github/workflows/strix.yml - git -C "$fixture_repo" commit -m "base" >/dev/null - base_sha="$(git -C "$fixture_repo" rev-parse HEAD)" - - cat >"$fixture_repo/.github/workflows/strix.yml" <<'EOF' -name: Strix Security Scan -concurrency: - group: strix-${{ github.event.pull_request.number || github.ref }} - cancel-in-progress: false -EOF - git -C "$fixture_repo" add .github/workflows/strix.yml - git -C "$fixture_repo" commit -m "head" >/dev/null - head_sha="$(git -C "$fixture_repo" rev-parse HEAD)" - - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -Conclusion: cancelled - -No GitHub Actions job log is available for this failed workflow run. -EOF - - PR_BASE_SHA="$base_sha" PR_HEAD_SHA="$head_sha" \ - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" - - assert_file_contains "$output_file" "cancelled pull_request_target run still used the base branch copies" "fallback explains trusted-base workflow execution" - assert_file_contains "$output_file" "Re-run Strix after the trusted base branch contains the workflow/gate change or capture equivalent temporary evidence tied to this head SHA" "fallback directs reviewers to trusted-base rerun or equivalent evidence" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_does_not_treat_no_report_summary_as_report() { - local tmp_dir - local evidence_file - local output_file - tmp_dir="$(mktemp -d)" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -### Failed log signal summary - -```text -strix Run Strix (quick) openai.RateLimitError: Too many requests. -strix Run Strix (quick) httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://api.deepseek.com/beta/chat/completions' -strix Run Strix (quick) litellm.BadRequestError: DeepseekException - {"error":{"message":"Authentication Fails, Your api key is invalid"}} -strix Run Strix (quick) Configured model and fallback models were unavailable. -``` - -No Strix vulnerability report windows were detected in the failed log. -EOF - - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$REPO_ROOT" >"$output_file" - - assert_file_contains "$output_file" "Strix provider failure blocked current-head security evidence" "fallback treats no-report summary as provider blocker" - assert_file_contains "$output_file" "api.deepseek.com" "fallback preserves direct DeepSeek endpoint failure evidence" - assert_file_contains "$output_file" "Authentication Fails" "fallback preserves direct DeepSeek authentication failure evidence" - assert_file_contains "$output_file" "github_models/deepseek/deepseek-v3-0324 github_models/deepseek/deepseek-r1-0528" "fallback gives exact GitHub Models fallback list" - assert_file_contains "$output_file" "Suggested edit: \`.github/workflows/strix.yml" "fallback gives a line-specific suggested edit for provider routing" - assert_file_not_contains "$output_file" "Strix provider signal left current-head security evidence incomplete" "fallback does not invent vulnerability report windows from a no-report summary" - assert_file_not_contains "$output_file" "after vulnerability reports" "fallback does not contradict no-report evidence" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_handles_deepseek_auth_only_signal() { - local tmp_dir - local evidence_file - local output_file - tmp_dir="$(mktemp -d)" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -### Failed log signal summary - -```text -strix Run Strix (quick) httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://api.deepseek.com/beta/chat/completions' -strix Run Strix (quick) litellm.BadRequestError: DeepseekException - {"error":{"message":"Authentication Fails, Your api key is invalid"}} -``` - -No Strix vulnerability report windows were detected in the failed log. -EOF - - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$REPO_ROOT" >"$output_file" - - assert_file_contains "$output_file" "Strix provider failure blocked current-head security evidence" "fallback treats DeepSeek auth-only logs as provider blockers" - assert_file_contains "$output_file" "api.deepseek.com" "fallback preserves DeepSeek auth-only endpoint evidence" - assert_file_contains "$output_file" "Authentication Fails" "fallback preserves DeepSeek auth-only failure evidence" - assert_file_contains "$output_file" "Suggested edit: \`.github/workflows/strix.yml" "fallback gives suggested edit for DeepSeek auth-only provider routing" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_handles_pg_erd_cloud_strix_log_shape() { - local tmp_dir - local fixture_repo - local evidence_file - local output_file - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - - mkdir -p "$fixture_repo/backend/app" "$fixture_repo/frontend" - for line_number in $(seq 1 150); do - printf '# auth fixture line %s\n' "$line_number" - done >"$fixture_repo/backend/app/auth.py" - cat >"$fixture_repo/frontend/next.config.ts" <<'EOF' -import type { NextConfig } from "next"; - -const nextConfig: NextConfig = { - async headers() { - return []; - }, -}; - -export default nextConfig; -EOF - - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -### Failed log signal summary - -```text -strix Run Strix (quick) Strix run failed for model 'deepseek/deepseek-r1-0528' after 206s (exit code 2). -strix Run Strix (quick) Below-threshold findings detected, but infrastructure errors occurred during this pipeline run; refusing bypass due to potentially incomplete scan. -strix Run Strix (quick) Unable to map Strix findings to changed files; failing closed for pull request. -``` - -### Strix vulnerability report window 1 - -│ Vulnerability Report │ -│ Title: Authentication Bypass via X-Dev-User Header │ -│ Severity: CRITICAL │ -│ Target: /workspace/strix-pr-scope.I4RF8w │ -│ Endpoint: /api/me │ -│ Method: GET │ -│ Code Locations │ -│ Location 1: backend/app/auth.py:132-135 │ -│ Model deepseek/deepseek-r1-0528 │ -│ Vulnerabilities 1 │ - -### Strix vulnerability report window 2 - -│ Vulnerability Report │ -│ Title: Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure │ -│ Data Handling │ -│ Severity: HIGH │ -│ Target: /workspace/strix-pr-scope.I4RF8w/frontend │ -│ Model deepseek/deepseek-v3-0324 │ -│ Vulnerabilities 1 │ -EOF - - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" - - assert_file_contains "$output_file" "Strix report from deepseek/deepseek-r1-0528: Authentication Bypass via X-Dev-User Header" "fallback includes pg-erd-cloud first model report" - assert_file_contains "$output_file" "backend/app/auth.py:132" "fallback maps pg-erd-cloud auth report to exact line" - assert_file_contains "$output_file" "Endpoint: /api/me. Method: GET" "fallback preserves pg-erd-cloud endpoint and method" - assert_file_contains "$output_file" "Strix report from deepseek/deepseek-v3-0324: Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure Data Handling" "fallback preserves wrapped pg-erd-cloud frontend title" - assert_file_contains "$output_file" "frontend/next.config.ts:3" "fallback anchors locationless frontend report to a concrete frontend hardening line" - assert_file_contains "$output_file" "Suggested edit: change \`frontend/next.config.ts:3\`" "fallback provides pg-erd-cloud frontend suggested edit" - assert_file_contains "$output_file" "Unable to map Strix findings" "fallback preserves failed Strix mapping signal" - assert_file_contains "$output_file" "Strix provider signal left current-head security evidence incomplete" "fallback reports incomplete Strix evidence after model findings" - assert_file_not_contains "$output_file" "failed before producing vulnerability reports" "fallback does not erase model findings after provider signals" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_handles_split_code_location_lines() { - local tmp_dir - local fixture_repo - local evidence_file - local output_file - local migration_file - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - migration_file="$fixture_repo/backend/alembic/versions/0002_provider_writeback_retry_queue.py" - - mkdir -p "$(dirname "$migration_file")" - for line_number in $(seq 1 80); do - if [ "$line_number" -eq 43 ]; then - printf '\tlegacy_index_execution_placeholder(statement)\n' - else - printf '# migration fixture line %s\n' "$line_number" - fi - done >"$migration_file" - - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -### Failed log signal summary - -```text -strix Run Strix (quick) Strix fallback model 'github_models/deepseek/deepseek-r1-0528' emitted provider infrastructure or failure-signal output; trying next configured fallback if available. -strix Run Strix (quick) Strix reported zero vulnerabilities before provider infrastructure failure; failing closed because provider infrastructure failures are not clean scan evidence. -``` - -### Strix vulnerability report window 1 - -│ Vulnerability Report │ -│ Title: SQL Injection Vulnerability in Database Script │ -│ Severity: HIGH │ -│ Target: │ -│ /workspace/strix-pr-scope.e0AHf4/backend/alembic/versions/0002_provider_wr │ -│ iteback_retry_queue.py │ -│ Code Locations │ -│ │ -│ Location 1: │ -│ backend/alembic/versions/0002_provider_writeback_retry_queue.py:43 │ -│ Vulnerable code location │ -│ legacy_index_execution_placeholder(statement) │ -│ Model openai/deepseek/deepseek-r1-0528 │ -│ Vulnerabilities 1 │ -EOF - - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" - - assert_file_contains "$output_file" "Strix report from openai/deepseek/deepseek-r1-0528: SQL Injection Vulnerability in Database Script" "fallback includes split-location Strix report" - assert_file_contains "$output_file" "backend/alembic/versions/0002_provider_writeback_retry_queue.py:43" "fallback maps split Code Locations path to exact line" - assert_file_contains "$output_file" "Code location evidence: backend/alembic/versions/0002_provider_writeback_retry_queue.py:43" "fallback preserves split Code Locations evidence" - assert_file_contains "$output_file" "Suggested edit: change \`backend/alembic/versions/0002_provider_writeback_retry_queue.py:43\`" "fallback gives suggested edit for split Code Locations" - assert_file_not_contains "$output_file" "Strix report did not include a mappable Code Location" "fallback does not misclassify split Code Locations as unmapped" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_does_not_anchor_unmapped_strix_reports_to_workflow() { - local tmp_dir - local fixture_repo - local evidence_file - local output_file - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - - mkdir -p "$fixture_repo/.github/workflows" "$fixture_repo/scripts/ci" - cat >"$fixture_repo/.github/workflows/strix.yml" <<'EOF' -name: Strix Security Scan -jobs: - strix: - steps: - - name: Run Strix - env: - STRIX_FALLBACK_MODELS: github_models/deepseek/deepseek-v3-0324 github_models/deepseek/deepseek-r1-0528 -EOF - - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -### Failed log signal summary - -```text -strix Run Strix (quick) Below-threshold findings detected, but infrastructure errors occurred during this pipeline run; refusing bypass due to potentially incomplete scan. -strix Run Strix (quick) Unable to map Strix findings to changed files; failing closed for pull request. -``` - -### Strix vulnerability report window 1 - -│ Vulnerability Report │ -│ Title: Insecure Direct Object Reference (IDOR) in User Profile API │ -│ Severity: MEDIUM │ -│ Target: /workspace/strix-pr-scope.mVhTAV/backend │ -│ Code Locations │ -│ Location 1: backend/api/users.py:45-52 │ -│ Model github_models/deepseek/deepseek-v3-0324 │ -│ Vulnerabilities 1 │ -EOF - - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" - - assert_file_contains "$output_file" "Strix provider signal left current-head security evidence incomplete" "fallback reports incomplete Strix evidence for unmapped report" - assert_file_contains "$output_file" "did not map to an existing repository file" "fallback explains unmapped Strix report" - assert_file_contains "$output_file" "Insecure Direct Object Reference (IDOR) in User Profile API" "fallback preserves unmapped report title as diagnostic evidence" - assert_file_not_contains "$output_file" "Strix report from github_models/deepseek/deepseek-v3-0324" "fallback does not convert unmapped report into source finding" - assert_file_not_contains "$output_file" "Inspect and patch .github/workflows/strix.yml" "fallback does not anchor unmapped report to workflow line" - assert_file_not_contains "$output_file" "backend/api/users.py:45" "fallback does not cite nonexistent source path as actionable line" - - rm -rf "$tmp_dir" -} - -assert_opencode_failed_check_fallback_maps_strix_status_permission_smoke_failure() { - local tmp_dir - local fixture_repo - local evidence_file - local output_file - tmp_dir="$(mktemp -d)" - fixture_repo="$tmp_dir/repo" - evidence_file="$tmp_dir/failed-check-evidence.md" - output_file="$tmp_dir/fallback.md" - - mkdir -p "$fixture_repo/.github/workflows" "$fixture_repo/scripts/ci" - cat >"$fixture_repo/.github/workflows/strix.yml" <<'EOF' -name: Strix Security Scan -jobs: - strix: - permissions: - contents: read - statuses: write -EOF - - cat >"$evidence_file" <<'EOF' -## Failed check: Strix Security Scan/strix - -### Failed log signal summary - -```text -strix Self-test Strix required workflow contract Running bounded Strix required-workflow smoke test. -strix Self-test Strix required workflow contract FAIL: Strix workflow keeps GITHUB_TOKEN status permissions read-only (unexpected 'statuses: write') -strix Self-test Strix required workflow contract Strix required workflow smoke test failed with 1 failure(s). -``` -EOF - - bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ - "$evidence_file" "$fixture_repo" >"$output_file" - - assert_file_contains "$output_file" "Strix required workflow must keep GITHUB_TOKEN statuses read-only" "fallback maps Strix smoke permission failure" - assert_file_contains "$output_file" ".github/workflows/strix.yml:6" "fallback cites the exact statuses write line" - assert_file_contains "$output_file" 'change `.github/workflows/strix.yml:6` from `statuses: write` to `statuses: read`' "fallback gives a concrete status-permission repair" - assert_file_not_contains "$output_file" "No source-backed failed-check fallback finding matched" "fallback does not leave Strix smoke failure undiagnosed" - - rm -rf "$tmp_dir" -} - -assert_internal_pr_scope_targets() { - local target_log_file="$1" - local repo_root_dir="$2" - local expected_count="$3" - - if [ ! -f "$target_log_file" ]; then - record_failure "internal PR scope target log should exist" - return - fi - - local actual_count=0 - local target_path - while IFS= read -r target_path; do - actual_count=$((actual_count + 1)) - case "$target_path" in - "$repo_root_dir" | "$repo_root_dir"/*) - record_failure "internal PR scope target should not reuse repository path: $target_path" - ;; - esac - case "$(basename -- "$target_path")" in - strix-pr-scope.*) - ;; - *) - record_failure "internal PR scope target should be generated by build_pull_request_scope_dir: $target_path" - ;; - esac - done <"$target_log_file" - - assert_equals "$expected_count" "$actual_count" "internal PR scope target count" -} - -run_gate_case() { - local scenario="$1" - local initial_model="$2" - local fallback_models="$3" - local expected_exit="$4" - local expected_message="$5" - local expected_calls="$6" - local expected_model_sequence="${7:-}" - local expected_api_base_sequence="${8:-}" - local default_provider="${9-vertex_ai}" - local raw_llm_api_base_override="${10-__DEFAULT__}" - local initial_llm_api_base="${11-}" - - local raw_llm_api_base="https://example.invalid/generateContent" - if [ "$raw_llm_api_base_override" != "__DEFAULT__" ]; then - raw_llm_api_base="$raw_llm_api_base_override" - elif [ "$default_provider" = "openai" ]; then - raw_llm_api_base="" - fi - local transient_retry_per_model="${12-0}" - local min_fail_severity="${13-CRITICAL}" - local transient_retry_backoff_seconds="${14:-0}" - local custom_target_path="${15-}" - local custom_source_dirs="${16-}" - local process_timeout_seconds="${17-1200}" - local total_timeout_seconds="${18-0}" - local github_event_name="${19-}" - local changed_files_override="${20-}" - local event_name_override="${21-}" - local legacy_scope_size_ignored="${22-}" - local disable_pr_scoping="${23-0}" - local test_pr_sca_status_override="${24-}" - local current_pr_number="${25-}" - local authoritative_sca_runs_json="${26-}" - local gemini_fallback_models="${27-__SAME_AS_FALLBACK_MODELS__}" - local generic_fallback_models="${28-}" - local fail_on_provider_signal="${29-1}" - if [ "$default_provider" = "openai" ] && [ -z "$generic_fallback_models" ] && [ -n "$fallback_models" ]; then - generic_fallback_models="$fallback_models" - fallback_models="" - fi - - if [ -n "${STRIX_TEST_CASE_FILTER:-}" ] && [ "$scenario" != "$STRIX_TEST_CASE_FILTER" ]; then - return - fi - if [ "${STRIX_TEST_TRACE_CASES:-0}" = "1" ]; then - printf 'RUN_GATE_CASE: %s\n' "$scenario" >&2 - fi - - local tmp_dir - tmp_dir="$(mktemp -d)" - # Separate bin/ (fake strix + helper files) from workspace/ (target path) - # so grep -r over the target path never matches the fake strix script itself. - local bin_dir="$tmp_dir/bin" - local untrusted_bin_dir="$tmp_dir/untrusted-bin" - local workspace_dir="$tmp_dir/workspace" - local repo_root_dir="$workspace_dir/smart-crawling-server" - mkdir -p "$bin_dir" "$untrusted_bin_dir" "$repo_root_dir/src" - mkdir -p "$repo_root_dir/scripts/ci" - local gate_under_test="$repo_root_dir/scripts/ci/strix_quick_gate.sh" - cp "$GATE_SCRIPT" "$gate_under_test" - cp "$REPO_ROOT/scripts/ci/strix_model_utils.sh" "$repo_root_dir/scripts/ci/strix_model_utils.sh" - chmod +x "$gate_under_test" - local fake_strix="$bin_dir/strix" - local path_hijack_log="$tmp_dir/path-hijack.log" - cat >"$untrusted_bin_dir/strix" <<'EOF' -#!/usr/bin/env bash -printf 'inherited PATH executable was invoked\n' >"${FAKE_STRIX_PATH_HIJACK_LOG:?}" -exit 99 -EOF - chmod +x "$untrusted_bin_dir/strix" - local call_log="$tmp_dir/calls.log" - local api_base_log="$tmp_dir/api_base.log" - local target_log="$tmp_dir/target.log" - local runtime_env_log="$tmp_dir/runtime_env.log" - local state_file="$tmp_dir/state.log" - local strix_llm_file="$tmp_dir/strix_llm.txt" - local llm_api_key_file="$tmp_dir/llm_api_key.txt" - local llm_api_base_file="$tmp_dir/llm_api_base.txt" - local output_log="$tmp_dir/output.log" - local fake_gh="$bin_dir/gh" - local gh_token_log="$tmp_dir/gh_token.log" - local event_payload_file="$tmp_dir/github_event.json" - - # Resolve target path: use repo-local relative defaults to mirror the real workflow. - local effective_target_path="." - if [ "$custom_target_path" = "__USE_SUBDIR_SRC__" ]; then - # Simulate STRIX_TARGET_PATH=./src with a repo-local relative path. - effective_target_path="./src" - elif [ -n "$custom_target_path" ]; then - effective_target_path="$custom_target_path" - # Ensure the custom target path exists - mkdir -p "$effective_target_path" - fi - - cat >"$fake_strix" <<'EOF' -#!/usr/bin/env bash -set -euo pipefail - -printf '%s\n' "${STRIX_LLM:-}" >> "${FAKE_STRIX_CALL_LOG:?}" -printf '%s\n' "${LLM_API_BASE:-}" >> "${FAKE_STRIX_API_BASE_LOG:?}" -if [ -n "${FAKE_STRIX_RUNTIME_ENV_LOG:-}" ]; then - printf 'LLM_TIMEOUT=%s;STRIX_MEMORY_COMPRESSOR_TIMEOUT=%s;STRIX_REASONING_EFFORT=%s;STRIX_LLM_MAX_RETRIES=%s;GEMINI_LOCATION=%s;PYTHONWARNINGS=%s;NPM_CONFIG_IGNORE_SCRIPTS=%s;PNPM_CONFIG_IGNORE_SCRIPTS=%s;YARN_ENABLE_SCRIPTS=%s;UNRELATED_SECRET=%s;LLM_DISABLE_STREAMING=%s\n' \ - "${LLM_TIMEOUT:-}" \ - "${STRIX_MEMORY_COMPRESSOR_TIMEOUT:-}" \ - "${STRIX_REASONING_EFFORT:-}" \ - "${STRIX_LLM_MAX_RETRIES:-}" \ - "${GEMINI_LOCATION:-}" \ - "${PYTHONWARNINGS:-}" \ - "${NPM_CONFIG_IGNORE_SCRIPTS:-}" \ - "${PNPM_CONFIG_IGNORE_SCRIPTS:-}" \ - "${YARN_ENABLE_SCRIPTS:-}" \ - "${UNRELATED_SECRET:-}" \ - "${LLM_DISABLE_STREAMING:-}" >> "${FAKE_STRIX_RUNTIME_ENV_LOG:?}" -fi - -target_path="" -while [ "$#" -gt 0 ]; do - if [ "$1" = "-t" ] && [ "$#" -ge 2 ]; then - target_path="$2" - break - fi - shift -done -if [ "$target_path" = "." ]; then - target_path="$PWD" -fi -printf '%s\n' "$target_path" >> "${FAKE_STRIX_TARGET_LOG:?}" - -STRIX_REPORTS_DIR="${STRIX_REPORTS_DIR:-strix_runs}" - -case "${FAKE_STRIX_SCENARIO:?}" in -success|runtime-env-forwarding|custom-openai-compatible-preserves-effort|vertex-primary-success-timing-message|direct-openai-gpt-does-not-require-github-models-api-base|pr-executable-integrity-mismatch|pr-executable-group-writable) - echo "scan ok" - exit 0 - ;; - contextual-orchestrator-gateway-model-qualification) - if [ "${STRIX_LLM:-}" != "openai/orchestrator/free" ]; then - echo "gateway model was not provider-qualified for LiteLLM" >&2 - exit 10 - fi - if [ "${LLM_API_BASE:-}" != "http://127.0.0.1:18080/v1" ]; then - echo "gateway API base was not preserved" >&2 - exit 11 - fi - echo "scan ok through contextual-orchestrator gateway" - exit 0 - ;; - scan-working-directory-isolated) - if [ "$PWD" = "$target_path" ] || [[ "$PWD" == "$target_path"/* ]]; then - echo "Error: Strix process inherited the untrusted scan target as cwd" >&2 - exit 81 - fi - if [ ! -f "$target_path/backend/app/pg_introspect/dsn_guard.py" ]; then - echo "Error: PostgreSQL DSN guard context missing from PR scope" >&2 - exit 82 - fi - echo "scan ok with isolated Strix working directory" - exit 0 - ;; - success-with-critical-report) - mkdir -p "$STRIX_REPORTS_DIR/fake-success/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-success/vulnerabilities/vuln-0001.md" <<'REPORT' -# Vulnerability Report - -- Severity: CRITICAL -- Title: Successful process still emitted a blocking vulnerability -REPORT - echo "Vulnerabilities 1" - exit 0 - ;; - slow-timeout) - sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" - exit 0 - ;; - timeout-disabled-success) - sleep 1 - echo "scan ok with timeout disabled" - exit 0 - ;; - vertex-primary-notfound-fallback-success|github-models-fallback-success|github-models-fallback-success-deepseek-v3|github-models-token-limit-fallback-success|github-models-fallback-requires-api-base|github-models-model-prefix-with-api-base-succeeds|github-models-meta-prefix-with-api-base-succeeds|github-models-mistral-prefix-with-api-base-succeeds) - case "${STRIX_LLM:-}" in - vertex_ai/missing-primary) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok with fallback" - exit 0 - ;; - openai/gpt-5|openai/openai/gpt-5.4|openai/meta/test-github-model|openai/mistral-ai/test-github-model) - if [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-token-limit-fallback-success" ]; then - echo "openai.APIStatusError: Error code: 413 - {'error': {'code': 'tokens_limit_reached', 'message': 'Request body too large for gpt-5 model. Max size: 4000 tokens.'}}" - exit 1 - fi - echo "scan ok with GitHub Models fallback" - exit 0 - ;; - openai/deepseek/deepseek-r1-0528) - if [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-fallback-success-deepseek-v3" ]; then - echo "LLM CONNECTION FAILED" - echo "Could not establish connection to the language model." - echo "Error: litellm.BadRequestError: OpenAIException - Unavailable model: deepseek-r1-0528" - exit 1 - fi - echo "scan ok with GitHub Models fallback" - exit 0 - ;; - openai/deepseek/deepseek-v3-0324) - echo "scan ok with GitHub Models fallback" - exit 0 - ;; - *) - echo "unexpected model ${STRIX_LLM:-}" >&2 - exit 9 - ;; - esac - ;; - nvidia-rate-limit-openai-direct-fallback-clears-api-base) - case "${STRIX_LLM:-}" in - nvidia_nim/nvidia/rate-limited-primary) - echo "LLM CONNECTION FAILED" - echo "Error: litellm.RateLimitError: Nvidia_nimException - Error code: 429 Too Many Requests" - exit 1 - ;; - openai/gpt-5.4) - if [ "${STRIX_REASONING_EFFORT:-}" != "none" ]; then - echo "direct OpenAI function-tools fallback requires reasoning effort none" >&2 - exit 29 - fi - if [ "${LLM_API_KEY:-}" != "openai-fallback-token" ]; then - echo "unexpected direct-OpenAI fallback key (${LLM_API_KEY:-})" >&2 - exit 26 - fi - if [ -n "${LLM_API_BASE:-}" ]; then - echo "direct OpenAI fallback inherited foreign API base ${LLM_API_BASE}" >&2 - exit 27 - fi - echo "scan ok after direct-OpenAI fallback" - exit 0 - ;; - *) - echo "unexpected cross-provider model ${STRIX_LLM:-}" >&2 - exit 28 - ;; - esac - ;; - openai-direct-quota-github-models-fallback-success) - case "${STRIX_LLM:-}" in - openai/gpt-5.4) - if [ "${LLM_API_KEY:-}" != "dummy" ]; then - echo "unexpected direct-OpenAI key for primary (${LLM_API_KEY:-})" >&2 - exit 15 - fi - echo "Error getting response: Error code: 429 - {'error': {'message': 'You exceeded your current quota, please check your plan and billing details.', 'type': 'insufficient_quota', 'code': 'insufficient_quota'}}" - echo "openai.RateLimitError: Error code: 429" - exit 1 - ;; - openai/o3) - if [ "${LLM_API_KEY:-}" != "github-models-fallback-token" ]; then - echo "unexpected GitHub Models key for fallback (${LLM_API_KEY:-})" >&2 - exit 16 - fi - echo "scan ok with GitHub Models fallback" - exit 0 - ;; - *) - echo "unexpected model ${STRIX_LLM:-}" >&2 - exit 9 - ;; - esac - ;; - vertex-all-notfound) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - nonrecoverable) - echo "Error: transport timeout" - exit 1 - ;; - provider-prefix-required) - if [ "${STRIX_LLM:-}" = "vertex_ai/gemini-2.5-pro" ]; then - echo "scan ok with normalized provider" - exit 0 - fi - echo "Error: provider prefix not normalized (${STRIX_LLM:-})" >&2 - exit 10 - ;; - provider-prefix-fallback-normalization) - case "${STRIX_LLM:-}" in - vertex_ai/missing-primary) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after fallback normalization" - exit 0 - ;; - *) - echo "Error: fallback provider prefix not normalized (${STRIX_LLM:-})" >&2 - exit 11 - ;; - esac - ;; - provider-prefix-required-resource-path-primary-implicit-default-provider | provider-prefix-required-resource-path-primary-explicit-empty-default-provider) - if [ "${STRIX_LLM:-}" = "vertex_ai/gemini-2.5-pro" ]; then - echo "scan ok with resource-path normalization" - exit 0 - fi - echo "Error: resource-path model not normalized (${STRIX_LLM:-})" >&2 - exit 12 - ;; - provider-prefix-resource-path-primary-notfound-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/missing-primary) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after resource-path fallback" - exit 0 - ;; - *) - echo "Error: resource-path fallback model not normalized (${STRIX_LLM:-})" >&2 - exit 13 - ;; - esac - ;; - vertex-custom-model-resource-path) - # projects/

/locations//models/ (no publishers/ segment) - if [ "${STRIX_LLM:-}" = "vertex_ai/my-custom-model-123" ]; then - echo "scan ok with custom model resource-path normalization" - exit 0 - fi - echo "Error: custom model resource-path not normalized (${STRIX_LLM:-})" >&2 - exit 40 - ;; - vertex-notfound-without-status-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/missing-primary) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after status-less not found fallback" - exit 0 - ;; - *) - echo "Error: status-less fallback model not normalized (${STRIX_LLM:-})" >&2 - exit 14 - ;; - esac - ;; - vertex-notfound-compact-status-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/missing-primary) - echo 'litellm.exceptions.NotFoundError: VertexAI error' - echo '{"error":{"status":"NOT_FOUND"}}' - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after compact-status not found fallback" - exit 0 - ;; - *) - echo "Error: compact-status fallback model not normalized (${STRIX_LLM:-})" >&2 - exit 17 - ;; - esac - ;; - nonvertex-slash-model-passthrough) - if [ "${STRIX_LLM:-}" = "foo/bar" ]; then - echo "scan ok with non-vertex slash model passthrough" - exit 0 - fi - echo "Error: non-vertex slash model was rewritten (${STRIX_LLM:-})" >&2 - exit 18 - ;; - primary-duplicate-in-fallback) - case "${STRIX_LLM:-}" in - vertex_ai/missing-primary) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after duplicate-primary skip" - exit 0 - ;; - *) - echo "Error: duplicate-primary path unexpected (${STRIX_LLM:-})" >&2 - exit 15 - ;; - esac - ;; - multiline-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/missing-primary) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - vertex_ai/fallback-one) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - vertex_ai/fallback-two) - echo "scan ok after multiline fallback parsing" - exit 0 - ;; - *) - echo "Error: multiline fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 19 - ;; - esac - ;; - vertex-primary-ratelimit-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/ratelimit-primary) - echo "Penetration test failed: LLM request failed: RateLimitError" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after rate-limit fallback" - exit 0 - ;; - *) - echo "Error: ratelimit fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 21 - ;; - esac - ;; - vertex-primary-resource-exhausted-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/resource-exhausted-primary) - echo '{"error":{"status":"RESOURCE_EXHAUSTED"}}' - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after resource exhausted fallback" - exit 0 - ;; - *) - echo "Error: resource exhausted fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 23 - ;; - esac - ;; - openai-primary-quota-fallback-success) - case "${STRIX_LLM:-}" in - openai/quota-primary) - echo "openai.agents: Error streaming response: You exceeded your current quota, please check your plan and billing details." - exit 1 - ;; - openai/fallback-one) - echo "scan ok after quota fallback" - exit 0 - ;; - *) - echo "Error: quota fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 24 - ;; - esac - ;; - vertex-primary-429-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/http429-primary) - echo "litellm: HTTP 429 Too Many Requests" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after 429 fallback" - exit 0 - ;; - *) - echo "Error: 429 fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 24 - ;; - esac - ;; - vertex-primary-midstream-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/midstream-primary) - echo "Penetration test failed: LLM request failed: MidStreamFallbackError" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after midstream fallback" - exit 0 - ;; - *) - echo "Error: midstream fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 25 - ;; - esac - ;; - vertex-primary-midstream-retry-same-model-success) - case "${STRIX_LLM:-}" in - vertex_ai/retry-midstream-primary) - attempt="0" - if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then - attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" - fi - attempt="$((attempt + 1))" - echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" - if [ "$attempt" -eq 1 ]; then - echo "Penetration test failed: LLM request failed: MidStreamFallbackError" - exit 1 - fi - echo "scan ok after same-model retry" - exit 0 - ;; - vertex_ai/fallback-one) - echo "Error: fallback should not be needed for same-model retry scenario" >&2 - exit 30 - ;; - *) - echo "Error: midstream fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 30 - ;; - esac - ;; - vertex-primary-ratelimit-retry-same-model-success|vertex-primary-ratelimit-retry-reason-message) - case "${STRIX_LLM:-}" in - vertex_ai/retry-ratelimit-primary) - attempt="0" - if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then - attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" - fi - attempt="$((attempt + 1))" - echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" - if [ "$attempt" -eq 1 ]; then - echo "Penetration test failed: LLM request failed: RateLimitError" - exit 1 - fi - echo "scan ok after same-model rate-limit retry" - exit 0 - ;; - vertex_ai/fallback-one) - echo "Error: fallback should not be needed for same-model rate-limit retry scenario" >&2 - exit 31 - ;; - *) - echo "Error: rate-limit fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 31 - ;; - esac - ;; - vertex-primary-api-connection-retry-same-model-success|github-models-internal-server-connection-retry-same-model-success|internal-server-error-unrelated-output-nonretryable|internal-server-error-many-blocks-retry-same-model-success) - case "${STRIX_LLM:-}" in - gemini/retry-api-connection-primary|vertex_ai/retry-api-connection-primary|openai/openai/retry-api-connection-primary) - attempt="0" - if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then - attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" - fi - attempt="$((attempt + 1))" - echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" - if [ "$attempt" -eq 1 ]; then - if [ "${STRIX_LLM:-}" = "openai/openai/retry-api-connection-primary" ]; then - if [ "${FAKE_STRIX_SCENARIO:?}" = "internal-server-error-unrelated-output-nonretryable" ]; then - echo "Error: litellm.InternalServerError: upstream request failed" - for filler in 1 2 3 4 5 6; do - echo "target application diagnostic $filler" - done - echo "Internal Server Error" - exit 1 - fi - if [ "${FAKE_STRIX_SCENARIO:?}" = "internal-server-error-many-blocks-retry-same-model-success" ]; then - # Regression for the SIGPIPE race (Devin finding on - # PR #1394): emit enough matching - # litellm.InternalServerError blocks that the bounded - # awk scan's piped output exceeds a single pipe - # buffer, so a `grep -q` that stops reading at the - # first match cannot SIGPIPE the still-writing awk - # producer into a false non-match under - # `set -o pipefail`. - for _ in $(seq 1 2000); do - echo "line filler some unrelated target application output padding padding padding" - echo "Error: litellm.InternalServerError: upstream request failed" - echo "Internal Server Error" - echo "more filler after context one" - echo "more filler after context two" - done - exit 1 - fi - echo "LLM CONNECTION FAILED" - echo "Could not establish connection to the language model." - echo "Error: litellm.InternalServerError: upstream request failed" - else - echo "LLM CONNECTION FAILED" - echo "litellm.APIConnectionError: GeminiException - Server disconnected without sending a response." - fi - exit 1 - fi - echo "scan ok after same-model api connection retry" - exit 0 - ;; - vertex_ai/fallback-one) - echo "Error: fallback should not be needed for API connection retry scenario" >&2 - exit 36 - ;; - *) - echo "Error: API connection retry path unexpected (${STRIX_LLM:-})" >&2 - exit 36 - ;; - esac - ;; - openrouter-502-fallback-retry-same-model-success) - case "${STRIX_LLM:-}" in - vertex_ai/missing-primary) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - openrouter/free) - attempt="0" - if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then - attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" - fi - attempt="$((attempt + 1))" - echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" - if [ "$attempt" -eq 1 ]; then - echo "Error: litellm.APIError: APIError:" - echo "OpenrouterException -" - echo '{"error":{"message":"Invalid URL:' - echo '","code":502,"metadata":{"provider_name":"Stealth"}}}' - exit 1 - fi - echo "scan ok after OpenRouter 502 same-model retry" - exit 0 - ;; - vertex_ai/fallback-two) - echo "Error: second fallback should not be needed after transient OpenRouter 502" >&2 - exit 38 - ;; - *) - echo "Error: OpenRouter 502 fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 38 - ;; - esac - ;; - openrouter-502-distant-target-output-nonretryable) - case "${STRIX_LLM:-}" in - vertex_ai/missing-primary) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - openrouter/free) - echo "Error: litellm.APIError: APIError: OpenrouterException -" - printf 'target output\n%.0s' 1 2 3 4 5 6 - echo '{"code":502,"metadata":{"provider_name":"spoof"}}' - exit 1 - ;; - vertex_ai/fallback-two) - echo "scan ok after distant target output" - exit 0 - ;; - esac - ;; - github-models-primary-unavailable-fallback-success|github-models-primary-denied-fallback-success) - case "${STRIX_LLM:-}" in - openai/gpt-5) - echo "LLM CONNECTION FAILED" - echo "Could not establish connection to the language model." - if [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-primary-denied-fallback-success" ]; then - echo "openai.PermissionDeniedError: Error code: 403" - else - echo "Error: litellm.BadRequestError: OpenAIException - Unavailable model: gpt-5" - fi - exit 1 - ;; - openai/deepseek/deepseek-r1-0528) - echo "scan ok after GitHub Models unavailable fallback" - exit 0 - ;; - *) - echo "Error: GitHub Models unavailable fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 37 - ;; - esac - ;; - github-models-http410-authenticated-fallback-success | github-models-http410-missing-http-token | github-models-http410-missing-provider-error | github-models-http410-numeric-continuation-4100 | github-models-http410-numeric-continuation-4104 | github-models-http410-target-output-spoof | github-models-retirement-brownout-phrase-only) - case "${STRIX_LLM:-}" in - openai/gpt-5) - case "${FAKE_STRIX_SCENARIO:?}" in - github-models-http410-authenticated-fallback-success) - echo "Error: litellm.BadRequestError: GitHub Models provider error at models.github.ai/inference: HTTP 410 Gone" - ;; - github-models-http410-missing-http-token) - echo "Error: litellm.BadRequestError: GitHub Models provider retirement at models.github.ai/inference" - ;; - github-models-http410-missing-provider-error) - echo "GitHub Models response at models.github.ai/inference: HTTP 410 Gone" - ;; - github-models-http410-numeric-continuation-4100) - echo "Error: litellm.BadRequestError: GitHub Models provider error at models.github.ai/inference: HTTP 4100" - ;; - github-models-http410-numeric-continuation-4104) - echo "Error: litellm.BadRequestError: GitHub Models provider error at models.github.ai/inference: HTTP 4104" - ;; - github-models-http410-target-output-spoof) - echo "TARGET OUTPUT: Error: litellm.BadRequestError: GitHub Models provider error HTTP 410" - ;; - github-models-retirement-brownout-phrase-only) - echo "GitHub Models retirement brownout" - ;; - esac - exit 1 - ;; - openai/deepseek/deepseek-r1-0528) - echo "scan ok after authenticated GitHub Models HTTP 410 retirement" - exit 0 - ;; - *) - echo "Error: GitHub Models HTTP 410 fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 39 - ;; - esac - ;; - github-models-primary-ratelimit-fallback-success) - case "${STRIX_LLM:-}" in - openai/gpt-5) - echo "LLM CONNECTION FAILED" - echo "Could not establish connection to the language model." - echo "Error: litellm.RateLimitError: RateLimitError: OpenAIException - Too many requests. For more on scraping GitHub and how it may affect your rights, please review our Terms of Service." - exit 1 - ;; - openai/deepseek/deepseek-r1-0528) - echo "scan ok after GitHub Models rate-limit fallback" - exit 0 - ;; - *) - echo "Error: GitHub Models rate-limit fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 38 - ;; - esac - ;; - github-models-fallback-provider-signal-tries-next | github-models-fallback-baseline-vulnerability-before-next-success-continues | github-models-exhausted-after-baseline-vulnerability-fails-closed | github-models-fallback-changed-vulnerability-before-next-success-blocks | github-models-fallback-dockerfile-test-baseline-before-next-success-continues) - case "${STRIX_LLM:-}" in - openai/gpt-5) - echo "LLM CONNECTION FAILED" - echo "Could not establish connection to the language model." - echo "Error: litellm.RateLimitError: RateLimitError: OpenAIException - Too many requests." - exit 1 - ;; - openai/deepseek/deepseek-r1-0528) - if [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-fallback-baseline-vulnerability-before-next-success-continues" ] || - [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-exhausted-after-baseline-vulnerability-fails-closed" ]; then - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-provider-signal/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-provider-signal/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Location 1: -sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl/SysUserServiceImpl.java:5 -EOS - elif [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-fallback-changed-vulnerability-before-next-success-blocks" ]; then - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-provider-signal/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-changed-provider-signal/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Location 1: -sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java:12 -EOS - elif [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-fallback-dockerfile-test-baseline-before-next-success-continues" ]; then - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-dockerfile-test-provider-signal/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-dockerfile-test-provider-signal/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: MEDIUM -Location 1: -Dockerfile.test:1 -EOS - else - echo "LLM CONNECTION FAILED" - echo "Could not establish connection to the language model." - echo "Error: litellm.BadRequestError: OpenAIException - Unavailable model: deepseek-r1-0528" - fi - exit 2 - ;; - openai/deepseek/deepseek-v3-0324) - if [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-exhausted-after-baseline-vulnerability-fails-closed" ]; then - echo "LLM CONNECTION FAILED" - echo "Could not establish connection to the language model." - echo "Error: provider retirement brownout" - exit 1 - fi - echo "scan ok after second GitHub Models fallback" - exit 0 - ;; - *) - echo "Error: GitHub Models provider-signal fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 38 - ;; - esac - ;; - gemini-high-demand-retry-same-model-success) - case "${STRIX_LLM:-}" in - gemini/retry-high-demand-primary) - attempt="0" - if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then - attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" - fi - attempt="$((attempt + 1))" - echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" - if [ "$attempt" -eq 1 ]; then - echo "LLM CONNECTION FAILED" - echo 'litellm.ServiceUnavailableError: GeminiException - {"error":{"code":503,"message":"This model is currently experiencing high demand. Spikes in demand are usually temporary. Please try again later.","status":"UNAVAILABLE"}}' - exit 1 - fi - echo "scan ok after same-model high-demand retry" - exit 0 - ;; - *) - echo "Error: high-demand retry path unexpected (${STRIX_LLM:-})" >&2 - exit 37 - ;; - esac - ;; - nvidia-overloaded-direct-fallback-success) - case "${STRIX_LLM:-}" in - nvidia_nim/nvidia/overloaded-primary) - echo "LLM CONNECTION FAILED" - echo "Could not establish connection to the language model." - echo "Error: litellm.ServiceUnavailableError: Nvidia_nimException - Service temporarily overloaded" - exit 1 - ;; - nvidia_nim/nvidia/fallback-one) - echo "scan ok after NVIDIA overload fallback" - exit 0 - ;; - *) - echo "Error: NVIDIA overload fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 37 - ;; - esac - ;; - gemini-timeout-direct-fallback-success) - case "${STRIX_LLM:-}" in - gemini/retry-timeout-primary) - echo "LLM CONNECTION FAILED" - echo "Error: litellm.Timeout: Connection timed out after None seconds." - exit 1 - ;; - gemini/fallback-one) - echo "scan ok after timeout fallback" - exit 0 - ;; - *) - echo "Error: gemini timeout fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 38 - ;; - esac - ;; - gemini-timeout-fallback-success|gemini-generic-fallback-success) - case "${STRIX_LLM:-}" in - gemini/timeout-fallback-primary) - echo "LLM CONNECTION FAILED" - echo "Error: litellm.Timeout: Connection timed out after None seconds." - exit 1 - ;; - gemini/fallback-one) - echo "scan ok after gemini fallback" - exit 0 - ;; - *) - echo "Error: gemini timeout fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 39 - ;; - esac - ;; - gemini-zero-findings-timeout-fallback-allows-pr) - case "${STRIX_LLM:-}" in - gemini/zero-timeout-primary|gemini/fallback-one) - echo "Vulnerabilities 0" - echo "LLM CONNECTION FAILED" - echo "Error: litellm.Timeout: Connection timed out after None seconds." - exit 1 - ;; - *) - echo "Error: gemini zero-finding fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 40 - ;; - esac - ;; - pr-scope-zero-finding-does-not-leak) - if [ -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" ]; then - echo "Vulnerabilities 0" - echo "LLM CONNECTION FAILED" - echo "Error: litellm.Timeout: Connection timed out after None seconds." - exit 1 - fi - if [ -f "$target_path/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java" ]; then - echo "LLM CONNECTION FAILED" - echo "Error: litellm.Timeout: Connection timed out after None seconds." - exit 1 - fi - echo "Error: unexpected PR scope zero-finding leak target layout ($target_path)" >&2 - exit 41 - ;; - service-unavailable-no-llm-marker-nonrecoverable) - echo 'ServiceUnavailableError: {"error":{"code":503,"status":"UNAVAILABLE"}}' - echo '{"error":{"code":502,"metadata":{"provider_name":"Stealth"}}}' - echo 'target application high demand response' - exit 1 - ;; - server-disconnect-no-llm-marker-nonrecoverable) - echo "ConnectionError: Server disconnected without sending a response." - exit 1 - ;; - vertex-all-ratelimited) - echo "Penetration test failed: LLM request failed: RateLimitError" - exit 1 - ;; - vertex-primary-hallucinated-endpoint-fallback-success|target-path-src-default-source-dirs) - case "${STRIX_LLM:-}" in - vertex_ai/hallucination-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-hallucinated/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-hallucinated/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Endpoint:** /api/ghost-admin -EOS - echo "Penetration test failed: CRITICAL finding on /api/ghost-admin" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after hallucinated-endpoint fallback" - exit 0 - ;; - *) - echo "Error: hallucinated-endpoint fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 26 - ;; - esac - ;; - opencode-documented-env-api-key-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/opencode-env-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-opencode-env/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-opencode-env/vulnerabilities/vuln-0001.md" <&2 - exit 27 - ;; - esac - ;; - generic-github-actions-workflow-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/generic-actions-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-generic-actions/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-generic-actions/vulnerabilities/vuln-0001.md" <<'EOS' -# Insecure Configurations in GitHub Actions Workflows - -**Severity:** CRITICAL -**Target:** local_code: /workspace/strix-pr-scope.fake -**Endpoint:** CI/CD Pipeline -**CWE:** CWE-732 - -## Description - -/workspace/strix-pr-scope.fake/.github/workflows/strix.yml - -## Technical Analysis - -The GitHub Actions configuration contains several security weaknesses: -1. Secrets are written to temporary files without proper access controls -2. API keys are passed through environment variables without adequate masking -3. Excessive permissions granted to workflows -4. Insufficient input validation for workflow parameters - -## Code Analysis - -**Location 1:** `.github/workflows/strix.yml` (lines 1-300) - ``` - Full file content - ``` - - **Suggested Fix:** -```diff -- Current content -+ Secured version -``` -EOS - echo "Penetration test failed: generic GitHub Actions workflow finding" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after generic GitHub Actions workflow false positive" - exit 0 - ;; - *) - echo "Error: generic GitHub Actions workflow fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 37 - ;; - esac - ;; - vertex-primary-existing-endpoint-nonrecoverable|multi-source-dirs-existing-endpoint) - case "${STRIX_LLM:-}" in - vertex_ai/existing-endpoint-primary|vertex_ai/multi-dir-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-existing-endpoint/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-existing-endpoint/vulnerabilities/vuln-0001.md" <<'EOS' -**Endpoint:** /api/status -EOS - echo "Penetration test failed: CRITICAL finding on /api/status" - exit 1 - ;; - vertex_ai/fallback-one|vertex_ai/fallback-two) - echo "Error: existing endpoint findings must remain non-recoverable (${STRIX_LLM:-})" >&2 - exit 27 - ;; - *) - echo "Error: existing-endpoint scenario unexpected model (${STRIX_LLM:-})" >&2 - exit 28 - ;; - esac - ;; - pr-stale-source-claim-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/stale-source-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-stale-source/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-stale-source/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** HIGH -**Target:** backend/db/models.py - -The `WorkspaceRunnerConfig.registration_token` field stores the token as plain text. -The vulnerable line is `registration_token: Mapped[str | None] = mapped_column(String, nullable=True)`. -EOS - echo "Penetration test failed: stale HIGH finding on backend/db/models.py" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after stale-source fallback" - exit 0 - ;; - *) - echo "Error: stale-source scenario unexpected model (${STRIX_LLM:-})" >&2 - exit 30 - ;; - esac - ;; - pr-stale-snapshot-snippet-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/stale-snapshot-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-stale-snapshot/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-stale-snapshot/vulnerabilities/vuln-0001.md" <<'EOS' -# IDOR in /api/snapshots endpoint allows unauthorized access to database schemas - -**Severity:** MEDIUM -**Target:** backend/app/api/snapshots.py - -## Code Analysis - -**Location 1:** `backend/app/api/snapshots.py` (lines 78-81) - Missing ownership check - ``` - snapshot = await get_snapshot_by_uuid(snapshot_uuid) -if not snapshot: - raise HTTPException(status_code=404) -return snapshot - ``` - -**Location 2:** `backend/app/api/snapshots.py` (lines 78-81) - **Suggested Fix:** -```diff -- snapshot = await get_snapshot_by_uuid(snapshot_uuid) -- if not snapshot: -- raise HTTPException(status_code=404) -- return snapshot -+ snapshot = await get_snapshot_by_uuid(snapshot_uuid) -+ if not snapshot: -+ raise HTTPException(status_code=404) -+ if not await is_project_member(current_user.user_account_uuid, snapshot.project_space_uuid): -+ raise HTTPException(status_code=403) -+ return snapshot -``` -EOS - echo "Penetration test failed: stale MEDIUM snapshot snippet" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after stale snapshot snippet fallback" - exit 0 - ;; - *) - echo "Error: stale-snapshot scenario unexpected model (${STRIX_LLM:-})" >&2 - exit 38 - ;; - esac - ;; - pr-stale-source-plus-real-finding-blocks) - case "${STRIX_LLM:-}" in - vertex_ai/stale-source-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-mixed-findings/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-mixed-findings/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** HIGH -**Target:** backend/db/models.py - -The `WorkspaceRunnerConfig.registration_token` field stores the token as plain text. -The vulnerable line is `registration_token: Mapped[str | None] = mapped_column(String, nullable=True)`. -EOS - cat >"$STRIX_REPORTS_DIR/fake-mixed-findings/vulnerabilities/vuln-0002.md" <<'EOS' -**Severity:** HIGH -**Target:** backend/api/emails.py - -This is a concrete changed-file finding that must remain blocking. -EOS - echo "Penetration test failed: mixed stale and real HIGH findings" - exit 1 - ;; - vertex_ai/fallback-one) - echo "Error: mixed real findings must not reach fallback" >&2 - exit 31 - ;; - *) - echo "Error: mixed-findings scenario unexpected model (${STRIX_LLM:-})" >&2 - exit 32 - ;; - esac - ;; - pr-changed-finding-with-retry-marker-blocks) - case "${STRIX_LLM:-}" in - vertex_ai/changed-finding-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-changed-retry-marker/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-changed-retry-marker/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** HIGH -**Target:** backend/api/emails.py - -This changed-file finding must remain blocking even when the model log also contains retryable provider text. -EOS - echo "litellm.exceptions.Timeout: provider timed out after writing a HIGH changed-file finding" - exit 1 - ;; - vertex_ai/fallback-one) - echo "Error: changed-file findings with retry markers must not reach fallback" >&2 - exit 33 - ;; - *) - echo "Error: changed-retry-marker scenario unexpected model (${STRIX_LLM:-})" >&2 - exit 34 - ;; - esac - ;; - pr-stale-report-plus-inline-changed-finding-blocks) - case "${STRIX_LLM:-}" in - vertex_ai/stale-inline-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-stale-report-inline-changed/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-stale-report-inline-changed/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** HIGH -**Target:** backend/db/models.py - -The `WorkspaceRunnerConfig.registration_token` field stores the token as plain text. -The vulnerable line is `registration_token: Mapped[str | None] = mapped_column(String, nullable=True)`. -EOS - echo "Severity: HIGH" - echo "Target: backend/api/emails.py" - echo "Penetration test failed: stale report plus inline changed-file HIGH finding" - exit 1 - ;; - vertex_ai/fallback-one) - echo "Error: inline changed-file findings must not reach fallback" >&2 - exit 35 - ;; - *) - echo "Error: stale-inline scenario unexpected model (${STRIX_LLM:-})" >&2 - exit 36 - ;; - esac - ;; - endpoint-in-excluded-dir) - case "${STRIX_LLM:-}" in - vertex_ai/excluded-dir-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-excluded-dir/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-excluded-dir/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Endpoint:** /api/hidden-secret -EOS - echo "Penetration test failed: CRITICAL finding on /api/hidden-secret" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after excluded-dir hallucination fallback" - exit 0 - ;; - *) - echo "Error: excluded-dir scenario unexpected model (${STRIX_LLM:-})" >&2 - exit 29 - ;; - esac - ;; - empty-fallback-models) - # Output must match is_vertex_not_found_error() patterns so the gate - # proceeds to the fallback loop (where empty array triggers the message). - echo "Publisher Model vertex_ai/empty-fb-primary was not found in project." - exit 1 - ;; - high-vuln-below-threshold) - mkdir -p "$STRIX_REPORTS_DIR/fake-high/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-high/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: HIGH -EOS - echo "Penetration test failed: simulated high finding" - exit 1 - ;; - multi-severity-low-then-critical) - mkdir -p "$STRIX_REPORTS_DIR/fake-multi-severity/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-multi-severity/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: LOW - -Related issue severity: CRITICAL -EOS - echo "Penetration test failed: report contains LOW followed by CRITICAL" - exit 1 - ;; - inline-medium-below-threshold) - echo "╭─ VULN-0001 ──────────────────────────────────────────────────────────────────╮" - echo "│ Vulnerability Report │" - echo "│ Severity: MEDIUM │" - echo "╰──────────────────────────────────────────────────────────────────────────────╯" - echo "Penetration test failed: simulated inline medium finding" - exit 2 - ;; - medium-vuln-default-threshold) - mkdir -p "$STRIX_REPORTS_DIR/fake-medium-default/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-medium-default/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: MEDIUM -EOS - echo "Penetration test failed: simulated medium finding" - exit 1 - ;; - critical-vuln-at-threshold) - mkdir -p "$STRIX_REPORTS_DIR/fake-critical/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-critical/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -EOS - echo "Penetration test failed: simulated critical finding" - exit 1 - ;; - malformed-severity-marker-nonrecoverable) - mkdir -p "$STRIX_REPORTS_DIR/fake-malformed/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-malformed/vulnerabilities/vuln-0001.md" <<'EOS' -Severity details: high confidence marker only -EOS - echo "Penetration test failed: malformed severity marker" - exit 1 - ;; - model-disagreement-critical-in-earlier-report) - case "${STRIX_LLM:-}" in - vertex_ai/model-a) - mkdir -p "$STRIX_REPORTS_DIR/run-001/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/run-001/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -EOS - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - echo "Penetration test failed: CRITICAL finding by model-a" - exit 1 - ;; - vertex_ai/model-b) - mkdir -p "$STRIX_REPORTS_DIR/run-002/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/run-002/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: LOW -EOS - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - echo "Penetration test failed: LOW finding by model-b" - exit 1 - ;; - *) - echo "Error: model-disagreement unexpected model (${STRIX_LLM:-})" >&2 - exit 32 - ;; - esac - ;; - nonvertex-slash-model-not-rewritten) - if [ "${STRIX_LLM:-}" = "deepseek/models/deepseek-r1" ]; then - echo "scan ok with deepseek model passthrough" - exit 0 - fi - echo "Error: deepseek model was rewritten (${STRIX_LLM:-})" >&2 - exit 33 - ;; - preserve-existing-api-base) - if [ "${LLM_API_BASE:-}" = "https://preexisting.invalid" ]; then - echo "scan ok with preserved api base" - exit 0 - fi - echo "Error: existing LLM_API_BASE was not preserved (${LLM_API_BASE:-})" >&2 - exit 20 - ;; - default-fallback-order-fast-first) - case "${STRIX_LLM:-}" in - vertex_ai/missing-primary) - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - vertex_ai/gemini-2.5-pro) - echo "scan ok with default fast fallback" - exit 0 - ;; - *) - echo "Error: default fallback order unexpected (${STRIX_LLM:-})" >&2 - exit 16 - ;; - esac - ;; - vertex-primary-timeout-retry-same-model-success|vertex-primary-timeout-retry-reason-message) - case "${STRIX_LLM:-}" in - vertex_ai/retry-timeout-primary) - echo "litellm.exceptions.Timeout: litellm.Timeout: Connection timed out after None seconds." - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after timeout fallback" - exit 0 - ;; - *) - echo "Error: timeout fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 34 - ;; - esac - ;; - all-fallbacks-same-as-primary) - # Bug 13: All fallback models are the same as the primary model. - # The gate should emit an ERROR and exit 1. - echo "Error: litellm.NotFoundError: Vertex_aiException - x" - echo '"status": "NOT_FOUND"' - exit 1 - ;; - vertex-primary-timeout-exhausted-fallback-success) - # Primary always times out (even after retries). Fallback succeeds. - case "${STRIX_LLM:-}" in - vertex_ai/timeout-exhaust-primary) - echo "litellm.exceptions.Timeout: litellm.Timeout: Connection timed out after None seconds." - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after timeout-exhausted fallback" - exit 0 - ;; - *) - echo "Error: timeout-exhausted-fallback unexpected model (${STRIX_LLM:-})" >&2 - exit 35 - ;; - esac - ;; - zero-findings-timeout-all-models|strict-zero-findings-timeout-fails-pr) - case "${STRIX_LLM:-}" in - vertex_ai/zero-timeout-primary|vertex_ai/fallback-one) - echo "╭─ STRIX ──────────────────────────────────────────────────────────────────────╮" - echo "│ Penetration test in progress │" - echo "│ Vulnerabilities 0 │" - echo "╰──────────────────────────────────────────────────────────────────────────────╯" - sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" - exit 0 - ;; - *) - echo "Error: zero-findings-timeout unexpected model (${STRIX_LLM:-})" >&2 - exit 57 - ;; - esac - ;; - zero-findings-sticky-across-fallback) - case "${STRIX_LLM:-}" in - vertex_ai/zero-sticky-primary) - echo "╭─ STRIX ──────────────────────────────────────────────────────────────────────╮" - echo "│ Penetration test in progress │" - echo "│ Vulnerabilities 0 │" - echo "╰──────────────────────────────────────────────────────────────────────────────╯" - sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" - exit 0 - ;; - vertex_ai/fallback-one) - sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" - exit 0 - ;; - *) - echo "Error: zero-findings-sticky unexpected model (${STRIX_LLM:-})" >&2 - exit 58 - ;; - esac - ;; - zero-findings-with-low-report-timeout) - case "${STRIX_LLM:-}" in - vertex_ai/zero-low-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-zero-low/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-zero-low/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: LOW -EOS - echo "╭─ STRIX ──────────────────────────────────────────────────────────────────────╮" - echo "│ Penetration test in progress │" - echo "│ Vulnerabilities 0 │" - echo "╰──────────────────────────────────────────────────────────────────────────────╯" - sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" - exit 0 - ;; - vertex_ai/fallback-one) - sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" - exit 0 - ;; - *) - echo "Error: zero-findings-with-low-report unexpected model (${STRIX_LLM:-})" >&2 - exit 59 - ;; - esac - ;; - provider-fatal-success-signal) - echo "Fatal: provider stream aborted" - exit 0 - ;; - provider-warning-success-signal) - echo "Warning: provider response included incomplete scan state" - exit 0 - ;; - provider-denied-success-signal) - echo "Denied: provider credentials were rejected" - exit 0 - ;; - provider-report-rate-limit-fallback-success) - case "${STRIX_LLM:-}" in - vertex_ai/report-rate-limit-primary) - mkdir -p "$STRIX_REPORTS_DIR/fake-report-rate-limit" - cat >"$STRIX_REPORTS_DIR/fake-report-rate-limit/strix.log" <<'EOS' -2026-08-21 04:00:00.000 WARNING strix-pr-scope-example - strix.provider: RateLimitError: provider response was exhausted -EOS - echo "scan aborted after provider report-rate-limit signal" - exit 1 - ;; - vertex_ai/fallback-one) - mkdir -p "$STRIX_REPORTS_DIR/fake-report-rate-limit-fallback" - echo "scan ok after report-only provider fallback" - exit 0 - ;; - *) - echo "Error: report-only provider fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 60 - ;; - esac - ;; - report-known-internal-warning-sanitized) - printf '%s\n' '│ MODEL QUALITY WARNING │' - echo 'Warning: You are sending unauthenticated requests to the HF Hub.' - mkdir -p "$STRIX_REPORTS_DIR/fake-known-internal-warning" - cat >"$STRIX_REPORTS_DIR/fake-known-internal-warning/strix.log" <<'EOS' -2026-06-18 13:08:05.986 WARNING strix-pr-scope-example - strix.core.execution: agent a9fb4033 produced non-lifecycle final output in non-interactive mode; forcing tool continuation (1/500): internal agent coordination note -2026-06-18 13:10:44.089 INFO strix-pr-scope-example - strix.tools.finish.tool: finish_scan: completed scan with 0 vulnerability report(s) -EOS - mkdir -p strix_runs/fake-known-internal-warning-relative - cat >strix_runs/fake-known-internal-warning-relative/strix.log <<'EOS' -2026-06-18 13:08:05.986 WARNING strix-pr-scope-example - strix.core.execution: agent a9fb4033 produced non-lifecycle final output in non-interactive mode; forcing tool continuation (1/500): relative internal agent coordination note -2026-06-18 13:10:44.089 INFO strix-pr-scope-example - strix.tools.finish.tool: finish_scan: completed scan with 0 vulnerability report(s) -EOS - outside_report_dir="${FAKE_STRIX_OUTSIDE_REPORT_DIR:-$(dirname -- "$STRIX_REPORTS_DIR")/outside-strix-report}" - mkdir -p "$outside_report_dir" - cat >"$outside_report_dir/strix.log" <<'EOS' -2026-06-18 13:08:05.986 WARNING strix-pr-scope-example - strix.core.execution: agent a9fb4033 produced non-lifecycle final output in non-interactive mode; forcing tool continuation (1/500): outside report should not be rewritten -EOS - ln -s "$outside_report_dir" "$STRIX_REPORTS_DIR/fake-known-internal-warning/linked-outside" - echo "scan ok with sanitized internal Strix report notice" - exit 0 - ;; - report-known-internal-warning-variant-sanitized) - mkdir -p "$STRIX_REPORTS_DIR/fake-known-internal-warning-variant" - cat >"$STRIX_REPORTS_DIR/fake-known-internal-warning-variant/strix.log" <<'EOS' -2026-08-22 09:53:26.193 WARNING strix-pr-scope-example - strix.core.execution: agent 673f770f ended a turn without a lifecycle tool call (interactive=False); forcing tool continuation (1/500): -2026-06-18 13:10:44.089 INFO strix-pr-scope-example - strix.tools.finish.tool: finish_scan: completed scan with 0 vulnerability report(s) -EOS - echo "scan ok with sanitized internal Strix report notice variant" - exit 0 - ;; - report-unknown-warning-fails) - mkdir -p "$STRIX_REPORTS_DIR/fake-unknown-warning" - cat >"$STRIX_REPORTS_DIR/fake-unknown-warning/strix.log" <<'EOS' -2026-06-18 13:08:05.986 WARNING strix-pr-scope-example - strix.provider: provider returned incomplete scan state -EOS - echo "scan ok but unknown report warning remains" - exit 0 - ;; - bare-timeout-with-provider-marker) - # Emit bare "Connection timed out" alongside a provider marker so - # is_timeout_error() matches the Tier 3 branch gated on - # LLM_PROVIDER_ONLY_REGEX. Does NOT include - # litellm.exceptions.Timeout / httpx.ReadTimeout to ensure we - # exercise the provider-marker fallback path specifically. - # Primary times out; fallback model succeeds. - case "${STRIX_LLM:-}" in - vertex_ai/bare-timeout-primary) - echo "Connection timed out" - echo "vertex_ai model invocation failed" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after bare-timeout fallback" - exit 0 - ;; - *) - echo "Error: bare-timeout fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 47 - ;; - esac - ;; - bare-timeout-no-provider-marker) - # Emit "Connection timed out" with transport library names (httpx, - # httpcore, requests) but WITHOUT any real LLM provider marker. - # is_timeout_error() Tier 3 uses LLM_PROVIDER_ONLY_REGEX which - # excludes transport libs, so this should NOT match. - echo "Connection timed out" - echo "httpx transport layer connection reset" - echo "httpcore pool timeout" - echo "requests transport timeout" - exit 1 - ;; - below-threshold-with-timeout) - # Produce a below-threshold (LOW) finding but also emit a timeout error - # so the infrastructure guard detects an incomplete scan. - mkdir -p "$STRIX_REPORTS_DIR/fake-low-timeout/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-low-timeout/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: LOW -EOS - echo "litellm.exceptions.Timeout: litellm.Timeout: Connection timed out after None seconds." - echo "Penetration test failed: simulated timeout with low finding" - exit 1 - ;; - below-threshold-with-ratelimit) - # Produce a below-threshold (LOW) finding but also emit a rate-limit error. - mkdir -p "$STRIX_REPORTS_DIR/fake-low-ratelimit/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-low-ratelimit/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: LOW -EOS - echo "Penetration test failed: LLM request failed: RateLimitError" - echo "Penetration test failed: simulated ratelimit with low finding" - exit 1 - ;; - below-threshold-with-connection-error) - # Produce a below-threshold (INFO) finding but also emit a - # ConnectionError WITH an LLM-provider context marker so the - # infrastructure guard detects an incomplete scan. - # The two-grep guard requires BOTH a transport error class AND an - # LLM_PROVIDER_ONLY_REGEX marker (litellm, openai, anthropic, etc.). - mkdir -p "$STRIX_REPORTS_DIR/fake-info-conn/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-info-conn/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: INFO -EOS - echo "litellm.exceptions.APIConnectionError: ConnectionError - connection refused" - echo "Penetration test failed: simulated connection error with info finding" - exit 1 - ;; - below-threshold-with-connection-error-no-provider) - # Produce a below-threshold (INFO) finding and emit a ConnectionError - # WITHOUT any LLM-provider context marker. The infra-error detector - # should NOT match because the log lacks provider markers like - # "litellm", "openai", "anthropic", etc. This validates that the - # two-grep guard avoids false positives from target-application logs. - mkdir -p "$STRIX_REPORTS_DIR/fake-info-conn-noprov/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-info-conn-noprov/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: INFO -EOS - echo "ConnectionError: target server refused connection on port 8443" - echo "Penetration test failed: simulated app-level connection error" - exit 1 - ;; - below-threshold-with-requests-connection-error) - # Produce a below-threshold (INFO) finding with a - # requests.exceptions.ConnectionError — the transport library prefix - # "requests" matches the broad PROVIDER_CONTEXT_REGEX but is - # intentionally excluded from LLM_PROVIDER_ONLY_REGEX. - # - # Before commit 0e90d48, the connection-error path used - # has_provider_context_marker() (PROVIDER_CONTEXT_REGEX) and would - # have incorrectly classified this as an LLM infrastructure error. - # After that fix, LLM_PROVIDER_ONLY_REGEX is used, so "requests" - # alone does NOT satisfy the provider check → below-threshold bypass - # succeeds → exit 0. - mkdir -p "$STRIX_REPORTS_DIR/fake-info-conn-requests/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-info-conn-requests/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: INFO -EOS - echo "requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.example.com', port=443): Max retries exceeded with url: /v1/scan" - echo "Penetration test failed: simulated requests transport error" - exit 1 - ;; - below-threshold-with-midstream) - # Produce a below-threshold (MEDIUM) finding below CRITICAL threshold - # but also emit a MidStreamFallbackError. - mkdir -p "$STRIX_REPORTS_DIR/fake-medium-midstream/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-medium-midstream/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: MEDIUM -EOS - echo "Penetration test failed: LLM request failed: MidStreamFallbackError" - echo "Penetration test failed: simulated midstream with medium finding" - exit 1 - ;; - bare-timeout-provider-marker-exhausted-fallback) - # Bare "Connection timed out" + provider marker: primary fails once, - # then the gate falls back to fallback-one which succeeds. - case "${STRIX_LLM:-}" in - vertex_ai/bare-timeout-exhaust-primary) - echo "Connection timed out" - echo "vertex_ai model invocation failed" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after bare-timeout-exhaust fallback" - exit 0 - ;; - *) - echo "Error: bare-timeout-exhaust-fallback unexpected model (${STRIX_LLM:-})" >&2 - exit 35 - ;; - esac - ;; - httpx-read-timeout-with-provider-marker) - # Tier 2: httpx.ReadTimeout + provider-context marker (litellm). - # Primary times out; fallback model succeeds. - case "${STRIX_LLM:-}" in - vertex_ai/httpx-timeout-primary) - echo "httpx.ReadTimeout: timed out" - echo "litellm.proxy: connection to upstream model failed" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after httpx-timeout fallback" - exit 0 - ;; - *) - echo "Error: httpx-timeout fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 45 - ;; - esac - ;; - httpx-read-timeout-no-provider-marker) - # Tier 2 negative: httpx.ReadTimeout WITHOUT any provider-context - # marker. Should NOT be classified as retryable timeout. - echo "httpx.ReadTimeout: timed out" - echo "application server connection pool exhausted" - exit 1 - ;; - httpcore-read-timeout-with-provider-marker) - # Tier 2b: httpcore.ReadTimeout + provider-context marker. - # Primary times out; fallback model succeeds. - case "${STRIX_LLM:-}" in - vertex_ai/httpcore-timeout-primary) - echo "httpcore.ReadTimeout: timed out" - echo "litellm.proxy: connection to upstream model failed" - exit 1 - ;; - vertex_ai/fallback-one) - echo "scan ok after httpcore-timeout fallback" - exit 0 - ;; - *) - echo "Error: httpcore-timeout fallback path unexpected (${STRIX_LLM:-})" >&2 - exit 46 - ;; - esac - ;; - httpcore-read-timeout-no-provider-marker) - # Tier 2b negative: httpcore.ReadTimeout WITHOUT any provider-context - # marker. Should NOT be classified as retryable timeout. - echo "httpcore.ReadTimeout: timed out" - echo "application server connection pool exhausted" - exit 1 - ;; - infra-error-sticky-flag) - # Sticky flag test: first call hits infra error (rate limit), - # second call fails on the first fallback model but produces a - # LOW finding report. After exhausting retries, the gate checks - # has_only_below_threshold_vulnerabilities — which finds LOW - # findings but sees INFRA_ERROR_DETECTED=1 (set from the first - # call's rate-limit error) and refuses the below-threshold bypass. - case "${STRIX_LLM:-}" in - vertex_ai/sticky-flag-primary) - touch "$FAKE_STRIX_STATE_FILE" - echo "RateLimitError: rate limit exceeded" - echo "litellm.proxy: rate limit on vertex_ai model" - exit 1 - ;; - vertex_ai/gemini-2.5-pro) - mkdir -p "$STRIX_REPORTS_DIR/run-sticky/vulnerabilities" - cat > "$STRIX_REPORTS_DIR/run-sticky/vulnerabilities/vuln-0001.md" <<'FINDINGS' -Severity: LOW -FINDINGS - echo "non-retryable scan error with partial results" - exit 1 - ;; - *) - echo "Error: infra-error-sticky-flag unexpected model (${STRIX_LLM:-})" >&2 - exit 35 - ;; - esac - ;; - pr-baseline-critical-unchanged) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-baseline/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Location 1: -sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl/SysUserServiceImpl.java:5 -EOS - echo "Penetration test failed: baseline critical finding" - exit 1 - ;; - pr-critical-changed) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-changed/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Location 1: -sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java:12 -EOS - echo "Penetration test failed: changed critical finding" - exit 1 - ;; - pr-changed-file-nonintersecting-line) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-nonintersecting-line/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-nonintersecting-line/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Location 1: -frontend/src/App.tsx:1 -EOS - echo "Penetration test failed: same changed file but baseline line finding" - exit 1 - ;; - pr-critical-changed-bracketed-next-route) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-bracketed-next-route/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-changed-bracketed-next-route/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Location 1: -frontend/src/app/labels/[slug]/page.tsx:12 -EOS - echo "Penetration test failed: changed bracketed Next.js route finding" - exit 1 - ;; - pr-critical-changed-xml-file-location) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-xml/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-changed-xml/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: HIGH - - - sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java - 120 - 124 - - -EOS - echo "Penetration test failed: changed XML file location finding" - exit 1 - ;; - pr-critical-changed-xml-file-location-space) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-xml-space/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-changed-xml-space/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: HIGH - - - src/unsafe name.py - 7 - 9 - - -EOS - echo "Penetration test failed: changed XML file location finding with space" - exit 1 - ;; - pr-baseline-critical-narrative-backticked-service-file) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-narrative-service/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-narrative-service/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Technical Analysis -The `backend/services/email_parser.py` file extracts HTML email bodies without sanitizing script tags. -EOS - echo "Penetration test failed: baseline critical narrative service finding" - exit 1 - ;; - pr-critical-unmapped-arbitrary-backticked-service-file) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-unmapped-arbitrary-backtick/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-unmapped-arbitrary-backtick/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Description: location data unavailable, but the report also mentions `backend/services/email_parser.py` as unrelated context. -EOS - echo "Penetration test failed: unmapped critical finding with arbitrary backticked file mention" - exit 1 - ;; - pr-critical-unmapped) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-unmapped/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-unmapped/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Description: location data unavailable -EOS - echo "Penetration test failed: unmapped critical finding" - exit 1 - ;; - pr-baseline-critical-absolute-target) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-absolute/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-absolute/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** File: /workspace/smart-crawling-server/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java -EOS - echo "Penetration test failed: baseline critical finding with absolute target" - exit 1 - ;; - pr-baseline-critical-extensionless-dockerfile-target) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-dockerfile/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-dockerfile/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** File: /workspace/smart-crawling-server/Dockerfile -EOS - echo "Penetration test failed: baseline critical finding with extensionless Dockerfile target" - exit 1 - ;; - pr-baseline-critical-subdir-target) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-subdir/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-subdir/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** File: /workspace/flyway/V16__hash_oauth2_registered_client_secret.sql -EOS - echo "Penetration test failed: baseline critical finding with narrowed subdir target" - exit 1 - ;; - pr-baseline-critical-subdir-boxed-target) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-boxed-target/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-boxed-target/vulnerabilities/vuln-0001.md" <<'EOS' -│ Severity: CRITICAL │ -│ Target: /workspace/flyway/V16__hash_oauth2_registered_client_secret.sql │ -│ Endpoint: N/A (database migration script) │ -EOS - echo "Penetration test failed: baseline critical finding with boxed narrowed subdir target" - exit 1 - ;; - pr-baseline-critical-subdir-endpoint) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-endpoint/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-endpoint/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** Local Codebase: /workspace/flyway -**Endpoint:** /workspace/flyway/V16__hash_oauth2_registered_client_secret.sql -EOS - echo "Penetration test failed: baseline critical finding with narrowed subdir endpoint" - exit 1 - ;; - pr-baseline-critical-subdir-endpoint-bare-filename) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-endpoint-bare-filename/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-endpoint-bare-filename/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** Local Codebase: /workspace/flyway -**Endpoint:** V16__hash_oauth2_registered_client_secret.sql -EOS - echo "Penetration test failed: baseline critical finding with narrowed subdir bare filename endpoint" - exit 1 - ;; - pr-baseline-critical-subdir-narrative-backticked-file) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-narrative-backticked-file/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-narrative-backticked-file/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** Local Codebase: /workspace/flyway -The issue appears in file `V4__ccf_scenario.sql`. -EOS - echo "Penetration test failed: baseline critical finding with narrowed subdir narrative backticked file" - exit 1 - ;; - pr-critical-relative-path-escape-subdir-narrative-backticked-file) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-relative-path-escape-subdir-narrative/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-relative-path-escape-subdir-narrative/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** Local Codebase: /workspace/flyway -The issue appears in file `../V24__update_search_expression_team_keyword_id.sql`. -EOS - echo "Penetration test failed: relative path escape critical finding with narrowed subdir narrative backticked file" - exit 1 - ;; - pr-critical-changed-absolute-target) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-absolute/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-changed-absolute/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** File: /workspace/smart-crawling-server/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java -EOS - echo "Penetration test failed: changed critical finding with absolute target" - exit 1 - ;; - pr-critical-changed-internal-dotdir-target) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-internal-dotdir/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-changed-internal-dotdir/vulnerabilities/vuln-0001.md" <"$STRIX_REPORTS_DIR/fake-pr-changed-json-target/vulnerabilities/vuln-0001.md" <"$STRIX_REPORTS_DIR/fake-pr-changed-subdir/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** File: /workspace/flyway/V24__update_search_expression_team_keyword_id.sql -EOS - echo "Penetration test failed: changed critical finding with narrowed subdir target" - exit 1 - ;; - pr-critical-changed-subdir-endpoint) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-subdir-endpoint/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-changed-subdir-endpoint/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** Local Codebase: /workspace/flyway -**Endpoint:** /workspace/flyway/V24__update_search_expression_team_keyword_id.sql -EOS - echo "Penetration test failed: changed critical finding with narrowed subdir endpoint" - exit 1 - ;; - pr-critical-path-escape-subdir-target) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-path-escape-subdir/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-path-escape-subdir/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** File: /workspace/flyway/../../../../../smart-crawling-common/src/main/java/org/empasy/sync/common/system/util/JwtUtil.java -EOS - echo "Penetration test failed: path escape critical finding with narrowed subdir target" - exit 1 - ;; - pr-critical-unmapped-narrative-target) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-unmapped-narrative/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-unmapped-narrative/vulnerabilities/vuln-0001.md" <<'EOS' -**Severity:** CRITICAL -**Target:** Multiple files in the codebase, particularly `org.empasy.sync.common.system.util.JwtUtil.java` (for signing) and its callers. -EOS - echo "Penetration test failed: unmapped narrative critical finding" - exit 1 - ;; - pr-critical-unmapped-other-workspace-repo) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-other-workspace-repo/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-other-workspace-repo/vulnerabilities/vuln-0001.md" <<'EOS' - **Severity:** CRITICAL - **Target:** File: /workspace/other-repo/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java -EOS - echo "Penetration test failed: other workspace repo target" - exit 1 - ;; - pr-critical-manifest-only-pom|pr-critical-manifest-only-pom-test-override|pr-critical-manifest-only-pom-same-head-different-pr|pr-critical-manifest-only-pom-current-pr-authoritative) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-manifest-only/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-manifest-only/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Location 1: -pom.xml:8 -EOS - echo "Penetration test failed: manifest-only critical finding" - exit 1 - ;; - pr-critical-manifest-only-pom-after-fallback-authoritative) - case "${STRIX_LLM:-}" in - vertex_ai/timeout-primary) - echo "litellm.exceptions.Timeout: primary model timed out" - exit 1 - ;; - vertex_ai/fallback-one) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-manifest-only-after-fallback/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-manifest-only-after-fallback/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: CRITICAL -Location 1: -pom.xml:8 -EOS - echo "Penetration test failed: manifest-only critical finding after fallback" - exit 1 - ;; - *) - echo "Error: pr-critical-manifest-only-pom-after-fallback-authoritative unexpected model (${STRIX_LLM:-})" >&2 - exit 53 - ;; - esac - ;; - pr-critical-manifest-only-pom-console-only-after-fallback-authoritative) - case "${STRIX_LLM:-}" in - vertex_ai/timeout-primary) - echo "litellm.exceptions.Timeout: primary model timed out" - exit 1 - ;; - vertex_ai/fallback-one) - echo "Severity: CRITICAL" - echo "Location 1:" - echo "pom.xml:59" - echo "Penetration test failed: manifest-only critical finding after fallback (console-only)" - exit 1 - ;; - *) - echo "Error: pr-critical-manifest-only-pom-console-only-after-fallback-authoritative unexpected model (${STRIX_LLM:-})" >&2 - exit 54 - ;; - esac - ;; - pr-critical-manifest-only-pom-console-target-only-after-fallback-authoritative) - case "${STRIX_LLM:-}" in - vertex_ai/timeout-primary) - echo "litellm.exceptions.Timeout: primary model timed out" - exit 1 - ;; - vertex_ai/fallback-one) - echo "Severity: CRITICAL" - echo "Target: /workspace/$(basename "$target_path")/pom.xml" - echo "Penetration test failed: manifest-only critical finding after fallback (console target-only)" - exit 1 - ;; - *) - echo "Error: pr-critical-manifest-only-pom-console-target-only-after-fallback-authoritative unexpected model (${STRIX_LLM:-})" >&2 - exit 56 - ;; - esac - ;; - pr-low-markdown-plus-console-critical-manifest-after-fallback-authoritative) - case "${STRIX_LLM:-}" in - vertex_ai/timeout-primary) - echo "litellm.exceptions.Timeout: primary model timed out" - exit 1 - ;; - vertex_ai/fallback-one) - mkdir -p "$STRIX_REPORTS_DIR/fake-pr-manifest-mixed-after-fallback/vulnerabilities" - cat >"$STRIX_REPORTS_DIR/fake-pr-manifest-mixed-after-fallback/vulnerabilities/vuln-0001.md" <<'EOS' -Severity: LOW -Location 1: -pom.xml:8 -EOS - echo "Severity: CRITICAL" - echo "Location 1:" - echo "pom.xml:59" - echo "Penetration test failed: manifest-only critical finding after fallback (mixed file+console)" - exit 1 - ;; - *) - echo "Error: pr-low-markdown-plus-console-critical-manifest-after-fallback-authoritative unexpected model (${STRIX_LLM:-})" >&2 - exit 55 - ;; - esac - ;; - pr-changed-scope-bounded) - if [ -z "$target_path" ]; then - echo "Error: target path missing" >&2 - exit 41 - fi - if [ ! -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" ]; then - echo "Error: changed file missing from bounded target path ($target_path)" >&2 - exit 42 - fi - if [ -e "$target_path/sync-module-system/smart-crawling-common/src/main/java/org/empasy/sync/common/system/util/JwtUtil.java" ]; then - echo "Error: unrelated file leaked into bounded target path ($target_path)" >&2 - exit 43 - fi - echo "scan ok with bounded changed-file scope" - exit 0 - ;; - pr-python-scope-context) - if [ ! -f "$target_path/backend/api/emails.py" ]; then - echo "Error: changed backend file missing from scoped target ($target_path)" >&2 - exit 57 - fi - if [ ! -f "$target_path/backend/core/config.py" ]; then - echo "Error: backend core config context missing from scoped target ($target_path)" >&2 - exit 58 - fi - if [ ! -f "$target_path/backend/core/runtime_secrets.py" ]; then - echo "Error: backend runtime secrets context missing from scoped target ($target_path)" >&2 - exit 62 - fi - if [ ! -f "$target_path/backend/api/search.py" ]; then - echo "Error: backend search router context missing from scoped target ($target_path)" >&2 - exit 63 - fi - if [ ! -f "$target_path/backend/db/session.py" ]; then - echo "Error: backend db session context missing from scoped target ($target_path)" >&2 - exit 59 - fi - if [ ! -f "$target_path/backend/services/exceptions.py" ]; then - echo "Error: backend service exceptions context missing from scoped target ($target_path)" >&2 - exit 60 - fi - if ! grep -Fq -- 'ensure_organization_access(auth_context, config.organization_id)' "$target_path/backend/api/runner_config.py"; then - echo "Error: backend organization access context missing from scoped target ($target_path)" >&2 - exit 61 - fi - echo "scan ok with python dependency scope" - exit 0 - ;; - pr-changed-scope-full) - attempt="0" - if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then - attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" - fi - attempt="$((attempt + 1))" - echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" - if [ "$attempt" -eq 1 ]; then - if [ ! -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" ]; then - echo "Error: full-set scope missing controller file ($target_path)" >&2 - exit 44 - fi - if [ ! -f "$target_path/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java" ]; then - echo "Error: full-set scope missing playwright file ($target_path)" >&2 - exit 45 - fi - if [ ! -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl/SysUserServiceImpl.java" ]; then - echo "Error: full-set scope missing service impl file ($target_path)" >&2 - exit 46 - fi - echo "scan ok with full changed-file scope" - exit 0 - fi - echo "Error: unexpected full-scope scan attempt $attempt" >&2 - exit 50 - ;; - pr-changed-scope-full-set) - attempt="0" - if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then - attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" - fi - attempt="$((attempt + 1))" - echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" - if [ "$attempt" -eq 1 ] && \ - [ -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" ] && \ - [ -f "$target_path/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java" ] && \ - [ -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl/SysUserServiceImpl.java" ] && \ - [ -f "$target_path/sync-module-system/smart-crawling-common/src/main/java/org/empasy/sync/common/system/util/JwtUtil.java" ]; then - echo "scan ok with full configured PR scope" - exit 0 - fi - echo "Error: PR changed-file scope did not include the complete changed-file set on one scan attempt $attempt ($target_path)" >&2 - exit 54 - ;; - pr-large-scope-full-set) - echo "scan ok with large full PR scope" - exit 0 - ;; - pr-changed-scope-includes-ci-dependency) - if [ -f "$target_path/scripts/ci/strix_quick_gate.sh" ] && [ -f "$target_path/scripts/ci/strix_model_utils.sh" ]; then - echo "scan ok with CI support dependency" - exit 0 - fi - echo "Error: PR changed-file scope missing CI support dependency ($target_path)" >&2 - exit 55 - ;; - pr-deployment-scope-entrypoint-context) - if [ ! -f "$target_path/Dockerfile" ]; then - echo "Error: deployment scope missing Dockerfile ($target_path)" >&2 - exit 56 - fi - if [ ! -f "$target_path/backend/scripts/docker_entrypoint.sh" ]; then - echo "Error: deployment scope missing backend/scripts/docker_entrypoint.sh ($target_path)" >&2 - exit 57 - fi - if [ ! -f "$target_path/backend/core/runtime_secrets.py" ]; then - echo "Error: deployment scope missing backend/core/runtime_secrets.py ($target_path)" >&2 - exit 60 - fi - if ! grep -Fq -- 'CMD ["/app/scripts/docker_entrypoint.sh"]' "$target_path/Dockerfile"; then - echo "Error: deployment Dockerfile does not reference docker_entrypoint.sh ($target_path)" >&2 - exit 58 - fi - if ! grep -Fq -- 'Starting backend (uvicorn :8000)' "$target_path/backend/scripts/docker_entrypoint.sh"; then - echo "Error: deployment entrypoint context did not include trusted script content ($target_path)" >&2 - exit 59 - fi - echo "scan ok with deployment entrypoint context" - exit 0 - ;; - pr-rust-workspace-context) - for rust_context in Cargo.toml Cargo.lock rust-toolchain.toml deny.toml; do - if [ ! -f "$target_path/$rust_context" ]; then - echo "Error: Rust workflow scope missing $rust_context ($target_path)" >&2 - exit 61 - fi - done - if ! grep -Fq -- 'name = "trusted-workspace"' "$target_path/Cargo.toml"; then - echo "Error: Rust workflow context did not preserve trusted Cargo content ($target_path)" >&2 - exit 62 - fi - echo "scan ok with Rust workspace context" - exit 0 - ;; - *) - echo "unknown scenario ${FAKE_STRIX_SCENARIO:?}" >&2 - exit 8 - ;; -esac -EOF - chmod +x "$fake_strix" - - cat >"$fake_gh" <<'EOF' -#!/usr/bin/env bash -set -euo pipefail - -printf '%s\n' "${GH_TOKEN-}" >> "${FAKE_GH_TOKEN_LOG:?}" - -if [ "${1-}" != "api" ]; then - echo "unexpected gh command: $*" >&2 - exit 90 -fi - -if [ -z "${FAKE_GH_API_RESPONSE_FILE:-}" ]; then - echo "missing FAKE_GH_API_RESPONSE_FILE" >&2 - exit 91 -fi - -cat -- "${FAKE_GH_API_RESPONSE_FILE}" -EOF - chmod +x "$fake_gh" - - local effective_event_name="$github_event_name" - if [ -z "$effective_event_name" ]; then - effective_event_name="$event_name_override" - fi - - # Scenario-specific source-tree setup so is_hallucinated_endpoint_finding() - # can locate "real" endpoints inside the self-contained temp workspace. - if [ "$effective_event_name" = "pull_request" ]; then - mkdir -p "$repo_root_dir/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller" - mkdir -p "$repo_root_dir/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl" - mkdir -p "$repo_root_dir/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service" - mkdir -p "$repo_root_dir/sync-module-system/smart-crawling-common/src/main/java/org/empasy/sync/common/system/util" - echo '' >"$repo_root_dir/pom.xml" - mkdir -p "$repo_root_dir/sync-module-system/smart-crawling-server/src/main/resources/flyway" - echo 'class ChangedController {}' >"$repo_root_dir/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" - echo 'class BaselineUserService {}' >"$repo_root_dir/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl/SysUserServiceImpl.java" - echo 'class ChangedPlaywright {}' >"$repo_root_dir/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java" - echo 'class ChangedJwtUtil {}' >"$repo_root_dir/sync-module-system/smart-crawling-common/src/main/java/org/empasy/sync/common/system/util/JwtUtil.java" - mkdir -p "$repo_root_dir/frontend/src/app/labels/[slug]" - echo 'export default function Page() { return null }' >"$repo_root_dir/frontend/src/app/labels/[slug]/page.tsx" - mkdir -p "$repo_root_dir/src" - echo 'print("unsafe name")' >"$repo_root_dir/src/unsafe name.py" - mkdir -p "$repo_root_dir/backend/services" - echo 'async def send_email(*args, **kwargs): return None' >"$repo_root_dir/backend/services/email_client.py" - echo 'def parse_eml(*args): return {}' >"$repo_root_dir/backend/services/email_parser.py" - if [ -n "$current_pr_number" ]; then - cat >"$event_payload_file" <"$repo_root_dir/sync-module-system/smart-crawling-server/src/main/resources/flyway/V4__ccf_scenario.sql" - echo '-- legacy flyway file' >"$repo_root_dir/sync-module-system/smart-crawling-server/src/main/resources/flyway/V16__hash_oauth2_registered_client_secret.sql" - echo '-- changed flyway file' >"$repo_root_dir/sync-module-system/smart-crawling-server/src/main/resources/flyway/V24__update_search_expression_team_keyword_id.sql" - fi - - if [ "$scenario" = "vertex-primary-existing-endpoint-nonrecoverable" ]; then - echo 'GET /api/status' >"$repo_root_dir/src/routes.txt" - elif [ "$scenario" = "multi-source-dirs-existing-endpoint" ]; then - # Endpoint lives in api/ (not src/), validating multi-dir scanning. - mkdir -p "$repo_root_dir/api" - echo 'GET /api/status' >"$repo_root_dir/api/routes.txt" - elif [ "$scenario" = "endpoint-in-excluded-dir" ]; then - # Endpoint /api/hidden-secret exists ONLY inside excluded directories - # (.git/ and node_modules/). The grep excludes must prevent matching, - # so the finding is treated as hallucinated → fallback allowed. - mkdir -p "$repo_root_dir/.git/refs" - echo 'GET /api/hidden-secret' >"$repo_root_dir/.git/refs/leaked.txt" - mkdir -p "$repo_root_dir/node_modules/fake-pkg" - echo 'GET /api/hidden-secret' >"$repo_root_dir/node_modules/fake-pkg/index.js" - elif [ "$scenario" = "pr-stale-source-claim-fallback-success" ]; then - mkdir -p "$repo_root_dir/backend/db" - cat >"$repo_root_dir/backend/db/models.py" <<'EOS' -from sqlalchemy.orm import Mapped, mapped_column - -class EncryptedString: - pass - -class WorkspaceRunnerConfig: - registration_token: Mapped[str | None] = mapped_column( - EncryptedString, nullable=True - ) -EOS - elif [ "$scenario" = "pr-stale-snapshot-snippet-fallback-success" ]; then - mkdir -p "$repo_root_dir/backend/app/api" - cat >"$repo_root_dir/backend/app/api/snapshots.py" <<'EOS' -from fastapi import HTTPException - - -async def _get_authorized_snapshot(session, schema_snapshot_uuid, user): - project_space_uuid = await session.scalar("select project space") - if project_space_uuid is None: - return None - try: - await require_project_member(session, project_space_uuid, user.user_account_uuid) - except HTTPException as exc: - if exc.status_code == 403: - return None - raise - return await session.get("SchemaSnapshot", schema_snapshot_uuid) - - -async def get_snapshot(schema_snapshot_uuid, user, session): - snap = await _get_authorized_snapshot(session, schema_snapshot_uuid, user) - if snap is None: - return {"status": "not_found", "snapshot_json": None} - data = await session.get("SchemaSnapshotData", schema_snapshot_uuid) - return {"status": snap.status, "snapshot_json": data.snapshot_json if data else None} -EOS - elif [ "$scenario" = "pr-stale-source-plus-real-finding-blocks" ]; then - mkdir -p "$repo_root_dir/backend/db" "$repo_root_dir/backend/api" - cat >"$repo_root_dir/backend/db/models.py" <<'EOS' -from sqlalchemy.orm import Mapped, mapped_column - -class EncryptedString: - pass - -class WorkspaceRunnerConfig: - registration_token: Mapped[str | None] = mapped_column( - EncryptedString, nullable=True - ) -EOS - echo 'def real_changed_endpoint(): pass' >"$repo_root_dir/backend/api/emails.py" - elif [ "$scenario" = "pr-changed-finding-with-retry-marker-blocks" ]; then - mkdir -p "$repo_root_dir/backend/api" - echo 'def real_changed_endpoint(): pass' >"$repo_root_dir/backend/api/emails.py" - elif [ "$scenario" = "pr-stale-report-plus-inline-changed-finding-blocks" ]; then - mkdir -p "$repo_root_dir/backend/db" "$repo_root_dir/backend/api" - cat >"$repo_root_dir/backend/db/models.py" <<'EOS' -from sqlalchemy.orm import Mapped, mapped_column - -class EncryptedString: - pass - -class WorkspaceRunnerConfig: - registration_token: Mapped[str | None] = mapped_column( - EncryptedString, nullable=True - ) -EOS - echo 'def real_changed_endpoint(): pass' >"$repo_root_dir/backend/api/emails.py" - elif [ "$scenario" = "pr-changed-scope-bounded" ]; then - echo 'class Unrelated {}' >"$repo_root_dir/sync-module-system/smart-crawling-common/src/main/java/org/empasy/sync/common/system/util/JwtUtil.java" - elif [ "$scenario" = "pr-python-scope-context" ]; then - mkdir -p "$repo_root_dir/backend/api" "$repo_root_dir/backend/core" "$repo_root_dir/backend/db" "$repo_root_dir/backend/services" - touch "$repo_root_dir/backend/api/__init__.py" - touch "$repo_root_dir/backend/core/__init__.py" - touch "$repo_root_dir/backend/db/__init__.py" - touch "$repo_root_dir/backend/services/__init__.py" - echo 'from db.session import get_db' >"$repo_root_dir/backend/api/emails.py" - echo 'from api.auth import ensure_organization_access' >"$repo_root_dir/backend/api/runner_config.py" - echo 'ensure_organization_access(auth_context, config.organization_id)' >>"$repo_root_dir/backend/api/runner_config.py" - echo 'router = object()' >"$repo_root_dir/backend/api/search.py" - echo 'TRUSTED_CONFIG = True' >"$repo_root_dir/backend/core/config.py" - echo 'class LocalError(Exception): pass' >"$repo_root_dir/backend/core/exceptions.py" - echo 'def validate_auth_session_hmac_secret_value(value): return value' >"$repo_root_dir/backend/core/runtime_secrets.py" - echo 'engine = object()' >"$repo_root_dir/backend/db/session.py" - echo 'class Email: pass' >"$repo_root_dir/backend/db/models.py" - echo 'class ServiceError(Exception): pass' >"$repo_root_dir/backend/services/exceptions.py" - echo 'async def extract_backup_async(*args): return []' >"$repo_root_dir/backend/services/archive.py" - echo 'def parse_eml(*args): return {}' >"$repo_root_dir/backend/services/email_parser.py" - echo 'async def generate_embeddings(*args): return []' >"$repo_root_dir/backend/services/embedding.py" - echo 'async def assign_thread_id(*args, **kwargs): return "thread"' >"$repo_root_dir/backend/services/threading_service.py" - echo 'async def send_email(*args, **kwargs): return None' >"$repo_root_dir/backend/services/email_client.py" - echo 'pytest==0' >"$repo_root_dir/backend/requirements.txt" - elif [ "$scenario" = "pr-deployment-scope-entrypoint-context" ] || [ "$scenario" = "pr-baseline-critical-extensionless-dockerfile-target" ]; then - mkdir -p "$repo_root_dir/.github/workflows" "$repo_root_dir/backend/api" "$repo_root_dir/backend/core" "$repo_root_dir/backend/scripts" "$repo_root_dir/frontend" - echo 'name: OpenCode Review' >"$repo_root_dir/.github/workflows/opencode-review.yml" - cat >"$repo_root_dir/Dockerfile" <<'EOS' -FROM python:3.11-slim AS backend-runtime -WORKDIR /app -COPY backend /app/ -FROM backend-runtime -RUN chmod +x /app/scripts/docker_entrypoint.sh -CMD ["/app/scripts/docker_entrypoint.sh"] -EOS - cat >"$repo_root_dir/backend/scripts/docker_entrypoint.sh" <<'EOS' -#!/usr/bin/env bash -echo "Starting backend (uvicorn :8000)" -EOS - echo 'router = object()' >"$repo_root_dir/backend/api/auth.py" - echo 'class Settings: pass' >"$repo_root_dir/backend/core/config.py" - echo 'def validate_auth_session_hmac_secret_value(value): return value' >"$repo_root_dir/backend/core/runtime_secrets.py" - echo 'app = object()' >"$repo_root_dir/backend/main.py" - touch "$repo_root_dir/frontend/Dockerfile" - echo '{"scripts":{"start":"next start"}}' >"$repo_root_dir/frontend/package.json" - touch "$repo_root_dir/frontend/next.config.ts" - touch "$repo_root_dir/frontend/postcss.config.mjs" - touch "$repo_root_dir/docker-compose.yml" - touch "$repo_root_dir/render.yaml" - echo '0.0.0' >"$repo_root_dir/VERSION" - elif [ "$scenario" = "pr-rust-workspace-context" ]; then - mkdir -p "$repo_root_dir/.github/workflows" "$repo_root_dir/src" - echo 'name: Rust CI' >"$repo_root_dir/.github/workflows/rust.yml" - cat >"$repo_root_dir/Cargo.toml" <<'EOS' -[package] -name = "trusted-workspace" -version = "0.1.0" -EOS - echo '# trusted lock' >"$repo_root_dir/Cargo.lock" - echo '[toolchain]' >"$repo_root_dir/rust-toolchain.toml" - echo '[advisories]' >"$repo_root_dir/deny.toml" - echo 'fn main() {}' >"$repo_root_dir/src/main.rs" - elif [ "$scenario" = "github-models-fallback-dockerfile-test-baseline-before-next-success-continues" ]; then - mkdir -p "$repo_root_dir/.github/workflows" - cat >"$repo_root_dir/.github/workflows/build-ci-image.yml" <<'EOS' -name: Build CI image -jobs: - build: - steps: - - uses: docker/build-push-action@example - with: - file: ./Dockerfile.test -EOS - cat >"$repo_root_dir/Dockerfile.test" <<'EOS' -FROM python:3.13-slim -HEALTHCHECK CMD python -V || exit 1 -EOS - elif [ "$scenario" = "pr-critical-changed-internal-dotdir-target" ]; then - mkdir -p "$repo_root_dir/.github/workflows" - echo 'name: OpenCode Review' >"$repo_root_dir/.github/workflows/opencode-review.yml" - elif [ "$scenario" = "pr-critical-changed-json-target" ]; then - mkdir -p "$repo_root_dir/frontend/src/components" - echo 'export function CalendarLayout() { return null }' >"$repo_root_dir/frontend/src/components/CalendarLayout.tsx" - elif [ "$scenario" = "pr-changed-file-nonintersecting-line" ]; then - mkdir -p "$repo_root_dir/frontend/src" - { - echo 'import React from "react";' - for line_number in $(seq 2 140); do - printf 'const value%s = %s;\n' "$line_number" "$line_number" - done - } >"$repo_root_dir/frontend/src/App.tsx" - elif [ "$scenario" = "opencode-documented-env-api-key-fallback-success" ]; then - mkdir -p "$repo_root_dir/.github/workflows" - cat >"$repo_root_dir/.github/workflows/opencode-review.yml" <<'EOS' -name: OpenCode Review -config: | - { - "provider": { - "github-models": { - "options": { - "apiKey": "{env:STRIX_GITHUB_MODELS_TOKEN}" - } - } - } - } -EOS - elif [ "$scenario" = "generic-github-actions-workflow-fallback-success" ]; then - mkdir -p "$repo_root_dir/.github/workflows" - cat >"$repo_root_dir/.github/workflows/strix.yml" <<'EOS' -name: Strix Security Scan - -permissions: - actions: read - contents: read - models: read - -jobs: - strix: - steps: - - name: Fetch pull request head for trusted scan - run: | - if ! [[ "$PR_HEAD_SHA" =~ ^[0-9a-fA-F]{40}$ ]]; then - exit 1 - fi - if [ -n "$PR_BASE_SHA" ] && ! [[ "$PR_BASE_SHA" =~ ^[0-9a-fA-F]{40}$ ]]; then - exit 1 - fi - - name: Gate Strix secrets - run: | - echo '::error::STRIX_LLM must select GitHub Models openai/gpt-5 or newer, direct OpenAI GPT-5.4 or newer, OpenRouter openrouter/free, or an approved organization Vertex AI model.' - - name: Mask LLM API key - run: | - sanitized="$(printf '%s' "$LLM_API_KEY" | tr -d '\r\n')" - echo "::add-mask::${sanitized}" - - name: Prepare LLM API key input file - run: | - umask 077 - printf '%s' "$sanitized" > "$RUNNER_TEMP/llm_api_key.txt" -EOS - elif [ "$scenario" = "pr-large-scope-full-set" ]; then - mkdir -p "$repo_root_dir/backend/large-scope" - local large_scope_index - for large_scope_index in $(seq 1 38); do - printf 'file %s\n' "$large_scope_index" >"$repo_root_dir/backend/large-scope/file-$large_scope_index.py" - done - elif [ "$scenario" = "scan-working-directory-isolated" ]; then - mkdir -p "$repo_root_dir/backend/app/pg_introspect" - printf '%s\n' 'HEAD_INTROSPECT_SHOULD_BE_SCANNED' >"$repo_root_dir/backend/app/pg_introspect/introspect.py" - printf '%s\n' 'TRUSTED_DSN_GUARD_CONTEXT_SHOULD_BE_SCANNED' >"$repo_root_dir/backend/app/pg_introspect/dsn_guard.py" - fi - - local scenario_base_sha="" - local scenario_head_sha="" - if [ "$scenario" = "pr-changed-file-nonintersecting-line" ]; then - ( - cd "$repo_root_dir" - git init -q - git config user.email "ci@example.com" - git config user.name "CI" - git add frontend/src/App.tsx - git commit -qm 'base commit' - python3 - <<'PY' -from pathlib import Path - -path = Path("frontend/src/App.tsx") -lines = path.read_text(encoding="utf-8").splitlines() -lines[119] = f"{lines[119]} // changed search line" -path.write_text("\n".join(lines) + "\n", encoding="utf-8") -PY - git add frontend/src/App.tsx - git commit -qm 'head commit' - ) - scenario_base_sha="$(git -C "$repo_root_dir" rev-list --max-parents=0 HEAD)" - scenario_head_sha="$(git -C "$repo_root_dir" rev-parse HEAD)" - fi - - set +e - local env_cmd=( - PATH="$untrusted_bin_dir:$bin_dir:$PATH" - STRIX_EXECUTABLE_PATH="$fake_strix" - FAKE_STRIX_PATH_HIJACK_LOG="$path_hijack_log" - STRIX_INPUT_FILE_ROOT="$tmp_dir" - GITHUB_EVENT_NAME="" - GITHUB_EVENT_PATH="" - FAKE_STRIX_SCENARIO="$scenario" - FAKE_STRIX_CALL_LOG="$call_log" - FAKE_STRIX_API_BASE_LOG="$api_base_log" - FAKE_STRIX_TARGET_LOG="$target_log" - FAKE_STRIX_RUNTIME_ENV_LOG="$runtime_env_log" - FAKE_STRIX_TIMEOUT_SLEEP_SECONDS="$TIMEOUT_TEST_FAKE_SLEEP_SECONDS" - STRIX_LLM_DEFAULT_PROVIDER="$default_provider" - FAKE_STRIX_STATE_FILE="$state_file" - STRIX_TRANSIENT_RETRY_PER_MODEL="$transient_retry_per_model" - STRIX_TRANSIENT_RETRY_BACKOFF_SECONDS="$transient_retry_backoff_seconds" - STRIX_PROCESS_TIMEOUT_SECONDS="$process_timeout_seconds" - STRIX_TOTAL_TIMEOUT_SECONDS="$total_timeout_seconds" - STRIX_FAIL_ON_MIN_SEVERITY="$min_fail_severity" - STRIX_REPORTS_DIR="$repo_root_dir/strix_runs" - STRIX_TARGET_PATH="$effective_target_path" - ) - if [ "$scenario" = "runtime-env-forwarding" ] || [ "$scenario" = "custom-openai-compatible-preserves-effort" ]; then - env_cmd+=( - LLM_TIMEOUT="90" - STRIX_MEMORY_COMPRESSOR_TIMEOUT="10" - STRIX_REASONING_EFFORT="minimal" - STRIX_LLM_MAX_RETRIES="1" - GEMINI_LOCATION="GLOBAL" - UNRELATED_SECRET="should-not-forward" - ) - fi - if [ "$scenario" = "pr-executable-integrity-mismatch" ]; then - env_cmd+=( - IS_PR_EVIDENCE_RUN="true" - STRIX_EXECUTABLE_ROOT="$bin_dir" - STRIX_EXECUTABLE_SHA256="0000000000000000000000000000000000000000000000000000000000000000" - ) - fi - if [ "$scenario" = "pr-executable-root-group-writable" ]; then - local fake_strix_sha256 - fake_strix_sha256="$(python3 - "$fake_strix" <<'PY' -import hashlib -from pathlib import Path -import sys - -print(hashlib.sha256(Path(sys.argv[1]).read_bytes()).hexdigest()) -PY -)" - env_cmd+=( - IS_PR_EVIDENCE_RUN="true" - STRIX_EXECUTABLE_ROOT="$bin_dir" - STRIX_EXECUTABLE_SHA256="$fake_strix_sha256" - ) - chmod 0775 "$bin_dir" - fi - if [ "$scenario" = "pr-executable-group-writable" ]; then - chmod 0775 "$fake_strix" - fi - if [ "$scenario" = "report-known-internal-warning-sanitized" ]; then - env_cmd+=( - FAKE_STRIX_OUTSIDE_REPORT_DIR="$repo_root_dir/outside-strix-report" - ) - fi - if [ "$scenario" = "nvidia-rate-limit-openai-direct-fallback-clears-api-base" ]; then - printf '%s' 'openai-fallback-token' >"$tmp_dir/openai_fallback_key.txt" - env_cmd+=(STRIX_OPENAI_FALLBACK_KEY_FILE="$tmp_dir/openai_fallback_key.txt") - env_cmd+=(STRIX_REASONING_EFFORT="high") - fi - if [ "$scenario" = "openai-direct-quota-github-models-fallback-success" ]; then - printf '%s' 'https://models.github.ai/inference' >"$tmp_dir/github_models_api_base.txt" - printf '%s' 'github-models-fallback-token' >"$tmp_dir/github_models_key.txt" - env_cmd+=(STRIX_GITHUB_MODELS_API_BASE_FILE="$tmp_dir/github_models_api_base.txt") - env_cmd+=(STRIX_GITHUB_MODELS_KEY_FILE="$tmp_dir/github_models_key.txt") - fi - if [ "$min_fail_severity" = "__UNSET__" ]; then - local next_env_cmd=() - local env_pair - for env_pair in "${env_cmd[@]}"; do - case "$env_pair" in - STRIX_FAIL_ON_MIN_SEVERITY=*) - continue - ;; - esac - next_env_cmd+=("$env_pair") - done - env_cmd=("${next_env_cmd[@]}") - fi - printf '%s' "$initial_model" >"$strix_llm_file" - env_cmd+=(STRIX_LLM_FILE="$strix_llm_file") - printf '%s' 'dummy' >"$llm_api_key_file" - env_cmd+=(LLM_API_KEY_FILE="$llm_api_key_file") - env_cmd+=(STRIX_DISABLE_PR_SCOPING="$disable_pr_scoping") - env_cmd+=(STRIX_FAIL_ON_PROVIDER_SIGNAL="$fail_on_provider_signal") - local llm_api_base_source="$raw_llm_api_base" - if [ -z "$llm_api_base_source" ] && [ -n "$initial_llm_api_base" ]; then - llm_api_base_source="$initial_llm_api_base" - fi - if [ -n "$llm_api_base_source" ]; then - printf '%s' "$llm_api_base_source" >"$llm_api_base_file" - env_cmd+=(LLM_API_BASE_FILE="$llm_api_base_file") - fi - # Only export fallback variables when a non-empty value is provided so the - # gate's ${VAR+x} checks correctly distinguish "unset → use defaults" from - # "set to empty → disable fallbacks". - if [ -n "$fallback_models" ]; then - env_cmd+=(STRIX_VERTEX_FALLBACK_MODELS="$fallback_models") - fi - case "$gemini_fallback_models" in - __SAME_AS_FALLBACK_MODELS__) - if [ -n "$fallback_models" ]; then - env_cmd+=(STRIX_GEMINI_FALLBACK_MODELS="$fallback_models") - fi - ;; - __UNSET__) - ;; - *) - if [ -n "$gemini_fallback_models" ]; then - env_cmd+=(STRIX_GEMINI_FALLBACK_MODELS="$gemini_fallback_models") - fi - ;; - esac - if [ -n "$generic_fallback_models" ]; then - env_cmd+=(STRIX_FALLBACK_MODELS="$generic_fallback_models") - fi - if [ -n "$custom_source_dirs" ]; then - env_cmd+=(STRIX_SOURCE_DIRS="$custom_source_dirs") - fi - : "$legacy_scope_size_ignored" - if [ -n "$github_event_name" ]; then - env_cmd+=(GITHUB_EVENT_NAME="$github_event_name") - fi - if [ -n "$event_name_override" ]; then - env_cmd+=(EVENT_NAME="$event_name_override") - fi - if [ -n "$test_pr_sca_status_override" ]; then - env_cmd+=(STRIX_TEST_PR_SCA_STATUS_OVERRIDE="$test_pr_sca_status_override") - fi - if [ -n "$current_pr_number" ]; then - env_cmd+=(GITHUB_EVENT_PATH="$event_payload_file") - env_cmd+=(GITHUB_REPOSITORY="octo-org/smart-crawling-server") - env_cmd+=(PR_BASE_SHA="test-base-sha") - env_cmd+=(PR_HEAD_SHA="test-head-sha") - env_cmd+=(GH_TOKEN="g""hs_test_token") - fi - if [ -n "$scenario_base_sha" ] && [ -n "$scenario_head_sha" ]; then - env_cmd+=(PR_BASE_SHA="$scenario_base_sha") - env_cmd+=(PR_HEAD_SHA="$scenario_head_sha") - fi - if [ -n "$authoritative_sca_runs_json" ]; then - local gh_api_response_file="$tmp_dir/gh-api-response.json" - printf '%s\n' "$authoritative_sca_runs_json" >"$gh_api_response_file" - env_cmd+=(FAKE_GH_API_RESPONSE_FILE="$gh_api_response_file") - env_cmd+=(FAKE_GH_TOKEN_LOG="$gh_token_log") - fi - if [ "$changed_files_override" = "__SET_EMPTY__" ]; then - env_cmd+=(STRIX_TEST_CHANGED_FILES_OVERRIDE="") - elif [ -n "$changed_files_override" ]; then - env_cmd+=(STRIX_TEST_CHANGED_FILES_OVERRIDE="$changed_files_override") - fi - ( - cd "$repo_root_dir" - env \ - -u GITHUB_EVENT_NAME \ - -u GITHUB_EVENT_PATH \ - -u STRIX_TEST_CHANGED_FILES_OVERRIDE \ - -u STRIX_VERTEX_FALLBACK_MODELS \ - -u STRIX_GEMINI_FALLBACK_MODELS \ - -u STRIX_FALLBACK_MODELS \ - -u STRIX_OPENAI_FALLBACK_KEY_FILE \ - -u STRIX_OPENAI_FALLBACK_API_BASE_FILE \ - "${env_cmd[@]}" \ - bash "./scripts/ci/strix_quick_gate.sh" >"$output_log" 2>&1 - ) - local rc=$? - set -e - - assert_equals "$expected_exit" "$rc" "scenario=$scenario exit code" - if [ "$expected_exit" != "$rc" ]; then - echo "scenario=$scenario gate output:" >&2 - sed 's/^/ | /' "$output_log" >&2 - fi - - if [ -n "$expected_message" ]; then - case "$expected_message" in - REGEX:*) - assert_file_matches "$output_log" "${expected_message#REGEX:}" "scenario=$scenario output" - ;; - *) - assert_file_contains "$output_log" "$expected_message" "scenario=$scenario output" - ;; - esac - fi - - local call_count - call_count="0" - if [ -f "$call_log" ]; then - call_count="$(wc -l <"$call_log" | tr -d ' ')" - fi - assert_equals "$expected_calls" "$call_count" "scenario=$scenario strix call count" - if [ -e "$path_hijack_log" ]; then - record_failure "scenario=$scenario selected a PATH-controlled Strix executable instead of STRIX_EXECUTABLE_PATH" - fi - - if [ -n "$expected_model_sequence" ]; then - local actual_model_sequence="" - if [ -f "$call_log" ]; then - while IFS= read -r model; do - if [ -n "$actual_model_sequence" ]; then - actual_model_sequence="${actual_model_sequence}|$model" - else - actual_model_sequence="$model" - fi - done <"$call_log" - fi - - assert_equals "$expected_model_sequence" "$actual_model_sequence" "scenario=$scenario STRIX_LLM sequence" - fi - - if [ -n "$expected_api_base_sequence" ]; then - local actual_api_base_sequence="" - if [ -f "$api_base_log" ]; then - while IFS= read -r api_base; do - if [ -n "$actual_api_base_sequence" ]; then - actual_api_base_sequence="${actual_api_base_sequence}|$api_base" - else - actual_api_base_sequence="$api_base" - fi - done <"$api_base_log" - fi - - assert_equals "$expected_api_base_sequence" "$actual_api_base_sequence" "scenario=$scenario LLM_API_BASE sequence" - fi - - if [ "$scenario" = "runtime-env-forwarding" ]; then - assert_file_contains \ - "$runtime_env_log" \ - "LLM_TIMEOUT=90;STRIX_MEMORY_COMPRESSOR_TIMEOUT=10;STRIX_REASONING_EFFORT=minimal;STRIX_LLM_MAX_RETRIES=1;GEMINI_LOCATION=GLOBAL;PYTHONWARNINGS=ignore:Pydantic serializer warnings:UserWarning:pydantic.main;NPM_CONFIG_IGNORE_SCRIPTS=true;PNPM_CONFIG_IGNORE_SCRIPTS=true;YARN_ENABLE_SCRIPTS=false;UNRELATED_SECRET=" \ - "scenario=$scenario runtime env forwarding" - # Non-contextual-orchestrator providers (gemini here) never see the - # stream-disabling opt-in: it is scoped narrowly to the gateway that - # rejects stream_options.include_usage alongside tools. - assert_file_contains \ - "$runtime_env_log" \ - "LLM_DISABLE_STREAMING=" \ - "scenario=$scenario non-gateway providers keep real streaming" - fi - if [ "$scenario" = "custom-openai-compatible-preserves-effort" ]; then - assert_file_contains \ - "$runtime_env_log" \ - "STRIX_REASONING_EFFORT=minimal" \ - "scenario=$scenario custom compatible endpoint effort" - fi - if [ "$scenario" = "contextual-orchestrator-gateway-model-qualification" ]; then - # contextual-orchestrator rejects stream_options.include_usage=true - # alongside tools; Strix's agent loop always sends both, so the gate - # routes this gateway through Strix's own LLM_DISABLE_STREAMING opt-in - # (single non-streaming get_response per turn) instead of streaming. - assert_file_contains \ - "$runtime_env_log" \ - "LLM_DISABLE_STREAMING=true" \ - "scenario=$scenario contextual-orchestrator gateway disables SDK streaming to avoid the stream_options+tools rejection" - fi - - if [ "$scenario" = "report-known-internal-warning-sanitized" ]; then - assert_file_not_contains \ - "$repo_root_dir/strix_runs/fake-known-internal-warning/strix.log" \ - "produced non-lifecycle final output" \ - "scenario=$scenario strips the known internal Strix warning from published artifacts" - assert_file_contains \ - "$repo_root_dir/strix_runs/fake-known-internal-warning/strix.log" \ - "finish_scan: completed scan with 0 vulnerability report(s)" \ - "scenario=$scenario keeps non-warning Strix report evidence" - assert_file_not_contains \ - "$repo_root_dir/strix_runs/fake-known-internal-warning-relative/strix.log" \ - "produced non-lifecycle final output" \ - "scenario=$scenario sanitizes relative scanner output before publication" - assert_file_contains \ - "$repo_root_dir/strix_runs/fake-known-internal-warning-relative/strix.log" \ - "finish_scan: completed scan with 0 vulnerability report(s)" \ - "scenario=$scenario publishes sanitized relative scanner evidence" - assert_file_contains \ - "$repo_root_dir/outside-strix-report/strix.log" \ - "outside report should not be rewritten" \ - "scenario=$scenario does not rewrite logs through symlinked report directories" - fi - - if [ "$scenario" = "report-known-internal-warning-variant-sanitized" ]; then - assert_file_not_contains \ - "$repo_root_dir/strix_runs/fake-known-internal-warning-variant/strix.log" \ - "ended a turn without a lifecycle tool call" \ - "scenario=$scenario strips the newer-wording known internal Strix warning from published artifacts" - assert_file_contains \ - "$repo_root_dir/strix_runs/fake-known-internal-warning-variant/strix.log" \ - "finish_scan: completed scan with 0 vulnerability report(s)" \ - "scenario=$scenario keeps non-warning Strix report evidence" - fi - - if [ "$scenario" = "github-models-primary-ratelimit-fallback-success" ]; then - assert_file_contains \ - "$output_log" \ - "GitHub Models rate limit detected for model 'openai/gpt-5'; skipping same-model retry and moving directly to fallback models or current-head neutral classification." \ - "scenario=$scenario logs why same-model retry was skipped" - assert_file_not_contains \ - "$output_log" \ - "Retrying model 'openai/gpt-5' due to rate limit" \ - "scenario=$scenario does not sleep in same-model retry after GitHub Models rate limiting" - fi - - if [ "$scenario" = "pr-changed-scope-full-set" ]; then - assert_internal_pr_scope_targets "$target_log" "$repo_root_dir" "$expected_calls" - fi - - rm -rf "$tmp_dir" -} - -run_gate_case_with_provider_signal_mode() { - local provider_signal_mode="$1" - shift - local args=("$@") - local default_args=( - "vertex_ai" - "__DEFAULT__" - "" - "0" - "CRITICAL" - "0" - "" - "" - "1200" - "0" - "" - "" - "" - "" - "0" - "" - "" - "" - "__SAME_AS_FALLBACK_MODELS__" - "" - ) - - while [ "${#args[@]}" -lt 28 ]; do - args+=("${default_args[${#args[@]} - 8]}") - done - args+=("$provider_signal_mode") - run_gate_case "${args[@]}" -} - -run_gate_case_allow_provider_signal() { - run_gate_case_with_provider_signal_mode "0" "$@" -} - -run_github_models_http410_case() { - local scenario="$1" - local expected_exit="$2" - local expected_calls="$3" - local expected_models="$4" - local expected_api_bases="$5" - local expected_message="${6-}" - - run_gate_case "$scenario" \ - "openai/gpt-5" \ - "" \ - "$expected_exit" \ - "$expected_message" \ - "$expected_calls" \ - "$expected_models" \ - "$expected_api_bases" \ - "openai" \ - "https://models.github.ai/inference" \ - "" \ - "0" \ - "CRITICAL" \ - "0" \ - "" \ - "" \ - "1200" \ - "0" \ - "" \ - "" \ - "" \ - "" \ - "0" \ - "" \ - "" \ - "" \ - "__SAME_AS_FALLBACK_MODELS__" \ - "deepseek/deepseek-r1-0528" \ - "1" -} - -run_filtered_gate_case_if_requested() { - case "${STRIX_TEST_CASE_FILTER:-}" in - "") - return 0 - ;; - success) - run_gate_case "success" \ - "vertex_ai/ready-primary" \ - "vertex_ai/fallback-one vertex_ai/fallback-two" \ - "0" \ - "scan ok" \ - "1" \ - "vertex_ai/ready-primary" \ - "" - ;; - contextual-orchestrator-missing-api-base-fails-closed) - run_gate_case "contextual-orchestrator-missing-api-base-fails-closed" \ - "orchestrator/free" \ - "" \ - "2" \ - "require LLM_API_BASE_FILE to select the pinned loopback gateway" \ - "0" \ - "" \ - "" \ - "contextual_orchestrator" \ - "" - ;; - contextual-orchestrator-gateway-model-qualification) - run_gate_case "contextual-orchestrator-gateway-model-qualification" \ - "orchestrator/free" \ - "" \ - "0" \ - "scan ok through contextual-orchestrator gateway" \ - "1" \ - "openai/orchestrator/free" \ - "http://127.0.0.1:18080/v1" \ - "contextual_orchestrator" \ - "http://127.0.0.1:18080/v1" - ;; - pr-rust-workspace-context) - run_gate_case "pr-rust-workspace-context" \ - "openai/gpt-4o-mini" \ - "" \ - "0" \ - "scan ok with Rust workspace context" \ - "1" \ - "openai/gpt-4o-mini" \ - "https://example.invalid" \ - "vertex_ai" \ - "__DEFAULT__" \ - "" \ - "0" \ - "CRITICAL" \ - "0" \ - "" \ - "" \ - "1200" \ - "0" \ - "pull_request" \ - ".github/workflows/rust.yml" - ;; - success-with-critical-report) - run_gate_case "success-with-critical-report" \ - "vertex_ai/ready-primary" \ - "" \ - "1" \ - "Strix exited successfully but emitted a vulnerability at or above 'CRITICAL'" \ - "1" \ - "vertex_ai/ready-primary" \ - "" - ;; - pr-executable-integrity-mismatch) - run_gate_case "pr-executable-integrity-mismatch" \ - "vertex_ai/ready-primary" \ - "" \ - "1" \ - "did not match the pinned SHA-256 digest" \ - "0" \ - "" \ - "" - ;; - pr-executable-group-writable) - run_gate_case "pr-executable-group-writable" \ - "vertex_ai/ready-primary" \ - "" \ - "1" \ - "must not be group/world writable" \ - "0" \ - "" \ - "" - ;; - pr-executable-root-group-writable) - run_gate_case "pr-executable-root-group-writable" \ - "vertex_ai/ready-primary" \ - "" \ - "1" \ - "pinned Strix installation root must not be group/world writable" \ - "0" \ - "" \ - "" - ;; - vertex-primary-hallucinated-endpoint-fallback-success) - run_gate_case "vertex-primary-hallucinated-endpoint-fallback-success" \ - "vertex_ai/hallucination-primary" \ - "vertex_ai/fallback-one vertex_ai/fallback-two" \ - "1" \ - "Strix quick scan failed with a non-recoverable error." \ - "1" \ - "vertex_ai/hallucination-primary" \ - "" - ;; - target-path-src-default-source-dirs) - run_gate_case "target-path-src-default-source-dirs" \ - "vertex_ai/hallucination-primary" \ - "vertex_ai/fallback-one vertex_ai/fallback-two" \ - "1" \ - "Strix quick scan failed with a non-recoverable error." \ - "1" \ - "vertex_ai/hallucination-primary" \ - "" \ - "vertex_ai" \ - "__DEFAULT__" \ - "" \ - "1" \ - "CRITICAL" \ - "0" \ - "__USE_SUBDIR_SRC__" \ - "" - ;; - vertex-ignores-untrusted-llm-api-base-file) - run_vertex_model_ignores_untrusted_llm_api_base_file_case - ;; - input-file-root-override-precedence) - run_input_file_root_override_takes_precedence_over_runner_temp_case - ;; - vertex-without-llm-api-key) - run_vertex_without_llm_api_key_case - ;; - vertex-with-llm-api-key-file-not-forwarded) - run_vertex_with_llm_api_key_file_does_not_forward_case - ;; - stale-report-does-not-bypass) - run_stale_report_case - ;; - symlink-report-does-not-bypass) - run_symlink_report_case - ;; - github-models-token-limit-fallback-success) - run_gate_case "github-models-token-limit-fallback-success" \ - "openai/gpt-5" \ - "" \ - "0" \ - "REGEX:Strix quick scan succeeded with fallback model 'github_models/deepseek/deepseek-v3-0324' in [0-9]+s\\." \ - "2" \ - "openai/gpt-5|openai/deepseek/deepseek-v3-0324" \ - "https://models.github.ai/inference|https://models.github.ai/inference" \ - "openai" \ - "https://models.github.ai/inference" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "github_models/deepseek/deepseek-v3-0324 github_models/deepseek/deepseek-r1-0528" - ;; - openrouter-502-fallback-retry-same-model-success) - run_gate_case "openrouter-502-fallback-retry-same-model-success" \ - "vertex_ai/missing-primary" \ - "openrouter/free vertex_ai/fallback-two" \ - "0" \ - "scan ok after OpenRouter 502 same-model retry" \ - "3" \ - "vertex_ai/missing-primary|openrouter/free|openrouter/free" \ - "|https://example.invalid|https://example.invalid" \ - "vertex_ai" \ - "__DEFAULT__" \ - "" \ - "1" - ;; - openrouter-502-distant-target-output-nonretryable) - run_gate_case "openrouter-502-distant-target-output-nonretryable" \ - "vertex_ai/missing-primary" \ - "openrouter/free vertex_ai/fallback-two" \ - "1" \ - "Strix quick scan failed with a non-recoverable error." \ - "2" \ - "vertex_ai/missing-primary|openrouter/free" \ - "|https://example.invalid" \ - "vertex_ai" \ - "__DEFAULT__" \ - "" \ - "1" - ;; - service-unavailable-no-llm-marker-nonrecoverable) - run_gate_case "service-unavailable-no-llm-marker-nonrecoverable" \ - "custom/service-unavailable-primary" \ - "vertex_ai/fallback-one vertex_ai/fallback-two" \ - "1" \ - "Strix quick scan failed with a non-recoverable error." \ - "1" \ - "custom/service-unavailable-primary" \ - "https://example.invalid" \ - "custom" \ - "__DEFAULT__" \ - "" \ - "1" - ;; - custom-openai-compatible-preserves-effort) - run_gate_case "custom-openai-compatible-preserves-effort" \ - "openai-direct/gpt-5.4" \ - "" \ - "0" \ - "scan ok" \ - "1" \ - "openai/gpt-5.4" \ - "https://compatible.example/v1" \ - "openai" \ - "https://compatible.example/v1" - ;; - nvidia-rate-limit-openai-direct-fallback-clears-api-base) - run_gate_case_allow_provider_signal "nvidia-rate-limit-openai-direct-fallback-clears-api-base" \ - "nvidia_nim/nvidia/rate-limited-primary" \ - "" \ - "0" \ - "REGEX:Strix quick scan succeeded with fallback model 'openai-direct/gpt-5.4' in [0-9]+s\\." \ - "2" \ - "nvidia_nim/nvidia/rate-limited-primary|openai/gpt-5.4" \ - "https://integrate.api.nvidia.com/v1|" \ - "nvidia_nim" \ - "https://integrate.api.nvidia.com/v1" \ - "" \ - "0" \ - "CRITICAL" \ - "0" \ - "" \ - "" \ - "1200" \ - "0" \ - "" \ - "" \ - "" \ - "" \ - "0" \ - "" \ - "" \ - "" \ - "__SAME_AS_FALLBACK_MODELS__" \ - "openai-direct/gpt-5.4" - ;; - openai-direct-quota-github-models-fallback-success) - run_gate_case "openai-direct-quota-github-models-fallback-success" \ - "openai_direct/gpt-5.4" \ - "" \ - "0" \ - "REGEX:Strix quick scan succeeded with fallback model 'github_models/openai/o3' in [0-9]+s\\." \ - "2" \ - "openai/gpt-5.4|openai/o3" \ - "|https://models.github.ai/inference" \ - "vertex_ai" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "" \ - "github_models/openai/o3" - ;; - gemini-timeout-fallback-success) - run_gate_case_allow_provider_signal "gemini-timeout-fallback-success" \ - "gemini/timeout-fallback-primary" \ - "gemini/fallback-one gemini/fallback-two" \ - "0" \ - "REGEX:Strix quick scan succeeded with fallback model 'gemini/fallback-one' in [0-9]+s\\." \ - "2" \ - "gemini/timeout-fallback-primary|gemini/fallback-one" \ - "https://example.invalid|https://example.invalid" \ - "vertex_ai" \ - "__DEFAULT__" \ - "" \ - "1" - ;; - zero-findings-with-low-report-timeout) - run_gate_case_allow_provider_signal "zero-findings-with-low-report-timeout" \ - "vertex_ai/zero-low-primary" \ - "vertex_ai/fallback-one" \ - "1" \ - "Configured Vertex model and fallback models were unavailable." \ - "2" \ - "vertex_ai/zero-low-primary|vertex_ai/fallback-one" \ - "|" \ - "vertex_ai" \ - "__DEFAULT__" \ - "" \ - "0" \ - "CRITICAL" \ - "0" \ - "" \ - "" \ - "$TIMEOUT_TEST_PROCESS_SECONDS" \ - "0" \ - "pull_request" \ - "sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" - ;; - zero-findings-timeout-all-models) - run_gate_case_allow_provider_signal "zero-findings-timeout-all-models" \ - "vertex_ai/zero-timeout-primary" \ - "vertex_ai/fallback-one" \ - "1" \ - "Strix reported zero vulnerabilities before provider infrastructure failure; failing closed because provider infrastructure failures are not clean scan evidence." \ - "2" \ - "vertex_ai/zero-timeout-primary|vertex_ai/fallback-one" \ - "|" \ - "vertex_ai" \ - "__DEFAULT__" \ - "" \ - "0" \ - "CRITICAL" \ - "0" \ - "" \ - "" \ - "$TIMEOUT_TEST_PROCESS_SECONDS" \ - "0" \ - "pull_request" \ - "sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" - run_gate_case_allow_provider_signal "zero-findings-timeout-all-models" \ - "vertex_ai/zero-timeout-primary" \ - "vertex_ai/fallback-one" \ - "1" \ - "Configured Vertex model and fallback models were unavailable." \ - "2" \ - "vertex_ai/zero-timeout-primary|vertex_ai/fallback-one" \ - "|" \ - "vertex_ai" \ - "__DEFAULT__" \ - "" \ - "0" \ - "CRITICAL" \ - "0" \ - "" \ - "" \ - "$TIMEOUT_TEST_PROCESS_SECONDS" \ - "0" \ - "push" - ;; - slow-timeout) - run_gate_case_allow_provider_signal "slow-timeout" \ - "vertex_ai/slow-primary" \ - "" \ - "1" \ - "Strix run timed out after ${TIMEOUT_TEST_PROCESS_SECONDS}s." \ + assert_file_contains "$work…61342 tokens truncated…CONDS}s." \ "3" \ "vertex_ai/slow-primary|vertex_ai/gemini-2.5-pro|vertex_ai/gemini-2.5-flash" \ "||" \ From c6305c9732faceb8a21561db11dc4717296a9a1c Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 31 Aug 2026 20:01:48 +0900 Subject: [PATCH 5/5] fix(ci): preserve the complete quick-gate regression --- scripts/ci/test_strix_quick_gate.sh | 5051 ++++++++++++++++++++++++++- 1 file changed, 5047 insertions(+), 4 deletions(-) diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index 719851a5ed..bd8fe9b854 100644 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -1,6 +1,3 @@ -Warning: truncated output (original token count: 161342) -Total output lines: 13105 - #!/usr/bin/env bash set -euo pipefail @@ -1476,7 +1473,5053 @@ assert_file_contains "$REPO_ROOT/scripts/ci/run_opencode_review_model_pool.sh" ' assert_file_contains "$workflow_file" ".github/workflows/strix.yml" "opencode inline fallback watches Strix workflow changes" assert_file_contains "$workflow_file" "self_modifying_strix_base_failure" "opencode approval detects trusted-base Strix failures for self-modifying workflow PRs" assert_file_contains "$workflow_file" 'local source_root="${OPENCODE_SOURCE_WORKDIR:-${GITHUB_WORKSPACE:-$PWD}}"' "opencode trusted-base Strix lag detection inspects the PR-head worktree" - assert_file_contains "$work…61342 tokens truncated…CONDS}s." \ + assert_file_contains "$workflow_file" 'git -C "$source_root" diff --quiet' "opencode trusted-base Strix lag detection compares trusted-input changes in the PR-head worktree" + assert_file_contains "$workflow_file" "opencode.jsonc: No such file or directory" "opencode approval recognizes base-workflow Strix self-test evidence that cannot see PR-head OpenCode config" + assert_file_contains "$workflow_file" "latest_current_head_manual_strix_run" "opencode approval inspects same-head manual Strix repository_dispatch runs before suppressing trusted-base Strix failures" + assert_file_contains "$workflow_file" 'wait_for_peer_github_checks "$pending_checks_file"' "opencode approval waits for pending same-head manual Strix evidence before failing self-modifying workflow PRs" + assert_file_contains "$workflow_file" "Current-head default-branch repository_dispatch Strix evidence completed with" "opencode approval resumes normal failed-check handling after same-head manual Strix completes" + assert_file_contains "$workflow_file" "Leaving the PR review unchanged; rerun same-head repository_dispatch Strix evidence" "opencode approval avoids false request-changes reviews for trusted-base Strix self-test lag" + assert_file_contains "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" "opencode.jsonc" "failed-check fallback treats OpenCode config as a trusted Strix input" + assert_file_contains "$workflow_file" "scripts/ci/strix_quick_gate.sh" "opencode inline fallback watches trusted Strix gate changes" + assert_file_contains "$workflow_file" "scripts/ci/test_strix_quick_gate.sh" "opencode inline fallback watches trusted Strix self-test changes" + assert_file_contains "$workflow_file" "requirements-strix-ci.txt" "opencode inline fallback watches trusted Strix dependency changes" + assert_file_contains "$workflow_file" "requirements-strix-ci-hashes.txt" "opencode inline fallback watches trusted Strix hash lockfile changes" + assert_file_contains "$workflow_file" "self_healed_strix_dependency_base_failure" "opencode approval can classify trusted-base Strix dependency failures fixed by the current head" + assert_file_contains "$workflow_file" 'Ignoring trusted-base Strix protobuf resolver failure because current head updates requirements-strix-ci-hashes.txt away from protobuf==7.35.1.' "opencode approval ignores self-healed trusted-base Strix dependency failures after model approval" + assert_file_contains "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" "Strix provider failure blocked current-head security evidence" "failed-check fallback does not label non-quota provider routing/auth failures as quota" + assert_file_not_contains "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" "Strix provider quota blocked current-head security evidence" "failed-check fallback avoids misleading quota-only provider blocker title" + assert_file_contains "$workflow_file" "- Root cause:" "opencode review request-changes body includes root cause per finding" + assert_file_contains "$workflow_file" "- Regression test:" "opencode review request-changes body includes regression test direction per finding" + assert_file_contains "$workflow_file" "- Suggested diff:" "opencode review request-changes body includes suggested diff per finding" + assert_file_contains "$workflow_file" "OpenCode reviewed the current-head bounded evidence and found source-backed failed-check findings that must be addressed before merge." "opencode review workflow requests changes only when current-head failed checks are mapped to source-backed findings" + assert_file_contains "$workflow_file" "OpenCode reviewed the current-head evidence but could not verify peer GitHub Checks before approval." "opencode review workflow explains check lookup failures instead of approving" + assert_file_contains "$workflow_file" '["FAILURE","TIMED_OUT","ACTION_REQUIRED","CANCELLED","STARTUP_FAILURE"]' "opencode review workflow treats failed check-run conclusions as request-changes blockers" + assert_file_contains "$workflow_file" '["FAILURE","ERROR"]' "opencode review workflow treats failed status contexts as request-changes blockers" + assert_file_not_contains "$workflow_file" "MODEL: github-models/gpt-4.1" "opencode review must not fall back to GPT-4.1" + assert_file_contains "$opencode_config" '"enabled_providers": ["contextual-orchestrator"]' "opencode config enables only the contextual-orchestrator provider" + assert_file_not_contains "$workflow_file" "github-models/openai/gpt-5-mini" "opencode review excludes GitHub Models GPT-5 mini from the high-sensitivity review pool" + + assert_file_contains "$opencode_config" '"mcp": {}' "opencode config disables all model-runtime MCP servers" + assert_file_not_contains "$opencode_config" '"@upstash/context7-mcp' "opencode config does not install Context7 at runtime" + assert_file_not_contains "$opencode_config" '"@guhcostan/web-search-mcp' "opencode config does not install web-search MCP at runtime" + assert_file_not_contains "$opencode_config" '"serve"' "opencode config does not launch CodeGraph inside the credentialed model process" + assert_file_contains "$opencode_config" '"small_model": "contextual-orchestrator/orchestrator/free"' "opencode config routes the small model through the contextual-orchestrator free pool" + assert_file_contains "$opencode_config" '"model": "contextual-orchestrator/orchestrator/free"' "opencode config defaults review sessions to the contextual-orchestrator free pool" + assert_file_not_contains "$opencode_config" '"small_model": "nvidia-nim/meta/llama-3.3-70b-instruct"' "opencode config no longer pins the NVIDIA NIM small model" + assert_file_not_contains "$opencode_config" '"model": "nvidia-nim/nvidia/llama-3.3-nemotron-super-49b-v1.5"' "opencode config no longer pins the NVIDIA NIM Nemotron Super default" +assert_file_contains "$opencode_config" '"nvidia-nim"' "opencode config enables nvidia-nim provider" +assert_file_contains "$opencode_config" 'integrate.api.nvidia.com' "opencode config points nvidia-nim at NIM API" + assert_file_contains "$opencode_config" '"openai/gpt-5"' "opencode config defines GitHub Models GPT-5 with full model id" + assert_file_contains "$opencode_config" '"openai/gpt-5-chat"' "opencode config defines GPT-5 Chat catalog fallback" + assert_file_contains "$opencode_config" '"openai/gpt-5-mini"' "opencode config defines GPT-5 Mini catalog fallback" + assert_file_contains "$opencode_config" '"deepseek/deepseek-r1-0528"' "opencode config defines DeepSeek R1 fallback" + assert_file_contains "$opencode_config" '"deepseek/deepseek-v3-0324"' "opencode config defines DeepSeek V3 fallback" + assert_file_contains "$opencode_config" '"context": 200000' "opencode config uses the GitHub Models GPT-5 200k context window" + assert_file_contains "$opencode_config" '"output": 100000' "opencode config uses the GitHub Models GPT-5 100k output window" + assert_file_contains "$opencode_config" '"openai/gpt-4.1"' "opencode config defines the GitHub Models GPT-4.1 fallback" + assert_file_contains "$opencode_config" '"reasoningEffort": "high"' "opencode config keeps high reasoning effort for capable review models" +} + +assert_opencode_review_posts_suggested_diffs_inline() { + local workflow_file="$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" + + assert_file_contains "$workflow_file" "create_pull_review_with_payload" "opencode review can post custom review payloads" + assert_file_contains "$workflow_file" "comments: [" "opencode review payload includes inline review comments" + assert_file_contains "$workflow_file" '#### Suggested diff\n```diff\n' "opencode review puts suggested diffs inside inline review comments" + assert_file_contains "$workflow_file" "GitHub did not accept the inline review comments" "opencode review explains anchor failures instead of copying diffs to the PR body" + assert_file_contains "$workflow_file" "publish_request_changes_from_control" "opencode review REQUEST_CHANGES path publishes findings from the control JSON" + + # Same SIGPIPE-under-pipefail shape as the required-workflow-bootstrap + # check above: read the piped awk range to completion instead of letting + # `grep -q` close the pipe on its first match, which could otherwise + # SIGPIPE a still-writing awk and flip this check's exit status. + if awk '/format_request_changes_body\(\)/,/build_request_changes_review_payload\(\)/ { print }' "$workflow_file" | + grep -F '```diff' >/dev/null; then + record_failure "opencode review PR-level REQUEST_CHANGES body must not contain fenced suggested diffs" + fi +} + +assert_pr_review_merge_scheduler_uses_github_actions_bot_token() { + local workflow_file="$REPO_ROOT/.github/workflows/pr-review-merge-scheduler.yml" + local fix_workflow_file="$REPO_ROOT/.github/workflows/pr-review-fix-scheduler.yml" + local autofix_workflow_file="$REPO_ROOT/.github/workflows/pr-review-autofix.yml" + local scheduler_file="$REPO_ROOT/scripts/ci/pr_review_merge_scheduler.py" + local fix_scheduler_file="$REPO_ROOT/scripts/ci/pr_review_fix_scheduler.py" + local readme_file="$REPO_ROOT/README.md" + local procedure_file="$REPO_ROOT/docs/pr-review-and-merge-procedure.md" + + assert_file_contains "$autofix_workflow_file" "Autofix allowed paths, authoritative:" "autofix prompt includes allowed paths outside the truncated review context" + assert_file_contains "$autofix_workflow_file" "" "autofix prompt has a dedicated allowed-paths block" + assert_file_contains "$autofix_workflow_file" 'git ls-files --others --exclude-standard' "autofix validation rejects untracked files outside allowed paths" + assert_file_contains "$workflow_file" 'workflow_call:' "scheduler can run as the central reusable workflow contract" + assert_file_contains "$workflow_file" 'push:' "scheduler wakes when a protected base branch advances and PR branches may become stale" + assert_file_contains "$workflow_file" 'branches: [main, develop, master]' "scheduler scans GitHub Flow and Git Flow default branches after base pushes" + assert_file_contains "$workflow_file" 'pull_request_target:' "scheduler can run as an organization required workflow without repository-local copies" + assert_file_contains "$workflow_file" 'auto_merge_enabled' "scheduler rechecks already stale PRs as soon as native auto-merge is enabled" + assert_file_contains "$workflow_file" 'workflows: ["Required OpenCode Review", "Strix Security Scan"]' "scheduler reruns after review or security evidence completion so approvals can trigger merge/update actions" + assert_file_contains "$workflow_file" 'cron: "*/30 * * * *"' "scheduler wakes frequently enough to clear auto-merge PRs that become stale after their initial PR events" + assert_file_not_contains "$workflow_file" "github.event.pull_request.number == 240" "scheduler must not hard-code repository-specific PR bypasses" + assert_file_contains "$workflow_file" "github.event_name == 'pull_request_target' && format('pr-{0}', github.event.pull_request.number)" "scheduler scopes pull_request_target concurrency to the active PR" + assert_file_contains "$workflow_file" "github.event_name == 'workflow_run' && github.event.workflow_run.pull_requests[0].number && format('pr-{0}', github.event.workflow_run.pull_requests[0].number)" "scheduler scopes workflow_run concurrency to the completed review PR" + assert_file_contains "$workflow_file" "github.event_name == 'schedule' && format('schedule-{0}', github.event.schedule)" "scheduler isolates the 15-minute organization sweep from the separate 30-minute scheduled scan" + assert_file_contains "$workflow_file" "github.event_name == 'repository_dispatch' && github.event.client_payload.target_repository != '' && github.event.client_payload.pr_number != ''" "scheduler scopes targeted manual queue scans to the requested PR" + assert_file_contains "$workflow_file" "cancel-in-progress: \${{ github.event_name == 'pull_request_target' || github.event_name == 'pull_request_review' || github.event_name == 'repository_dispatch' || (github.event_name == 'workflow_run' && !github.event.workflow_run.pull_requests[0].number) }}" "scheduler cancels stale PR/review/manual queue scans instead of accumulating merge/update attempts" + assert_file_contains "$workflow_file" "timeout-minutes: 60" "organization sweep has enough headroom to finish the complete repository walk" + assert_file_contains "$workflow_file" "ORG_SWEEP_TRIGGER_REVIEWS: \${{ github.event_name == 'schedule' ||" "scheduled organization sweeps retry missing current-head OpenCode reviews" + assert_file_contains "$workflow_file" "ORG_SWEEP_ENABLE_AUTO_MERGE: \${{ github.event_name == 'schedule' ||" "scheduled organization sweeps merge approved current heads" + assert_file_contains "$workflow_file" "ORG_SWEEP_UPDATE_BRANCHES: \${{ github.event_name == 'schedule' ||" "scheduled organization sweeps refresh eligible stale branches" + assert_file_contains "$workflow_file" 'github.event.workflow_run.pull_requests[0].number' "scheduler scopes OpenCode workflow_run events to the completed review PR" + assert_file_contains "$workflow_file" "github.event.client_payload.trigger_reviews != false" "scheduler enables review dispatch by default for default-branch dispatch events" + assert_file_contains "$workflow_file" "github.event_name == 'workflow_run' || github.event_name == 'push'" "scheduler can dispatch a bounded follow-up OpenCode review after review workflow completion" + assert_file_contains "$workflow_file" "github.event_name == 'push' || github.event_name == 'pull_request_target'" "scheduler treats base-branch pushes as queue-maintenance events" + assert_file_contains "$workflow_file" "github.event.client_payload.enable_auto_merge != false" "scheduler enables auto-merge by default for default-branch dispatch events" + assert_file_contains "$workflow_file" "github.event_name == 'workflow_run' || (github.event_name == 'repository_dispatch' && github.event.client_payload.update_branches != false) || inputs.update_branches == true" "scheduler enables branch updates after review completion or an explicit default-branch dispatch" + assert_file_contains "$workflow_file" "review_dispatch_limit:" "scheduler exposes a bounded review dispatch budget" + assert_file_contains "$workflow_file" "REVIEW_DISPATCH_LIMIT_INPUT" "scheduler forwards the review dispatch budget to the canonical script" + assert_file_contains "$workflow_file" 'review_dispatch_limit="-1"' "scheduler dispatches every eligible same-head review or Strix evidence job immediately unless an explicit budget overrides it" + assert_file_not_contains "$workflow_file" 'review_dispatch_limit="0"' "scheduler must not silently suppress eligible review dispatches on base-branch push events" + assert_file_contains "$workflow_file" "--review-dispatch-limit" "scheduler passes the dispatch budget to the canonical script" + assert_file_contains "$workflow_file" "branch_update_limit:" "scheduler exposes a bounded branch-update budget" + assert_file_contains "$workflow_file" "BRANCH_UPDATE_LIMIT_INPUT" "scheduler forwards the branch-update budget to the canonical script" + assert_file_contains "$workflow_file" "ORG_SWEEP_BRANCH_UPDATE_LIMIT" "organization sweeps bound branch updates per repository" + assert_file_contains "$workflow_file" "--branch-update-limit" "scheduler passes the branch-update budget to the canonical script" + assert_file_contains "$workflow_file" 'GH_TOKEN: ${{ github.token }}' "scheduler uses the caller workflow token so mutations are attributed to GitHub Actions in the target repository" + assert_file_not_contains "$workflow_file" "INPUT_CANONICAL_REF" "scheduler trusted source checkout must not be controlled by workflow input" + assert_file_not_contains "$workflow_file" "inputs.canonical_ref" "scheduler no longer accepts checkout-ref override input" + assert_file_contains "$workflow_file" "Materialize trusted scheduler" "scheduler materializes the trusted central implementation without privileged checkout" + assert_file_contains "$workflow_file" 'repos/ContextualWisdomLab/.github/tarball/${TRUSTED_SOURCE_REF}' "scheduler downloads the central implementation archive by trusted source ref" + assert_file_contains "$workflow_file" "Trusted scheduler source ref must resolve to the immutable workflow commit SHA before archive materialization." "scheduler fails closed when the trusted source is not pinned to a workflow SHA" + assert_file_not_contains "$workflow_file" "uses: actions/checkout" "scheduler does not use checkout in privileged pull_request_target or workflow_run contexts" + assert_file_not_contains "$workflow_file" 'repository: ContextualWisdomLab/.github' "scheduler no longer uses checkout repository configuration in privileged contexts" + assert_file_not_contains "$workflow_file" 'repository: ${{ steps.trusted_source.outputs.repository }}' "scheduler does not pass a dynamic repository expression to privileged checkout" + assert_file_contains "$workflow_file" 'TRUSTED_SOURCE_REF: ${{ steps.trusted_source.outputs.ref }}' "scheduler materializes the resolved central ref" + assert_file_contains "$workflow_file" "contents: write" "scheduler has write permission for GitHub Actions bot branch updates" + assert_file_contains "$workflow_file" "pull-requests: write" "scheduler has pull-request write permission for update-branch and auto-merge" + assert_file_not_contains "$workflow_file" "format('pr-{0}-{1}', github.event.pull_request.number, github.event.pull_request.head.sha)" "scheduler does not keep stale head-specific concurrency groups" + assert_file_contains "$scheduler_file" "update-branch" "scheduler calls the GitHub update-branch API for outdated approved PRs" + assert_file_contains "$scheduler_file" "expected_head_sha={head}" "scheduler guards branch updates with the current PR head SHA" + assert_file_contains "$scheduler_file" "squash is disabled; retrying" "scheduler logs and retries with merge commit when repository settings reject squash" + assert_file_contains "$scheduler_file" 'merge_args.extend(["--merge", "--match-head-commit", head])' "scheduler preserves the exact-head guard when falling back from squash" + assert_file_contains "$scheduler_file" "shell=False" "scheduler subprocess wrapper forbids shell command execution" + assert_file_contains "$scheduler_file" "check=True" "scheduler subprocess wrapper raises on failed commands" + assert_file_contains "$REPO_ROOT/tests/test_pr_review_merge_scheduler.py" "test_run_passes_shell_metacharacters_as_plain_arguments" "scheduler tests prove branch-like shell metacharacters stay argv data" + assert_file_contains "$scheduler_file" "dispatch_strix_evidence" "scheduler dispatches same-head Strix evidence before OpenCode review" + assert_file_contains "$scheduler_file" '"--method"' "scheduler reads active workflow runs with GET query parameters" + assert_file_contains "$scheduler_file" "--security-workflow" "scheduler allows the canonical Strix workflow name to be configured" + assert_file_contains "$scheduler_file" "same-head OpenCode dispatched" "scheduler records review dispatch after completed security evidence" + assert_file_contains "$workflow_file" "--pr-number" "scheduler scopes required-workflow PR events to the current pull request" + assert_file_contains "$workflow_file" "--review-workflow \"Required OpenCode Review\"" "scheduler dispatches the canonical required OpenCode Review workflow" + assert_file_contains "$readme_file" "docs/pr-review-and-merge-procedure.md" "README points operators to the bot/agent review procedure instead of embedding it" + assert_file_contains "$procedure_file" "PR_REVIEW_MERGE_TOKEN" "review procedure documents that mechanical branch updates and merges use the central mutation credential" + assert_file_contains "$fix_workflow_file" 'workflow_call:' "fix scheduler can run as the central reusable autofix-dispatch workflow" + assert_file_contains "$fix_workflow_file" 'repository: ContextualWisdomLab/.github' "fix scheduler checks out the canonical implementation instead of relying on repo-local scheduler code" + assert_file_contains "$fix_workflow_file" 'AUTOFIX_REPOSITORY' "fix scheduler can dispatch the central autofix worker without per-repository workflow copies" + assert_file_contains "$fix_workflow_file" 'GH_TOKEN: ${{ secrets.PR_REVIEW_MERGE_TOKEN || secrets.OPENCODE_APPROVE_TOKEN || github.token }}' "fix scheduler uses central mutation credentials before falling back to the workflow token" + assert_file_contains "$fix_workflow_file" "python3 scripts/ci/pr_review_fix_scheduler.py --self-test" "fix scheduler self-tests the central dispatch contract before scanning" + assert_file_contains "$autofix_workflow_file" "github.event.client_payload.target_repository" "central autofix worker accepts the repository that owns the PR through default-branch repository dispatch" + assert_file_contains "$autofix_workflow_file" "types: [pr-review-autofix]" "central autofix worker exposes only the default-branch repository-dispatch entrypoint" + assert_file_not_contains "$autofix_workflow_file" "workflow_dispatch:" "central autofix worker cannot load privileged code from a caller-selected ref" + assert_file_contains "$autofix_workflow_file" "Autofix only supports same-repository PR heads." "central autofix worker refuses external heads before mutation" + assert_file_contains "$autofix_workflow_file" "reasoningEffort" "central autofix worker raises reasoning effort for models that support it" + assert_file_contains "$fix_scheduler_file" "current-head OpenCode requested changes" "fix scheduler dispatches only for current-head actionable review evidence" + assert_file_contains "$fix_scheduler_file" "DEFAULT_AUTOFIX_REPOSITORY" "fix scheduler defaults to the central autofix workflow repository" + assert_file_contains "$fix_scheduler_file" '"target_repository": repo' "fix scheduler passes the target repository in the central repository-dispatch JSON payload" + assert_file_contains "$fix_scheduler_file" "recent autofix marker exists for this head" "fix scheduler avoids repeated autofix loops for the same head" + assert_file_contains "$fix_scheduler_file" "external PR head is not writable" "fix scheduler refuses external heads for bot autofix" + assert_file_contains "$procedure_file" "PR Review Fix Scheduler" "review procedure documents the central autofix scheduler contract" + assert_file_contains "$procedure_file" "Scratch PoC files are not" "review procedure documents PoC proof artifacts are scratch evidence, not committed changes" + assert_file_contains "$procedure_file" "committed." "review procedure documents scratch PoC proof artifacts are not committed" + assert_file_contains "$procedure_file" "Failed GitHub Checks are not reviewed as URL lists." "review procedure documents failed-check reviews require explanations, not URL-only bullets" +} + +assert_opencode_review_normalizer_accepts_transcript_json() { + local tmp_dir + local output_file + local changed_files_file + local rc + local gate_result + tmp_dir="$(mktemp -d)" + output_file="$tmp_dir/opencode-output.md" + changed_files_file="$tmp_dir/opencode-changed-files.txt" + + cat >"$changed_files_file" <<'EOF' +.github/workflows/opencode-review.yml +scripts/ci/opencode_review_normalize_output.py +scripts/ci/test_strix_quick_gate.sh +EOF + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after structural exploration of .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/test_strix_quick_gate.sh self-test evidence passed. Coverage: Coverage execution evidence reported 100% test coverage. Docstring coverage: Coverage execution evidence reported 100% docstring coverage. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to GitHub Actions review job and verification path. PoC/execution: scratch PoC executed bash scripts/ci/test_strix_quick_gate.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and shell conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} +EOF + + set +e + RUNNER_TEMP="$tmp_dir" OPENCODE_CHANGED_FILES_FILE="$changed_files_file" \ + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" + rc=$? + set -e + + assert_equals "0" "$rc" "opencode review normalizer accepts transcript-embedded current-run JSON" + assert_file_contains "$output_file" "" "opencode review normalizer writes the gate sentinel" + assert_file_contains "$output_file" "" + + cat >"$changed_files_file" <<'EOF' +.github/workflows/opencode-review.yml +scripts/ci/opencode_review_normalize_output.py +scripts/ci/test_strix_quick_gate.sh +EOF + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" + + cat >"$output_file" <<'EOF' + + + + +But that is not meticulous. + +We should request changes. +EOF + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" + + set +e + gate_result="$( + RUNNER_TEMP="$tmp_dir" OPENCODE_CHANGED_FILES_FILE="$changed_files_file" \ + bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ + "abc123" "42" "1" "$output_file" "$normalized_json" + )" + rc=$? + set -e + + assert_equals "0" "$rc" "opencode publish sanitizer accepts the first valid control block" + assert_equals "APPROVE" "$gate_result" "opencode publish sanitizer preserves the valid gate result" + + { + printf '%s\n\n' "$sentinel" + printf '\n' + } >"$comment_body_file" + + assert_file_contains "$comment_body_file" '"result":"APPROVE"' "opencode publish sanitizer keeps normalized approval JSON" + assert_file_not_contains "$comment_body_file" "But that is not meticulous." "opencode publish sanitizer drops trailing model prose" + assert_file_not_contains "$comment_body_file" "We should request changes." "opencode publish sanitizer drops contradictory trailing model prose" + + rm -rf "$tmp_dir" +} + +assert_opencode_review_gate_rejects_missing_structural_exploration_approval() { + local tmp_dir + local output_file + local changed_files_file + local RUNNER_TEMP + local OPENCODE_CHANGED_FILES_FILE + local rc + local gate_result + tmp_dir="$(mktemp -d)" + output_file="$tmp_dir/opencode-output.md" + changed_files_file="$tmp_dir/opencode-changed-files.txt" + RUNNER_TEMP="$tmp_dir" + OPENCODE_CHANGED_FILES_FILE="$changed_files_file" + export RUNNER_TEMP OPENCODE_CHANGED_FILES_FILE + cat >"$changed_files_file" <<'EOF' +.github/workflows/opencode-review.yml +scripts/ci/opencode_review_normalize_output.py +scripts/ci/test_strix_quick_gate.sh +EOF + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found, but structural exploration was not possible.","summary":"This docs-only PR does not require structural review and the evidence was truncated.","findings":[]} +EOF + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects approvals that admit missing structural exploration" + assert_file_contains "$tmp_dir/normalize.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for missing structural exploration" + + cat >"$output_file" <<'EOF' + + + +EOF + + set +e + gate_result="$( + bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ + "abc123" "42" "1" "$output_file" + )" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode approval gate rejects approvals that admit missing structural exploration" + assert_equals "NO_CONCLUSION" "$gate_result" "missing structural exploration rejection gate result" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after structural exploration of changed files.","summary":"CodeGraph evidence was insufficient for one generated artifact, but local inspection covered the changed workflow, scripts, and tests.","findings":[]} +EOF + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize-valid.out" 2>"$tmp_dir/normalize-valid.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects approvals that omit concrete changed-file evidence" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after structural exploration of .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/test_strix_quick_gate.sh self-test evidence passed. Coverage: Coverage execution evidence reported 100% test coverage. Docstring coverage: Coverage execution evidence reported 100% docstring coverage. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to GitHub Actions review job and verification path. PoC/execution: scratch PoC executed bash scripts/ci/test_strix_quick_gate.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and shell conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} +EOF + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize-valid.out" 2>"$tmp_dir/normalize-valid.err" + rc=$? + set -e + + assert_equals "0" "$rc" "opencode normalizer accepts approvals that name concrete changed-file evidence after structural inspection" + + rm -rf "$tmp_dir" +} + +assert_opencode_review_gate_rejects_unmeasured_coverage_approval() { + local tmp_dir + local output_file + local changed_files_file + local RUNNER_TEMP + local OPENCODE_CHANGED_FILES_FILE + local rc + local gate_result + tmp_dir="$(mktemp -d)" + output_file="$tmp_dir/opencode-output.md" + changed_files_file="$tmp_dir/opencode-changed-files.txt" + RUNNER_TEMP="$tmp_dir" + OPENCODE_CHANGED_FILES_FILE="$changed_files_file" + export RUNNER_TEMP OPENCODE_CHANGED_FILES_FILE + printf '%s\n' '.github/workflows/opencode-review.yml' >"$changed_files_file" + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/test_strix_quick_gate.sh self-test evidence passed. Coverage: not measured. Docstring coverage: not measured. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to GitHub Actions review job and verification path. PoC/execution: scratch PoC executed bash scripts/ci/test_strix_quick_gate.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and shell conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} +EOF + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects approvals with unmeasured coverage" + assert_file_contains "$tmp_dir/normalize.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for unmeasured coverage approval" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/test_strix_quick_gate.sh self-test evidence passed. Coverage: Not applicable. Docstring coverage: Not applicable. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to GitHub Actions review job and verification path. PoC/execution: scratch PoC executed bash scripts/ci/test_strix_quick_gate.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and shell conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} +EOF + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize-na.out" 2>"$tmp_dir/normalize-na.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects approvals with not-applicable coverage" + assert_file_contains "$tmp_dir/normalize-na.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for not-applicable coverage approval" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/test_strix_quick_gate.sh self-test evidence passed. Coverage: Coverage execution evidence reports test coverage as not applicable because no supported changed source files or package manifests were found. Docstring coverage: Coverage execution evidence reports docstring coverage as not applicable because no supported changed source files or package manifests were found. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to GitHub Actions review job and verification path. PoC/execution: scratch PoC executed bash scripts/ci/test_strix_quick_gate.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and shell conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} +EOF + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize-no-source.out" 2>"$tmp_dir/normalize-no-source.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects no-source coverage claims for source-like changes" + assert_file_contains "$tmp_dir/normalize-no-source.err" "NO_CONCLUSION" "opencode normalizer exposes the contradictory no-source coverage rejection" + + cat >"$output_file" <<'EOF' + + + +EOF + + set +e + gate_result="$( + bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ + "abc123" "42" "1" "$output_file" + )" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode approval gate rejects approvals when coverage evidence did not run" + assert_equals "NO_CONCLUSION" "$gate_result" "unmeasured coverage approval rejection gate result" + + rm -rf "$tmp_dir" +} + +assert_opencode_review_gate_rejects_no_changes_approval() { + local tmp_dir + local output_file + local RUNNER_TEMP + local rc + local gate_result + tmp_dir="$(mktemp -d)" + output_file="$tmp_dir/opencode-output.md" + RUNNER_TEMP="$tmp_dir" + export RUNNER_TEMP + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No changes detected in the PR head source directory.","summary":"No files or changes were found in the PR head source directory, indicating no actionable changes to review.","findings":[]} +EOF + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects no-changes approvals" + assert_file_contains "$tmp_dir/normalize.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for no-changes approval" + + cat >"$output_file" <<'EOF' + + + +EOF + + set +e + gate_result="$( + bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ + "abc123" "42" "1" "$output_file" + )" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode approval gate rejects no-changes approvals" + assert_equals "NO_CONCLUSION" "$gate_result" "no-changes approval rejection gate result" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "Never approve with a reason or summary that says no changes" "opencode prompt rejects no-changes approvals when bounded evidence lists changed files" + + rm -rf "$tmp_dir" +} + +assert_opencode_review_gate_rejects_approve_without_changed_file_evidence() { + local tmp_dir + local output_file + local changed_files_file + local RUNNER_TEMP + local OPENCODE_CHANGED_FILES_FILE + local rc + local gate_result + tmp_dir="$(mktemp -d)" + output_file="$tmp_dir/opencode-output.md" + changed_files_file="$tmp_dir/opencode-changed-files.txt" + RUNNER_TEMP="$tmp_dir" + OPENCODE_CHANGED_FILES_FILE="$changed_files_file" + export RUNNER_TEMP OPENCODE_CHANGED_FILES_FILE + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blocking issues found; changes improve CI configuration and documentation.","summary":"PR enhances OpenCode review workflow with clearer guidance and validation. Changes are well-contained with no security or functional regressions detected.","findings":[]} +EOF + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects approvals without changed-file evidence" + assert_file_contains "$tmp_dir/normalize.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for approvals without changed-file evidence" + + cat >"$output_file" <<'EOF' + + + +EOF + + set +e + gate_result="$( + bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ + "abc123" "42" "1" "$output_file" + )" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode approval gate rejects approvals without changed-file evidence" + assert_equals "NO_CONCLUSION" "$gate_result" "missing changed-file evidence rejection gate result" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "Before APPROVE, the summary must include at least one exact changed file path inspected as changed-file evidence" "opencode prompt requires changed-file evidence before approval" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "when result is APPROVE the JSON findings value must be exactly []" "opencode prompt keeps approval findings empty" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "Put all required Verification posture labels inside the JSON summary string itself" "opencode prompt keeps approval evidence inside the control JSON" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "never say no source files changed, no test files changed, or no executable changes when exact changed-file evidence lists workflow, script, source, or test files" "opencode prompt rejects contradictory changed-file kind claims" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "Never approve material workflow, script, source, config, package, or test changes with a reason or summary that says simple typo fix" "opencode prompt rejects trivial approval claims for material changes" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "OPENCODE_CHANGED_FILES_FILE" "opencode workflow exports exact current-head changed files" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'git -C "$OPENCODE_SOURCE_WORKDIR" diff --name-only --find-renames "$PR_MERGE_BASE" "$PR_HEAD_SHA" |' "opencode workflow derives exact changed files from the PR-head worktree" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'awk '\''NF > 0 && $0 !~ /^\// && $0 !~ /(^|\/)\.\.($|\/)/ { print }'\'' >"$OPENCODE_CHANGED_FILES_FILE"' "opencode workflow writes path-safe exact changed files for the normalizer" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" "changed-files.txt" "opencode workflow copies exact changed-file evidence into the isolated review workspace" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'A["text"]' "opencode prompt requires quoted Mermaid labels" + assert_file_contains "$REPO_ROOT/scripts/ci/opencode_review_comment_helpers.sh" 'S%s["%s"]' "opencode generated Mermaid surface labels are quoted" + assert_file_contains "$REPO_ROOT/scripts/ci/opencode_review_comment_helpers.sh" 'R%s["Review risk: %s"]' "opencode generated Mermaid risk labels are quoted" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'emit_review_body_to_action_log "$event" "$body"' "opencode PR-level review bodies are mirrored to the Actions log" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'emit_review_body_to_action_log "$event" "$body" "$review_payload_file"' "opencode inline review bodies are mirrored to the Actions log" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" 'OpenCode is publishing this review content to PR #%s.' "opencode Actions log includes the review body that is being posted" + assert_file_contains "$REPO_ROOT/.github/workflows/opencode-review-dispatch.yml" '## OpenCode %s review body' "opencode Step Summary includes the review body that is being posted" + + cat >"$changed_files_file" <<'EOF' +.github/workflows/opencode-review.yml +scripts/ci/opencode_review_normalize_output.py +scripts/ci/test_strix_quick_gate.sh +EOF + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting README.md.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed README.md. Verification posture: Linter/static: actionlint and bash syntax evidence passed. TDD/regression: scripts/ci/other_gate_test.sh self-test evidence passed. Coverage: Coverage execution evidence reported 100% test coverage. Docstring coverage: Coverage execution evidence reported 100% docstring coverage. DAG: CodeGraph behavior DAG rendered README.md to docs review path. PoC/execution: scratch PoC executed bash scripts/ci/other_gate_test.sh and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions docs. Compatibility/convention: conventions match existing code. Breaking-change/backcompat: no public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web docs and review-comment output was checked. Accessibility/i18n: human-readable docs and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token boundaries preserved.","findings":[]} +EOF + + set +e + OPENCODE_CHANGED_FILES_FILE="$changed_files_file" \ + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/nonchanged-normalize.out" 2>"$tmp_dir/nonchanged-normalize.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects approvals that cite non-changed files when exact changed-file evidence is available" + assert_file_contains "$tmp_dir/nonchanged-normalize.err" "NO_CONCLUSION" "opencode normalizer reports no conclusion for non-changed-file approval evidence" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: Not applicable (no source files changed). TDD/regression: Not applicable (no test files changed). Coverage: Coverage execution evidence reported 100% test coverage. Docstring coverage: Coverage execution evidence reported 100% docstring coverage. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to review decision path. PoC/execution: Not applicable (no executable changes). DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and Python conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} +EOF + + set +e + OPENCODE_CHANGED_FILES_FILE="$changed_files_file" \ + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/contradictory-normalize.out" 2>"$tmp_dir/contradictory-normalize.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects approvals that deny changed source/test/executable surfaces" + assert_file_contains "$tmp_dir/contradictory-normalize.err" "NO_CONCLUSION" "opencode normalizer reports no conclusion for contradictory changed-file kind claims" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"APPROVE","reason":"No blockers found after inspecting .github/workflows/opencode-review.yml.","summary":"Approval sufficiency: affirmative evidence supported approval beyond absence of blockers. Reviewed .github/workflows/opencode-review.yml, scripts/ci/opencode_review_normalize_output.py, and scripts/ci/test_strix_quick_gate.sh. Verification posture: Linter/static: actionlint and Python syntax evidence passed. TDD/regression: normalizer self-test evidence passed. Coverage: Coverage execution evidence reported 100% test coverage. Docstring coverage: Coverage execution evidence reported 100% docstring coverage. DAG: CodeGraph behavior DAG rendered .github/workflows/opencode-review.yml to scripts/ci/opencode_review_normalize_output.py to review decision path. PoC/execution: scratch PoC executed the normalizer with exact changed-file evidence and passed. DDD/domain: no product domain boundary changed. CDD/context: CodeGraph structural MCP evidence covered the workflow and script blast radius. Similar issues: checked related OpenCode gate cases. Claim/concept check: no unverified user concept accepted. Standards search: checked current GitHub Actions/OpenCode docs where applicable. Compatibility/convention: workflow naming and Python conventions match existing code. Breaking-change/backcompat: no deployed public contract changed. Performance: no runtime path affected. Developer experience: review automation remains clear to maintainers and contributors. User experience: no user-facing UI affected. Visual/DOM: non-web workflow and review-comment output was checked. Accessibility/i18n: human-readable workflow and review text was checked. Supply-chain/license: dependency and external-tool risk was checked. Packaging: package and workflow contracts were checked. Security/privacy: token and pull_request_target boundaries preserved.","findings":[]} +EOF + + set +e + OPENCODE_CHANGED_FILES_FILE="$changed_files_file" \ + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/changed-normalize.out" 2>"$tmp_dir/changed-normalize.err" + rc=$? + set -e + + assert_equals "0" "$rc" "opencode normalizer accepts approvals that cite exact current changed files" + + rm -rf "$tmp_dir" +} + +assert_opencode_review_gate_rejects_line_zero_findings() { + local tmp_dir + local output_file + local RUNNER_TEMP + local rc + local gate_result + tmp_dir="$(mktemp -d)" + output_file="$tmp_dir/opencode-output.md" + RUNNER_TEMP="$tmp_dir" + export RUNNER_TEMP + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" + + cat >"$output_file" <<'EOF' + + + +EOF + + set +e + gate_result="$( + bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ + "abc123" "42" "1" "$output_file" + )" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode approval gate rejects line zero findings" + assert_equals "NO_CONCLUSION" "$gate_result" "line zero rejection gate result" + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/normalize.out" 2>"$tmp_dir/normalize.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects line zero findings" + assert_file_contains "$tmp_dir/normalize.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for line zero findings" + + cat >"$output_file" <<'EOF' +OpenCode transcript text before the review control block. + +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Boolean line blocker","summary":"Boolean line values are not concrete source locations.","findings":[{"path":"scripts/ci/example.sh","line":true,"severity":"HIGH","title":"Boolean line","problem":"Boolean line values are not actionable.","root_cause":"The review did not inspect a concrete line.","fix_direction":"Inspect the actual file and cite a positive integer line number.","regression_test_direction":"Add a gate test for boolean line rejection.","suggested_diff":"diff --git a/scripts/ci/example.sh b/scripts/ci/example.sh\n--- a/scripts/ci/example.sh\n+++ b/scripts/ci/example.sh\n@@ -1 +1 @@\n-old\n+new"}]} +EOF + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/bool-line.out" 2>"$tmp_dir/bool-line.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects boolean line findings" + assert_file_contains "$tmp_dir/bool-line.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for boolean line findings" + + rm -rf "$tmp_dir" +} + +assert_opencode_review_gate_rejects_placeholder_findings() { + local tmp_dir + local output_file + local RUNNER_TEMP + local rc + local gate_result + tmp_dir="$(mktemp -d)" + output_file="$tmp_dir/opencode-output.md" + RUNNER_TEMP="$tmp_dir" + export RUNNER_TEMP + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" + + cat >"$output_file" <<'EOF' + + + +EOF + + set +e + gate_result="$( + bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ + "abc123" "42" "1" "$output_file" + )" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode approval gate rejects placeholder findings" + assert_equals "NO_CONCLUSION" "$gate_result" "placeholder finding rejection gate result" + + rm -rf "$tmp_dir" +} + +assert_opencode_review_gate_rejects_non_source_backed_findings() { + local tmp_dir + local output_file + local stderr_file + local changed_files_file + local RUNNER_TEMP + local OPENCODE_CHANGED_FILES_FILE + local rc + local gate_result + tmp_dir="$(mktemp -d)" + output_file="$tmp_dir/opencode-output.md" + stderr_file="$tmp_dir/gate.err" + changed_files_file="$tmp_dir/opencode-changed-files.txt" + RUNNER_TEMP="$tmp_dir" + OPENCODE_CHANGED_FILES_FILE="$changed_files_file" + export RUNNER_TEMP OPENCODE_CHANGED_FILES_FILE + printf '%s\n' 'scripts/ci/opencode_review_approve_gate.sh' >"$changed_files_file" + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" "$changed_files_file" + + cat >"$output_file" <<'EOF' + + + +EOF + + set +e + gate_result="$( + bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ + "abc123" "42" "1" "$output_file" 2>"$stderr_file" + )" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode approval gate rejects non-source-backed findings" + assert_equals "NO_CONCLUSION" "$gate_result" "non-source-backed finding rejection gate result" + assert_file_contains "$stderr_file" "REQUEST_CHANGES finding is not source-backed by the current-head diff" "non-source-backed finding rejection explains the invalid model result" + + rm -rf "$tmp_dir" +} + +assert_opencode_review_gate_rejects_generic_failed_check_deflection() { + local tmp_dir + local output_file + local RUNNER_TEMP + local rc + local gate_result + tmp_dir="$(mktemp -d)" + output_file="$tmp_dir/opencode-output.md" + RUNNER_TEMP="$tmp_dir" + export RUNNER_TEMP + seal_opencode_test_artifacts "$tmp_dir" "abc123" "42" "1" + + cat >"$output_file" <<'EOF' + + + +EOF + + set +e + gate_result="$( + bash "$REPO_ROOT/scripts/ci/opencode_review_approve_gate.sh" \ + "abc123" "42" "1" "$output_file" + )" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode approval gate rejects generic failed-check deflections" + assert_equals "NO_CONCLUSION" "$gate_result" "generic failed-check deflection rejection gate result" + + set +e + python3 "$REPO_ROOT/scripts/ci/opencode_review_normalize_output.py" \ + "abc123" "42" "1" "$output_file" >"$tmp_dir/generic-deflection.out" 2>"$tmp_dir/generic-deflection.err" + rc=$? + set -e + + assert_equals "4" "$rc" "opencode normalizer rejects generic failed-check deflections" + assert_file_contains "$tmp_dir/generic-deflection.err" "NO_CONCLUSION" "opencode normalizer reports no valid conclusion for generic failed-check deflections" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_review_validator_rejects_unrelated_findings() { + local tmp_dir + local control_json + local failed_checks_file + local evidence_file + local rc + tmp_dir="$(mktemp -d)" + control_json="$tmp_dir/control.json" + failed_checks_file="$tmp_dir/failed-checks.txt" + evidence_file="$tmp_dir/failed-check-evidence.md" + + cat >"$failed_checks_file" <<'EOF' +- Strix Security Scan/strix: FAILURE (https://github.com/example/repo/actions/runs/1/job/2) +EOF + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +### Failed job steps + +- step 6: Self-test Strix gate script (failure) + +### Strix vulnerability report window 1 + +Model github-models/openai/gpt-5 Vulnerabilities 1 +│ Vulnerability Report │ +│ Title: Authentication Bypass via X-Dev-User Header │ +│ Severity: CRITICAL │ +│ Endpoint: /api/me │ +│ Method: GET │ +│ Location 1: backend/app/auth.py:132-135 │ + +### Strix vulnerability report window 2 + +Model deepseek/deepseek-v3-0324 Vulnerabilities 1 +│ Vulnerability Report │ +│ Title: Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure │ +│ Severity: HIGH │ + +### Failed log excerpt + +FAIL: strix workflow defaults PR Strix scans to GitHub Models GPT-5 (missing 'github.event.client_payload.strix_llm || 'openai/gpt-5'') +FAIL: strix workflow rejects unsupported model inputs (missing 'STRIX_LLM must select GitHub Models openai/gpt-5 or newer, direct OpenAI GPT-5.4 or newer, OpenRouter openrouter/free, or an approved organization Vertex AI model') +FAIL: opencode failed-check diagnosis prefers DeepSeek V3 (missing 'MODEL: github-models/deepseek/deepseek-v3-0324') +EOF + cat >"$control_json" <<'EOF' +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Generic security concern","summary":"Generic speculative CI issues.","findings":[{"path":"scripts/ci/collect_failed_check_evidence.sh","line":15,"severity":"HIGH","title":"Generic finding","problem":"Speculative input validation issue unrelated to failed checks.","root_cause":"The review did not use the failed Strix evidence.","fix_direction":"Add generic validation.","regression_test_direction":"Add a generic test.","suggested_diff":"diff --git a/scripts/ci/collect_failed_check_evidence.sh b/scripts/ci/collect_failed_check_evidence.sh\n--- a/scripts/ci/collect_failed_check_evidence.sh\n+++ b/scripts/ci/collect_failed_check_evidence.sh\n@@ -1 +1 @@\n-old\n+new"}]} +EOF + + set +e + bash "$REPO_ROOT/scripts/ci/validate_opencode_failed_check_review.sh" \ + "$control_json" "$failed_checks_file" "$evidence_file" >"$tmp_dir/bad.out" 2>"$tmp_dir/bad.err" + rc=$? + set -e + assert_equals "4" "$rc" "failed-check review validator rejects unrelated findings" + assert_file_contains "$tmp_dir/bad.out" "FAILED_CHECK_EVIDENCE_NOT_REFERENCED" "failed-check validator explains unrelated finding rejection" + assert_file_contains "$tmp_dir/bad.out" "review does not" "failed-check validator logs the missing evidence linkage" + + cat >"$control_json" <<'EOF' +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Strix Security Scan/strix failed","summary":"No deterministic missing-string markers or Strix report locations were recognized. Use the failed-check evidence below to map each failed check to exact local source lines before approving.","findings":[{"path":"scripts/ci/collect_failed_check_evidence.sh","line":15,"severity":"HIGH","title":"Generic failed-check deflection","problem":"No deterministic missing-string markers or Strix report locations were recognized.","root_cause":"The review did not map Strix Security Scan/strix to failed log evidence and concrete local source lines.","fix_direction":"Inspect the failed-check evidence and produce source-backed findings instead of handing the mapping back to the reader.","regression_test_direction":"Reject generic failed-check deflections before publishing reviews.","suggested_diff":"diff --git a/scripts/ci/collect_failed_check_evidence.sh b/scripts/ci/collect_failed_check_evidence.sh\n--- a/scripts/ci/collect_failed_check_evidence.sh\n+++ b/scripts/ci/collect_failed_check_evidence.sh\n@@ -1 +1 @@\n-old\n+new"}]} +EOF + set +e + bash "$REPO_ROOT/scripts/ci/validate_opencode_failed_check_review.sh" \ + "$control_json" "$failed_checks_file" "$evidence_file" >"$tmp_dir/generic.out" 2>"$tmp_dir/generic.err" + rc=$? + set -e + assert_equals "4" "$rc" "failed-check review validator rejects generic failed-check deflections" + assert_file_contains "$tmp_dir/generic.out" "FAILED_CHECK_EVIDENCE_NOT_REFERENCED" "failed-check validator blocks generic deflection review text" + assert_file_contains "$tmp_dir/generic.out" "punts failed-check diagnosis back to the reader" "failed-check validator logs generic deflection reason" + + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +### Strix vulnerability report window 1 + +Model github-models/openai/gpt-5 Vulnerabilities 1 +│ Vulnerability Report │ +│ Title: Authentication Bypass via X-Dev-User Header │ +│ Severity: CRITICAL │ +│ Endpoint: /api/me │ +│ Method: GET │ +│ Location 1: backend/app/auth.py:132-135 │ + +### Strix vulnerability report window 2 + +Model deepseek/deepseek-v3-0324 Vulnerabilities 1 +│ Vulnerability Report │ +│ Title: Authentication Bypass via X-Dev-User Header │ +│ Severity: CRITICAL │ +│ Endpoint: /api/me │ +│ Method: GET │ +│ Location 1: backend/app/auth.py:132-135 │ +EOF + cat >"$control_json" <<'EOF' +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Strix Security Scan/strix failed","summary":"Strix Security Scan/strix failed and reported github-models/openai/gpt-5 plus deepseek/deepseek-v3-0324 Authentication Bypass via X-Dev-User Header with Severity: CRITICAL, /api/me, Method: GET, backend/app/auth.py:132-135.","findings":[{"path":"backend/app/auth.py","line":132,"severity":"CRITICAL","title":"Authentication Bypass via X-Dev-User Header","problem":"Strix Security Scan/strix failed with github-models/openai/gpt-5 and deepseek/deepseek-v3-0324 reports for Authentication Bypass via X-Dev-User Header, Severity: CRITICAL, /api/me, Method: GET, backend/app/auth.py:132-135.","root_cause":"The review collapsed two Strix model reports into one finding.","fix_direction":"Remove the unauthenticated fallback at backend/app/auth.py:132-135.","regression_test_direction":"Add auth tests for both request paths.","suggested_diff":"diff --git a/backend/app/auth.py b/backend/app/auth.py\n--- a/backend/app/auth.py\n+++ b/backend/app/auth.py\n@@ -132 +132 @@\n-old\n+new"}]} +EOF + set +e + bash "$REPO_ROOT/scripts/ci/validate_opencode_failed_check_review.sh" \ + "$control_json" "$failed_checks_file" "$evidence_file" >"$tmp_dir/collapsed.out" 2>"$tmp_dir/collapsed.err" + rc=$? + set -e + assert_equals "4" "$rc" "failed-check review validator rejects collapsed duplicate Strix model reports" + assert_file_contains "$tmp_dir/collapsed.out" "FAILED_CHECK_EVIDENCE_NOT_REFERENCED" "failed-check validator requires one Strix-specific finding per model report" + assert_file_contains "$tmp_dir/collapsed.out" "distinct source-backed findings" "failed-check validator logs collapsed Strix report reason" + + cat >"$control_json" <<'EOF' +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Strix Security Scan/strix failed","summary":"Strix Security Scan/strix failed and mentioned github-models/openai/gpt-5 plus deepseek/deepseek-v3-0324, but the model reports were still collapsed.","findings":[{"path":".github/workflows/strix.yml","line":120,"severity":"HIGH","title":"Strix self-test failed","problem":"Strix Security Scan/strix failed in Self-test Strix gate script while github-models/openai/gpt-5 and deepseek/deepseek-v3-0324 model reports were present elsewhere in the evidence.","root_cause":"The workflow finding is about CI self-test evidence, not a distinct model vulnerability report.","fix_direction":"Fix the workflow default.","regression_test_direction":"Keep the self-test assertion.","suggested_diff":"diff --git a/.github/workflows/strix.yml b/.github/workflows/strix.yml\n--- a/.github/workflows/strix.yml\n+++ b/.github/workflows/strix.yml\n@@ -120 +120 @@\n-old\n+new"},{"path":"backend/app/auth.py","line":132,"severity":"CRITICAL","title":"Authentication Bypass via X-Dev-User Header","problem":"Strix Security Scan/strix failed with github-models/openai/gpt-5 and deepseek/deepseek-v3-0324 reports for Authentication Bypass via X-Dev-User Header, Severity: CRITICAL, /api/me, Method: GET, backend/app/auth.py:132-135.","root_cause":"This finding still collapses two Strix model reports into one item even though the titles and locations match.","fix_direction":"Remove the unauthenticated fallback at backend/app/auth.py:132-135.","regression_test_direction":"Add auth tests for both request paths.","suggested_diff":"diff --git a/backend/app/auth.py b/backend/app/auth.py\n--- a/backend/app/auth.py\n+++ b/backend/app/auth.py\n@@ -132 +132 @@\n-old\n+new"}]} +EOF + set +e + bash "$REPO_ROOT/scripts/ci/validate_opencode_failed_check_review.sh" \ + "$control_json" "$failed_checks_file" "$evidence_file" >"$tmp_dir/collapsed-with-count.out" 2>"$tmp_dir/collapsed-with-count.err" + rc=$? + set -e + assert_equals "4" "$rc" "failed-check review validator rejects collapsed Strix reports even when finding count matches" + assert_file_contains "$tmp_dir/collapsed-with-count.out" "FAILED_CHECK_EVIDENCE_NOT_REFERENCED" "failed-check validator requires distinct matching findings, not only matching counts" + + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +### Failed job steps + +- step 6: Self-test Strix gate script (failure) + +### Strix vulnerability report window 1 + +Model github-models/openai/gpt-5 Vulnerabilities 1 +│ Vulnerability Report │ +│ Title: Authentication Bypass via X-Dev-User Header │ +│ Severity: CRITICAL │ +│ Endpoint: /api/me │ +│ Method: GET │ +│ Location 1: backend/app/auth.py:132-135 │ + +### Strix vulnerability report window 2 + +Model deepseek/deepseek-v3-0324 Vulnerabilities 1 +│ Vulnerability Report │ +│ Title: Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure │ +│ Severity: HIGH │ + +### Failed log excerpt + +FAIL: strix workflow defaults PR Strix scans to GitHub Models GPT-5 (missing 'github.event.client_payload.strix_llm || 'openai/gpt-5'') +FAIL: strix workflow rejects unsupported model inputs (missing 'STRIX_LLM must select GitHub Models openai/gpt-5 or newer, direct OpenAI GPT-5.4 or newer, OpenRouter openrouter/free, or an approved organization Vertex AI model') +FAIL: opencode failed-check diagnosis prefers DeepSeek V3 (missing 'MODEL: github-models/deepseek/deepseek-v3-0324') +EOF + + cat >"$control_json" <<'EOF' +{"head_sha":"abc123","run_id":"42","run_attempt":"1","result":"REQUEST_CHANGES","reason":"Strix Security Scan/strix failed","summary":"Strix Security Scan/strix failed in Self-test Strix gate script and reported github-models/openai/gpt-5 Authentication Bypass via X-Dev-User Header with Severity: CRITICAL at backend/app/auth.py:132-135 plus deepseek/deepseek-v3-0324 Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure with Severity: HIGH.","findings":[{"path":".github/workflows/strix.yml","line":120,"severity":"HIGH","title":"Strix workflow default is not visible to trusted self-test","problem":"Strix Security Scan/strix failed in Self-test Strix gate script: strix workflow defaults PR Strix scans to GitHub Models GPT-5 (missing 'github.event.client_payload.strix_llm || 'openai/gpt-5''); strix workflow rejects unsupported model inputs (missing 'STRIX_LLM must select GitHub Models openai/gpt-5 or newer, direct OpenAI GPT-5.4 or newer, OpenRouter openrouter/free, or an approved organization Vertex AI model'); opencode failed-check diagnosis prefers DeepSeek V3 (missing 'MODEL: github-models/deepseek/deepseek-v3-0324'). The same failed Strix evidence includes github-models/openai/gpt-5 report Authentication Bypass via X-Dev-User Header, Severity: CRITICAL, /api/me, Method: GET, backend/app/auth.py:132-135.","root_cause":"The failed check evidence shows Self-test Strix gate script could not find github.event.client_payload.strix_llm, STRIX_LLM must select, and MODEL: github-models/deepseek/deepseek-v3-0324 in trusted-base files, and the model report identifies the backend auth fallback line.","fix_direction":"Update the workflow lines that provide the Strix model default and OpenCode model env so the trusted self-test can find those exact strings, then remove the unauthenticated X-Dev-User fallback at backend/app/auth.py:132-135.","regression_test_direction":"Keep the static self-test assertions for all three missing strings and add auth tests proving /api/me rejects forged X-Dev-User requests without signed auth.","suggested_diff":"diff --git a/.github/workflows/strix.yml b/.github/workflows/strix.yml\n--- a/.github/workflows/strix.yml\n+++ b/.github/workflows/strix.yml\n@@ -120 +120 @@\n- STRIX_MODEL: old\n+ STRIX_MODEL: ${{ github.event.client_payload.strix_llm || 'openai/gpt-5' }}"},{"path":"frontend/src/app/page.tsx","line":1,"severity":"HIGH","title":"Strix frontend model report must be reviewed separately","problem":"Strix Security Scan/strix failed with a separate deepseek/deepseek-v3-0324 report: Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure, Severity: HIGH.","root_cause":"The failed Strix evidence contains a second model vulnerability report, so OpenCode must not collapse it into the first backend finding.","fix_direction":"Inspect the frontend source lines responsible for token storage, hardcoded credentials, dynamic error rendering, and missing CSP, then remove or harden each concrete line before approval.","regression_test_direction":"Add frontend tests covering safe token/session handling, output encoding, and security headers for the affected route.","suggested_diff":"diff --git a/frontend/src/app/page.tsx b/frontend/src/app/page.tsx\n--- a/frontend/src/app/page.tsx\n+++ b/frontend/src/app/page.tsx\n@@ -1 +1 @@\n-export default function Page() { return null }\n+export default function Page() { return null }"}]} +EOF + set +e + bash "$REPO_ROOT/scripts/ci/validate_opencode_failed_check_review.sh" \ + "$control_json" "$failed_checks_file" "$evidence_file" >"$tmp_dir/good.out" 2>"$tmp_dir/good.err" + rc=$? + set -e + assert_equals "0" "$rc" "failed-check review validator accepts Strix log-backed findings" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_emits_each_strix_report() { + local tmp_dir + local fixture_repo + local evidence_file + local output_file + local stderr_file + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + stderr_file="$tmp_dir/fallback.err" + mkdir -p "$fixture_repo/backend/services" "$fixture_repo/frontend/src/app/prompt-studio" "$fixture_repo/frontend" + + { + for _ in $(seq 1 59); do + printf '# filler\n' + done + printf 'filename = part.get_filename()\n' + } >"$fixture_repo/backend/services/email_parser.py" + { + for _ in $(seq 1 28); do + printf '// filler\n' + done + printf 'setTestResult(await apiClient.post("/prompt-studio", payload));\n' + } >"$fixture_repo/frontend/src/app/prompt-studio/page.tsx" + { + for _ in $(seq 1 34); do + printf '// filler\n' + done + printf 'const nextConfig = {};\n' + } >"$fixture_repo/frontend/next.config.ts" + + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +### Failed log signal summary + +```text +strix Run Strix (quick) LLM CONNECTION FAILED +strix Run Strix (quick) Strix fallback model 'deepseek/deepseek-r1-0528' emitted provider infrastructure or failure-signal output; trying next configured fallback if available. +``` + +### Strix vulnerability report window 1 + +Model deepseek/deepseek-r1-0528 Vulnerabilities 2 +│ Vulnerability Report │ +│ Title: Path Traversal in Email Attachment Handling │ +│ Severity: CRITICAL │ +│ Endpoint: /services/email_parser.py │ +│ Location 1: backend/services/email_parser.py:60-72 │ +│ Vulnerability Report │ +│ Title: Prompt Injection and XSS in AI Prompt Studio │ +│ Severity: HIGH │ +│ Endpoint: /prompt-studio │ +│ Location 1: frontend/src/app/prompt-studio/page.tsx:29-32 │ + +### Strix vulnerability report window 2 + +Model deepseek/deepseek-v3-0324 Vulnerabilities 1 +│ Vulnerability Report │ +│ Title: Missing Content Security Policy in Next.js Frontend │ +│ Severity: HIGH │ +│ Endpoint: all frontend pages │ +EOF + + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" + + assert_file_contains "$output_file" "Strix report from deepseek/deepseek-r1-0528: Path Traversal in Email Attachment Handling" "fallback includes first model report" + assert_file_contains "$output_file" "backend/services/email_parser.py:60" "fallback maps first report to exact source line" + assert_file_contains "$output_file" "Strix report from deepseek/deepseek-r1-0528: Prompt Injection and XSS in AI Prompt Studio" "fallback includes second report from same model" + assert_file_contains "$output_file" "frontend/src/app/prompt-studio/page.tsx:29" "fallback maps second report to exact source line" + assert_file_contains "$output_file" "Strix report from deepseek/deepseek-v3-0324: Missing Content Security Policy in Next.js Frontend" "fallback includes report from second model" + assert_file_contains "$output_file" "frontend/next.config.ts:35" "fallback derives a concrete CSP hardening line" + assert_file_contains "$output_file" "Suggested edit: change \`frontend/next.config.ts:35\`" "fallback provides a concrete suggested edit for model reports" + assert_file_contains "$output_file" "Strix provider signal left current-head security evidence incomplete" "fallback still reports provider failure after vulnerability reports" + assert_file_not_contains "$output_file" "failed before producing vulnerability reports" "fallback does not contradict preserved Strix report windows" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_explains_pytest_and_cancelled_checks() { + local tmp_dir + local fixture_repo + local evidence_file + local output_file + local stderr_file + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + stderr_file="$tmp_dir/fallback.err" + mkdir -p "$fixture_repo/tests/live" + + cat >"$fixture_repo/tests/live/test_live_api_sequence.py" <<'EOF' +"""Live HTTP integration harness tests.""" + +from pathlib import Path + + +def test_live_harness_avoids_broad_url_opener_pattern() -> None: + source = Path(__file__).read_text(encoding="utf-8") + unsafe_terms = ("urllib.request", "urlopen") + + for unsafe_term in unsafe_terms: + assert unsafe_term not in source +EOF + + cat >"$evidence_file" <<'EOF' +# Failed GitHub Check Evidence + +- PR: #744 +- Head SHA: `fc6d263e9fcfdcf4d710427618ee511b64331dd0` +- Repository: `ContextualWisdomLab/naruon` + +## Failed check: Application CI/backend (Python 3.14) + +- Type: `check_run` +- Conclusion: `FAILURE` +- Details URL: https://github.com/ContextualWisdomLab/naruon/actions/runs/27946373277/job/82692061303 + +### Failed job steps + +- step 6: Run backend tests (failure) + +### Failed log excerpt + +```text +backend (Python 3.14) Run backend tests pytest -q +backend (Python 3.14) Run backend tests =================================== FAILURES =================================== +backend (Python 3.14) Run backend tests ______________ test_live_harness_avoids_broad_url_opener_pattern _______________ +backend (Python 3.14) Run backend tests def test_live_harness_avoids_broad_url_opener_pattern() -> None: +backend (Python 3.14) Run backend tests unsafe_terms = ("urllib.request", "urlopen") +backend (Python 3.14) Run backend tests > assert unsafe_term not in source +backend (Python 3.14) Run backend tests E assert 'urllib.request' not in '"""Live HTT... in source\n' +backend (Python 3.14) Run backend tests E 'urllib.request' is contained here: +backend (Python 3.14) Run backend tests E terms = ("urllib.request", "urlopen") +backend (Python 3.14) Run backend tests tests/live/test_live_api_sequence.py:10: AssertionError +backend (Python 3.14) Run backend tests FAILED tests/live/test_live_api_sequence.py::test_live_harness_avoids_broad_url_opener_pattern - assert 'urllib.request' not in '"""Live HTT... in source\n' +backend (Python 3.14) Run backend tests 1 failed, 965 passed, 15 skipped in 7.28s +``` + +## Failed check: PR Governance/metadata-only gate evaluation + +- Type: `check_run` +- Conclusion: `CANCELLED` +- Details URL: https://github.com/ContextualWisdomLab/naruon/actions/runs/27946373334/job/82692061348 + +### Check annotations + +- .github:1-1 [failure] Canceling since a higher priority waiting request for PR Governance-744 exists +EOF + + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" + + assert_file_contains "$output_file" "Failed GitHub Check needs a source-backed pytest fix for test_live_harness_avoids_broad_url_opener_pattern" "fallback explains pytest failure with the test name" + assert_file_contains "$output_file" "tests/live/test_live_api_sequence.py:" "fallback maps pytest failure to a source file and line" + assert_file_contains "$output_file" "urllib.request" "fallback preserves the assertion term that caused the pytest failure" + assert_file_contains "$output_file" "cd backend && python -m pytest tests/live/test_live_api_sequence.py::test_live_harness_avoids_broad_url_opener_pattern -q" "fallback gives a focused pytest rerun command" + assert_file_not_contains "$output_file" "GitHub Checks queue - PR Governance/metadata-only gate evaluation was cancelled by a newer queued request" "fallback does not publish cancelled queue states as source-backed findings" + assert_file_contains "$stderr_file" "Non-source-backed cancelled check queue state" "fallback explains cancelled governance checks outside source-backed findings" + assert_file_contains "$stderr_file" "no repository source edit is justified by this cancelled check alone" "fallback does not invent source fixes for cancelled queue state" + assert_file_not_contains "$output_file" "No deterministic missing-string markers" "fallback must not fall back to generic evidence-dump text when pytest evidence is actionable" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_maps_supply_chain_vulnerabilities() { + local tmp_dir + local fixture_repo + local evidence_file + local output_file + local stderr_file + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + stderr_file="$tmp_dir/fallback.err" + mkdir -p "$fixture_repo" + + cat >"$fixture_repo/requirements.txt" <<'EOF' +flask==2.0.1 +requests==2.19.0 +urllib3==1.25.0 +EOF + + cat >"$evidence_file" <<'EOF' +# Failed GitHub Check Evidence + +- PR: #23 +- Head SHA: `abc123def456abc123def456abc123def456abcd` +- Repository: `ContextualWisdomLab/clearfolio` + +## Failed check: OSV-Scanner/osv-scan + +- Type: `check_run` +- Conclusion: `FAILURE` +- Details URL: https://github.com/ContextualWisdomLab/clearfolio/actions/runs/28863381355 + +### Supply-chain vulnerability findings + +- Supply-chain vulnerability: id=GHSA-j8r2-6x86-q33q severity=HIGH package=requests installed=2.19.0 fixed=2.31.0 manifest=requirements.txt + +## Failed check: Security Scan/trivy-fs + +- Type: `check_run` +- Conclusion: `FAILURE` +- Details URL: https://github.com/ContextualWisdomLab/clearfolio/actions/runs/28863381999 + +### Failed log excerpt + +```text +requirements.txt (pip) +======================= +Total: 1 (HIGH: 1, CRITICAL: 0) + +┌──────────┬────────────────┬──────────┬────────┬───────────────────┬───────────────┐ +│ Library │ Vulnerability │ Severity │ Status │ Installed Version │ Fixed Version │ +├──────────┼────────────────┼──────────┼────────┼───────────────────┼───────────────┤ +│ urllib3 │ CVE-2023-43804 │ HIGH │ fixed │ 1.25.0 │ 1.26.18 │ +└──────────┴────────────────┴──────────┴────────┴───────────────────┴───────────────┘ +``` +EOF + + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" + + # osv-scanner canonical evidence: source-backed finding with the exact manifest line and from->to bump. + assert_file_contains "$output_file" "requirements.txt:2 - Supply-chain vulnerability GHSA-j8r2-6x86-q33q in requests" "supply-chain fallback maps the osv-scanner advisory to the exact manifest line" + assert_file_contains "$output_file" "bump \`requests\` from 2.19.0 to 2.31.0" "supply-chain fallback states the concrete requests version bump" + assert_file_contains "$output_file" "OSV-Scanner/osv-scan" "supply-chain fallback preserves the failed osv-scanner check label as evidence" + # trivy-fs job-log table: source-backed finding located under the manifest header. + assert_file_contains "$output_file" "requirements.txt:3 - Supply-chain vulnerability CVE-2023-43804 in urllib3" "supply-chain fallback maps the trivy table row to the exact manifest line" + assert_file_contains "$output_file" "bump \`urllib3\` from 1.25.0 to 1.26.18" "supply-chain fallback states the concrete urllib3 version bump" + assert_file_contains "$output_file" "urllib3==1.26.18" "supply-chain fallback offers a GitHub-suggestion-ready pin for the trivy finding" + assert_file_contains "$output_file" "requests==2.31.0" "supply-chain fallback offers a GitHub-suggestion-ready pin for the osv finding" + # Never line 0, and no URL-only deflection. + assert_file_not_contains "$output_file" ":0 - Supply-chain" "supply-chain fallback never emits a line-zero finding" + assert_file_not_contains "$output_file" "see the Actions run URL" "supply-chain fallback does not post URL-only supply-chain reviews" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_preserves_empty_supply_chain_columns() { + # Regression for the record-delimiter bug: the internal per-vulnerability + # record was joined with a TAB and read back with `IFS=$'\t'`. Tab is an + # IFS-whitespace character, so `read` collapsed consecutive tabs and any empty + # interior field (missing installed OR missing fixed) shifted every later + # column left by one — producing garbled findings such as a severity word in + # the advisory-id slot and a CVE id in the version slot. The collector appends + # installed=/fixed= only when present, so both are common real inputs. + local tmp_dir + local fixture_repo + local evidence_file + local output_file + local stderr_file + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + stderr_file="$tmp_dir/fallback.err" + mkdir -p "$fixture_repo" + + cat >"$fixture_repo/requirements.txt" <<'EOF' +flask==2.0.1 +requests==2.19.0 +EOF + + # Record 1: installed is MISSING (osv/trivy SARIF alert with no installed + # version). Record 2: fixed is MISSING (no-fix advisory). Both interior gaps + # used to collapse and shift columns. + cat >"$evidence_file" <<'EOF' +# Failed GitHub Check Evidence + +- PR: #77 +- Head SHA: `abc123def456abc123def456abc123def456abcd` +- Repository: `ContextualWisdomLab/clearfolio` + +## Failed check: OSV-Scanner/osv-scan + +- Type: `check_run` +- Conclusion: `FAILURE` +- Details URL: https://github.com/ContextualWisdomLab/clearfolio/actions/runs/28863381355 + +### Supply-chain vulnerability findings + +- Supply-chain vulnerability: id=CVE-2020-0001 severity=CRITICAL package=flask fixed=2.0.2 manifest=requirements.txt +- Supply-chain vulnerability: id=GHSA-aaaa-bbbb-cccc severity=HIGH package=requests installed=2.19.0 manifest=requirements.txt +EOF + + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" + + # Record 1 (installed missing): the advisory id must be the CVE (NOT the + # severity word), the package must be flask, and the fix target must be the + # fixed VERSION (2.0.2), never the CVE id in the version slot. + assert_file_contains "$output_file" "Supply-chain vulnerability CVE-2020-0001 in flask" "empty installed keeps the advisory id in the title, not the severity word" + assert_file_not_contains "$output_file" "Supply-chain vulnerability CRITICAL in flask" "empty installed does not shift the severity word into the advisory-id slot" + assert_file_contains "$output_file" "upgrade \`flask\` to 2.0.2" "empty installed still names the concrete fixed version as the upgrade target" + assert_file_not_contains "$output_file" "to CVE-2020-0001" "the CVE id never appears in the upgrade/version slot" + + # Record 2 (fixed missing): the advisory id must be the GHSA (NOT the severity + # word), installed must be the real version, and the fix must say no upstream + # fix is available — never 'bump ... to '. + assert_file_contains "$output_file" "Supply-chain vulnerability GHSA-aaaa-bbbb-cccc in requests" "empty fixed keeps the advisory id in the title, not the severity word" + assert_file_contains "$output_file" "no fixed version is available upstream for \`requests\` 2.19.0" "empty fixed produces a sensible no-fix instruction with the real installed version" + assert_file_not_contains "$output_file" "to GHSA-aaaa-bbbb-cccc" "the GHSA id never appears in the upgrade/version slot" + assert_file_not_contains "$output_file" "from GHSA-aaaa-bbbb-cccc" "the GHSA id never appears in the from-version slot" + + # Columns are not shifted: severity lands in the severity slot for both. + assert_file_contains "$output_file" "CRITICAL requirements.txt" "record 1 severity stays in the severity column" + assert_file_contains "$output_file" "HIGH requirements.txt" "record 2 severity stays in the severity column" + + # Line numbers stay positive (never 0), even with empty interior fields. + assert_file_not_contains "$output_file" ":0 - Supply-chain" "empty interior fields never produce a line-zero finding" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_rejects_url_only_supply_chain() { + local tmp_dir + local fixture_repo + local evidence_file + local output_file + local stderr_file + local rc + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + stderr_file="$tmp_dir/fallback.err" + mkdir -p "$fixture_repo" + + # A supply-chain check failed, but the evidence carries only the check name + # and a run URL — no package, advisory id, manifest, or fixed version. This + # must stay fail-closed: no source-backed finding can be invented. + cat >"$evidence_file" <<'EOF' +# Failed GitHub Check Evidence + +- PR: #24 +- Head SHA: `abc123def456abc123def456abc123def456abcd` +- Repository: `ContextualWisdomLab/clearfolio` + +## Failed check: OSV-Scanner/osv-scan + +- Type: `check_run` +- Conclusion: `FAILURE` +- Details URL: https://github.com/ContextualWisdomLab/clearfolio/actions/runs/28863381355 +EOF + + set +e + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" + rc=$? + set -e + + assert_equals "1" "$rc" "URL-only supply-chain evidence does not produce a REQUEST_CHANGES finding" + assert_file_not_contains "$output_file" "Supply-chain vulnerability" "URL-only supply-chain evidence emits no supply-chain finding" + assert_file_contains "$stderr_file" "No source-backed failed-check fallback finding matched" "URL-only supply-chain evidence stays fail-closed and asks for rerun or newer logs" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_rejects_cancelled_queue_only_reviews() { + local tmp_dir + local fixture_repo + local evidence_file + local output_file + local stderr_file + local rc + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + stderr_file="$tmp_dir/fallback.err" + mkdir -p "$fixture_repo" + + cat >"$evidence_file" <<'EOF' +# Failed GitHub Check Evidence + +- PR: #119 +- Head SHA: `96ce73d581b4ddeb8668f93768deb2b106b8f55a` +- Repository: `ContextualWisdomLab/.github` + +## Failed check: PR Review Merge Scheduler/scan-pr-queue + +- Type: `check_run` +- Conclusion: `CANCELLED` +- Details URL: https://github.com/ContextualWisdomLab/.github/actions/runs/28354829112/job/83995330163 + +### Check annotations + +- .github:1-1 [failure] Canceling since a higher priority waiting request for central-pr-review-merge-scheduler-ContextualWisdomLab/.github exists +EOF + + set +e + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" 2>"$stderr_file" + rc=$? + set -e + + assert_equals "1" "$rc" "cancelled queue-only evidence does not produce REQUEST_CHANGES findings" + assert_file_contains "$stderr_file" "Non-source-backed cancelled check queue state" "cancelled queue-only evidence is explained as non-source-backed" + assert_file_contains "$stderr_file" "No source-backed failed-check fallback finding matched" "cancelled queue-only evidence asks for rerun or newer logs" + assert_file_not_contains "$output_file" "GitHub Checks queue" "cancelled queue-only evidence does not emit a finding" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_explains_trusted_base_strix_prs() { + local tmp_dir + local fixture_repo + local evidence_file + local output_file + local base_sha + local head_sha + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + + mkdir -p "$fixture_repo/.github/workflows" + cat >"$fixture_repo/.github/workflows/strix.yml" <<'EOF' +name: Strix Security Scan +concurrency: + cancel-in-progress: false +EOF + + git init -q "$fixture_repo" >/dev/null + git -C "$fixture_repo" config user.email "copilot@example.com" + git -C "$fixture_repo" config user.name "copilot" + git -C "$fixture_repo" add .github/workflows/strix.yml + git -C "$fixture_repo" commit -m "base" >/dev/null + base_sha="$(git -C "$fixture_repo" rev-parse HEAD)" + + cat >"$fixture_repo/.github/workflows/strix.yml" <<'EOF' +name: Strix Security Scan +concurrency: + group: strix-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: false +EOF + git -C "$fixture_repo" add .github/workflows/strix.yml + git -C "$fixture_repo" commit -m "head" >/dev/null + head_sha="$(git -C "$fixture_repo" rev-parse HEAD)" + + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +Conclusion: cancelled + +No GitHub Actions job log is available for this failed workflow run. +EOF + + PR_BASE_SHA="$base_sha" PR_HEAD_SHA="$head_sha" \ + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" + + assert_file_contains "$output_file" "cancelled pull_request_target run still used the base branch copies" "fallback explains trusted-base workflow execution" + assert_file_contains "$output_file" "Re-run Strix after the trusted base branch contains the workflow/gate change or capture equivalent temporary evidence tied to this head SHA" "fallback directs reviewers to trusted-base rerun or equivalent evidence" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_does_not_treat_no_report_summary_as_report() { + local tmp_dir + local evidence_file + local output_file + tmp_dir="$(mktemp -d)" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +### Failed log signal summary + +```text +strix Run Strix (quick) openai.RateLimitError: Too many requests. +strix Run Strix (quick) httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://api.deepseek.com/beta/chat/completions' +strix Run Strix (quick) litellm.BadRequestError: DeepseekException - {"error":{"message":"Authentication Fails, Your api key is invalid"}} +strix Run Strix (quick) Configured model and fallback models were unavailable. +``` + +No Strix vulnerability report windows were detected in the failed log. +EOF + + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$REPO_ROOT" >"$output_file" + + assert_file_contains "$output_file" "Strix provider failure blocked current-head security evidence" "fallback treats no-report summary as provider blocker" + assert_file_contains "$output_file" "api.deepseek.com" "fallback preserves direct DeepSeek endpoint failure evidence" + assert_file_contains "$output_file" "Authentication Fails" "fallback preserves direct DeepSeek authentication failure evidence" + assert_file_contains "$output_file" "github_models/deepseek/deepseek-v3-0324 github_models/deepseek/deepseek-r1-0528" "fallback gives exact GitHub Models fallback list" + assert_file_contains "$output_file" "Suggested edit: \`.github/workflows/strix.yml" "fallback gives a line-specific suggested edit for provider routing" + assert_file_not_contains "$output_file" "Strix provider signal left current-head security evidence incomplete" "fallback does not invent vulnerability report windows from a no-report summary" + assert_file_not_contains "$output_file" "after vulnerability reports" "fallback does not contradict no-report evidence" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_handles_deepseek_auth_only_signal() { + local tmp_dir + local evidence_file + local output_file + tmp_dir="$(mktemp -d)" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +### Failed log signal summary + +```text +strix Run Strix (quick) httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://api.deepseek.com/beta/chat/completions' +strix Run Strix (quick) litellm.BadRequestError: DeepseekException - {"error":{"message":"Authentication Fails, Your api key is invalid"}} +``` + +No Strix vulnerability report windows were detected in the failed log. +EOF + + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$REPO_ROOT" >"$output_file" + + assert_file_contains "$output_file" "Strix provider failure blocked current-head security evidence" "fallback treats DeepSeek auth-only logs as provider blockers" + assert_file_contains "$output_file" "api.deepseek.com" "fallback preserves DeepSeek auth-only endpoint evidence" + assert_file_contains "$output_file" "Authentication Fails" "fallback preserves DeepSeek auth-only failure evidence" + assert_file_contains "$output_file" "Suggested edit: \`.github/workflows/strix.yml" "fallback gives suggested edit for DeepSeek auth-only provider routing" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_handles_pg_erd_cloud_strix_log_shape() { + local tmp_dir + local fixture_repo + local evidence_file + local output_file + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + + mkdir -p "$fixture_repo/backend/app" "$fixture_repo/frontend" + for line_number in $(seq 1 150); do + printf '# auth fixture line %s\n' "$line_number" + done >"$fixture_repo/backend/app/auth.py" + cat >"$fixture_repo/frontend/next.config.ts" <<'EOF' +import type { NextConfig } from "next"; + +const nextConfig: NextConfig = { + async headers() { + return []; + }, +}; + +export default nextConfig; +EOF + + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +### Failed log signal summary + +```text +strix Run Strix (quick) Strix run failed for model 'deepseek/deepseek-r1-0528' after 206s (exit code 2). +strix Run Strix (quick) Below-threshold findings detected, but infrastructure errors occurred during this pipeline run; refusing bypass due to potentially incomplete scan. +strix Run Strix (quick) Unable to map Strix findings to changed files; failing closed for pull request. +``` + +### Strix vulnerability report window 1 + +│ Vulnerability Report │ +│ Title: Authentication Bypass via X-Dev-User Header │ +│ Severity: CRITICAL │ +│ Target: /workspace/strix-pr-scope.I4RF8w │ +│ Endpoint: /api/me │ +│ Method: GET │ +│ Code Locations │ +│ Location 1: backend/app/auth.py:132-135 │ +│ Model deepseek/deepseek-r1-0528 │ +│ Vulnerabilities 1 │ + +### Strix vulnerability report window 2 + +│ Vulnerability Report │ +│ Title: Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure │ +│ Data Handling │ +│ Severity: HIGH │ +│ Target: /workspace/strix-pr-scope.I4RF8w/frontend │ +│ Model deepseek/deepseek-v3-0324 │ +│ Vulnerabilities 1 │ +EOF + + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" + + assert_file_contains "$output_file" "Strix report from deepseek/deepseek-r1-0528: Authentication Bypass via X-Dev-User Header" "fallback includes pg-erd-cloud first model report" + assert_file_contains "$output_file" "backend/app/auth.py:132" "fallback maps pg-erd-cloud auth report to exact line" + assert_file_contains "$output_file" "Endpoint: /api/me. Method: GET" "fallback preserves pg-erd-cloud endpoint and method" + assert_file_contains "$output_file" "Strix report from deepseek/deepseek-v3-0324: Frontend Security Issues: XSS, Hardcoded Credentials, and Insecure Data Handling" "fallback preserves wrapped pg-erd-cloud frontend title" + assert_file_contains "$output_file" "frontend/next.config.ts:3" "fallback anchors locationless frontend report to a concrete frontend hardening line" + assert_file_contains "$output_file" "Suggested edit: change \`frontend/next.config.ts:3\`" "fallback provides pg-erd-cloud frontend suggested edit" + assert_file_contains "$output_file" "Unable to map Strix findings" "fallback preserves failed Strix mapping signal" + assert_file_contains "$output_file" "Strix provider signal left current-head security evidence incomplete" "fallback reports incomplete Strix evidence after model findings" + assert_file_not_contains "$output_file" "failed before producing vulnerability reports" "fallback does not erase model findings after provider signals" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_handles_split_code_location_lines() { + local tmp_dir + local fixture_repo + local evidence_file + local output_file + local migration_file + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + migration_file="$fixture_repo/backend/alembic/versions/0002_provider_writeback_retry_queue.py" + + mkdir -p "$(dirname "$migration_file")" + for line_number in $(seq 1 80); do + if [ "$line_number" -eq 43 ]; then + printf '\tlegacy_index_execution_placeholder(statement)\n' + else + printf '# migration fixture line %s\n' "$line_number" + fi + done >"$migration_file" + + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +### Failed log signal summary + +```text +strix Run Strix (quick) Strix fallback model 'github_models/deepseek/deepseek-r1-0528' emitted provider infrastructure or failure-signal output; trying next configured fallback if available. +strix Run Strix (quick) Strix reported zero vulnerabilities before provider infrastructure failure; failing closed because provider infrastructure failures are not clean scan evidence. +``` + +### Strix vulnerability report window 1 + +│ Vulnerability Report │ +│ Title: SQL Injection Vulnerability in Database Script │ +│ Severity: HIGH │ +│ Target: │ +│ /workspace/strix-pr-scope.e0AHf4/backend/alembic/versions/0002_provider_wr │ +│ iteback_retry_queue.py │ +│ Code Locations │ +│ │ +│ Location 1: │ +│ backend/alembic/versions/0002_provider_writeback_retry_queue.py:43 │ +│ Vulnerable code location │ +│ legacy_index_execution_placeholder(statement) │ +│ Model openai/deepseek/deepseek-r1-0528 │ +│ Vulnerabilities 1 │ +EOF + + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" + + assert_file_contains "$output_file" "Strix report from openai/deepseek/deepseek-r1-0528: SQL Injection Vulnerability in Database Script" "fallback includes split-location Strix report" + assert_file_contains "$output_file" "backend/alembic/versions/0002_provider_writeback_retry_queue.py:43" "fallback maps split Code Locations path to exact line" + assert_file_contains "$output_file" "Code location evidence: backend/alembic/versions/0002_provider_writeback_retry_queue.py:43" "fallback preserves split Code Locations evidence" + assert_file_contains "$output_file" "Suggested edit: change \`backend/alembic/versions/0002_provider_writeback_retry_queue.py:43\`" "fallback gives suggested edit for split Code Locations" + assert_file_not_contains "$output_file" "Strix report did not include a mappable Code Location" "fallback does not misclassify split Code Locations as unmapped" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_does_not_anchor_unmapped_strix_reports_to_workflow() { + local tmp_dir + local fixture_repo + local evidence_file + local output_file + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + + mkdir -p "$fixture_repo/.github/workflows" "$fixture_repo/scripts/ci" + cat >"$fixture_repo/.github/workflows/strix.yml" <<'EOF' +name: Strix Security Scan +jobs: + strix: + steps: + - name: Run Strix + env: + STRIX_FALLBACK_MODELS: github_models/deepseek/deepseek-v3-0324 github_models/deepseek/deepseek-r1-0528 +EOF + + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +### Failed log signal summary + +```text +strix Run Strix (quick) Below-threshold findings detected, but infrastructure errors occurred during this pipeline run; refusing bypass due to potentially incomplete scan. +strix Run Strix (quick) Unable to map Strix findings to changed files; failing closed for pull request. +``` + +### Strix vulnerability report window 1 + +│ Vulnerability Report │ +│ Title: Insecure Direct Object Reference (IDOR) in User Profile API │ +│ Severity: MEDIUM │ +│ Target: /workspace/strix-pr-scope.mVhTAV/backend │ +│ Code Locations │ +│ Location 1: backend/api/users.py:45-52 │ +│ Model github_models/deepseek/deepseek-v3-0324 │ +│ Vulnerabilities 1 │ +EOF + + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" + + assert_file_contains "$output_file" "Strix provider signal left current-head security evidence incomplete" "fallback reports incomplete Strix evidence for unmapped report" + assert_file_contains "$output_file" "did not map to an existing repository file" "fallback explains unmapped Strix report" + assert_file_contains "$output_file" "Insecure Direct Object Reference (IDOR) in User Profile API" "fallback preserves unmapped report title as diagnostic evidence" + assert_file_not_contains "$output_file" "Strix report from github_models/deepseek/deepseek-v3-0324" "fallback does not convert unmapped report into source finding" + assert_file_not_contains "$output_file" "Inspect and patch .github/workflows/strix.yml" "fallback does not anchor unmapped report to workflow line" + assert_file_not_contains "$output_file" "backend/api/users.py:45" "fallback does not cite nonexistent source path as actionable line" + + rm -rf "$tmp_dir" +} + +assert_opencode_failed_check_fallback_maps_strix_status_permission_smoke_failure() { + local tmp_dir + local fixture_repo + local evidence_file + local output_file + tmp_dir="$(mktemp -d)" + fixture_repo="$tmp_dir/repo" + evidence_file="$tmp_dir/failed-check-evidence.md" + output_file="$tmp_dir/fallback.md" + + mkdir -p "$fixture_repo/.github/workflows" "$fixture_repo/scripts/ci" + cat >"$fixture_repo/.github/workflows/strix.yml" <<'EOF' +name: Strix Security Scan +jobs: + strix: + permissions: + contents: read + statuses: write +EOF + + cat >"$evidence_file" <<'EOF' +## Failed check: Strix Security Scan/strix + +### Failed log signal summary + +```text +strix Self-test Strix required workflow contract Running bounded Strix required-workflow smoke test. +strix Self-test Strix required workflow contract FAIL: Strix workflow keeps GITHUB_TOKEN status permissions read-only (unexpected 'statuses: write') +strix Self-test Strix required workflow contract Strix required workflow smoke test failed with 1 failure(s). +``` +EOF + + bash "$REPO_ROOT/scripts/ci/emit_opencode_failed_check_fallback_findings.sh" \ + "$evidence_file" "$fixture_repo" >"$output_file" + + assert_file_contains "$output_file" "Strix required workflow must keep GITHUB_TOKEN statuses read-only" "fallback maps Strix smoke permission failure" + assert_file_contains "$output_file" ".github/workflows/strix.yml:6" "fallback cites the exact statuses write line" + assert_file_contains "$output_file" 'change `.github/workflows/strix.yml:6` from `statuses: write` to `statuses: read`' "fallback gives a concrete status-permission repair" + assert_file_not_contains "$output_file" "No source-backed failed-check fallback finding matched" "fallback does not leave Strix smoke failure undiagnosed" + + rm -rf "$tmp_dir" +} + +assert_internal_pr_scope_targets() { + local target_log_file="$1" + local repo_root_dir="$2" + local expected_count="$3" + + if [ ! -f "$target_log_file" ]; then + record_failure "internal PR scope target log should exist" + return + fi + + local actual_count=0 + local target_path + while IFS= read -r target_path; do + actual_count=$((actual_count + 1)) + case "$target_path" in + "$repo_root_dir" | "$repo_root_dir"/*) + record_failure "internal PR scope target should not reuse repository path: $target_path" + ;; + esac + case "$(basename -- "$target_path")" in + strix-pr-scope.*) + ;; + *) + record_failure "internal PR scope target should be generated by build_pull_request_scope_dir: $target_path" + ;; + esac + done <"$target_log_file" + + assert_equals "$expected_count" "$actual_count" "internal PR scope target count" +} + +run_gate_case() { + local scenario="$1" + local initial_model="$2" + local fallback_models="$3" + local expected_exit="$4" + local expected_message="$5" + local expected_calls="$6" + local expected_model_sequence="${7:-}" + local expected_api_base_sequence="${8:-}" + local default_provider="${9-vertex_ai}" + local raw_llm_api_base_override="${10-__DEFAULT__}" + local initial_llm_api_base="${11-}" + + local raw_llm_api_base="https://example.invalid/generateContent" + if [ "$raw_llm_api_base_override" != "__DEFAULT__" ]; then + raw_llm_api_base="$raw_llm_api_base_override" + elif [ "$default_provider" = "openai" ]; then + raw_llm_api_base="" + fi + local transient_retry_per_model="${12-0}" + local min_fail_severity="${13-CRITICAL}" + local transient_retry_backoff_seconds="${14:-0}" + local custom_target_path="${15-}" + local custom_source_dirs="${16-}" + local process_timeout_seconds="${17-1200}" + local total_timeout_seconds="${18-0}" + local github_event_name="${19-}" + local changed_files_override="${20-}" + local event_name_override="${21-}" + local legacy_scope_size_ignored="${22-}" + local disable_pr_scoping="${23-0}" + local test_pr_sca_status_override="${24-}" + local current_pr_number="${25-}" + local authoritative_sca_runs_json="${26-}" + local gemini_fallback_models="${27-__SAME_AS_FALLBACK_MODELS__}" + local generic_fallback_models="${28-}" + local fail_on_provider_signal="${29-1}" + if [ "$default_provider" = "openai" ] && [ -z "$generic_fallback_models" ] && [ -n "$fallback_models" ]; then + generic_fallback_models="$fallback_models" + fallback_models="" + fi + + if [ -n "${STRIX_TEST_CASE_FILTER:-}" ] && [ "$scenario" != "$STRIX_TEST_CASE_FILTER" ]; then + return + fi + if [ "${STRIX_TEST_TRACE_CASES:-0}" = "1" ]; then + printf 'RUN_GATE_CASE: %s\n' "$scenario" >&2 + fi + + local tmp_dir + tmp_dir="$(mktemp -d)" + # Separate bin/ (fake strix + helper files) from workspace/ (target path) + # so grep -r over the target path never matches the fake strix script itself. + local bin_dir="$tmp_dir/bin" + local untrusted_bin_dir="$tmp_dir/untrusted-bin" + local workspace_dir="$tmp_dir/workspace" + local repo_root_dir="$workspace_dir/smart-crawling-server" + mkdir -p "$bin_dir" "$untrusted_bin_dir" "$repo_root_dir/src" + mkdir -p "$repo_root_dir/scripts/ci" + local gate_under_test="$repo_root_dir/scripts/ci/strix_quick_gate.sh" + cp "$GATE_SCRIPT" "$gate_under_test" + cp "$REPO_ROOT/scripts/ci/strix_model_utils.sh" "$repo_root_dir/scripts/ci/strix_model_utils.sh" + chmod +x "$gate_under_test" + local fake_strix="$bin_dir/strix" + local path_hijack_log="$tmp_dir/path-hijack.log" + cat >"$untrusted_bin_dir/strix" <<'EOF' +#!/usr/bin/env bash +printf 'inherited PATH executable was invoked\n' >"${FAKE_STRIX_PATH_HIJACK_LOG:?}" +exit 99 +EOF + chmod +x "$untrusted_bin_dir/strix" + local call_log="$tmp_dir/calls.log" + local api_base_log="$tmp_dir/api_base.log" + local target_log="$tmp_dir/target.log" + local runtime_env_log="$tmp_dir/runtime_env.log" + local state_file="$tmp_dir/state.log" + local strix_llm_file="$tmp_dir/strix_llm.txt" + local llm_api_key_file="$tmp_dir/llm_api_key.txt" + local llm_api_base_file="$tmp_dir/llm_api_base.txt" + local output_log="$tmp_dir/output.log" + local fake_gh="$bin_dir/gh" + local gh_token_log="$tmp_dir/gh_token.log" + local event_payload_file="$tmp_dir/github_event.json" + + # Resolve target path: use repo-local relative defaults to mirror the real workflow. + local effective_target_path="." + if [ "$custom_target_path" = "__USE_SUBDIR_SRC__" ]; then + # Simulate STRIX_TARGET_PATH=./src with a repo-local relative path. + effective_target_path="./src" + elif [ -n "$custom_target_path" ]; then + effective_target_path="$custom_target_path" + # Ensure the custom target path exists + mkdir -p "$effective_target_path" + fi + + cat >"$fake_strix" <<'EOF' +#!/usr/bin/env bash +set -euo pipefail + +printf '%s\n' "${STRIX_LLM:-}" >> "${FAKE_STRIX_CALL_LOG:?}" +printf '%s\n' "${LLM_API_BASE:-}" >> "${FAKE_STRIX_API_BASE_LOG:?}" +if [ -n "${FAKE_STRIX_RUNTIME_ENV_LOG:-}" ]; then + printf 'LLM_TIMEOUT=%s;STRIX_MEMORY_COMPRESSOR_TIMEOUT=%s;STRIX_REASONING_EFFORT=%s;STRIX_LLM_MAX_RETRIES=%s;GEMINI_LOCATION=%s;PYTHONWARNINGS=%s;NPM_CONFIG_IGNORE_SCRIPTS=%s;PNPM_CONFIG_IGNORE_SCRIPTS=%s;YARN_ENABLE_SCRIPTS=%s;UNRELATED_SECRET=%s;LLM_DISABLE_STREAMING=%s\n' \ + "${LLM_TIMEOUT:-}" \ + "${STRIX_MEMORY_COMPRESSOR_TIMEOUT:-}" \ + "${STRIX_REASONING_EFFORT:-}" \ + "${STRIX_LLM_MAX_RETRIES:-}" \ + "${GEMINI_LOCATION:-}" \ + "${PYTHONWARNINGS:-}" \ + "${NPM_CONFIG_IGNORE_SCRIPTS:-}" \ + "${PNPM_CONFIG_IGNORE_SCRIPTS:-}" \ + "${YARN_ENABLE_SCRIPTS:-}" \ + "${UNRELATED_SECRET:-}" \ + "${LLM_DISABLE_STREAMING:-}" >> "${FAKE_STRIX_RUNTIME_ENV_LOG:?}" +fi + +target_path="" +while [ "$#" -gt 0 ]; do + if [ "$1" = "-t" ] && [ "$#" -ge 2 ]; then + target_path="$2" + break + fi + shift +done +if [ "$target_path" = "." ]; then + target_path="$PWD" +fi +printf '%s\n' "$target_path" >> "${FAKE_STRIX_TARGET_LOG:?}" + +STRIX_REPORTS_DIR="${STRIX_REPORTS_DIR:-strix_runs}" + +case "${FAKE_STRIX_SCENARIO:?}" in +success|runtime-env-forwarding|custom-openai-compatible-preserves-effort|vertex-primary-success-timing-message|direct-openai-gpt-does-not-require-github-models-api-base|pr-executable-integrity-mismatch|pr-executable-group-writable) + echo "scan ok" + exit 0 + ;; + contextual-orchestrator-gateway-model-qualification) + if [ "${STRIX_LLM:-}" != "openai/orchestrator/free" ]; then + echo "gateway model was not provider-qualified for LiteLLM" >&2 + exit 10 + fi + if [ "${LLM_API_BASE:-}" != "http://127.0.0.1:18080/v1" ]; then + echo "gateway API base was not preserved" >&2 + exit 11 + fi + echo "scan ok through contextual-orchestrator gateway" + exit 0 + ;; + scan-working-directory-isolated) + if [ "$PWD" = "$target_path" ] || [[ "$PWD" == "$target_path"/* ]]; then + echo "Error: Strix process inherited the untrusted scan target as cwd" >&2 + exit 81 + fi + if [ ! -f "$target_path/backend/app/pg_introspect/dsn_guard.py" ]; then + echo "Error: PostgreSQL DSN guard context missing from PR scope" >&2 + exit 82 + fi + echo "scan ok with isolated Strix working directory" + exit 0 + ;; + success-with-critical-report) + mkdir -p "$STRIX_REPORTS_DIR/fake-success/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-success/vulnerabilities/vuln-0001.md" <<'REPORT' +# Vulnerability Report + +- Severity: CRITICAL +- Title: Successful process still emitted a blocking vulnerability +REPORT + echo "Vulnerabilities 1" + exit 0 + ;; + slow-timeout) + sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" + exit 0 + ;; + timeout-disabled-success) + sleep 1 + echo "scan ok with timeout disabled" + exit 0 + ;; + vertex-primary-notfound-fallback-success|github-models-fallback-success|github-models-fallback-success-deepseek-v3|github-models-token-limit-fallback-success|github-models-fallback-requires-api-base|github-models-model-prefix-with-api-base-succeeds|github-models-meta-prefix-with-api-base-succeeds|github-models-mistral-prefix-with-api-base-succeeds) + case "${STRIX_LLM:-}" in + vertex_ai/missing-primary) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok with fallback" + exit 0 + ;; + openai/gpt-5|openai/openai/gpt-5.4|openai/meta/test-github-model|openai/mistral-ai/test-github-model) + if [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-token-limit-fallback-success" ]; then + echo "openai.APIStatusError: Error code: 413 - {'error': {'code': 'tokens_limit_reached', 'message': 'Request body too large for gpt-5 model. Max size: 4000 tokens.'}}" + exit 1 + fi + echo "scan ok with GitHub Models fallback" + exit 0 + ;; + openai/deepseek/deepseek-r1-0528) + if [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-fallback-success-deepseek-v3" ]; then + echo "LLM CONNECTION FAILED" + echo "Could not establish connection to the language model." + echo "Error: litellm.BadRequestError: OpenAIException - Unavailable model: deepseek-r1-0528" + exit 1 + fi + echo "scan ok with GitHub Models fallback" + exit 0 + ;; + openai/deepseek/deepseek-v3-0324) + echo "scan ok with GitHub Models fallback" + exit 0 + ;; + *) + echo "unexpected model ${STRIX_LLM:-}" >&2 + exit 9 + ;; + esac + ;; + nvidia-rate-limit-openai-direct-fallback-clears-api-base) + case "${STRIX_LLM:-}" in + nvidia_nim/nvidia/rate-limited-primary) + echo "LLM CONNECTION FAILED" + echo "Error: litellm.RateLimitError: Nvidia_nimException - Error code: 429 Too Many Requests" + exit 1 + ;; + openai/gpt-5.4) + if [ "${STRIX_REASONING_EFFORT:-}" != "none" ]; then + echo "direct OpenAI function-tools fallback requires reasoning effort none" >&2 + exit 29 + fi + if [ "${LLM_API_KEY:-}" != "openai-fallback-token" ]; then + echo "unexpected direct-OpenAI fallback key (${LLM_API_KEY:-})" >&2 + exit 26 + fi + if [ -n "${LLM_API_BASE:-}" ]; then + echo "direct OpenAI fallback inherited foreign API base ${LLM_API_BASE}" >&2 + exit 27 + fi + echo "scan ok after direct-OpenAI fallback" + exit 0 + ;; + *) + echo "unexpected cross-provider model ${STRIX_LLM:-}" >&2 + exit 28 + ;; + esac + ;; + openai-direct-quota-github-models-fallback-success) + case "${STRIX_LLM:-}" in + openai/gpt-5.4) + if [ "${LLM_API_KEY:-}" != "dummy" ]; then + echo "unexpected direct-OpenAI key for primary (${LLM_API_KEY:-})" >&2 + exit 15 + fi + echo "Error getting response: Error code: 429 - {'error': {'message': 'You exceeded your current quota, please check your plan and billing details.', 'type': 'insufficient_quota', 'code': 'insufficient_quota'}}" + echo "openai.RateLimitError: Error code: 429" + exit 1 + ;; + openai/o3) + if [ "${LLM_API_KEY:-}" != "github-models-fallback-token" ]; then + echo "unexpected GitHub Models key for fallback (${LLM_API_KEY:-})" >&2 + exit 16 + fi + echo "scan ok with GitHub Models fallback" + exit 0 + ;; + *) + echo "unexpected model ${STRIX_LLM:-}" >&2 + exit 9 + ;; + esac + ;; + vertex-all-notfound) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + nonrecoverable) + echo "Error: transport timeout" + exit 1 + ;; + provider-prefix-required) + if [ "${STRIX_LLM:-}" = "vertex_ai/gemini-2.5-pro" ]; then + echo "scan ok with normalized provider" + exit 0 + fi + echo "Error: provider prefix not normalized (${STRIX_LLM:-})" >&2 + exit 10 + ;; + provider-prefix-fallback-normalization) + case "${STRIX_LLM:-}" in + vertex_ai/missing-primary) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after fallback normalization" + exit 0 + ;; + *) + echo "Error: fallback provider prefix not normalized (${STRIX_LLM:-})" >&2 + exit 11 + ;; + esac + ;; + provider-prefix-required-resource-path-primary-implicit-default-provider | provider-prefix-required-resource-path-primary-explicit-empty-default-provider) + if [ "${STRIX_LLM:-}" = "vertex_ai/gemini-2.5-pro" ]; then + echo "scan ok with resource-path normalization" + exit 0 + fi + echo "Error: resource-path model not normalized (${STRIX_LLM:-})" >&2 + exit 12 + ;; + provider-prefix-resource-path-primary-notfound-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/missing-primary) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after resource-path fallback" + exit 0 + ;; + *) + echo "Error: resource-path fallback model not normalized (${STRIX_LLM:-})" >&2 + exit 13 + ;; + esac + ;; + vertex-custom-model-resource-path) + # projects/

/locations//models/ (no publishers/ segment) + if [ "${STRIX_LLM:-}" = "vertex_ai/my-custom-model-123" ]; then + echo "scan ok with custom model resource-path normalization" + exit 0 + fi + echo "Error: custom model resource-path not normalized (${STRIX_LLM:-})" >&2 + exit 40 + ;; + vertex-notfound-without-status-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/missing-primary) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after status-less not found fallback" + exit 0 + ;; + *) + echo "Error: status-less fallback model not normalized (${STRIX_LLM:-})" >&2 + exit 14 + ;; + esac + ;; + vertex-notfound-compact-status-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/missing-primary) + echo 'litellm.exceptions.NotFoundError: VertexAI error' + echo '{"error":{"status":"NOT_FOUND"}}' + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after compact-status not found fallback" + exit 0 + ;; + *) + echo "Error: compact-status fallback model not normalized (${STRIX_LLM:-})" >&2 + exit 17 + ;; + esac + ;; + nonvertex-slash-model-passthrough) + if [ "${STRIX_LLM:-}" = "foo/bar" ]; then + echo "scan ok with non-vertex slash model passthrough" + exit 0 + fi + echo "Error: non-vertex slash model was rewritten (${STRIX_LLM:-})" >&2 + exit 18 + ;; + primary-duplicate-in-fallback) + case "${STRIX_LLM:-}" in + vertex_ai/missing-primary) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after duplicate-primary skip" + exit 0 + ;; + *) + echo "Error: duplicate-primary path unexpected (${STRIX_LLM:-})" >&2 + exit 15 + ;; + esac + ;; + multiline-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/missing-primary) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + vertex_ai/fallback-one) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + vertex_ai/fallback-two) + echo "scan ok after multiline fallback parsing" + exit 0 + ;; + *) + echo "Error: multiline fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 19 + ;; + esac + ;; + vertex-primary-ratelimit-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/ratelimit-primary) + echo "Penetration test failed: LLM request failed: RateLimitError" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after rate-limit fallback" + exit 0 + ;; + *) + echo "Error: ratelimit fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 21 + ;; + esac + ;; + vertex-primary-resource-exhausted-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/resource-exhausted-primary) + echo '{"error":{"status":"RESOURCE_EXHAUSTED"}}' + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after resource exhausted fallback" + exit 0 + ;; + *) + echo "Error: resource exhausted fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 23 + ;; + esac + ;; + openai-primary-quota-fallback-success) + case "${STRIX_LLM:-}" in + openai/quota-primary) + echo "openai.agents: Error streaming response: You exceeded your current quota, please check your plan and billing details." + exit 1 + ;; + openai/fallback-one) + echo "scan ok after quota fallback" + exit 0 + ;; + *) + echo "Error: quota fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 24 + ;; + esac + ;; + vertex-primary-429-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/http429-primary) + echo "litellm: HTTP 429 Too Many Requests" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after 429 fallback" + exit 0 + ;; + *) + echo "Error: 429 fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 24 + ;; + esac + ;; + vertex-primary-midstream-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/midstream-primary) + echo "Penetration test failed: LLM request failed: MidStreamFallbackError" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after midstream fallback" + exit 0 + ;; + *) + echo "Error: midstream fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 25 + ;; + esac + ;; + vertex-primary-midstream-retry-same-model-success) + case "${STRIX_LLM:-}" in + vertex_ai/retry-midstream-primary) + attempt="0" + if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then + attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" + fi + attempt="$((attempt + 1))" + echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" + if [ "$attempt" -eq 1 ]; then + echo "Penetration test failed: LLM request failed: MidStreamFallbackError" + exit 1 + fi + echo "scan ok after same-model retry" + exit 0 + ;; + vertex_ai/fallback-one) + echo "Error: fallback should not be needed for same-model retry scenario" >&2 + exit 30 + ;; + *) + echo "Error: midstream fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 30 + ;; + esac + ;; + vertex-primary-ratelimit-retry-same-model-success|vertex-primary-ratelimit-retry-reason-message) + case "${STRIX_LLM:-}" in + vertex_ai/retry-ratelimit-primary) + attempt="0" + if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then + attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" + fi + attempt="$((attempt + 1))" + echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" + if [ "$attempt" -eq 1 ]; then + echo "Penetration test failed: LLM request failed: RateLimitError" + exit 1 + fi + echo "scan ok after same-model rate-limit retry" + exit 0 + ;; + vertex_ai/fallback-one) + echo "Error: fallback should not be needed for same-model rate-limit retry scenario" >&2 + exit 31 + ;; + *) + echo "Error: rate-limit fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 31 + ;; + esac + ;; + vertex-primary-api-connection-retry-same-model-success|github-models-internal-server-connection-retry-same-model-success|internal-server-error-unrelated-output-nonretryable|internal-server-error-many-blocks-retry-same-model-success) + case "${STRIX_LLM:-}" in + gemini/retry-api-connection-primary|vertex_ai/retry-api-connection-primary|openai/openai/retry-api-connection-primary) + attempt="0" + if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then + attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" + fi + attempt="$((attempt + 1))" + echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" + if [ "$attempt" -eq 1 ]; then + if [ "${STRIX_LLM:-}" = "openai/openai/retry-api-connection-primary" ]; then + if [ "${FAKE_STRIX_SCENARIO:?}" = "internal-server-error-unrelated-output-nonretryable" ]; then + echo "Error: litellm.InternalServerError: upstream request failed" + for filler in 1 2 3 4 5 6; do + echo "target application diagnostic $filler" + done + echo "Internal Server Error" + exit 1 + fi + if [ "${FAKE_STRIX_SCENARIO:?}" = "internal-server-error-many-blocks-retry-same-model-success" ]; then + # Regression for the SIGPIPE race (Devin finding on + # PR #1394): emit enough matching + # litellm.InternalServerError blocks that the bounded + # awk scan's piped output exceeds a single pipe + # buffer, so a `grep -q` that stops reading at the + # first match cannot SIGPIPE the still-writing awk + # producer into a false non-match under + # `set -o pipefail`. + for _ in $(seq 1 2000); do + echo "line filler some unrelated target application output padding padding padding" + echo "Error: litellm.InternalServerError: upstream request failed" + echo "Internal Server Error" + echo "more filler after context one" + echo "more filler after context two" + done + exit 1 + fi + echo "LLM CONNECTION FAILED" + echo "Could not establish connection to the language model." + echo "Error: litellm.InternalServerError: upstream request failed" + else + echo "LLM CONNECTION FAILED" + echo "litellm.APIConnectionError: GeminiException - Server disconnected without sending a response." + fi + exit 1 + fi + echo "scan ok after same-model api connection retry" + exit 0 + ;; + vertex_ai/fallback-one) + echo "Error: fallback should not be needed for API connection retry scenario" >&2 + exit 36 + ;; + *) + echo "Error: API connection retry path unexpected (${STRIX_LLM:-})" >&2 + exit 36 + ;; + esac + ;; + openrouter-502-fallback-retry-same-model-success) + case "${STRIX_LLM:-}" in + vertex_ai/missing-primary) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + openrouter/free) + attempt="0" + if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then + attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" + fi + attempt="$((attempt + 1))" + echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" + if [ "$attempt" -eq 1 ]; then + echo "Error: litellm.APIError: APIError:" + echo "OpenrouterException -" + echo '{"error":{"message":"Invalid URL:' + echo '","code":502,"metadata":{"provider_name":"Stealth"}}}' + exit 1 + fi + echo "scan ok after OpenRouter 502 same-model retry" + exit 0 + ;; + vertex_ai/fallback-two) + echo "Error: second fallback should not be needed after transient OpenRouter 502" >&2 + exit 38 + ;; + *) + echo "Error: OpenRouter 502 fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 38 + ;; + esac + ;; + openrouter-502-distant-target-output-nonretryable) + case "${STRIX_LLM:-}" in + vertex_ai/missing-primary) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + openrouter/free) + echo "Error: litellm.APIError: APIError: OpenrouterException -" + printf 'target output\n%.0s' 1 2 3 4 5 6 + echo '{"code":502,"metadata":{"provider_name":"spoof"}}' + exit 1 + ;; + vertex_ai/fallback-two) + echo "scan ok after distant target output" + exit 0 + ;; + esac + ;; + github-models-primary-unavailable-fallback-success|github-models-primary-denied-fallback-success) + case "${STRIX_LLM:-}" in + openai/gpt-5) + echo "LLM CONNECTION FAILED" + echo "Could not establish connection to the language model." + if [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-primary-denied-fallback-success" ]; then + echo "openai.PermissionDeniedError: Error code: 403" + else + echo "Error: litellm.BadRequestError: OpenAIException - Unavailable model: gpt-5" + fi + exit 1 + ;; + openai/deepseek/deepseek-r1-0528) + echo "scan ok after GitHub Models unavailable fallback" + exit 0 + ;; + *) + echo "Error: GitHub Models unavailable fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 37 + ;; + esac + ;; + github-models-http410-authenticated-fallback-success | github-models-http410-missing-http-token | github-models-http410-missing-provider-error | github-models-http410-numeric-continuation-4100 | github-models-http410-numeric-continuation-4104 | github-models-http410-target-output-spoof | github-models-retirement-brownout-phrase-only) + case "${STRIX_LLM:-}" in + openai/gpt-5) + case "${FAKE_STRIX_SCENARIO:?}" in + github-models-http410-authenticated-fallback-success) + echo "Error: litellm.BadRequestError: GitHub Models provider error at models.github.ai/inference: HTTP 410 Gone" + ;; + github-models-http410-missing-http-token) + echo "Error: litellm.BadRequestError: GitHub Models provider retirement at models.github.ai/inference" + ;; + github-models-http410-missing-provider-error) + echo "GitHub Models response at models.github.ai/inference: HTTP 410 Gone" + ;; + github-models-http410-numeric-continuation-4100) + echo "Error: litellm.BadRequestError: GitHub Models provider error at models.github.ai/inference: HTTP 4100" + ;; + github-models-http410-numeric-continuation-4104) + echo "Error: litellm.BadRequestError: GitHub Models provider error at models.github.ai/inference: HTTP 4104" + ;; + github-models-http410-target-output-spoof) + echo "TARGET OUTPUT: Error: litellm.BadRequestError: GitHub Models provider error HTTP 410" + ;; + github-models-retirement-brownout-phrase-only) + echo "GitHub Models retirement brownout" + ;; + esac + exit 1 + ;; + openai/deepseek/deepseek-r1-0528) + echo "scan ok after authenticated GitHub Models HTTP 410 retirement" + exit 0 + ;; + *) + echo "Error: GitHub Models HTTP 410 fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 39 + ;; + esac + ;; + github-models-primary-ratelimit-fallback-success) + case "${STRIX_LLM:-}" in + openai/gpt-5) + echo "LLM CONNECTION FAILED" + echo "Could not establish connection to the language model." + echo "Error: litellm.RateLimitError: RateLimitError: OpenAIException - Too many requests. For more on scraping GitHub and how it may affect your rights, please review our Terms of Service." + exit 1 + ;; + openai/deepseek/deepseek-r1-0528) + echo "scan ok after GitHub Models rate-limit fallback" + exit 0 + ;; + *) + echo "Error: GitHub Models rate-limit fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 38 + ;; + esac + ;; + github-models-fallback-provider-signal-tries-next | github-models-fallback-baseline-vulnerability-before-next-success-continues | github-models-exhausted-after-baseline-vulnerability-fails-closed | github-models-fallback-changed-vulnerability-before-next-success-blocks | github-models-fallback-dockerfile-test-baseline-before-next-success-continues) + case "${STRIX_LLM:-}" in + openai/gpt-5) + echo "LLM CONNECTION FAILED" + echo "Could not establish connection to the language model." + echo "Error: litellm.RateLimitError: RateLimitError: OpenAIException - Too many requests." + exit 1 + ;; + openai/deepseek/deepseek-r1-0528) + if [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-fallback-baseline-vulnerability-before-next-success-continues" ] || + [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-exhausted-after-baseline-vulnerability-fails-closed" ]; then + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-provider-signal/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-provider-signal/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Location 1: +sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl/SysUserServiceImpl.java:5 +EOS + elif [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-fallback-changed-vulnerability-before-next-success-blocks" ]; then + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-provider-signal/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-changed-provider-signal/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Location 1: +sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java:12 +EOS + elif [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-fallback-dockerfile-test-baseline-before-next-success-continues" ]; then + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-dockerfile-test-provider-signal/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-dockerfile-test-provider-signal/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: MEDIUM +Location 1: +Dockerfile.test:1 +EOS + else + echo "LLM CONNECTION FAILED" + echo "Could not establish connection to the language model." + echo "Error: litellm.BadRequestError: OpenAIException - Unavailable model: deepseek-r1-0528" + fi + exit 2 + ;; + openai/deepseek/deepseek-v3-0324) + if [ "${FAKE_STRIX_SCENARIO:?}" = "github-models-exhausted-after-baseline-vulnerability-fails-closed" ]; then + echo "LLM CONNECTION FAILED" + echo "Could not establish connection to the language model." + echo "Error: provider retirement brownout" + exit 1 + fi + echo "scan ok after second GitHub Models fallback" + exit 0 + ;; + *) + echo "Error: GitHub Models provider-signal fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 38 + ;; + esac + ;; + gemini-high-demand-retry-same-model-success) + case "${STRIX_LLM:-}" in + gemini/retry-high-demand-primary) + attempt="0" + if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then + attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" + fi + attempt="$((attempt + 1))" + echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" + if [ "$attempt" -eq 1 ]; then + echo "LLM CONNECTION FAILED" + echo 'litellm.ServiceUnavailableError: GeminiException - {"error":{"code":503,"message":"This model is currently experiencing high demand. Spikes in demand are usually temporary. Please try again later.","status":"UNAVAILABLE"}}' + exit 1 + fi + echo "scan ok after same-model high-demand retry" + exit 0 + ;; + *) + echo "Error: high-demand retry path unexpected (${STRIX_LLM:-})" >&2 + exit 37 + ;; + esac + ;; + nvidia-overloaded-direct-fallback-success) + case "${STRIX_LLM:-}" in + nvidia_nim/nvidia/overloaded-primary) + echo "LLM CONNECTION FAILED" + echo "Could not establish connection to the language model." + echo "Error: litellm.ServiceUnavailableError: Nvidia_nimException - Service temporarily overloaded" + exit 1 + ;; + nvidia_nim/nvidia/fallback-one) + echo "scan ok after NVIDIA overload fallback" + exit 0 + ;; + *) + echo "Error: NVIDIA overload fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 37 + ;; + esac + ;; + gemini-timeout-direct-fallback-success) + case "${STRIX_LLM:-}" in + gemini/retry-timeout-primary) + echo "LLM CONNECTION FAILED" + echo "Error: litellm.Timeout: Connection timed out after None seconds." + exit 1 + ;; + gemini/fallback-one) + echo "scan ok after timeout fallback" + exit 0 + ;; + *) + echo "Error: gemini timeout fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 38 + ;; + esac + ;; + gemini-timeout-fallback-success|gemini-generic-fallback-success) + case "${STRIX_LLM:-}" in + gemini/timeout-fallback-primary) + echo "LLM CONNECTION FAILED" + echo "Error: litellm.Timeout: Connection timed out after None seconds." + exit 1 + ;; + gemini/fallback-one) + echo "scan ok after gemini fallback" + exit 0 + ;; + *) + echo "Error: gemini timeout fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 39 + ;; + esac + ;; + gemini-zero-findings-timeout-fallback-allows-pr) + case "${STRIX_LLM:-}" in + gemini/zero-timeout-primary|gemini/fallback-one) + echo "Vulnerabilities 0" + echo "LLM CONNECTION FAILED" + echo "Error: litellm.Timeout: Connection timed out after None seconds." + exit 1 + ;; + *) + echo "Error: gemini zero-finding fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 40 + ;; + esac + ;; + pr-scope-zero-finding-does-not-leak) + if [ -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" ]; then + echo "Vulnerabilities 0" + echo "LLM CONNECTION FAILED" + echo "Error: litellm.Timeout: Connection timed out after None seconds." + exit 1 + fi + if [ -f "$target_path/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java" ]; then + echo "LLM CONNECTION FAILED" + echo "Error: litellm.Timeout: Connection timed out after None seconds." + exit 1 + fi + echo "Error: unexpected PR scope zero-finding leak target layout ($target_path)" >&2 + exit 41 + ;; + service-unavailable-no-llm-marker-nonrecoverable) + echo 'ServiceUnavailableError: {"error":{"code":503,"status":"UNAVAILABLE"}}' + echo '{"error":{"code":502,"metadata":{"provider_name":"Stealth"}}}' + echo 'target application high demand response' + exit 1 + ;; + server-disconnect-no-llm-marker-nonrecoverable) + echo "ConnectionError: Server disconnected without sending a response." + exit 1 + ;; + vertex-all-ratelimited) + echo "Penetration test failed: LLM request failed: RateLimitError" + exit 1 + ;; + vertex-primary-hallucinated-endpoint-fallback-success|target-path-src-default-source-dirs) + case "${STRIX_LLM:-}" in + vertex_ai/hallucination-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-hallucinated/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-hallucinated/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Endpoint:** /api/ghost-admin +EOS + echo "Penetration test failed: CRITICAL finding on /api/ghost-admin" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after hallucinated-endpoint fallback" + exit 0 + ;; + *) + echo "Error: hallucinated-endpoint fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 26 + ;; + esac + ;; + opencode-documented-env-api-key-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/opencode-env-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-opencode-env/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-opencode-env/vulnerabilities/vuln-0001.md" <&2 + exit 27 + ;; + esac + ;; + generic-github-actions-workflow-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/generic-actions-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-generic-actions/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-generic-actions/vulnerabilities/vuln-0001.md" <<'EOS' +# Insecure Configurations in GitHub Actions Workflows + +**Severity:** CRITICAL +**Target:** local_code: /workspace/strix-pr-scope.fake +**Endpoint:** CI/CD Pipeline +**CWE:** CWE-732 + +## Description + +/workspace/strix-pr-scope.fake/.github/workflows/strix.yml + +## Technical Analysis + +The GitHub Actions configuration contains several security weaknesses: +1. Secrets are written to temporary files without proper access controls +2. API keys are passed through environment variables without adequate masking +3. Excessive permissions granted to workflows +4. Insufficient input validation for workflow parameters + +## Code Analysis + +**Location 1:** `.github/workflows/strix.yml` (lines 1-300) + ``` + Full file content + ``` + + **Suggested Fix:** +```diff +- Current content ++ Secured version +``` +EOS + echo "Penetration test failed: generic GitHub Actions workflow finding" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after generic GitHub Actions workflow false positive" + exit 0 + ;; + *) + echo "Error: generic GitHub Actions workflow fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 37 + ;; + esac + ;; + vertex-primary-existing-endpoint-nonrecoverable|multi-source-dirs-existing-endpoint) + case "${STRIX_LLM:-}" in + vertex_ai/existing-endpoint-primary|vertex_ai/multi-dir-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-existing-endpoint/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-existing-endpoint/vulnerabilities/vuln-0001.md" <<'EOS' +**Endpoint:** /api/status +EOS + echo "Penetration test failed: CRITICAL finding on /api/status" + exit 1 + ;; + vertex_ai/fallback-one|vertex_ai/fallback-two) + echo "Error: existing endpoint findings must remain non-recoverable (${STRIX_LLM:-})" >&2 + exit 27 + ;; + *) + echo "Error: existing-endpoint scenario unexpected model (${STRIX_LLM:-})" >&2 + exit 28 + ;; + esac + ;; + pr-stale-source-claim-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/stale-source-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-stale-source/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-stale-source/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** HIGH +**Target:** backend/db/models.py + +The `WorkspaceRunnerConfig.registration_token` field stores the token as plain text. +The vulnerable line is `registration_token: Mapped[str | None] = mapped_column(String, nullable=True)`. +EOS + echo "Penetration test failed: stale HIGH finding on backend/db/models.py" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after stale-source fallback" + exit 0 + ;; + *) + echo "Error: stale-source scenario unexpected model (${STRIX_LLM:-})" >&2 + exit 30 + ;; + esac + ;; + pr-stale-snapshot-snippet-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/stale-snapshot-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-stale-snapshot/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-stale-snapshot/vulnerabilities/vuln-0001.md" <<'EOS' +# IDOR in /api/snapshots endpoint allows unauthorized access to database schemas + +**Severity:** MEDIUM +**Target:** backend/app/api/snapshots.py + +## Code Analysis + +**Location 1:** `backend/app/api/snapshots.py` (lines 78-81) + Missing ownership check + ``` + snapshot = await get_snapshot_by_uuid(snapshot_uuid) +if not snapshot: + raise HTTPException(status_code=404) +return snapshot + ``` + +**Location 2:** `backend/app/api/snapshots.py` (lines 78-81) + **Suggested Fix:** +```diff +- snapshot = await get_snapshot_by_uuid(snapshot_uuid) +- if not snapshot: +- raise HTTPException(status_code=404) +- return snapshot ++ snapshot = await get_snapshot_by_uuid(snapshot_uuid) ++ if not snapshot: ++ raise HTTPException(status_code=404) ++ if not await is_project_member(current_user.user_account_uuid, snapshot.project_space_uuid): ++ raise HTTPException(status_code=403) ++ return snapshot +``` +EOS + echo "Penetration test failed: stale MEDIUM snapshot snippet" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after stale snapshot snippet fallback" + exit 0 + ;; + *) + echo "Error: stale-snapshot scenario unexpected model (${STRIX_LLM:-})" >&2 + exit 38 + ;; + esac + ;; + pr-stale-source-plus-real-finding-blocks) + case "${STRIX_LLM:-}" in + vertex_ai/stale-source-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-mixed-findings/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-mixed-findings/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** HIGH +**Target:** backend/db/models.py + +The `WorkspaceRunnerConfig.registration_token` field stores the token as plain text. +The vulnerable line is `registration_token: Mapped[str | None] = mapped_column(String, nullable=True)`. +EOS + cat >"$STRIX_REPORTS_DIR/fake-mixed-findings/vulnerabilities/vuln-0002.md" <<'EOS' +**Severity:** HIGH +**Target:** backend/api/emails.py + +This is a concrete changed-file finding that must remain blocking. +EOS + echo "Penetration test failed: mixed stale and real HIGH findings" + exit 1 + ;; + vertex_ai/fallback-one) + echo "Error: mixed real findings must not reach fallback" >&2 + exit 31 + ;; + *) + echo "Error: mixed-findings scenario unexpected model (${STRIX_LLM:-})" >&2 + exit 32 + ;; + esac + ;; + pr-changed-finding-with-retry-marker-blocks) + case "${STRIX_LLM:-}" in + vertex_ai/changed-finding-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-changed-retry-marker/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-changed-retry-marker/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** HIGH +**Target:** backend/api/emails.py + +This changed-file finding must remain blocking even when the model log also contains retryable provider text. +EOS + echo "litellm.exceptions.Timeout: provider timed out after writing a HIGH changed-file finding" + exit 1 + ;; + vertex_ai/fallback-one) + echo "Error: changed-file findings with retry markers must not reach fallback" >&2 + exit 33 + ;; + *) + echo "Error: changed-retry-marker scenario unexpected model (${STRIX_LLM:-})" >&2 + exit 34 + ;; + esac + ;; + pr-stale-report-plus-inline-changed-finding-blocks) + case "${STRIX_LLM:-}" in + vertex_ai/stale-inline-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-stale-report-inline-changed/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-stale-report-inline-changed/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** HIGH +**Target:** backend/db/models.py + +The `WorkspaceRunnerConfig.registration_token` field stores the token as plain text. +The vulnerable line is `registration_token: Mapped[str | None] = mapped_column(String, nullable=True)`. +EOS + echo "Severity: HIGH" + echo "Target: backend/api/emails.py" + echo "Penetration test failed: stale report plus inline changed-file HIGH finding" + exit 1 + ;; + vertex_ai/fallback-one) + echo "Error: inline changed-file findings must not reach fallback" >&2 + exit 35 + ;; + *) + echo "Error: stale-inline scenario unexpected model (${STRIX_LLM:-})" >&2 + exit 36 + ;; + esac + ;; + endpoint-in-excluded-dir) + case "${STRIX_LLM:-}" in + vertex_ai/excluded-dir-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-excluded-dir/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-excluded-dir/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Endpoint:** /api/hidden-secret +EOS + echo "Penetration test failed: CRITICAL finding on /api/hidden-secret" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after excluded-dir hallucination fallback" + exit 0 + ;; + *) + echo "Error: excluded-dir scenario unexpected model (${STRIX_LLM:-})" >&2 + exit 29 + ;; + esac + ;; + empty-fallback-models) + # Output must match is_vertex_not_found_error() patterns so the gate + # proceeds to the fallback loop (where empty array triggers the message). + echo "Publisher Model vertex_ai/empty-fb-primary was not found in project." + exit 1 + ;; + high-vuln-below-threshold) + mkdir -p "$STRIX_REPORTS_DIR/fake-high/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-high/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: HIGH +EOS + echo "Penetration test failed: simulated high finding" + exit 1 + ;; + multi-severity-low-then-critical) + mkdir -p "$STRIX_REPORTS_DIR/fake-multi-severity/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-multi-severity/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: LOW + +Related issue severity: CRITICAL +EOS + echo "Penetration test failed: report contains LOW followed by CRITICAL" + exit 1 + ;; + inline-medium-below-threshold) + echo "╭─ VULN-0001 ──────────────────────────────────────────────────────────────────╮" + echo "│ Vulnerability Report │" + echo "│ Severity: MEDIUM │" + echo "╰──────────────────────────────────────────────────────────────────────────────╯" + echo "Penetration test failed: simulated inline medium finding" + exit 2 + ;; + medium-vuln-default-threshold) + mkdir -p "$STRIX_REPORTS_DIR/fake-medium-default/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-medium-default/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: MEDIUM +EOS + echo "Penetration test failed: simulated medium finding" + exit 1 + ;; + critical-vuln-at-threshold) + mkdir -p "$STRIX_REPORTS_DIR/fake-critical/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-critical/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +EOS + echo "Penetration test failed: simulated critical finding" + exit 1 + ;; + malformed-severity-marker-nonrecoverable) + mkdir -p "$STRIX_REPORTS_DIR/fake-malformed/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-malformed/vulnerabilities/vuln-0001.md" <<'EOS' +Severity details: high confidence marker only +EOS + echo "Penetration test failed: malformed severity marker" + exit 1 + ;; + model-disagreement-critical-in-earlier-report) + case "${STRIX_LLM:-}" in + vertex_ai/model-a) + mkdir -p "$STRIX_REPORTS_DIR/run-001/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/run-001/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +EOS + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + echo "Penetration test failed: CRITICAL finding by model-a" + exit 1 + ;; + vertex_ai/model-b) + mkdir -p "$STRIX_REPORTS_DIR/run-002/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/run-002/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: LOW +EOS + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + echo "Penetration test failed: LOW finding by model-b" + exit 1 + ;; + *) + echo "Error: model-disagreement unexpected model (${STRIX_LLM:-})" >&2 + exit 32 + ;; + esac + ;; + nonvertex-slash-model-not-rewritten) + if [ "${STRIX_LLM:-}" = "deepseek/models/deepseek-r1" ]; then + echo "scan ok with deepseek model passthrough" + exit 0 + fi + echo "Error: deepseek model was rewritten (${STRIX_LLM:-})" >&2 + exit 33 + ;; + preserve-existing-api-base) + if [ "${LLM_API_BASE:-}" = "https://preexisting.invalid" ]; then + echo "scan ok with preserved api base" + exit 0 + fi + echo "Error: existing LLM_API_BASE was not preserved (${LLM_API_BASE:-})" >&2 + exit 20 + ;; + default-fallback-order-fast-first) + case "${STRIX_LLM:-}" in + vertex_ai/missing-primary) + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + vertex_ai/gemini-2.5-pro) + echo "scan ok with default fast fallback" + exit 0 + ;; + *) + echo "Error: default fallback order unexpected (${STRIX_LLM:-})" >&2 + exit 16 + ;; + esac + ;; + vertex-primary-timeout-retry-same-model-success|vertex-primary-timeout-retry-reason-message) + case "${STRIX_LLM:-}" in + vertex_ai/retry-timeout-primary) + echo "litellm.exceptions.Timeout: litellm.Timeout: Connection timed out after None seconds." + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after timeout fallback" + exit 0 + ;; + *) + echo "Error: timeout fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 34 + ;; + esac + ;; + all-fallbacks-same-as-primary) + # Bug 13: All fallback models are the same as the primary model. + # The gate should emit an ERROR and exit 1. + echo "Error: litellm.NotFoundError: Vertex_aiException - x" + echo '"status": "NOT_FOUND"' + exit 1 + ;; + vertex-primary-timeout-exhausted-fallback-success) + # Primary always times out (even after retries). Fallback succeeds. + case "${STRIX_LLM:-}" in + vertex_ai/timeout-exhaust-primary) + echo "litellm.exceptions.Timeout: litellm.Timeout: Connection timed out after None seconds." + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after timeout-exhausted fallback" + exit 0 + ;; + *) + echo "Error: timeout-exhausted-fallback unexpected model (${STRIX_LLM:-})" >&2 + exit 35 + ;; + esac + ;; + zero-findings-timeout-all-models|strict-zero-findings-timeout-fails-pr) + case "${STRIX_LLM:-}" in + vertex_ai/zero-timeout-primary|vertex_ai/fallback-one) + echo "╭─ STRIX ──────────────────────────────────────────────────────────────────────╮" + echo "│ Penetration test in progress │" + echo "│ Vulnerabilities 0 │" + echo "╰──────────────────────────────────────────────────────────────────────────────╯" + sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" + exit 0 + ;; + *) + echo "Error: zero-findings-timeout unexpected model (${STRIX_LLM:-})" >&2 + exit 57 + ;; + esac + ;; + zero-findings-sticky-across-fallback) + case "${STRIX_LLM:-}" in + vertex_ai/zero-sticky-primary) + echo "╭─ STRIX ──────────────────────────────────────────────────────────────────────╮" + echo "│ Penetration test in progress │" + echo "│ Vulnerabilities 0 │" + echo "╰──────────────────────────────────────────────────────────────────────────────╯" + sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" + exit 0 + ;; + vertex_ai/fallback-one) + sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" + exit 0 + ;; + *) + echo "Error: zero-findings-sticky unexpected model (${STRIX_LLM:-})" >&2 + exit 58 + ;; + esac + ;; + zero-findings-with-low-report-timeout) + case "${STRIX_LLM:-}" in + vertex_ai/zero-low-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-zero-low/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-zero-low/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: LOW +EOS + echo "╭─ STRIX ──────────────────────────────────────────────────────────────────────╮" + echo "│ Penetration test in progress │" + echo "│ Vulnerabilities 0 │" + echo "╰──────────────────────────────────────────────────────────────────────────────╯" + sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" + exit 0 + ;; + vertex_ai/fallback-one) + sleep "${FAKE_STRIX_TIMEOUT_SLEEP_SECONDS:?}" + exit 0 + ;; + *) + echo "Error: zero-findings-with-low-report unexpected model (${STRIX_LLM:-})" >&2 + exit 59 + ;; + esac + ;; + provider-fatal-success-signal) + echo "Fatal: provider stream aborted" + exit 0 + ;; + provider-warning-success-signal) + echo "Warning: provider response included incomplete scan state" + exit 0 + ;; + provider-denied-success-signal) + echo "Denied: provider credentials were rejected" + exit 0 + ;; + provider-report-rate-limit-fallback-success) + case "${STRIX_LLM:-}" in + vertex_ai/report-rate-limit-primary) + mkdir -p "$STRIX_REPORTS_DIR/fake-report-rate-limit" + cat >"$STRIX_REPORTS_DIR/fake-report-rate-limit/strix.log" <<'EOS' +2026-08-21 04:00:00.000 WARNING strix-pr-scope-example - strix.provider: RateLimitError: provider response was exhausted +EOS + echo "scan aborted after provider report-rate-limit signal" + exit 1 + ;; + vertex_ai/fallback-one) + mkdir -p "$STRIX_REPORTS_DIR/fake-report-rate-limit-fallback" + echo "scan ok after report-only provider fallback" + exit 0 + ;; + *) + echo "Error: report-only provider fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 60 + ;; + esac + ;; + report-known-internal-warning-sanitized) + printf '%s\n' '│ MODEL QUALITY WARNING │' + echo 'Warning: You are sending unauthenticated requests to the HF Hub.' + mkdir -p "$STRIX_REPORTS_DIR/fake-known-internal-warning" + cat >"$STRIX_REPORTS_DIR/fake-known-internal-warning/strix.log" <<'EOS' +2026-06-18 13:08:05.986 WARNING strix-pr-scope-example - strix.core.execution: agent a9fb4033 produced non-lifecycle final output in non-interactive mode; forcing tool continuation (1/500): internal agent coordination note +2026-06-18 13:10:44.089 INFO strix-pr-scope-example - strix.tools.finish.tool: finish_scan: completed scan with 0 vulnerability report(s) +EOS + mkdir -p strix_runs/fake-known-internal-warning-relative + cat >strix_runs/fake-known-internal-warning-relative/strix.log <<'EOS' +2026-06-18 13:08:05.986 WARNING strix-pr-scope-example - strix.core.execution: agent a9fb4033 produced non-lifecycle final output in non-interactive mode; forcing tool continuation (1/500): relative internal agent coordination note +2026-06-18 13:10:44.089 INFO strix-pr-scope-example - strix.tools.finish.tool: finish_scan: completed scan with 0 vulnerability report(s) +EOS + outside_report_dir="${FAKE_STRIX_OUTSIDE_REPORT_DIR:-$(dirname -- "$STRIX_REPORTS_DIR")/outside-strix-report}" + mkdir -p "$outside_report_dir" + cat >"$outside_report_dir/strix.log" <<'EOS' +2026-06-18 13:08:05.986 WARNING strix-pr-scope-example - strix.core.execution: agent a9fb4033 produced non-lifecycle final output in non-interactive mode; forcing tool continuation (1/500): outside report should not be rewritten +EOS + ln -s "$outside_report_dir" "$STRIX_REPORTS_DIR/fake-known-internal-warning/linked-outside" + echo "scan ok with sanitized internal Strix report notice" + exit 0 + ;; + report-known-internal-warning-variant-sanitized) + mkdir -p "$STRIX_REPORTS_DIR/fake-known-internal-warning-variant" + cat >"$STRIX_REPORTS_DIR/fake-known-internal-warning-variant/strix.log" <<'EOS' +2026-08-22 09:53:26.193 WARNING strix-pr-scope-example - strix.core.execution: agent 673f770f ended a turn without a lifecycle tool call (interactive=False); forcing tool continuation (1/500): +2026-06-18 13:10:44.089 INFO strix-pr-scope-example - strix.tools.finish.tool: finish_scan: completed scan with 0 vulnerability report(s) +EOS + echo "scan ok with sanitized internal Strix report notice variant" + exit 0 + ;; + report-unknown-warning-fails) + mkdir -p "$STRIX_REPORTS_DIR/fake-unknown-warning" + cat >"$STRIX_REPORTS_DIR/fake-unknown-warning/strix.log" <<'EOS' +2026-06-18 13:08:05.986 WARNING strix-pr-scope-example - strix.provider: provider returned incomplete scan state +EOS + echo "scan ok but unknown report warning remains" + exit 0 + ;; + bare-timeout-with-provider-marker) + # Emit bare "Connection timed out" alongside a provider marker so + # is_timeout_error() matches the Tier 3 branch gated on + # LLM_PROVIDER_ONLY_REGEX. Does NOT include + # litellm.exceptions.Timeout / httpx.ReadTimeout to ensure we + # exercise the provider-marker fallback path specifically. + # Primary times out; fallback model succeeds. + case "${STRIX_LLM:-}" in + vertex_ai/bare-timeout-primary) + echo "Connection timed out" + echo "vertex_ai model invocation failed" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after bare-timeout fallback" + exit 0 + ;; + *) + echo "Error: bare-timeout fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 47 + ;; + esac + ;; + bare-timeout-no-provider-marker) + # Emit "Connection timed out" with transport library names (httpx, + # httpcore, requests) but WITHOUT any real LLM provider marker. + # is_timeout_error() Tier 3 uses LLM_PROVIDER_ONLY_REGEX which + # excludes transport libs, so this should NOT match. + echo "Connection timed out" + echo "httpx transport layer connection reset" + echo "httpcore pool timeout" + echo "requests transport timeout" + exit 1 + ;; + below-threshold-with-timeout) + # Produce a below-threshold (LOW) finding but also emit a timeout error + # so the infrastructure guard detects an incomplete scan. + mkdir -p "$STRIX_REPORTS_DIR/fake-low-timeout/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-low-timeout/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: LOW +EOS + echo "litellm.exceptions.Timeout: litellm.Timeout: Connection timed out after None seconds." + echo "Penetration test failed: simulated timeout with low finding" + exit 1 + ;; + below-threshold-with-ratelimit) + # Produce a below-threshold (LOW) finding but also emit a rate-limit error. + mkdir -p "$STRIX_REPORTS_DIR/fake-low-ratelimit/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-low-ratelimit/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: LOW +EOS + echo "Penetration test failed: LLM request failed: RateLimitError" + echo "Penetration test failed: simulated ratelimit with low finding" + exit 1 + ;; + below-threshold-with-connection-error) + # Produce a below-threshold (INFO) finding but also emit a + # ConnectionError WITH an LLM-provider context marker so the + # infrastructure guard detects an incomplete scan. + # The two-grep guard requires BOTH a transport error class AND an + # LLM_PROVIDER_ONLY_REGEX marker (litellm, openai, anthropic, etc.). + mkdir -p "$STRIX_REPORTS_DIR/fake-info-conn/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-info-conn/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: INFO +EOS + echo "litellm.exceptions.APIConnectionError: ConnectionError - connection refused" + echo "Penetration test failed: simulated connection error with info finding" + exit 1 + ;; + below-threshold-with-connection-error-no-provider) + # Produce a below-threshold (INFO) finding and emit a ConnectionError + # WITHOUT any LLM-provider context marker. The infra-error detector + # should NOT match because the log lacks provider markers like + # "litellm", "openai", "anthropic", etc. This validates that the + # two-grep guard avoids false positives from target-application logs. + mkdir -p "$STRIX_REPORTS_DIR/fake-info-conn-noprov/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-info-conn-noprov/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: INFO +EOS + echo "ConnectionError: target server refused connection on port 8443" + echo "Penetration test failed: simulated app-level connection error" + exit 1 + ;; + below-threshold-with-requests-connection-error) + # Produce a below-threshold (INFO) finding with a + # requests.exceptions.ConnectionError — the transport library prefix + # "requests" matches the broad PROVIDER_CONTEXT_REGEX but is + # intentionally excluded from LLM_PROVIDER_ONLY_REGEX. + # + # Before commit 0e90d48, the connection-error path used + # has_provider_context_marker() (PROVIDER_CONTEXT_REGEX) and would + # have incorrectly classified this as an LLM infrastructure error. + # After that fix, LLM_PROVIDER_ONLY_REGEX is used, so "requests" + # alone does NOT satisfy the provider check → below-threshold bypass + # succeeds → exit 0. + mkdir -p "$STRIX_REPORTS_DIR/fake-info-conn-requests/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-info-conn-requests/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: INFO +EOS + echo "requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.example.com', port=443): Max retries exceeded with url: /v1/scan" + echo "Penetration test failed: simulated requests transport error" + exit 1 + ;; + below-threshold-with-midstream) + # Produce a below-threshold (MEDIUM) finding below CRITICAL threshold + # but also emit a MidStreamFallbackError. + mkdir -p "$STRIX_REPORTS_DIR/fake-medium-midstream/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-medium-midstream/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: MEDIUM +EOS + echo "Penetration test failed: LLM request failed: MidStreamFallbackError" + echo "Penetration test failed: simulated midstream with medium finding" + exit 1 + ;; + bare-timeout-provider-marker-exhausted-fallback) + # Bare "Connection timed out" + provider marker: primary fails once, + # then the gate falls back to fallback-one which succeeds. + case "${STRIX_LLM:-}" in + vertex_ai/bare-timeout-exhaust-primary) + echo "Connection timed out" + echo "vertex_ai model invocation failed" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after bare-timeout-exhaust fallback" + exit 0 + ;; + *) + echo "Error: bare-timeout-exhaust-fallback unexpected model (${STRIX_LLM:-})" >&2 + exit 35 + ;; + esac + ;; + httpx-read-timeout-with-provider-marker) + # Tier 2: httpx.ReadTimeout + provider-context marker (litellm). + # Primary times out; fallback model succeeds. + case "${STRIX_LLM:-}" in + vertex_ai/httpx-timeout-primary) + echo "httpx.ReadTimeout: timed out" + echo "litellm.proxy: connection to upstream model failed" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after httpx-timeout fallback" + exit 0 + ;; + *) + echo "Error: httpx-timeout fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 45 + ;; + esac + ;; + httpx-read-timeout-no-provider-marker) + # Tier 2 negative: httpx.ReadTimeout WITHOUT any provider-context + # marker. Should NOT be classified as retryable timeout. + echo "httpx.ReadTimeout: timed out" + echo "application server connection pool exhausted" + exit 1 + ;; + httpcore-read-timeout-with-provider-marker) + # Tier 2b: httpcore.ReadTimeout + provider-context marker. + # Primary times out; fallback model succeeds. + case "${STRIX_LLM:-}" in + vertex_ai/httpcore-timeout-primary) + echo "httpcore.ReadTimeout: timed out" + echo "litellm.proxy: connection to upstream model failed" + exit 1 + ;; + vertex_ai/fallback-one) + echo "scan ok after httpcore-timeout fallback" + exit 0 + ;; + *) + echo "Error: httpcore-timeout fallback path unexpected (${STRIX_LLM:-})" >&2 + exit 46 + ;; + esac + ;; + httpcore-read-timeout-no-provider-marker) + # Tier 2b negative: httpcore.ReadTimeout WITHOUT any provider-context + # marker. Should NOT be classified as retryable timeout. + echo "httpcore.ReadTimeout: timed out" + echo "application server connection pool exhausted" + exit 1 + ;; + infra-error-sticky-flag) + # Sticky flag test: first call hits infra error (rate limit), + # second call fails on the first fallback model but produces a + # LOW finding report. After exhausting retries, the gate checks + # has_only_below_threshold_vulnerabilities — which finds LOW + # findings but sees INFRA_ERROR_DETECTED=1 (set from the first + # call's rate-limit error) and refuses the below-threshold bypass. + case "${STRIX_LLM:-}" in + vertex_ai/sticky-flag-primary) + touch "$FAKE_STRIX_STATE_FILE" + echo "RateLimitError: rate limit exceeded" + echo "litellm.proxy: rate limit on vertex_ai model" + exit 1 + ;; + vertex_ai/gemini-2.5-pro) + mkdir -p "$STRIX_REPORTS_DIR/run-sticky/vulnerabilities" + cat > "$STRIX_REPORTS_DIR/run-sticky/vulnerabilities/vuln-0001.md" <<'FINDINGS' +Severity: LOW +FINDINGS + echo "non-retryable scan error with partial results" + exit 1 + ;; + *) + echo "Error: infra-error-sticky-flag unexpected model (${STRIX_LLM:-})" >&2 + exit 35 + ;; + esac + ;; + pr-baseline-critical-unchanged) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-baseline/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Location 1: +sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl/SysUserServiceImpl.java:5 +EOS + echo "Penetration test failed: baseline critical finding" + exit 1 + ;; + pr-critical-changed) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-changed/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Location 1: +sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java:12 +EOS + echo "Penetration test failed: changed critical finding" + exit 1 + ;; + pr-changed-file-nonintersecting-line) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-nonintersecting-line/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-nonintersecting-line/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Location 1: +frontend/src/App.tsx:1 +EOS + echo "Penetration test failed: same changed file but baseline line finding" + exit 1 + ;; + pr-critical-changed-bracketed-next-route) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-bracketed-next-route/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-changed-bracketed-next-route/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Location 1: +frontend/src/app/labels/[slug]/page.tsx:12 +EOS + echo "Penetration test failed: changed bracketed Next.js route finding" + exit 1 + ;; + pr-critical-changed-xml-file-location) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-xml/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-changed-xml/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: HIGH + + + sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java + 120 + 124 + + +EOS + echo "Penetration test failed: changed XML file location finding" + exit 1 + ;; + pr-critical-changed-xml-file-location-space) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-xml-space/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-changed-xml-space/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: HIGH + + + src/unsafe name.py + 7 + 9 + + +EOS + echo "Penetration test failed: changed XML file location finding with space" + exit 1 + ;; + pr-baseline-critical-narrative-backticked-service-file) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-narrative-service/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-narrative-service/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Technical Analysis +The `backend/services/email_parser.py` file extracts HTML email bodies without sanitizing script tags. +EOS + echo "Penetration test failed: baseline critical narrative service finding" + exit 1 + ;; + pr-critical-unmapped-arbitrary-backticked-service-file) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-unmapped-arbitrary-backtick/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-unmapped-arbitrary-backtick/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Description: location data unavailable, but the report also mentions `backend/services/email_parser.py` as unrelated context. +EOS + echo "Penetration test failed: unmapped critical finding with arbitrary backticked file mention" + exit 1 + ;; + pr-critical-unmapped) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-unmapped/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-unmapped/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Description: location data unavailable +EOS + echo "Penetration test failed: unmapped critical finding" + exit 1 + ;; + pr-baseline-critical-absolute-target) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-absolute/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-absolute/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** File: /workspace/smart-crawling-server/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java +EOS + echo "Penetration test failed: baseline critical finding with absolute target" + exit 1 + ;; + pr-baseline-critical-extensionless-dockerfile-target) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-dockerfile/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-dockerfile/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** File: /workspace/smart-crawling-server/Dockerfile +EOS + echo "Penetration test failed: baseline critical finding with extensionless Dockerfile target" + exit 1 + ;; + pr-baseline-critical-subdir-target) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-subdir/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-subdir/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** File: /workspace/flyway/V16__hash_oauth2_registered_client_secret.sql +EOS + echo "Penetration test failed: baseline critical finding with narrowed subdir target" + exit 1 + ;; + pr-baseline-critical-subdir-boxed-target) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-boxed-target/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-boxed-target/vulnerabilities/vuln-0001.md" <<'EOS' +│ Severity: CRITICAL │ +│ Target: /workspace/flyway/V16__hash_oauth2_registered_client_secret.sql │ +│ Endpoint: N/A (database migration script) │ +EOS + echo "Penetration test failed: baseline critical finding with boxed narrowed subdir target" + exit 1 + ;; + pr-baseline-critical-subdir-endpoint) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-endpoint/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-endpoint/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** Local Codebase: /workspace/flyway +**Endpoint:** /workspace/flyway/V16__hash_oauth2_registered_client_secret.sql +EOS + echo "Penetration test failed: baseline critical finding with narrowed subdir endpoint" + exit 1 + ;; + pr-baseline-critical-subdir-endpoint-bare-filename) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-endpoint-bare-filename/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-endpoint-bare-filename/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** Local Codebase: /workspace/flyway +**Endpoint:** V16__hash_oauth2_registered_client_secret.sql +EOS + echo "Penetration test failed: baseline critical finding with narrowed subdir bare filename endpoint" + exit 1 + ;; + pr-baseline-critical-subdir-narrative-backticked-file) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-narrative-backticked-file/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-baseline-subdir-narrative-backticked-file/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** Local Codebase: /workspace/flyway +The issue appears in file `V4__ccf_scenario.sql`. +EOS + echo "Penetration test failed: baseline critical finding with narrowed subdir narrative backticked file" + exit 1 + ;; + pr-critical-relative-path-escape-subdir-narrative-backticked-file) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-relative-path-escape-subdir-narrative/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-relative-path-escape-subdir-narrative/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** Local Codebase: /workspace/flyway +The issue appears in file `../V24__update_search_expression_team_keyword_id.sql`. +EOS + echo "Penetration test failed: relative path escape critical finding with narrowed subdir narrative backticked file" + exit 1 + ;; + pr-critical-changed-absolute-target) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-absolute/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-changed-absolute/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** File: /workspace/smart-crawling-server/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java +EOS + echo "Penetration test failed: changed critical finding with absolute target" + exit 1 + ;; + pr-critical-changed-internal-dotdir-target) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-internal-dotdir/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-changed-internal-dotdir/vulnerabilities/vuln-0001.md" <"$STRIX_REPORTS_DIR/fake-pr-changed-json-target/vulnerabilities/vuln-0001.md" <"$STRIX_REPORTS_DIR/fake-pr-changed-subdir/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** File: /workspace/flyway/V24__update_search_expression_team_keyword_id.sql +EOS + echo "Penetration test failed: changed critical finding with narrowed subdir target" + exit 1 + ;; + pr-critical-changed-subdir-endpoint) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-changed-subdir-endpoint/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-changed-subdir-endpoint/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** Local Codebase: /workspace/flyway +**Endpoint:** /workspace/flyway/V24__update_search_expression_team_keyword_id.sql +EOS + echo "Penetration test failed: changed critical finding with narrowed subdir endpoint" + exit 1 + ;; + pr-critical-path-escape-subdir-target) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-path-escape-subdir/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-path-escape-subdir/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** File: /workspace/flyway/../../../../../smart-crawling-common/src/main/java/org/empasy/sync/common/system/util/JwtUtil.java +EOS + echo "Penetration test failed: path escape critical finding with narrowed subdir target" + exit 1 + ;; + pr-critical-unmapped-narrative-target) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-unmapped-narrative/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-unmapped-narrative/vulnerabilities/vuln-0001.md" <<'EOS' +**Severity:** CRITICAL +**Target:** Multiple files in the codebase, particularly `org.empasy.sync.common.system.util.JwtUtil.java` (for signing) and its callers. +EOS + echo "Penetration test failed: unmapped narrative critical finding" + exit 1 + ;; + pr-critical-unmapped-other-workspace-repo) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-other-workspace-repo/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-other-workspace-repo/vulnerabilities/vuln-0001.md" <<'EOS' + **Severity:** CRITICAL + **Target:** File: /workspace/other-repo/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java +EOS + echo "Penetration test failed: other workspace repo target" + exit 1 + ;; + pr-critical-manifest-only-pom|pr-critical-manifest-only-pom-test-override|pr-critical-manifest-only-pom-same-head-different-pr|pr-critical-manifest-only-pom-current-pr-authoritative) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-manifest-only/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-manifest-only/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Location 1: +pom.xml:8 +EOS + echo "Penetration test failed: manifest-only critical finding" + exit 1 + ;; + pr-critical-manifest-only-pom-after-fallback-authoritative) + case "${STRIX_LLM:-}" in + vertex_ai/timeout-primary) + echo "litellm.exceptions.Timeout: primary model timed out" + exit 1 + ;; + vertex_ai/fallback-one) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-manifest-only-after-fallback/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-manifest-only-after-fallback/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: CRITICAL +Location 1: +pom.xml:8 +EOS + echo "Penetration test failed: manifest-only critical finding after fallback" + exit 1 + ;; + *) + echo "Error: pr-critical-manifest-only-pom-after-fallback-authoritative unexpected model (${STRIX_LLM:-})" >&2 + exit 53 + ;; + esac + ;; + pr-critical-manifest-only-pom-console-only-after-fallback-authoritative) + case "${STRIX_LLM:-}" in + vertex_ai/timeout-primary) + echo "litellm.exceptions.Timeout: primary model timed out" + exit 1 + ;; + vertex_ai/fallback-one) + echo "Severity: CRITICAL" + echo "Location 1:" + echo "pom.xml:59" + echo "Penetration test failed: manifest-only critical finding after fallback (console-only)" + exit 1 + ;; + *) + echo "Error: pr-critical-manifest-only-pom-console-only-after-fallback-authoritative unexpected model (${STRIX_LLM:-})" >&2 + exit 54 + ;; + esac + ;; + pr-critical-manifest-only-pom-console-target-only-after-fallback-authoritative) + case "${STRIX_LLM:-}" in + vertex_ai/timeout-primary) + echo "litellm.exceptions.Timeout: primary model timed out" + exit 1 + ;; + vertex_ai/fallback-one) + echo "Severity: CRITICAL" + echo "Target: /workspace/$(basename "$target_path")/pom.xml" + echo "Penetration test failed: manifest-only critical finding after fallback (console target-only)" + exit 1 + ;; + *) + echo "Error: pr-critical-manifest-only-pom-console-target-only-after-fallback-authoritative unexpected model (${STRIX_LLM:-})" >&2 + exit 56 + ;; + esac + ;; + pr-low-markdown-plus-console-critical-manifest-after-fallback-authoritative) + case "${STRIX_LLM:-}" in + vertex_ai/timeout-primary) + echo "litellm.exceptions.Timeout: primary model timed out" + exit 1 + ;; + vertex_ai/fallback-one) + mkdir -p "$STRIX_REPORTS_DIR/fake-pr-manifest-mixed-after-fallback/vulnerabilities" + cat >"$STRIX_REPORTS_DIR/fake-pr-manifest-mixed-after-fallback/vulnerabilities/vuln-0001.md" <<'EOS' +Severity: LOW +Location 1: +pom.xml:8 +EOS + echo "Severity: CRITICAL" + echo "Location 1:" + echo "pom.xml:59" + echo "Penetration test failed: manifest-only critical finding after fallback (mixed file+console)" + exit 1 + ;; + *) + echo "Error: pr-low-markdown-plus-console-critical-manifest-after-fallback-authoritative unexpected model (${STRIX_LLM:-})" >&2 + exit 55 + ;; + esac + ;; + pr-changed-scope-bounded) + if [ -z "$target_path" ]; then + echo "Error: target path missing" >&2 + exit 41 + fi + if [ ! -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" ]; then + echo "Error: changed file missing from bounded target path ($target_path)" >&2 + exit 42 + fi + if [ -e "$target_path/sync-module-system/smart-crawling-common/src/main/java/org/empasy/sync/common/system/util/JwtUtil.java" ]; then + echo "Error: unrelated file leaked into bounded target path ($target_path)" >&2 + exit 43 + fi + echo "scan ok with bounded changed-file scope" + exit 0 + ;; + pr-python-scope-context) + if [ ! -f "$target_path/backend/api/emails.py" ]; then + echo "Error: changed backend file missing from scoped target ($target_path)" >&2 + exit 57 + fi + if [ ! -f "$target_path/backend/core/config.py" ]; then + echo "Error: backend core config context missing from scoped target ($target_path)" >&2 + exit 58 + fi + if [ ! -f "$target_path/backend/core/runtime_secrets.py" ]; then + echo "Error: backend runtime secrets context missing from scoped target ($target_path)" >&2 + exit 62 + fi + if [ ! -f "$target_path/backend/api/search.py" ]; then + echo "Error: backend search router context missing from scoped target ($target_path)" >&2 + exit 63 + fi + if [ ! -f "$target_path/backend/db/session.py" ]; then + echo "Error: backend db session context missing from scoped target ($target_path)" >&2 + exit 59 + fi + if [ ! -f "$target_path/backend/services/exceptions.py" ]; then + echo "Error: backend service exceptions context missing from scoped target ($target_path)" >&2 + exit 60 + fi + if ! grep -Fq -- 'ensure_organization_access(auth_context, config.organization_id)' "$target_path/backend/api/runner_config.py"; then + echo "Error: backend organization access context missing from scoped target ($target_path)" >&2 + exit 61 + fi + echo "scan ok with python dependency scope" + exit 0 + ;; + pr-changed-scope-full) + attempt="0" + if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then + attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" + fi + attempt="$((attempt + 1))" + echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" + if [ "$attempt" -eq 1 ]; then + if [ ! -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" ]; then + echo "Error: full-set scope missing controller file ($target_path)" >&2 + exit 44 + fi + if [ ! -f "$target_path/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java" ]; then + echo "Error: full-set scope missing playwright file ($target_path)" >&2 + exit 45 + fi + if [ ! -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl/SysUserServiceImpl.java" ]; then + echo "Error: full-set scope missing service impl file ($target_path)" >&2 + exit 46 + fi + echo "scan ok with full changed-file scope" + exit 0 + fi + echo "Error: unexpected full-scope scan attempt $attempt" >&2 + exit 50 + ;; + pr-changed-scope-full-set) + attempt="0" + if [ -f "${FAKE_STRIX_STATE_FILE:?}" ]; then + attempt="$(cat "${FAKE_STRIX_STATE_FILE:?}")" + fi + attempt="$((attempt + 1))" + echo "$attempt" > "${FAKE_STRIX_STATE_FILE:?}" + if [ "$attempt" -eq 1 ] && \ + [ -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" ] && \ + [ -f "$target_path/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java" ] && \ + [ -f "$target_path/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl/SysUserServiceImpl.java" ] && \ + [ -f "$target_path/sync-module-system/smart-crawling-common/src/main/java/org/empasy/sync/common/system/util/JwtUtil.java" ]; then + echo "scan ok with full configured PR scope" + exit 0 + fi + echo "Error: PR changed-file scope did not include the complete changed-file set on one scan attempt $attempt ($target_path)" >&2 + exit 54 + ;; + pr-large-scope-full-set) + echo "scan ok with large full PR scope" + exit 0 + ;; + pr-changed-scope-includes-ci-dependency) + if [ -f "$target_path/scripts/ci/strix_quick_gate.sh" ] && [ -f "$target_path/scripts/ci/strix_model_utils.sh" ]; then + echo "scan ok with CI support dependency" + exit 0 + fi + echo "Error: PR changed-file scope missing CI support dependency ($target_path)" >&2 + exit 55 + ;; + pr-deployment-scope-entrypoint-context) + if [ ! -f "$target_path/Dockerfile" ]; then + echo "Error: deployment scope missing Dockerfile ($target_path)" >&2 + exit 56 + fi + if [ ! -f "$target_path/backend/scripts/docker_entrypoint.sh" ]; then + echo "Error: deployment scope missing backend/scripts/docker_entrypoint.sh ($target_path)" >&2 + exit 57 + fi + if [ ! -f "$target_path/backend/core/runtime_secrets.py" ]; then + echo "Error: deployment scope missing backend/core/runtime_secrets.py ($target_path)" >&2 + exit 60 + fi + if ! grep -Fq -- 'CMD ["/app/scripts/docker_entrypoint.sh"]' "$target_path/Dockerfile"; then + echo "Error: deployment Dockerfile does not reference docker_entrypoint.sh ($target_path)" >&2 + exit 58 + fi + if ! grep -Fq -- 'Starting backend (uvicorn :8000)' "$target_path/backend/scripts/docker_entrypoint.sh"; then + echo "Error: deployment entrypoint context did not include trusted script content ($target_path)" >&2 + exit 59 + fi + echo "scan ok with deployment entrypoint context" + exit 0 + ;; + pr-rust-workspace-context) + for rust_context in Cargo.toml Cargo.lock rust-toolchain.toml deny.toml; do + if [ ! -f "$target_path/$rust_context" ]; then + echo "Error: Rust workflow scope missing $rust_context ($target_path)" >&2 + exit 61 + fi + done + if ! grep -Fq -- 'name = "trusted-workspace"' "$target_path/Cargo.toml"; then + echo "Error: Rust workflow context did not preserve trusted Cargo content ($target_path)" >&2 + exit 62 + fi + echo "scan ok with Rust workspace context" + exit 0 + ;; + *) + echo "unknown scenario ${FAKE_STRIX_SCENARIO:?}" >&2 + exit 8 + ;; +esac +EOF + chmod +x "$fake_strix" + + cat >"$fake_gh" <<'EOF' +#!/usr/bin/env bash +set -euo pipefail + +printf '%s\n' "${GH_TOKEN-}" >> "${FAKE_GH_TOKEN_LOG:?}" + +if [ "${1-}" != "api" ]; then + echo "unexpected gh command: $*" >&2 + exit 90 +fi + +if [ -z "${FAKE_GH_API_RESPONSE_FILE:-}" ]; then + echo "missing FAKE_GH_API_RESPONSE_FILE" >&2 + exit 91 +fi + +cat -- "${FAKE_GH_API_RESPONSE_FILE}" +EOF + chmod +x "$fake_gh" + + local effective_event_name="$github_event_name" + if [ -z "$effective_event_name" ]; then + effective_event_name="$event_name_override" + fi + + # Scenario-specific source-tree setup so is_hallucinated_endpoint_finding() + # can locate "real" endpoints inside the self-contained temp workspace. + if [ "$effective_event_name" = "pull_request" ]; then + mkdir -p "$repo_root_dir/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller" + mkdir -p "$repo_root_dir/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl" + mkdir -p "$repo_root_dir/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service" + mkdir -p "$repo_root_dir/sync-module-system/smart-crawling-common/src/main/java/org/empasy/sync/common/system/util" + echo '' >"$repo_root_dir/pom.xml" + mkdir -p "$repo_root_dir/sync-module-system/smart-crawling-server/src/main/resources/flyway" + echo 'class ChangedController {}' >"$repo_root_dir/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" + echo 'class BaselineUserService {}' >"$repo_root_dir/sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/service/impl/SysUserServiceImpl.java" + echo 'class ChangedPlaywright {}' >"$repo_root_dir/sync-module-system/smart-crawling-playwright/src/main/java/org/empasy/sync/mcp/service/PlayWrightService.java" + echo 'class ChangedJwtUtil {}' >"$repo_root_dir/sync-module-system/smart-crawling-common/src/main/java/org/empasy/sync/common/system/util/JwtUtil.java" + mkdir -p "$repo_root_dir/frontend/src/app/labels/[slug]" + echo 'export default function Page() { return null }' >"$repo_root_dir/frontend/src/app/labels/[slug]/page.tsx" + mkdir -p "$repo_root_dir/src" + echo 'print("unsafe name")' >"$repo_root_dir/src/unsafe name.py" + mkdir -p "$repo_root_dir/backend/services" + echo 'async def send_email(*args, **kwargs): return None' >"$repo_root_dir/backend/services/email_client.py" + echo 'def parse_eml(*args): return {}' >"$repo_root_dir/backend/services/email_parser.py" + if [ -n "$current_pr_number" ]; then + cat >"$event_payload_file" <"$repo_root_dir/sync-module-system/smart-crawling-server/src/main/resources/flyway/V4__ccf_scenario.sql" + echo '-- legacy flyway file' >"$repo_root_dir/sync-module-system/smart-crawling-server/src/main/resources/flyway/V16__hash_oauth2_registered_client_secret.sql" + echo '-- changed flyway file' >"$repo_root_dir/sync-module-system/smart-crawling-server/src/main/resources/flyway/V24__update_search_expression_team_keyword_id.sql" + fi + + if [ "$scenario" = "vertex-primary-existing-endpoint-nonrecoverable" ]; then + echo 'GET /api/status' >"$repo_root_dir/src/routes.txt" + elif [ "$scenario" = "multi-source-dirs-existing-endpoint" ]; then + # Endpoint lives in api/ (not src/), validating multi-dir scanning. + mkdir -p "$repo_root_dir/api" + echo 'GET /api/status' >"$repo_root_dir/api/routes.txt" + elif [ "$scenario" = "endpoint-in-excluded-dir" ]; then + # Endpoint /api/hidden-secret exists ONLY inside excluded directories + # (.git/ and node_modules/). The grep excludes must prevent matching, + # so the finding is treated as hallucinated → fallback allowed. + mkdir -p "$repo_root_dir/.git/refs" + echo 'GET /api/hidden-secret' >"$repo_root_dir/.git/refs/leaked.txt" + mkdir -p "$repo_root_dir/node_modules/fake-pkg" + echo 'GET /api/hidden-secret' >"$repo_root_dir/node_modules/fake-pkg/index.js" + elif [ "$scenario" = "pr-stale-source-claim-fallback-success" ]; then + mkdir -p "$repo_root_dir/backend/db" + cat >"$repo_root_dir/backend/db/models.py" <<'EOS' +from sqlalchemy.orm import Mapped, mapped_column + +class EncryptedString: + pass + +class WorkspaceRunnerConfig: + registration_token: Mapped[str | None] = mapped_column( + EncryptedString, nullable=True + ) +EOS + elif [ "$scenario" = "pr-stale-snapshot-snippet-fallback-success" ]; then + mkdir -p "$repo_root_dir/backend/app/api" + cat >"$repo_root_dir/backend/app/api/snapshots.py" <<'EOS' +from fastapi import HTTPException + + +async def _get_authorized_snapshot(session, schema_snapshot_uuid, user): + project_space_uuid = await session.scalar("select project space") + if project_space_uuid is None: + return None + try: + await require_project_member(session, project_space_uuid, user.user_account_uuid) + except HTTPException as exc: + if exc.status_code == 403: + return None + raise + return await session.get("SchemaSnapshot", schema_snapshot_uuid) + + +async def get_snapshot(schema_snapshot_uuid, user, session): + snap = await _get_authorized_snapshot(session, schema_snapshot_uuid, user) + if snap is None: + return {"status": "not_found", "snapshot_json": None} + data = await session.get("SchemaSnapshotData", schema_snapshot_uuid) + return {"status": snap.status, "snapshot_json": data.snapshot_json if data else None} +EOS + elif [ "$scenario" = "pr-stale-source-plus-real-finding-blocks" ]; then + mkdir -p "$repo_root_dir/backend/db" "$repo_root_dir/backend/api" + cat >"$repo_root_dir/backend/db/models.py" <<'EOS' +from sqlalchemy.orm import Mapped, mapped_column + +class EncryptedString: + pass + +class WorkspaceRunnerConfig: + registration_token: Mapped[str | None] = mapped_column( + EncryptedString, nullable=True + ) +EOS + echo 'def real_changed_endpoint(): pass' >"$repo_root_dir/backend/api/emails.py" + elif [ "$scenario" = "pr-changed-finding-with-retry-marker-blocks" ]; then + mkdir -p "$repo_root_dir/backend/api" + echo 'def real_changed_endpoint(): pass' >"$repo_root_dir/backend/api/emails.py" + elif [ "$scenario" = "pr-stale-report-plus-inline-changed-finding-blocks" ]; then + mkdir -p "$repo_root_dir/backend/db" "$repo_root_dir/backend/api" + cat >"$repo_root_dir/backend/db/models.py" <<'EOS' +from sqlalchemy.orm import Mapped, mapped_column + +class EncryptedString: + pass + +class WorkspaceRunnerConfig: + registration_token: Mapped[str | None] = mapped_column( + EncryptedString, nullable=True + ) +EOS + echo 'def real_changed_endpoint(): pass' >"$repo_root_dir/backend/api/emails.py" + elif [ "$scenario" = "pr-changed-scope-bounded" ]; then + echo 'class Unrelated {}' >"$repo_root_dir/sync-module-system/smart-crawling-common/src/main/java/org/empasy/sync/common/system/util/JwtUtil.java" + elif [ "$scenario" = "pr-python-scope-context" ]; then + mkdir -p "$repo_root_dir/backend/api" "$repo_root_dir/backend/core" "$repo_root_dir/backend/db" "$repo_root_dir/backend/services" + touch "$repo_root_dir/backend/api/__init__.py" + touch "$repo_root_dir/backend/core/__init__.py" + touch "$repo_root_dir/backend/db/__init__.py" + touch "$repo_root_dir/backend/services/__init__.py" + echo 'from db.session import get_db' >"$repo_root_dir/backend/api/emails.py" + echo 'from api.auth import ensure_organization_access' >"$repo_root_dir/backend/api/runner_config.py" + echo 'ensure_organization_access(auth_context, config.organization_id)' >>"$repo_root_dir/backend/api/runner_config.py" + echo 'router = object()' >"$repo_root_dir/backend/api/search.py" + echo 'TRUSTED_CONFIG = True' >"$repo_root_dir/backend/core/config.py" + echo 'class LocalError(Exception): pass' >"$repo_root_dir/backend/core/exceptions.py" + echo 'def validate_auth_session_hmac_secret_value(value): return value' >"$repo_root_dir/backend/core/runtime_secrets.py" + echo 'engine = object()' >"$repo_root_dir/backend/db/session.py" + echo 'class Email: pass' >"$repo_root_dir/backend/db/models.py" + echo 'class ServiceError(Exception): pass' >"$repo_root_dir/backend/services/exceptions.py" + echo 'async def extract_backup_async(*args): return []' >"$repo_root_dir/backend/services/archive.py" + echo 'def parse_eml(*args): return {}' >"$repo_root_dir/backend/services/email_parser.py" + echo 'async def generate_embeddings(*args): return []' >"$repo_root_dir/backend/services/embedding.py" + echo 'async def assign_thread_id(*args, **kwargs): return "thread"' >"$repo_root_dir/backend/services/threading_service.py" + echo 'async def send_email(*args, **kwargs): return None' >"$repo_root_dir/backend/services/email_client.py" + echo 'pytest==0' >"$repo_root_dir/backend/requirements.txt" + elif [ "$scenario" = "pr-deployment-scope-entrypoint-context" ] || [ "$scenario" = "pr-baseline-critical-extensionless-dockerfile-target" ]; then + mkdir -p "$repo_root_dir/.github/workflows" "$repo_root_dir/backend/api" "$repo_root_dir/backend/core" "$repo_root_dir/backend/scripts" "$repo_root_dir/frontend" + echo 'name: OpenCode Review' >"$repo_root_dir/.github/workflows/opencode-review.yml" + cat >"$repo_root_dir/Dockerfile" <<'EOS' +FROM python:3.11-slim AS backend-runtime +WORKDIR /app +COPY backend /app/ +FROM backend-runtime +RUN chmod +x /app/scripts/docker_entrypoint.sh +CMD ["/app/scripts/docker_entrypoint.sh"] +EOS + cat >"$repo_root_dir/backend/scripts/docker_entrypoint.sh" <<'EOS' +#!/usr/bin/env bash +echo "Starting backend (uvicorn :8000)" +EOS + echo 'router = object()' >"$repo_root_dir/backend/api/auth.py" + echo 'class Settings: pass' >"$repo_root_dir/backend/core/config.py" + echo 'def validate_auth_session_hmac_secret_value(value): return value' >"$repo_root_dir/backend/core/runtime_secrets.py" + echo 'app = object()' >"$repo_root_dir/backend/main.py" + touch "$repo_root_dir/frontend/Dockerfile" + echo '{"scripts":{"start":"next start"}}' >"$repo_root_dir/frontend/package.json" + touch "$repo_root_dir/frontend/next.config.ts" + touch "$repo_root_dir/frontend/postcss.config.mjs" + touch "$repo_root_dir/docker-compose.yml" + touch "$repo_root_dir/render.yaml" + echo '0.0.0' >"$repo_root_dir/VERSION" + elif [ "$scenario" = "pr-rust-workspace-context" ]; then + mkdir -p "$repo_root_dir/.github/workflows" "$repo_root_dir/src" + echo 'name: Rust CI' >"$repo_root_dir/.github/workflows/rust.yml" + cat >"$repo_root_dir/Cargo.toml" <<'EOS' +[package] +name = "trusted-workspace" +version = "0.1.0" +EOS + echo '# trusted lock' >"$repo_root_dir/Cargo.lock" + echo '[toolchain]' >"$repo_root_dir/rust-toolchain.toml" + echo '[advisories]' >"$repo_root_dir/deny.toml" + echo 'fn main() {}' >"$repo_root_dir/src/main.rs" + elif [ "$scenario" = "github-models-fallback-dockerfile-test-baseline-before-next-success-continues" ]; then + mkdir -p "$repo_root_dir/.github/workflows" + cat >"$repo_root_dir/.github/workflows/build-ci-image.yml" <<'EOS' +name: Build CI image +jobs: + build: + steps: + - uses: docker/build-push-action@example + with: + file: ./Dockerfile.test +EOS + cat >"$repo_root_dir/Dockerfile.test" <<'EOS' +FROM python:3.13-slim +HEALTHCHECK CMD python -V || exit 1 +EOS + elif [ "$scenario" = "pr-critical-changed-internal-dotdir-target" ]; then + mkdir -p "$repo_root_dir/.github/workflows" + echo 'name: OpenCode Review' >"$repo_root_dir/.github/workflows/opencode-review.yml" + elif [ "$scenario" = "pr-critical-changed-json-target" ]; then + mkdir -p "$repo_root_dir/frontend/src/components" + echo 'export function CalendarLayout() { return null }' >"$repo_root_dir/frontend/src/components/CalendarLayout.tsx" + elif [ "$scenario" = "pr-changed-file-nonintersecting-line" ]; then + mkdir -p "$repo_root_dir/frontend/src" + { + echo 'import React from "react";' + for line_number in $(seq 2 140); do + printf 'const value%s = %s;\n' "$line_number" "$line_number" + done + } >"$repo_root_dir/frontend/src/App.tsx" + elif [ "$scenario" = "opencode-documented-env-api-key-fallback-success" ]; then + mkdir -p "$repo_root_dir/.github/workflows" + cat >"$repo_root_dir/.github/workflows/opencode-review.yml" <<'EOS' +name: OpenCode Review +config: | + { + "provider": { + "github-models": { + "options": { + "apiKey": "{env:STRIX_GITHUB_MODELS_TOKEN}" + } + } + } + } +EOS + elif [ "$scenario" = "generic-github-actions-workflow-fallback-success" ]; then + mkdir -p "$repo_root_dir/.github/workflows" + cat >"$repo_root_dir/.github/workflows/strix.yml" <<'EOS' +name: Strix Security Scan + +permissions: + actions: read + contents: read + models: read + +jobs: + strix: + steps: + - name: Fetch pull request head for trusted scan + run: | + if ! [[ "$PR_HEAD_SHA" =~ ^[0-9a-fA-F]{40}$ ]]; then + exit 1 + fi + if [ -n "$PR_BASE_SHA" ] && ! [[ "$PR_BASE_SHA" =~ ^[0-9a-fA-F]{40}$ ]]; then + exit 1 + fi + - name: Gate Strix secrets + run: | + echo '::error::STRIX_LLM must select GitHub Models openai/gpt-5 or newer, direct OpenAI GPT-5.4 or newer, OpenRouter openrouter/free, or an approved organization Vertex AI model.' + - name: Mask LLM API key + run: | + sanitized="$(printf '%s' "$LLM_API_KEY" | tr -d '\r\n')" + echo "::add-mask::${sanitized}" + - name: Prepare LLM API key input file + run: | + umask 077 + printf '%s' "$sanitized" > "$RUNNER_TEMP/llm_api_key.txt" +EOS + elif [ "$scenario" = "pr-large-scope-full-set" ]; then + mkdir -p "$repo_root_dir/backend/large-scope" + local large_scope_index + for large_scope_index in $(seq 1 38); do + printf 'file %s\n' "$large_scope_index" >"$repo_root_dir/backend/large-scope/file-$large_scope_index.py" + done + elif [ "$scenario" = "scan-working-directory-isolated" ]; then + mkdir -p "$repo_root_dir/backend/app/pg_introspect" + printf '%s\n' 'HEAD_INTROSPECT_SHOULD_BE_SCANNED' >"$repo_root_dir/backend/app/pg_introspect/introspect.py" + printf '%s\n' 'TRUSTED_DSN_GUARD_CONTEXT_SHOULD_BE_SCANNED' >"$repo_root_dir/backend/app/pg_introspect/dsn_guard.py" + fi + + local scenario_base_sha="" + local scenario_head_sha="" + if [ "$scenario" = "pr-changed-file-nonintersecting-line" ]; then + ( + cd "$repo_root_dir" + git init -q + git config user.email "ci@example.com" + git config user.name "CI" + git add frontend/src/App.tsx + git commit -qm 'base commit' + python3 - <<'PY' +from pathlib import Path + +path = Path("frontend/src/App.tsx") +lines = path.read_text(encoding="utf-8").splitlines() +lines[119] = f"{lines[119]} // changed search line" +path.write_text("\n".join(lines) + "\n", encoding="utf-8") +PY + git add frontend/src/App.tsx + git commit -qm 'head commit' + ) + scenario_base_sha="$(git -C "$repo_root_dir" rev-list --max-parents=0 HEAD)" + scenario_head_sha="$(git -C "$repo_root_dir" rev-parse HEAD)" + fi + + set +e + local env_cmd=( + PATH="$untrusted_bin_dir:$bin_dir:$PATH" + STRIX_EXECUTABLE_PATH="$fake_strix" + FAKE_STRIX_PATH_HIJACK_LOG="$path_hijack_log" + STRIX_INPUT_FILE_ROOT="$tmp_dir" + GITHUB_EVENT_NAME="" + GITHUB_EVENT_PATH="" + FAKE_STRIX_SCENARIO="$scenario" + FAKE_STRIX_CALL_LOG="$call_log" + FAKE_STRIX_API_BASE_LOG="$api_base_log" + FAKE_STRIX_TARGET_LOG="$target_log" + FAKE_STRIX_RUNTIME_ENV_LOG="$runtime_env_log" + FAKE_STRIX_TIMEOUT_SLEEP_SECONDS="$TIMEOUT_TEST_FAKE_SLEEP_SECONDS" + STRIX_LLM_DEFAULT_PROVIDER="$default_provider" + FAKE_STRIX_STATE_FILE="$state_file" + STRIX_TRANSIENT_RETRY_PER_MODEL="$transient_retry_per_model" + STRIX_TRANSIENT_RETRY_BACKOFF_SECONDS="$transient_retry_backoff_seconds" + STRIX_PROCESS_TIMEOUT_SECONDS="$process_timeout_seconds" + STRIX_TOTAL_TIMEOUT_SECONDS="$total_timeout_seconds" + STRIX_FAIL_ON_MIN_SEVERITY="$min_fail_severity" + STRIX_REPORTS_DIR="$repo_root_dir/strix_runs" + STRIX_TARGET_PATH="$effective_target_path" + ) + if [ "$scenario" = "runtime-env-forwarding" ] || [ "$scenario" = "custom-openai-compatible-preserves-effort" ]; then + env_cmd+=( + LLM_TIMEOUT="90" + STRIX_MEMORY_COMPRESSOR_TIMEOUT="10" + STRIX_REASONING_EFFORT="minimal" + STRIX_LLM_MAX_RETRIES="1" + GEMINI_LOCATION="GLOBAL" + UNRELATED_SECRET="should-not-forward" + ) + fi + if [ "$scenario" = "pr-executable-integrity-mismatch" ]; then + env_cmd+=( + IS_PR_EVIDENCE_RUN="true" + STRIX_EXECUTABLE_ROOT="$bin_dir" + STRIX_EXECUTABLE_SHA256="0000000000000000000000000000000000000000000000000000000000000000" + ) + fi + if [ "$scenario" = "pr-executable-root-group-writable" ]; then + local fake_strix_sha256 + fake_strix_sha256="$(python3 - "$fake_strix" <<'PY' +import hashlib +from pathlib import Path +import sys + +print(hashlib.sha256(Path(sys.argv[1]).read_bytes()).hexdigest()) +PY +)" + env_cmd+=( + IS_PR_EVIDENCE_RUN="true" + STRIX_EXECUTABLE_ROOT="$bin_dir" + STRIX_EXECUTABLE_SHA256="$fake_strix_sha256" + ) + chmod 0775 "$bin_dir" + fi + if [ "$scenario" = "pr-executable-group-writable" ]; then + chmod 0775 "$fake_strix" + fi + if [ "$scenario" = "report-known-internal-warning-sanitized" ]; then + env_cmd+=( + FAKE_STRIX_OUTSIDE_REPORT_DIR="$repo_root_dir/outside-strix-report" + ) + fi + if [ "$scenario" = "nvidia-rate-limit-openai-direct-fallback-clears-api-base" ]; then + printf '%s' 'openai-fallback-token' >"$tmp_dir/openai_fallback_key.txt" + env_cmd+=(STRIX_OPENAI_FALLBACK_KEY_FILE="$tmp_dir/openai_fallback_key.txt") + env_cmd+=(STRIX_REASONING_EFFORT="high") + fi + if [ "$scenario" = "openai-direct-quota-github-models-fallback-success" ]; then + printf '%s' 'https://models.github.ai/inference' >"$tmp_dir/github_models_api_base.txt" + printf '%s' 'github-models-fallback-token' >"$tmp_dir/github_models_key.txt" + env_cmd+=(STRIX_GITHUB_MODELS_API_BASE_FILE="$tmp_dir/github_models_api_base.txt") + env_cmd+=(STRIX_GITHUB_MODELS_KEY_FILE="$tmp_dir/github_models_key.txt") + fi + if [ "$min_fail_severity" = "__UNSET__" ]; then + local next_env_cmd=() + local env_pair + for env_pair in "${env_cmd[@]}"; do + case "$env_pair" in + STRIX_FAIL_ON_MIN_SEVERITY=*) + continue + ;; + esac + next_env_cmd+=("$env_pair") + done + env_cmd=("${next_env_cmd[@]}") + fi + printf '%s' "$initial_model" >"$strix_llm_file" + env_cmd+=(STRIX_LLM_FILE="$strix_llm_file") + printf '%s' 'dummy' >"$llm_api_key_file" + env_cmd+=(LLM_API_KEY_FILE="$llm_api_key_file") + env_cmd+=(STRIX_DISABLE_PR_SCOPING="$disable_pr_scoping") + env_cmd+=(STRIX_FAIL_ON_PROVIDER_SIGNAL="$fail_on_provider_signal") + local llm_api_base_source="$raw_llm_api_base" + if [ -z "$llm_api_base_source" ] && [ -n "$initial_llm_api_base" ]; then + llm_api_base_source="$initial_llm_api_base" + fi + if [ -n "$llm_api_base_source" ]; then + printf '%s' "$llm_api_base_source" >"$llm_api_base_file" + env_cmd+=(LLM_API_BASE_FILE="$llm_api_base_file") + fi + # Only export fallback variables when a non-empty value is provided so the + # gate's ${VAR+x} checks correctly distinguish "unset → use defaults" from + # "set to empty → disable fallbacks". + if [ -n "$fallback_models" ]; then + env_cmd+=(STRIX_VERTEX_FALLBACK_MODELS="$fallback_models") + fi + case "$gemini_fallback_models" in + __SAME_AS_FALLBACK_MODELS__) + if [ -n "$fallback_models" ]; then + env_cmd+=(STRIX_GEMINI_FALLBACK_MODELS="$fallback_models") + fi + ;; + __UNSET__) + ;; + *) + if [ -n "$gemini_fallback_models" ]; then + env_cmd+=(STRIX_GEMINI_FALLBACK_MODELS="$gemini_fallback_models") + fi + ;; + esac + if [ -n "$generic_fallback_models" ]; then + env_cmd+=(STRIX_FALLBACK_MODELS="$generic_fallback_models") + fi + if [ -n "$custom_source_dirs" ]; then + env_cmd+=(STRIX_SOURCE_DIRS="$custom_source_dirs") + fi + : "$legacy_scope_size_ignored" + if [ -n "$github_event_name" ]; then + env_cmd+=(GITHUB_EVENT_NAME="$github_event_name") + fi + if [ -n "$event_name_override" ]; then + env_cmd+=(EVENT_NAME="$event_name_override") + fi + if [ -n "$test_pr_sca_status_override" ]; then + env_cmd+=(STRIX_TEST_PR_SCA_STATUS_OVERRIDE="$test_pr_sca_status_override") + fi + if [ -n "$current_pr_number" ]; then + env_cmd+=(GITHUB_EVENT_PATH="$event_payload_file") + env_cmd+=(GITHUB_REPOSITORY="octo-org/smart-crawling-server") + env_cmd+=(PR_BASE_SHA="test-base-sha") + env_cmd+=(PR_HEAD_SHA="test-head-sha") + env_cmd+=(GH_TOKEN="g""hs_test_token") + fi + if [ -n "$scenario_base_sha" ] && [ -n "$scenario_head_sha" ]; then + env_cmd+=(PR_BASE_SHA="$scenario_base_sha") + env_cmd+=(PR_HEAD_SHA="$scenario_head_sha") + fi + if [ -n "$authoritative_sca_runs_json" ]; then + local gh_api_response_file="$tmp_dir/gh-api-response.json" + printf '%s\n' "$authoritative_sca_runs_json" >"$gh_api_response_file" + env_cmd+=(FAKE_GH_API_RESPONSE_FILE="$gh_api_response_file") + env_cmd+=(FAKE_GH_TOKEN_LOG="$gh_token_log") + fi + if [ "$changed_files_override" = "__SET_EMPTY__" ]; then + env_cmd+=(STRIX_TEST_CHANGED_FILES_OVERRIDE="") + elif [ -n "$changed_files_override" ]; then + env_cmd+=(STRIX_TEST_CHANGED_FILES_OVERRIDE="$changed_files_override") + fi + ( + cd "$repo_root_dir" + env \ + -u GITHUB_EVENT_NAME \ + -u GITHUB_EVENT_PATH \ + -u STRIX_TEST_CHANGED_FILES_OVERRIDE \ + -u STRIX_VERTEX_FALLBACK_MODELS \ + -u STRIX_GEMINI_FALLBACK_MODELS \ + -u STRIX_FALLBACK_MODELS \ + -u STRIX_OPENAI_FALLBACK_KEY_FILE \ + -u STRIX_OPENAI_FALLBACK_API_BASE_FILE \ + "${env_cmd[@]}" \ + bash "./scripts/ci/strix_quick_gate.sh" >"$output_log" 2>&1 + ) + local rc=$? + set -e + + assert_equals "$expected_exit" "$rc" "scenario=$scenario exit code" + if [ "$expected_exit" != "$rc" ]; then + echo "scenario=$scenario gate output:" >&2 + sed 's/^/ | /' "$output_log" >&2 + fi + + if [ -n "$expected_message" ]; then + case "$expected_message" in + REGEX:*) + assert_file_matches "$output_log" "${expected_message#REGEX:}" "scenario=$scenario output" + ;; + *) + assert_file_contains "$output_log" "$expected_message" "scenario=$scenario output" + ;; + esac + fi + + local call_count + call_count="0" + if [ -f "$call_log" ]; then + call_count="$(wc -l <"$call_log" | tr -d ' ')" + fi + assert_equals "$expected_calls" "$call_count" "scenario=$scenario strix call count" + if [ -e "$path_hijack_log" ]; then + record_failure "scenario=$scenario selected a PATH-controlled Strix executable instead of STRIX_EXECUTABLE_PATH" + fi + + if [ -n "$expected_model_sequence" ]; then + local actual_model_sequence="" + if [ -f "$call_log" ]; then + while IFS= read -r model; do + if [ -n "$actual_model_sequence" ]; then + actual_model_sequence="${actual_model_sequence}|$model" + else + actual_model_sequence="$model" + fi + done <"$call_log" + fi + + assert_equals "$expected_model_sequence" "$actual_model_sequence" "scenario=$scenario STRIX_LLM sequence" + fi + + if [ -n "$expected_api_base_sequence" ]; then + local actual_api_base_sequence="" + if [ -f "$api_base_log" ]; then + while IFS= read -r api_base; do + if [ -n "$actual_api_base_sequence" ]; then + actual_api_base_sequence="${actual_api_base_sequence}|$api_base" + else + actual_api_base_sequence="$api_base" + fi + done <"$api_base_log" + fi + + assert_equals "$expected_api_base_sequence" "$actual_api_base_sequence" "scenario=$scenario LLM_API_BASE sequence" + fi + + if [ "$scenario" = "runtime-env-forwarding" ]; then + assert_file_contains \ + "$runtime_env_log" \ + "LLM_TIMEOUT=90;STRIX_MEMORY_COMPRESSOR_TIMEOUT=10;STRIX_REASONING_EFFORT=minimal;STRIX_LLM_MAX_RETRIES=1;GEMINI_LOCATION=GLOBAL;PYTHONWARNINGS=ignore:Pydantic serializer warnings:UserWarning:pydantic.main;NPM_CONFIG_IGNORE_SCRIPTS=true;PNPM_CONFIG_IGNORE_SCRIPTS=true;YARN_ENABLE_SCRIPTS=false;UNRELATED_SECRET=" \ + "scenario=$scenario runtime env forwarding" + # Non-contextual-orchestrator providers (gemini here) never see the + # stream-disabling opt-in: it is scoped narrowly to the gateway that + # rejects stream_options.include_usage alongside tools. + assert_file_contains \ + "$runtime_env_log" \ + "LLM_DISABLE_STREAMING=" \ + "scenario=$scenario non-gateway providers keep real streaming" + fi + if [ "$scenario" = "custom-openai-compatible-preserves-effort" ]; then + assert_file_contains \ + "$runtime_env_log" \ + "STRIX_REASONING_EFFORT=minimal" \ + "scenario=$scenario custom compatible endpoint effort" + fi + if [ "$scenario" = "contextual-orchestrator-gateway-model-qualification" ]; then + # contextual-orchestrator rejects stream_options.include_usage=true + # alongside tools; Strix's agent loop always sends both, so the gate + # routes this gateway through Strix's own LLM_DISABLE_STREAMING opt-in + # (single non-streaming get_response per turn) instead of streaming. + assert_file_contains \ + "$runtime_env_log" \ + "LLM_DISABLE_STREAMING=true" \ + "scenario=$scenario contextual-orchestrator gateway disables SDK streaming to avoid the stream_options+tools rejection" + fi + + if [ "$scenario" = "report-known-internal-warning-sanitized" ]; then + assert_file_not_contains \ + "$repo_root_dir/strix_runs/fake-known-internal-warning/strix.log" \ + "produced non-lifecycle final output" \ + "scenario=$scenario strips the known internal Strix warning from published artifacts" + assert_file_contains \ + "$repo_root_dir/strix_runs/fake-known-internal-warning/strix.log" \ + "finish_scan: completed scan with 0 vulnerability report(s)" \ + "scenario=$scenario keeps non-warning Strix report evidence" + assert_file_not_contains \ + "$repo_root_dir/strix_runs/fake-known-internal-warning-relative/strix.log" \ + "produced non-lifecycle final output" \ + "scenario=$scenario sanitizes relative scanner output before publication" + assert_file_contains \ + "$repo_root_dir/strix_runs/fake-known-internal-warning-relative/strix.log" \ + "finish_scan: completed scan with 0 vulnerability report(s)" \ + "scenario=$scenario publishes sanitized relative scanner evidence" + assert_file_contains \ + "$repo_root_dir/outside-strix-report/strix.log" \ + "outside report should not be rewritten" \ + "scenario=$scenario does not rewrite logs through symlinked report directories" + fi + + if [ "$scenario" = "report-known-internal-warning-variant-sanitized" ]; then + assert_file_not_contains \ + "$repo_root_dir/strix_runs/fake-known-internal-warning-variant/strix.log" \ + "ended a turn without a lifecycle tool call" \ + "scenario=$scenario strips the newer-wording known internal Strix warning from published artifacts" + assert_file_contains \ + "$repo_root_dir/strix_runs/fake-known-internal-warning-variant/strix.log" \ + "finish_scan: completed scan with 0 vulnerability report(s)" \ + "scenario=$scenario keeps non-warning Strix report evidence" + fi + + if [ "$scenario" = "github-models-primary-ratelimit-fallback-success" ]; then + assert_file_contains \ + "$output_log" \ + "GitHub Models rate limit detected for model 'openai/gpt-5'; skipping same-model retry and moving directly to fallback models or current-head neutral classification." \ + "scenario=$scenario logs why same-model retry was skipped" + assert_file_not_contains \ + "$output_log" \ + "Retrying model 'openai/gpt-5' due to rate limit" \ + "scenario=$scenario does not sleep in same-model retry after GitHub Models rate limiting" + fi + + if [ "$scenario" = "pr-changed-scope-full-set" ]; then + assert_internal_pr_scope_targets "$target_log" "$repo_root_dir" "$expected_calls" + fi + + rm -rf "$tmp_dir" +} + +run_gate_case_with_provider_signal_mode() { + local provider_signal_mode="$1" + shift + local args=("$@") + local default_args=( + "vertex_ai" + "__DEFAULT__" + "" + "0" + "CRITICAL" + "0" + "" + "" + "1200" + "0" + "" + "" + "" + "" + "0" + "" + "" + "" + "__SAME_AS_FALLBACK_MODELS__" + "" + ) + + while [ "${#args[@]}" -lt 28 ]; do + args+=("${default_args[${#args[@]} - 8]}") + done + args+=("$provider_signal_mode") + run_gate_case "${args[@]}" +} + +run_gate_case_allow_provider_signal() { + run_gate_case_with_provider_signal_mode "0" "$@" +} + +run_github_models_http410_case() { + local scenario="$1" + local expected_exit="$2" + local expected_calls="$3" + local expected_models="$4" + local expected_api_bases="$5" + local expected_message="${6-}" + + run_gate_case "$scenario" \ + "openai/gpt-5" \ + "" \ + "$expected_exit" \ + "$expected_message" \ + "$expected_calls" \ + "$expected_models" \ + "$expected_api_bases" \ + "openai" \ + "https://models.github.ai/inference" \ + "" \ + "0" \ + "CRITICAL" \ + "0" \ + "" \ + "" \ + "1200" \ + "0" \ + "" \ + "" \ + "" \ + "" \ + "0" \ + "" \ + "" \ + "" \ + "__SAME_AS_FALLBACK_MODELS__" \ + "deepseek/deepseek-r1-0528" \ + "1" +} + +run_filtered_gate_case_if_requested() { + case "${STRIX_TEST_CASE_FILTER:-}" in + "") + return 0 + ;; + success) + run_gate_case "success" \ + "vertex_ai/ready-primary" \ + "vertex_ai/fallback-one vertex_ai/fallback-two" \ + "0" \ + "scan ok" \ + "1" \ + "vertex_ai/ready-primary" \ + "" + ;; + contextual-orchestrator-missing-api-base-fails-closed) + run_gate_case "contextual-orchestrator-missing-api-base-fails-closed" \ + "orchestrator/free" \ + "" \ + "2" \ + "require LLM_API_BASE_FILE to select the pinned loopback gateway" \ + "0" \ + "" \ + "" \ + "contextual_orchestrator" \ + "" + ;; + contextual-orchestrator-gateway-model-qualification) + run_gate_case "contextual-orchestrator-gateway-model-qualification" \ + "orchestrator/free" \ + "" \ + "0" \ + "scan ok through contextual-orchestrator gateway" \ + "1" \ + "openai/orchestrator/free" \ + "http://127.0.0.1:18080/v1" \ + "contextual_orchestrator" \ + "http://127.0.0.1:18080/v1" + ;; + pr-rust-workspace-context) + run_gate_case "pr-rust-workspace-context" \ + "openai/gpt-4o-mini" \ + "" \ + "0" \ + "scan ok with Rust workspace context" \ + "1" \ + "openai/gpt-4o-mini" \ + "https://example.invalid" \ + "vertex_ai" \ + "__DEFAULT__" \ + "" \ + "0" \ + "CRITICAL" \ + "0" \ + "" \ + "" \ + "1200" \ + "0" \ + "pull_request" \ + ".github/workflows/rust.yml" + ;; + success-with-critical-report) + run_gate_case "success-with-critical-report" \ + "vertex_ai/ready-primary" \ + "" \ + "1" \ + "Strix exited successfully but emitted a vulnerability at or above 'CRITICAL'" \ + "1" \ + "vertex_ai/ready-primary" \ + "" + ;; + pr-executable-integrity-mismatch) + run_gate_case "pr-executable-integrity-mismatch" \ + "vertex_ai/ready-primary" \ + "" \ + "1" \ + "did not match the pinned SHA-256 digest" \ + "0" \ + "" \ + "" + ;; + pr-executable-group-writable) + run_gate_case "pr-executable-group-writable" \ + "vertex_ai/ready-primary" \ + "" \ + "1" \ + "must not be group/world writable" \ + "0" \ + "" \ + "" + ;; + pr-executable-root-group-writable) + run_gate_case "pr-executable-root-group-writable" \ + "vertex_ai/ready-primary" \ + "" \ + "1" \ + "pinned Strix installation root must not be group/world writable" \ + "0" \ + "" \ + "" + ;; + vertex-primary-hallucinated-endpoint-fallback-success) + run_gate_case "vertex-primary-hallucinated-endpoint-fallback-success" \ + "vertex_ai/hallucination-primary" \ + "vertex_ai/fallback-one vertex_ai/fallback-two" \ + "1" \ + "Strix quick scan failed with a non-recoverable error." \ + "1" \ + "vertex_ai/hallucination-primary" \ + "" + ;; + target-path-src-default-source-dirs) + run_gate_case "target-path-src-default-source-dirs" \ + "vertex_ai/hallucination-primary" \ + "vertex_ai/fallback-one vertex_ai/fallback-two" \ + "1" \ + "Strix quick scan failed with a non-recoverable error." \ + "1" \ + "vertex_ai/hallucination-primary" \ + "" \ + "vertex_ai" \ + "__DEFAULT__" \ + "" \ + "1" \ + "CRITICAL" \ + "0" \ + "__USE_SUBDIR_SRC__" \ + "" + ;; + vertex-ignores-untrusted-llm-api-base-file) + run_vertex_model_ignores_untrusted_llm_api_base_file_case + ;; + input-file-root-override-precedence) + run_input_file_root_override_takes_precedence_over_runner_temp_case + ;; + vertex-without-llm-api-key) + run_vertex_without_llm_api_key_case + ;; + vertex-with-llm-api-key-file-not-forwarded) + run_vertex_with_llm_api_key_file_does_not_forward_case + ;; + stale-report-does-not-bypass) + run_stale_report_case + ;; + symlink-report-does-not-bypass) + run_symlink_report_case + ;; + github-models-token-limit-fallback-success) + run_gate_case "github-models-token-limit-fallback-success" \ + "openai/gpt-5" \ + "" \ + "0" \ + "REGEX:Strix quick scan succeeded with fallback model 'github_models/deepseek/deepseek-v3-0324' in [0-9]+s\\." \ + "2" \ + "openai/gpt-5|openai/deepseek/deepseek-v3-0324" \ + "https://models.github.ai/inference|https://models.github.ai/inference" \ + "openai" \ + "https://models.github.ai/inference" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "github_models/deepseek/deepseek-v3-0324 github_models/deepseek/deepseek-r1-0528" + ;; + openrouter-502-fallback-retry-same-model-success) + run_gate_case "openrouter-502-fallback-retry-same-model-success" \ + "vertex_ai/missing-primary" \ + "openrouter/free vertex_ai/fallback-two" \ + "0" \ + "scan ok after OpenRouter 502 same-model retry" \ + "3" \ + "vertex_ai/missing-primary|openrouter/free|openrouter/free" \ + "|https://example.invalid|https://example.invalid" \ + "vertex_ai" \ + "__DEFAULT__" \ + "" \ + "1" + ;; + openrouter-502-distant-target-output-nonretryable) + run_gate_case "openrouter-502-distant-target-output-nonretryable" \ + "vertex_ai/missing-primary" \ + "openrouter/free vertex_ai/fallback-two" \ + "1" \ + "Strix quick scan failed with a non-recoverable error." \ + "2" \ + "vertex_ai/missing-primary|openrouter/free" \ + "|https://example.invalid" \ + "vertex_ai" \ + "__DEFAULT__" \ + "" \ + "1" + ;; + service-unavailable-no-llm-marker-nonrecoverable) + run_gate_case "service-unavailable-no-llm-marker-nonrecoverable" \ + "custom/service-unavailable-primary" \ + "vertex_ai/fallback-one vertex_ai/fallback-two" \ + "1" \ + "Strix quick scan failed with a non-recoverable error." \ + "1" \ + "custom/service-unavailable-primary" \ + "https://example.invalid" \ + "custom" \ + "__DEFAULT__" \ + "" \ + "1" + ;; + custom-openai-compatible-preserves-effort) + run_gate_case "custom-openai-compatible-preserves-effort" \ + "openai-direct/gpt-5.4" \ + "" \ + "0" \ + "scan ok" \ + "1" \ + "openai/gpt-5.4" \ + "https://compatible.example/v1" \ + "openai" \ + "https://compatible.example/v1" + ;; + nvidia-rate-limit-openai-direct-fallback-clears-api-base) + run_gate_case_allow_provider_signal "nvidia-rate-limit-openai-direct-fallback-clears-api-base" \ + "nvidia_nim/nvidia/rate-limited-primary" \ + "" \ + "0" \ + "REGEX:Strix quick scan succeeded with fallback model 'openai-direct/gpt-5.4' in [0-9]+s\\." \ + "2" \ + "nvidia_nim/nvidia/rate-limited-primary|openai/gpt-5.4" \ + "https://integrate.api.nvidia.com/v1|" \ + "nvidia_nim" \ + "https://integrate.api.nvidia.com/v1" \ + "" \ + "0" \ + "CRITICAL" \ + "0" \ + "" \ + "" \ + "1200" \ + "0" \ + "" \ + "" \ + "" \ + "" \ + "0" \ + "" \ + "" \ + "" \ + "__SAME_AS_FALLBACK_MODELS__" \ + "openai-direct/gpt-5.4" + ;; + openai-direct-quota-github-models-fallback-success) + run_gate_case "openai-direct-quota-github-models-fallback-success" \ + "openai_direct/gpt-5.4" \ + "" \ + "0" \ + "REGEX:Strix quick scan succeeded with fallback model 'github_models/openai/o3' in [0-9]+s\\." \ + "2" \ + "openai/gpt-5.4|openai/o3" \ + "|https://models.github.ai/inference" \ + "vertex_ai" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "github_models/openai/o3" + ;; + gemini-timeout-fallback-success) + run_gate_case_allow_provider_signal "gemini-timeout-fallback-success" \ + "gemini/timeout-fallback-primary" \ + "gemini/fallback-one gemini/fallback-two" \ + "0" \ + "REGEX:Strix quick scan succeeded with fallback model 'gemini/fallback-one' in [0-9]+s\\." \ + "2" \ + "gemini/timeout-fallback-primary|gemini/fallback-one" \ + "https://example.invalid|https://example.invalid" \ + "vertex_ai" \ + "__DEFAULT__" \ + "" \ + "1" + ;; + zero-findings-with-low-report-timeout) + run_gate_case_allow_provider_signal "zero-findings-with-low-report-timeout" \ + "vertex_ai/zero-low-primary" \ + "vertex_ai/fallback-one" \ + "1" \ + "Configured Vertex model and fallback models were unavailable." \ + "2" \ + "vertex_ai/zero-low-primary|vertex_ai/fallback-one" \ + "|" \ + "vertex_ai" \ + "__DEFAULT__" \ + "" \ + "0" \ + "CRITICAL" \ + "0" \ + "" \ + "" \ + "$TIMEOUT_TEST_PROCESS_SECONDS" \ + "0" \ + "pull_request" \ + "sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" + ;; + zero-findings-timeout-all-models) + run_gate_case_allow_provider_signal "zero-findings-timeout-all-models" \ + "vertex_ai/zero-timeout-primary" \ + "vertex_ai/fallback-one" \ + "1" \ + "Strix reported zero vulnerabilities before provider infrastructure failure; failing closed because provider infrastructure failures are not clean scan evidence." \ + "2" \ + "vertex_ai/zero-timeout-primary|vertex_ai/fallback-one" \ + "|" \ + "vertex_ai" \ + "__DEFAULT__" \ + "" \ + "0" \ + "CRITICAL" \ + "0" \ + "" \ + "" \ + "$TIMEOUT_TEST_PROCESS_SECONDS" \ + "0" \ + "pull_request" \ + "sync-module-system/smart-crawling-biz/src/main/java/org/empasy/sync/modules/system/controller/SysPositionController.java" + run_gate_case_allow_provider_signal "zero-findings-timeout-all-models" \ + "vertex_ai/zero-timeout-primary" \ + "vertex_ai/fallback-one" \ + "1" \ + "Configured Vertex model and fallback models were unavailable." \ + "2" \ + "vertex_ai/zero-timeout-primary|vertex_ai/fallback-one" \ + "|" \ + "vertex_ai" \ + "__DEFAULT__" \ + "" \ + "0" \ + "CRITICAL" \ + "0" \ + "" \ + "" \ + "$TIMEOUT_TEST_PROCESS_SECONDS" \ + "0" \ + "push" + ;; + slow-timeout) + run_gate_case_allow_provider_signal "slow-timeout" \ + "vertex_ai/slow-primary" \ + "" \ + "1" \ + "Strix run timed out after ${TIMEOUT_TEST_PROCESS_SECONDS}s." \ "3" \ "vertex_ai/slow-primary|vertex_ai/gemini-2.5-pro|vertex_ai/gemini-2.5-flash" \ "||" \