diff --git a/.github/workflows/strix.yml b/.github/workflows/strix.yml index ace3f3352..514fd8a44 100644 --- a/.github/workflows/strix.yml +++ b/.github/workflows/strix.yml @@ -269,11 +269,21 @@ jobs: echo "::error::Strix target repository must belong to ContextualWisdomLab." exit 1 fi - is_private="$(gh api "repos/${TARGET_REPOSITORY}" --jq '.private')" + is_private="" + for target_visibility_attempt in 1 2 3 4 5 6; do + if is_private="$(gh api "repos/${TARGET_REPOSITORY}" --jq '.private')"; then + break + fi + is_private="" + if [ "$target_visibility_attempt" -lt 6 ]; then + echo "Repository visibility lookup failed (attempt ${target_visibility_attempt}/6), possibly a transient GitHub API rate limit; retrying after backoff." >&2 + sleep "$(( target_visibility_attempt * 5 ))" + fi + done case "$is_private" in true | false) ;; *) - echo "::error::Target repository visibility did not resolve to true or false." + echo "::error::Target repository visibility did not resolve to true or false after retries." exit 1 ;; esac diff --git a/CHANGELOG.md b/CHANGELOG.md index 35b938271..7bc40394c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -45,6 +45,7 @@ Semantic Versioning where the repository publishes a release. ### Fixed +- Retried the Strix scan up to `STRIX_TRANSIENT_RETRY_PER_MODEL` times, same model, when the log shows the upstream strix-agent Caido sandbox bootstrap timing race (`loginAsGuest failed after N attempts` / `Failed to connect to 127.0.0.1 port `; tracked upstream as usestrix/strix#1036, #1037, #1056). A slow CI runner can exceed strix-agent's fixed 10-attempt sandbox-login budget before its local intercepting proxy is reachable, even though the penetration test itself never started and no vulnerability evidence was produced or lost; the Docker image is already cached from the failed attempt, so a same-model retry is cheap and typically clears the one-off boot race. Not wired into cross-model fallback, since switching LLM models cannot change local sandbox container boot timing. - Replaced nonexistent `job.workflow_repository` / `job.workflow_sha` / `job.workflow_ref` / `job.workflow_file_path` context references (actionlint: "property ... is not defined in object type") in `pr-review-fix-scheduler.yml`'s called-workflow source verification and `exact-artifact-sbom-attestation.yml`'s trusted-verifier checkout. Both always failed closed on the missing properties (ContextualWisdomLab/.github#1212) or, for the SBOM attestation checkout, silently resolved an empty repository/ref instead of the pinned trusted source (downstream `gh attestation verify --signer-repo`/`--signer-workflow`, using the separately hardcoded `SIGNER_REPOSITORY` constant rather than any workflow_ref, still failed closed on the resulting empty signer identity). `github.workflow_ref`/`github.workflow_sha` are real, documented properties, but for a `workflow_call` target they reflect the top-level *calling* workflow, not the reusable workflow's own file — a prefix match against the reusable workflow's own path can never succeed. `exact-artifact-sbom-attestation.yml`'s checkout now uses `github.workflow_sha` (correct today: it has no callers yet); `pr-review-fix-scheduler.yml`'s identity check instead validates `github.repository`, since every current caller uses a local, same-repo `uses: ./...` where caller and callee share one commit and `github.workflow_sha` is still the right pin. Tracked follow-up for the SBOM attestation checkout once a real (potentially cross-repo) caller exists: ContextualWisdomLab/.github#1228. - Used the receiving repository's workflow token for same-repository scheduler Actions inventory and read calls, while retaining the established mutation @@ -80,6 +81,7 @@ Semantic Versioning where the repository publishes a release. - Bind reusable scheduler implementation to the validated called-workflow repository, SHA, ref, and file path, and verify the checked-out commit before executing privileged scheduler logic. - Removed the ambiguous central-repository schedule fallback that could scan `.github` instead of Clearfolio when no external variable was configured; the active product caller now names Clearfolio explicitly while the reusable engine retains caller and dispatch overrides. - Corrected the conflict-ordering regression contract to select the conflict-specific snapshot and verification after the ordinary path adopted the same trusted helper. +- Retried the Strix target-repository visibility lookup up to six times with linear backoff before failing closed, matching the existing PR-head-fetch retry convention in the same workflow. A single transient `gh api` failure (observed as a shared GitHub App installation token hitting its hourly rate limit while dozens of org repositories run hourly review schedulers concurrently) previously failed the entire required Strix check immediately, blocking otherwise mergeable, fully reviewed pull requests fleet-wide with no code defect involved. ### Security diff --git a/scripts/ci/strix_quick_gate.sh b/scripts/ci/strix_quick_gate.sh index 0f37f3460..649cdf552 100755 --- a/scripts/ci/strix_quick_gate.sh +++ b/scripts/ci/strix_quick_gate.sh @@ -169,9 +169,11 @@ import sys root = Path(sys.argv[1]) known_internal_warning = re.compile( r"^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d+ WARNING " - r"[^ ]+ - strix\.core\.execution: agent [0-9a-f]+ produced " - r"non-lifecycle final output in non-interactive mode; forcing tool " - r"continuation \(\d+/\d+\): " + r"[^ ]+ - strix\.core\.execution: agent [0-9a-f]+ " + r"(?:" + r"produced non-lifecycle final output in non-interactive mode" + r"|ended a turn without a lifecycle tool call \(interactive=False\)" + r"); forcing tool continuation \(\d+/\d+\): " ) @@ -2662,11 +2664,12 @@ is_nvidia_nim_not_found_error() { ## Determines whether the last strix failure is a transient error eligible ## for same-model retry (up to STRIX_TRANSIENT_RETRY_PER_MODEL times). -## Four error families qualify: +## Five error families qualify: ## - RateLimit / RESOURCE_EXHAUSTED / HTTP 429 ## - litellm API connection failures with LLM-provider evidence ## - litellm service-unavailable / high-demand provider failures ## - MidStreamFallbackError (litellm mid-stream provider switch) +## - Caido bootstrap timing failures (guest login before the local proxy is up) ## Timeouts are infrastructure failures. In strict CI mode they fail closed; ## otherwise the caller may still move to fallback model evaluation. is_transient_same_model_retry_error() { @@ -2686,6 +2689,9 @@ is_transient_same_model_retry_error() { if is_midstream_fallback_error; then return 0 fi + if is_caido_bootstrap_timing_error; then + return 0 + fi return 1 } @@ -2747,6 +2753,8 @@ run_strix_with_transient_retry() { retry_reason="LLM service unavailable" elif is_midstream_fallback_error; then retry_reason="midstream fallback" + elif is_caido_bootstrap_timing_error; then + retry_reason="Caido sandbox bootstrap timing" fi echo "Retrying model '$model' due to $retry_reason (attempt $((attempt + 1))/$max_attempts)." >&2 sleep "$STRIX_TRANSIENT_RETRY_BACKOFF_SECONDS" @@ -2924,6 +2932,23 @@ is_midstream_fallback_error() { return 1 } +## Detects the upstream strix-agent Caido sandbox bootstrap timing race +## (usestrix/strix#1036, #1037, #1056): the sandbox container runs a +## chown -R before starting caido-cli, and a slow CI runner can exceed +## Strix's fixed 10-attempt loginAsGuest budget before the proxy is +## reachable, even though the penetration test itself never started. This +## is local sandbox/container timing, not model-specific, so it is +## same-model-retry-eligible only; switching LLM models would not change +## how long the sandbox takes to boot. +is_caido_bootstrap_timing_error() { + if grep -Fq 'loginAsGuest failed after' "$STRIX_LOG" && + grep -Eq 'Failed to connect to 127\.0\.0\.1 port [0-9]+' "$STRIX_LOG"; then + return 0 + fi + + return 1 +} + # Narrower variant: LLM providers only, excluding HTTP transport libraries # (httpx, httpcore, requests). Used for generic transport failures where # library names alone are insufficient to prove the timeout/connection error @@ -2976,6 +3001,10 @@ has_detected_infrastructure_error() { return 0 fi + if is_caido_bootstrap_timing_error; then + return 0 + fi + # Generic strix non-zero exit with known transport/connection errors # that don't fall into the specific categories above. # Use LLM_PROVIDER_ONLY_REGEX (not PROVIDER_CONTEXT_REGEX) to avoid diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index cf451d365..5a37ffc0c 100755 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -4420,6 +4420,15 @@ EOS echo "scan ok with sanitized internal Strix report notice" exit 0 ;; + report-known-internal-warning-variant-sanitized) + mkdir -p "$STRIX_REPORTS_DIR/fake-known-internal-warning-variant" + cat >"$STRIX_REPORTS_DIR/fake-known-internal-warning-variant/strix.log" <<'EOS' +2026-08-22 09:53:26.193 WARNING strix-pr-scope-example - strix.core.execution: agent 673f770f ended a turn without a lifecycle tool call (interactive=False); forcing tool continuation (1/500): +2026-06-18 13:10:44.089 INFO strix-pr-scope-example - strix.tools.finish.tool: finish_scan: completed scan with 0 vulnerability report(s) +EOS + echo "scan ok with sanitized internal Strix report notice variant" + exit 0 + ;; report-unknown-warning-fails) mkdir -p "$STRIX_REPORTS_DIR/fake-unknown-warning" cat >"$STRIX_REPORTS_DIR/fake-unknown-warning/strix.log" <<'EOS' @@ -5683,6 +5692,17 @@ PY "scenario=$scenario does not rewrite logs through symlinked report directories" fi + if [ "$scenario" = "report-known-internal-warning-variant-sanitized" ]; then + assert_file_not_contains \ + "$repo_root_dir/strix_runs/fake-known-internal-warning-variant/strix.log" \ + "ended a turn without a lifecycle tool call" \ + "scenario=$scenario strips the newer-wording known internal Strix warning from published artifacts" + assert_file_contains \ + "$repo_root_dir/strix_runs/fake-known-internal-warning-variant/strix.log" \ + "finish_scan: completed scan with 0 vulnerability report(s)" \ + "scenario=$scenario keeps non-warning Strix report evidence" + fi + if [ "$scenario" = "github-models-primary-ratelimit-fallback-success" ]; then assert_file_contains \ "$output_log" \ @@ -9991,6 +10011,35 @@ run_gate_case "report-known-internal-warning-sanitized" \ "" \ "1" +run_gate_case "report-known-internal-warning-variant-sanitized" \ + "vertex_ai/report-known-internal-warning-variant-sanitized" \ + "" \ + "0" \ + "Strix run succeeded for model 'vertex_ai/report-known-internal-warning-variant-sanitized'" \ + "1" \ + "vertex_ai/report-known-internal-warning-variant-sanitized" \ + "" \ + "vertex_ai" \ + "__DEFAULT__" \ + "" \ + "0" \ + "CRITICAL" \ + "0" \ + "" \ + "" \ + "1200" \ + "0" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "" \ + "__SAME_AS_FALLBACK_MODELS__" \ + "" \ + "1" + run_gate_case "report-unknown-warning-fails" \ "vertex_ai/report-unknown-warning-fails" \ "" \ diff --git a/tests/test_strix_caido_bootstrap_timing_retry.py b/tests/test_strix_caido_bootstrap_timing_retry.py new file mode 100644 index 000000000..a60b9d801 --- /dev/null +++ b/tests/test_strix_caido_bootstrap_timing_retry.py @@ -0,0 +1,142 @@ +"""Regression contract for the strix-agent Caido sandbox bootstrap timing race. + +Upstream strix-agent (usestrix/strix#1036, #1037, #1056) runs a chown -R on +the sandbox container before starting caido-cli, and enforces a fixed +10-attempt loginAsGuest retry budget. A slow CI runner can exceed that +budget before the local proxy is reachable, even though the penetration +test itself never started and no security evidence was produced or lost. +This is local sandbox/container boot timing, not tied to any one LLM model, +so it must be retried same-model rather than treated as grounds to switch +models or as a genuine, non-backend scan failure. +""" + +from __future__ import annotations + +import re +import subprocess +import tempfile +import unittest +from pathlib import Path + + +REPOSITORY_ROOT = Path(__file__).resolve().parents[1] +STRIX_GATE = REPOSITORY_ROOT / "scripts" / "ci" / "strix_quick_gate.sh" + +# The exact strings strix-agent's caido_bootstrap.py emits on this failure. +OBSERVED_LOG = ( + "Error during penetration test: loginAsGuest failed after 10 attempts: " + "curl exit 7: curl: (7) Failed to connect to 127.0.0.1 port 48080 after " + "0 ms: Could not connect to server\n" + "Vulnerabilities 0\n" +) + + +def _function_block(source: str, function_name: str) -> str: + """Return one top-level Bash function, including its closing brace.""" + + match = re.search( + rf"(?ms)^{re.escape(function_name)}\(\) \{{\n.*?^\}}\n", + source, + ) + if match is None: + raise AssertionError(f"missing Bash function: {function_name}") + return match.group(0) + + +def _classifies_as_caido_bootstrap_timing(log_text: str) -> bool: + """Execute the production classifier against a bounded synthetic log.""" + + gate_source = STRIX_GATE.read_text(encoding="utf-8") + function_source = _function_block( + gate_source, + "is_caido_bootstrap_timing_error", + ) + with tempfile.TemporaryDirectory(prefix="strix-caido-timing-") as temp_dir: + log_path = Path(temp_dir) / "strix.log" + log_path.write_text(log_text, encoding="utf-8") + script = "\n".join( + ( + "set -euo pipefail", + 'STRIX_LOG="$1"', + function_source, + "is_caido_bootstrap_timing_error", + ) + ) + completed = subprocess.run( + ["bash", "-c", script, "strix-classifier", str(log_path)], + check=False, + capture_output=True, + text=True, + ) + if completed.returncode not in {0, 1}: + raise AssertionError(completed.stderr) + return completed.returncode == 0 + + +class StrixCaidoBootstrapTimingRetryTests(unittest.TestCase): + """Protect same-model retry for the upstream sandbox boot race.""" + + def test_observed_caido_login_failure_is_retryable(self) -> None: + """Recognize the exact loginAsGuest failure observed in required CI.""" + + self.assertTrue(_classifies_as_caido_bootstrap_timing(OBSERVED_LOG)) + + def test_different_port_is_still_recognized(self) -> None: + """Match the strix-agent message shape, not one hardcoded port.""" + + log = ( + "loginAsGuest failed after 10 attempts: curl exit 7: curl: (7) " + "Failed to connect to 127.0.0.1 port 51234 after 0 ms: Could " + "not connect to server\n" + ) + self.assertTrue(_classifies_as_caido_bootstrap_timing(log)) + + def test_unrelated_connection_refused_is_not_retryable(self) -> None: + """Do not let an unrelated target-application connection error match.""" + + log = "requests.exceptions.ConnectionError: Failed to establish a new connection\n" + self.assertFalse(_classifies_as_caido_bootstrap_timing(log)) + + def test_login_failure_without_the_connect_evidence_is_not_retryable(self) -> None: + """Require both the loginAsGuest phrase and the connect-failure line.""" + + log = "loginAsGuest failed after 10 attempts: unknown reason\n" + self.assertFalse(_classifies_as_caido_bootstrap_timing(log)) + + def test_wired_into_same_model_retry_and_infrastructure_not_cross_model( + self, + ) -> None: + """Retry the same model; do not treat this as a reason to switch models. + + Switching LLM models cannot change how long the local sandbox + container takes to boot, so cross-model fallback (`is_model_retryable_error`) + must stay untouched by this classifier. + """ + + gate_source = STRIX_GATE.read_text(encoding="utf-8") + infrastructure = _function_block( + gate_source, + "has_detected_infrastructure_error", + ) + same_model_retry = _function_block( + gate_source, + "is_transient_same_model_retry_error", + ) + cross_model_retry = _function_block(gate_source, "is_model_retryable_error") + + self.assertIn("is_caido_bootstrap_timing_error", infrastructure) + self.assertIn("is_caido_bootstrap_timing_error", same_model_retry) + self.assertNotIn("is_caido_bootstrap_timing_error", cross_model_retry) + + def test_retry_reason_is_logged_for_operators(self) -> None: + """Keep the diagnostic retry-reason message in sync with the classifier.""" + + gate_source = STRIX_GATE.read_text(encoding="utf-8") + self.assertIn( + 'retry_reason="Caido sandbox bootstrap timing"', + gate_source, + ) + + +if __name__ == "__main__": + unittest.main()