Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions .github/workflows/strix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ jobs:
- name: Gate Strix secrets
id: gate
env:
STRIX_MODEL: ${{ secrets.STRIX_LLM || 'vertex_ai/gemini-2.5-flash' }}
STRIX_MODEL: ${{ secrets.STRIX_LLM || 'vertex_ai/gemini-3.1-pro-preview-customtools' }}
STRIX_OPENAI_API_KEY: ${{ secrets.STRIX_OPENAI_API_KEY }}
STRIX_VERTEX_CREDENTIALS: ${{ secrets.GCP_SA_KEY }}
run: |
Expand All @@ -117,7 +117,7 @@ jobs:
exit 1
fi
;;
vertex_ai/gemini-2.5-flash)
vertex_ai/gemini-3.1-pro-preview-customtools | vertex_ai/gemini-2.5-flash)
echo 'enabled=true' >> "$GITHUB_OUTPUT"
echo 'provider_mode=vertex_ai' >> "$GITHUB_OUTPUT"
sanitized_vertex_credentials="$(printf '%s' "$STRIX_VERTEX_CREDENTIALS" | tr -d '\r')"
Expand Down Expand Up @@ -203,7 +203,7 @@ jobs:
- name: Prepare Strix model input file
if: steps.gate.outputs.enabled == 'true'
env:
STRIX_MODEL: ${{ secrets.STRIX_LLM || 'vertex_ai/gemini-2.5-flash' }}
STRIX_MODEL: ${{ secrets.STRIX_LLM || 'vertex_ai/gemini-3.1-pro-preview-customtools' }}
run: |
umask 077
strix_llm_file="$RUNNER_TEMP/strix_llm.txt"
Expand All @@ -215,7 +215,7 @@ jobs:
gpt-*)
printf 'openai/%s' "$strix_model" > "$strix_llm_file"
;;
vertex_ai/gemini-2.5-flash)
vertex_ai/gemini-3.1-pro-preview-customtools | vertex_ai/gemini-2.5-flash)
printf '%s' "$strix_model" > "$strix_llm_file"
;;
*)
Expand Down Expand Up @@ -249,6 +249,7 @@ jobs:
STRIX_MEMORY_COMPRESSOR_TIMEOUT: 10
STRIX_TRANSIENT_RETRY_PER_MODEL: 2
STRIX_TRANSIENT_RETRY_BACKOFF_SECONDS: 3
PYTHONWARNINGS: "ignore:Pydantic serializer warnings:UserWarning:pydantic.main"
STRIX_PROCESS_TIMEOUT_SECONDS: ${{ (github.event_name == 'pull_request_target' || github.event.inputs.pr_number != '') && '1200' || '2400' }}
STRIX_TOTAL_TIMEOUT_SECONDS: 4800
STRIX_PR_SCOPE_MAX_FILES_PER_BATCH: 12
Expand Down
23 changes: 13 additions & 10 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,16 +15,19 @@
packages unless compatibility evidence is recorded in the PR.
- Strix Security Scan must not route through GitHub Models, `github.token`,
generic `LLM_API_KEY`, GPT-4o, or GPT-4.1. The current organization-secret
route is `STRIX_LLM` with `GCP_SA_KEY`; `vertex_ai/gemini-2.5-flash` is the
validated operational model after run `26581416713` proved
`vertex_ai/gemini-3.1-pro-preview-customtools` returns Vertex 404/no-access in
this project. Expose Google/Vertex credentials only for Vertex provider mode.
Direct OpenAI GPT-5.4-or-newer scans remain supported only when selected
explicitly with `STRIX_OPENAI_API_KEY`. Do not silently fall back between
providers, and record provider evidence in the PR. Keep architecture docs and
reusable Strix gate tests aligned with this rule so stale GitHub Models,
OpenAI-only, unavailable-model, or generic-key examples cannot re-enter copied
workflow guidance.
route is `STRIX_LLM` with `GCP_SA_KEY`;
`vertex_ai/gemini-3.1-pro-preview-customtools` is the configured operational
model after organization-secret visibility correction, and
`vertex_ai/gemini-2.5-flash` remains the exact approved fallback. Expose
Google/Vertex credentials only for Vertex provider mode. Direct OpenAI
GPT-5.4-or-newer scans remain supported only when selected explicitly with
`STRIX_OPENAI_API_KEY`. Do not silently fall back between providers, and
record provider evidence in the PR. Known third-party Strix/Pydantic
serializer warnings must be filtered narrowly through the workflow/gate env
contract so Warn-class logs are not accepted as clean evidence. Keep
architecture docs and reusable Strix gate tests aligned with this rule so
stale GitHub Models, OpenAI-only, unavailable-model, blanket-warning, or
generic-key examples cannot re-enter copied workflow guidance.

## PR automation and review defaults

Expand Down
10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,15 @@ mail/calendar/file systems.
required checks. Human approval is not awaited by default under repo policy.
- Strix PR/security evidence uses the organization-secret provider selected by
`STRIX_LLM` with `GCP_SA_KEY`. The validated operational route is
`vertex_ai/gemini-2.5-flash`; direct OpenAI GPT-5.4-or-newer remains
`vertex_ai/gemini-3.1-pro-preview-customtools`, with
`vertex_ai/gemini-2.5-flash` kept as an exact approved fallback; direct OpenAI
GPT-5.4-or-newer remains
supported only with an explicit `STRIX_OPENAI_API_KEY`. The workflow fails
closed rather than falling back to GitHub Models, `github.token`, generic
`LLM_API_KEY`, or GPT-4-era models. Pending CodeRabbit or check evidence is a
wait state, not a hard blocker.
`LLM_API_KEY`, or GPT-4-era models. Known third-party Strix/Pydantic
serializer warnings are filtered narrowly instead of allowing Warn-class logs
into passing evidence. Pending CodeRabbit or check evidence is a wait state,
not a hard blocker.
- Security governance is source-backed through signed
`/api/security/access-surface`. The endpoint reads scoped WebDAV, CalDAV, and
connector evidence plus durable `security_audit_events`, reuses the deny-first
Expand Down
23 changes: 23 additions & 0 deletions backend/tests/test_release_governance.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,29 @@ def test_compose_log_scanner_exists_for_warning_policy() -> None:
assert "unexpected_count" in scanner


def test_strix_workflow_uses_configured_vertex_model_and_narrow_warning_filter() -> (
None
):
workflow = read_repo_text(".github/workflows/strix.yml")
gate_script = read_repo_text("scripts/ci/strix_quick_gate.sh")

assert "models: read" not in workflow
assert "provider_mode=github_models" not in workflow
assert "vertex_ai/gemini-3.1-pro-preview-customtools" in workflow
assert (
"vertex_ai/gemini-3.1-pro-preview-customtools | "
"vertex_ai/gemini-2.5-flash"
in workflow
)
assert "vertex_ai/* | vertex_ai_beta/*" not in workflow
assert (
'PYTHONWARNINGS: "ignore:Pydantic serializer warnings:UserWarning:pydantic.main"'
in workflow
)
assert "ignore::UserWarning" not in workflow
assert '"PYTHONWARNINGS",' in gate_script


def test_pr_governance_uses_metadata_only_events_without_checkout_or_admin_merge() -> (
None
):
Expand Down
35 changes: 35 additions & 0 deletions docs/plans/2026-05-29-strix-vertex-model-warning-filter.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Strix Vertex Model and Warning Filter Roadmap

## Evidence

- PR #302 Strix run `26633823390` completed successfully, but the job log
emitted a third-party `pydantic.main` `UserWarning` with the message
`Pydantic serializer warnings`.
- Project policy treats `Timeout`, `Fatal`, `Warn`, and `Denied` output as
failure evidence even when the GitHub job conclusion is success.
- The organization secret `STRIX_LLM` is expected to select
`vertex_ai/gemini-3.1-pro-preview-customtools`; GitHub Models must not be used
for this route.

## Plan

1. Keep the Strix workflow on trusted `pull_request_target` materialization with
no PR-head checkout and no GitHub Models permission.
2. Allow only exact organization-approved Vertex model names:
`vertex_ai/gemini-3.1-pro-preview-customtools` and
`vertex_ai/gemini-2.5-flash`.
3. Default missing `STRIX_LLM` to the configured organization Vertex model so
org-secret visibility issues do not silently route to GitHub Models.
4. Suppress only the known third-party `pydantic.main` serializer warning via
`PYTHONWARNINGS`; do not blanket-ignore all `UserWarning` output.
5. Forward that warning contract through the Strix gate to the Python child
process so the setting affects the actual scanner, not only the shell wrapper.
6. Lock the contract in shell and backend release-governance tests.

## Non-Goals

- Naruon application warnings remain failures in CI.
- Strix findings, failed auth, timeout, denial, or security scanner errors are
not suppressed.
- GitHub Models, `github.token`, generic `LLM_API_KEY`, and arbitrary Vertex
model patterns remain disallowed.
1 change: 1 addition & 0 deletions scripts/ci/strix_quick_gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -1951,6 +1951,7 @@ for key in (
"http_proxy",
"https_proxy",
"no_proxy",
"PYTHONWARNINGS",
):
value = os.environ.get(key)
if value:
Expand Down
19 changes: 13 additions & 6 deletions scripts/ci/test_strix_quick_gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,11 @@ assert_strix_workflow_pr_trigger_hardened() {
assert_file_contains "$workflow_file" "timeout-minutes: 90" "strix workflow job budget covers PR-scoped Strix batches"
assert_file_contains "$workflow_file" "STRIX_TOTAL_TIMEOUT_SECONDS: 4800" "strix workflow total Strix budget covers PR-scoped batches"
assert_file_contains "$workflow_file" "STRIX_PR_SCOPE_MAX_FILES_PER_BATCH: 12" "strix workflow reduces PR batch startup overhead"
assert_file_contains "$workflow_file" 'STRIX_MODEL: ${{ secrets.STRIX_LLM || '"'"'vertex_ai/gemini-2.5-flash'"'"' }}' "strix workflow defaults STRIX_LLM to the operational organization Vertex AI model"
assert_file_contains "$workflow_file" 'STRIX_MODEL: ${{ secrets.STRIX_LLM || '"'"'vertex_ai/gemini-3.1-pro-preview-customtools'"'"' }}' "strix workflow defaults STRIX_LLM to the operational organization Vertex AI model"
assert_file_contains "$workflow_file" "STRIX_LLM must select direct OpenAI GPT-5.4 or newer, or an approved organization Vertex AI model" "strix workflow rejects unsupported model inputs"
assert_file_contains "$workflow_file" "vertex_ai/gemini-2.5-flash" "strix workflow accepts the validated organization Vertex AI operational model"
assert_file_contains "$workflow_file" "vertex_ai/gemini-3.1-pro-preview-customtools | vertex_ai/gemini-2.5-flash" "strix workflow accepts exact approved organization Vertex AI operational models"
assert_file_contains "$workflow_file" 'PYTHONWARNINGS: "ignore:Pydantic serializer warnings:UserWarning:pydantic.main"' "strix workflow narrowly filters the known third-party Pydantic serializer warning"
assert_file_not_contains "$workflow_file" "ignore::UserWarning" "strix workflow must not blanket-suppress all UserWarning output"
assert_file_not_contains "$workflow_file" "vertex_ai/* | vertex_ai_beta/*" "strix workflow must not accept arbitrary Vertex models"
assert_file_contains "$workflow_file" "provider_mode=openai_direct" "strix workflow requires direct OpenAI GPT-5 credentials"
assert_file_contains "$workflow_file" 'LLM_API_KEY_SECRET: ${{ steps.gate.outputs.provider_mode == '"'"'openai_direct'"'"' && secrets.STRIX_OPENAI_API_KEY || '"'"''"'"' }}' "strix workflow uses provider-scoped LLM key material"
Expand Down Expand Up @@ -119,7 +121,7 @@ assert_strix_gpt54_model_guard_semantics() {
case "$model" in
gpt-5.[4-9]* | gpt-5.[1-9][0-9]* | gpt-[6-9]* | gpt-[1-9][0-9]* | \
openai/gpt-5.[4-9]* | openai/gpt-5.[1-9][0-9]* | openai/gpt-[6-9]* | openai/gpt-[1-9][0-9]* | \
vertex_ai/gemini-2.5-flash)
vertex_ai/gemini-3.1-pro-preview-customtools | vertex_ai/gemini-2.5-flash)
return 0
;;
*)
Expand All @@ -144,8 +146,11 @@ assert_strix_gpt54_model_guard_cases() {
if assert_strix_gpt54_model_guard_semantics "openai/openai/gpt-5.4"; then
record_failure "strix GPT-5.4 guard must reject GitHub Models openai/openai/gpt-5.4"
fi
if ! assert_strix_gpt54_model_guard_semantics "vertex_ai/gemini-3.1-pro-preview-customtools"; then
record_failure "strix guard must accept the configured organization Vertex AI operational model"
fi
if ! assert_strix_gpt54_model_guard_semantics "vertex_ai/gemini-2.5-flash"; then
record_failure "strix guard must accept the validated organization Vertex AI operational model"
record_failure "strix guard must accept the fallback approved organization Vertex AI model"
fi
if assert_strix_gpt54_model_guard_semantics "vertex_ai/gemini-2.5-pro"; then
record_failure "strix guard must reject arbitrary Vertex models"
Expand Down Expand Up @@ -293,12 +298,13 @@ set -euo pipefail
printf '%s\n' "${STRIX_LLM:-}" >> "${FAKE_STRIX_CALL_LOG:?}"
printf '%s\n' "${LLM_API_BASE:-<unset>}" >> "${FAKE_STRIX_API_BASE_LOG:?}"
if [ -n "${FAKE_STRIX_RUNTIME_ENV_LOG:-}" ]; then
printf 'LLM_TIMEOUT=%s;STRIX_MEMORY_COMPRESSOR_TIMEOUT=%s;STRIX_REASONING_EFFORT=%s;STRIX_LLM_MAX_RETRIES=%s;GEMINI_LOCATION=%s;UNRELATED_SECRET=%s\n' \
printf 'LLM_TIMEOUT=%s;STRIX_MEMORY_COMPRESSOR_TIMEOUT=%s;STRIX_REASONING_EFFORT=%s;STRIX_LLM_MAX_RETRIES=%s;GEMINI_LOCATION=%s;PYTHONWARNINGS=%s;UNRELATED_SECRET=%s\n' \
"${LLM_TIMEOUT:-<unset>}" \
"${STRIX_MEMORY_COMPRESSOR_TIMEOUT:-<unset>}" \
"${STRIX_REASONING_EFFORT:-<unset>}" \
"${STRIX_LLM_MAX_RETRIES:-<unset>}" \
"${GEMINI_LOCATION:-<unset>}" \
"${PYTHONWARNINGS:-<unset>}" \
"${UNRELATED_SECRET:-<unset>}" >> "${FAKE_STRIX_RUNTIME_ENV_LOG:?}"
fi

Expand Down Expand Up @@ -1964,6 +1970,7 @@ EOS
STRIX_REASONING_EFFORT="minimal"
STRIX_LLM_MAX_RETRIES="1"
GEMINI_LOCATION="GLOBAL"
PYTHONWARNINGS="ignore:Pydantic serializer warnings:UserWarning:pydantic.main"
UNRELATED_SECRET="should-not-forward"
)
fi
Expand Down Expand Up @@ -2110,7 +2117,7 @@ EOS
if [ "$scenario" = "runtime-env-forwarding" ]; then
assert_file_contains \
"$runtime_env_log" \
"LLM_TIMEOUT=90;STRIX_MEMORY_COMPRESSOR_TIMEOUT=10;STRIX_REASONING_EFFORT=minimal;STRIX_LLM_MAX_RETRIES=1;GEMINI_LOCATION=GLOBAL;UNRELATED_SECRET=<unset>" \
"LLM_TIMEOUT=90;STRIX_MEMORY_COMPRESSOR_TIMEOUT=10;STRIX_REASONING_EFFORT=minimal;STRIX_LLM_MAX_RETRIES=1;GEMINI_LOCATION=GLOBAL;PYTHONWARNINGS=ignore:Pydantic serializer warnings:UserWarning:pydantic.main;UNRELATED_SECRET=<unset>" \
"scenario=$scenario runtime env forwarding"
fi

Expand Down