Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions docs/doctoring/contextual-orchestrator-vendored-sidecar.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,3 +88,19 @@ the full local suite passed with `1689 passed, 1 skipped, 16 subtests passed`.
The PR-target Noema check still runs the trusted base copy until this trusted
workflow change is merged, so its reproduction of the old error is retained as
bootstrap evidence rather than treated as a current-head runtime result.

## 2026-08-28 post-#1370 runtime correction

Main push run `33141468804` confirmed that the catalog envelope correction
reached the Strix sidecar, but LiteLLM rejected the child model
`orchestrator/free` because it had no provider prefix. Follow-up commits
`9f58d74` and `5aa0a20` map only the pinned gateway request to
`openai/orchestrator/free`, fail closed when that gateway base is absent or
not loopback, and keep the loopback sidecar receiving `orchestrator/free`.

The same follow-up masks the dynamic sidecar bearer before writing `GITHUB_ENV`
and rejects carriage returns/newlines in an override. This closes the runtime
log exposure observed in the Noema step environment block. Focused contracts
pass (`32 passed`) and the full local suite passes (`1689 passed, 1 skipped`).
The main Strix rerun and an independently authorized Noema model verdict are
still required before claiming end-to-end review completion.
35 changes: 28 additions & 7 deletions docs/product-technical-gap-baseline.md
Original file line number Diff line number Diff line change
Expand Up @@ -240,8 +240,9 @@ flowchart LR

## 2026-08-28 current-main routing and runtime recheck

- Current protected-main candidate is `f8823a544c3c4c046977f8511f683e85f83eb496`,
the merge commit for #1364. #1360 is merged at
- Current protected main is `24ee38b097dbfc1a895e1199ade48cff36431d05`,
the merge commit for #1370. #1364 is merged at
`f8823a544c3c4c046977f8511f683e85f83eb496`; #1360 is merged at
`17052a7ca3c16db90932a4d6036b43165ddee418`.
- The current Required OpenCode dispatch, `noema-review.yml`, `strix.yml`,
and write-capable `pr-review-autofix.yml` all provision the pinned
Expand All @@ -260,11 +261,31 @@ flowchart LR
in commit `861463c11a7ca8b1f9179073e2a3db9eba5aa5ab`; its current head is
`38e0307c655823a1e474b29aae89f8cfcb1edbc0`. Focused tests and the full local
suite pass (`1689 passed, 1 skipped, 16 subtests passed`).
- #1370 remains open and blocked against main `f8823a5`. Its PR-target Noema
run `33140830199` executes the trusted base launcher and reproduces the
pre-fix bare-list error; its `opencode-review` check fails closed because no
current-head OpenCode verdict exists. These are bootstrap evidence gaps,
not proof that the #1370 catalog-envelope patch fails.
- #1370 merged on `24ee38b…`; its pre-merge PR-target Noema run
`33140830199` executed the trusted base launcher and reproduced the
pre-fix bare-list error. The post-merge push run below shows that the
catalog-envelope fix reached the Strix sidecar successfully.

## 2026-08-28 post-#1370 Strix runtime recheck

- Main push run `33141468804` reached `Provision contextual-orchestrator Strix
sidecar` successfully, then failed in `Run Strix (quick)`. LiteLLM rejected
the unqualified child model `orchestrator/free` with `LLM Provider NOT
provided`; this is a request-shape defect, not evidence that the sidecar
catalog failed.
- Follow-up commits `9f58d74` and `5aa0a20` qualify only the LiteLLM child
request as `openai/orchestrator/free` when the API base is the pinned
loopback gateway; the gateway still receives `orchestrator/free` and owns
discovery/failover. They also fail closed when that base is absent or not
the pinned loopback, register the dynamic bearer token with `::add-mask::`
before exporting `GITHUB_ENV`, and reject token newlines. Focused contracts
pass (`32 passed`); the full local suite passes (`1689 passed, 1 skipped`).
- A real current-main Noema run on #1369 (`33141494393`) also booted the
sidecar and executed the Noema gate, but correctly skipped the LLM verdict
because the exact head had no primary OpenCode approval. A successful
sidecar/bootstrap step is not counted as a model-review result; post-fix
Strix completion and an independently authorized Noema verdict remain
separate evidence items.

## 5. 실행 루프와 고객의 다음 행동

Expand Down
8 changes: 8 additions & 0 deletions scripts/ci/contextual_orchestrator_review_sidecar.sh
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,14 @@ fi
log "provider secrets present: $provider_secret_count of 5"

ORCHESTRATOR_TOKEN="${ORCHESTRATOR_TOKEN:-$(python3 -c 'import secrets; print(secrets.token_urlsafe(32))')}"
if [[ "$ORCHESTRATOR_TOKEN" == *$'\r'* || "$ORCHESTRATOR_TOKEN" == *$'\n'* ]]; then
fail "ORCHESTRATOR_TOKEN must not contain carriage returns or newlines"
fi
if [ -n "${GITHUB_ACTIONS:-}" ]; then
# Register the process-local bearer token before exporting it through
# GITHUB_ENV; later step environment blocks otherwise echo it verbatim.
echo "::add-mask::$ORCHESTRATOR_TOKEN"
fi

mkdir -p "$ORCHESTRATOR_WORK"
rm -rf "$ORCHESTRATOR_SOURCE"
Expand Down
23 changes: 23 additions & 0 deletions scripts/ci/strix_quick_gate.sh
Comment thread
seonghobae marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -2499,6 +2499,10 @@ resolved_llm_api_base_for_model() {
fi

if [ -z "$api_base_file" ]; then
if is_contextual_orchestrator_model "$model"; then
echo "ERROR: contextual-orchestrator Strix scans require LLM_API_BASE_FILE to select the pinned loopback gateway." >&2
return 2
fi
if is_github_models_model "$model"; then
echo "ERROR: GitHub Models Strix scans require LLM_API_BASE_FILE to select the GitHub Models inference endpoint." >&2
return 2
Expand All @@ -2516,8 +2520,17 @@ resolved_llm_api_base_for_model() {
llm_api_base_value="${llm_api_base_value%%:generateContent*}"
llm_api_base_value="$(trim_whitespace "$llm_api_base_value")"
if [ -z "$llm_api_base_value" ]; then
if is_contextual_orchestrator_model "$model"; then
echo "ERROR: contextual-orchestrator Strix scans require a non-empty pinned loopback API base." >&2
return 2
fi
return 0
fi
if is_contextual_orchestrator_model "$model" &&
! is_contextual_orchestrator_api_base "$llm_api_base_value"; then
echo "ERROR: contextual-orchestrator Strix scans require the pinned loopback API base." >&2
return 2
fi
if [[ "$llm_api_base_value" =~ [[:space:][:cntrl:]] ]]; then
echo "ERROR: LLM_API_BASE must not contain whitespace or control characters." >&2
return 2
Expand Down Expand Up @@ -2547,6 +2560,16 @@ child_model_for_api_base() {
local model="$1"
local llm_api_base_value="$2"

# LiteLLM requires an explicit provider prefix even when the gateway is an
# OpenAI-compatible local endpoint. Keep the public gateway model name, but
# qualify only the child process model so the request still carries
# orchestrator/free to contextual-orchestrator.
if is_contextual_orchestrator_model "$model" &&
is_contextual_orchestrator_api_base "$llm_api_base_value"; then
printf '%s\n' 'openai/orchestrator/free'
return 0
fi
Comment thread
seonghobae marked this conversation as resolved.
Comment thread
seonghobae marked this conversation as resolved.

if [ -n "$llm_api_base_value" ] && is_github_models_api_base "$llm_api_base_value"; then
case "$model" in
github_models/openai/*)
Expand Down
58 changes: 58 additions & 0 deletions scripts/ci/test_strix_quick_gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3321,6 +3321,18 @@ success|runtime-env-forwarding|custom-openai-compatible-preserves-effort|vertex-
echo "scan ok"
exit 0
;;
contextual-orchestrator-gateway-model-qualification)
if [ "${STRIX_LLM:-}" != "openai/orchestrator/free" ]; then
echo "gateway model was not provider-qualified for LiteLLM" >&2
exit 10
fi
if [ "${LLM_API_BASE:-}" != "http://127.0.0.1:18080/v1" ]; then
echo "gateway API base was not preserved" >&2
exit 11
fi
echo "scan ok through contextual-orchestrator gateway"
exit 0
;;
scan-working-directory-isolated)
if [ "$PWD" = "$target_path" ] || [[ "$PWD" == "$target_path"/* ]]; then
echo "Error: Strix process inherited the untrusted scan target as cwd" >&2
Expand Down Expand Up @@ -6068,6 +6080,30 @@ run_filtered_gate_case_if_requested() {
"vertex_ai/ready-primary" \
"<unset>"
;;
contextual-orchestrator-missing-api-base-fails-closed)
run_gate_case "contextual-orchestrator-missing-api-base-fails-closed" \
"orchestrator/free" \
"" \
"2" \
"require LLM_API_BASE_FILE to select the pinned loopback gateway" \
"0" \
"" \
"" \
"contextual_orchestrator" \
""
;;
contextual-orchestrator-gateway-model-qualification)
run_gate_case "contextual-orchestrator-gateway-model-qualification" \
"orchestrator/free" \
"" \
"0" \
"scan ok through contextual-orchestrator gateway" \
"1" \
"openai/orchestrator/free" \
"http://127.0.0.1:18080/v1" \
"contextual_orchestrator" \
"http://127.0.0.1:18080/v1"
;;
pr-rust-workspace-context)
run_gate_case "pr-rust-workspace-context" \
"openai/gpt-4o-mini" \
Expand Down Expand Up @@ -9747,6 +9783,28 @@ run_gate_case "success" \
"vertex_ai/ready-primary" \
"<unset>"

run_gate_case "contextual-orchestrator-missing-api-base-fails-closed" \
"orchestrator/free" \
"" \
"2" \
"require LLM_API_BASE_FILE to select the pinned loopback gateway" \
"0" \
"" \
"" \
"contextual_orchestrator" \
""

run_gate_case "contextual-orchestrator-gateway-model-qualification" \
"orchestrator/free" \
"" \
"0" \
"scan ok through contextual-orchestrator gateway" \
"1" \
"openai/orchestrator/free" \
"http://127.0.0.1:18080/v1" \
"contextual_orchestrator" \
"http://127.0.0.1:18080/v1"

run_gate_case "success-with-critical-report" \
"vertex_ai/ready-primary" \
"" \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,11 @@ def test_sidecar_feeds_discovery_and_policy_artifacts_to_the_launcher() -> None:
def test_sidecar_exports_gateway_env_for_review_steps() -> None:
"""The gateway address and bearer token land in GITHUB_ENV for later steps."""
text = _read(SIDECAR)
assert 'echo "::add-mask::$ORCHESTRATOR_TOKEN"' in text
assert "ORCHESTRATOR_TOKEN must not contain carriage returns or newlines" in text
assert text.index('echo "::add-mask::$ORCHESTRATOR_TOKEN"') < text.index(
'if [ -n "$ORCHESTRATOR_GITHUB_ENV" ]; then'
)
assert "CONTEXTUAL_ORCHESTRATOR_BASE_URL=http://%s:%s\\n' \"$ORCHESTRATOR_HOST\" \"$ORCHESTRATOR_PORT\"" in text
assert "CONTEXTUAL_ORCHESTRATOR_TOKEN=%s\\n' \"$ORCHESTRATOR_TOKEN\"" in text
assert "CONTEXTUAL_ORCHESTRATOR_EVIDENCE=%s\\n' \"$policy_report\"" in text
Expand Down
Loading