Skip to content

Revert "fix(strix): disable SDK streaming for the contextual-orchestrator gateway" - #1463

Merged
seonghobae merged 8 commits into
mainfrom
revert-strix-disable-streaming
Sep 1, 2026
Merged

Revert "fix(strix): disable SDK streaming for the contextual-orchestrator gateway"#1463
seonghobae merged 8 commits into
mainfrom
revert-strix-disable-streaming

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Why

Reverts #1448 (702392a), a deliberate, documented temporary workaround for a bug that is now fixed at the source.

#1448 routed around contextual-orchestrator's gateway unconditionally rejecting stream_options.include_usage=true combined with tools — the exact shape Strix's always-streaming, always-tools agent loop sends on every turn — by disabling Strix's own SDK streaming (LLM_DISABLE_STREAMING=true) whenever it talks to that gateway. That traded away real-time SSE output for Strix scans to unblock the org's required Strix gate (which was itself transitively blocking every PR's opencode-review, since that check requires completed Strix evidence before it will dispatch).

ContextualWisdomLab/contextual-orchestrator#925 (merged) fixes the actual root cause: the gateway's rejection is now narrowed to only the genuinely-unmeasured response_format-only (conduct mode) case. Single-agent tools passthrough — exactly Strix's request shape — now succeeds normally, with honest usage labeling (usage_source: "reported" when the provider returns it, "estimated" otherwise, never fabricated as "reported"). Both #1448's own PR body and contextual-orchestrator's docs/product-technical-gap-baseline.md explicitly tracked this as a workaround to revert once #925 landed — this PR is that follow-up.

Effect

Strix scans through the contextual-orchestrator gateway resume real SSE streaming instead of the single-non-streaming-call fallback. No other provider's behavior is affected — #1448's opt-in was already scoped narrowly to the contextual-orchestrator loopback, so reverting it only removes that one narrow branch; every other provider in Strix's fallback chain was never touched.

Verification

  • git revert --no-edit 702392a2fb80149d25734a4cae173ed5fecd6d83: clean, no conflicts.
  • bash -n on both touched scripts: clean.
  • bash scripts/ci/test_strix_quick_gate.sh: test_strix_quick_gate: PASS (full suite, including the reverted assertions' own now-absent counterparts).
  • PYTHONPATH=. coverage run -m pytest tests -q: 2023 passed, 1 skipped, 21 subtests passed.
  • coverage report --show-missing: TOTAL 10250/10250 statements, 4084/4084 branches, 100%.
  • interrogate .: PASSED (minimum: 100.0%, actual: 100.0%).

Generated by Claude Code


Devin Review

Summary by CodeRabbit

  • 변경 사항

    • Contextual Orchestrator 게이트웨이를 통한 Strix 실행에서 스트리밍이 기본 동작으로 유지됩니다.
    • 기존의 비스트리밍 강제 설정이 제거되어 게이트웨이 결함 재발 가능성을 줄였습니다.
    • 관련 기본값과 계약 검증 기준을 최신 게이트웨이 동작에 맞게 동기화했습니다.
  • 테스트

    • 스트리밍 환경 전달 및 게이트웨이 동작 검증을 새로운 기본 동작에 맞게 업데이트했습니다.

…ator gateway (#1448)"

This reverts commit 702392a.

contextual-orchestrator#925 (merged) fixes the actual root cause #1448
routed around: the gateway's server.py unconditionally rejected
stream_options.include_usage=true combined with tools before any
upstream call. That rejection is now narrowed to only the genuinely
unmeasured response_format-only (conduct mode) case; single-agent
tools passthrough -- exactly Strix's request shape -- succeeds with
real, honestly-labeled usage (usage_source: "reported" when the
provider returns it, "estimated" otherwise, never fabricated).

#1448 was always documented as a temporary workaround, not the fix
(see its own PR body and contextual-orchestrator's
docs/product-technical-gap-baseline.md), trading away Strix's
real-time SSE streaming to route around a bug that no longer exists.
With #925 live, Strix's tools + stream_options.include_usage=true
requests against orchestrator/free succeed on their own -- this
workaround is now unnecessary and reverted to restore real streaming.
@coderabbitai

coderabbitai Bot commented Aug 31, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

Next included review available in 36 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 94492da5-fe93-461c-8c68-5177b5c646fc

📥 Commits

Reviewing files that changed from the base of the PR and between 8159910 and 85b764c.

📒 Files selected for processing (1)
  • scripts/ci/test_strix_quick_gate.sh
📝 Walkthrough

Walkthrough

Strix의 contextual-orchestrator 스트리밍 비활성화 경로를 제거했습니다. 게이트웨이 핀 참조를 갱신했습니다. 자체 테스트에서 LLM_DISABLE_STREAMING 관련 로그와 검증을 제거했습니다.

Changes

Strix 스트리밍 경로

Layer / File(s) Summary
게이트웨이 핀과 관련 문서 동기화
docs/product-technical-gap-baseline.md
ORCHESTRATOR_PIN_SHAcontextual-orchestrator#925 수정 사항을 포함하는 커밋으로 갱신했습니다. 사이드카 기본값, 계약 테스트, ADR의 핀 참조를 함께 동기화했습니다.
Strix 스트리밍 우회 및 자체 테스트 제거
scripts/ci/strix_quick_gate.sh, scripts/ci/test_strix_quick_gate.sh
Strix 자식 프로세스에 LLM_DISABLE_STREAMING을 전달하는 로직을 제거했습니다. fake Strix 로그와 관련 자체 테스트 어서션을 갱신했습니다.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: 🟡 Moderate · up to 81599

The PR resumes streaming for Strix, but the sidecar may still use an older gateway revision that rejects the request shape, causing scans or required checks to fail; the related streaming regression checks are also weakened. Synchronize the pinned revision and retain explicit assertions before merging.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 contextual-orchestrator 게이트웨이의 Strix SDK 스트리밍 비활성화 변경을 되돌리는 주요 변경 사항을 정확하게 설명합니다.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (2 skipped: 1 unsupported, 1 too large.)

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch revert-strix-disable-streaming

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

devin-ai-integration[bot]

This comment was marked as resolved.

…ols fix

Devin Review correctly caught that this revert branch's sidecar still
vendored contextual-orchestrator at 30c6d716..., a commit predating
contextual-orchestrator#925's merge (7944a3c) -- confirmed via
git merge-base --is-ancestor. Removing the Strix-side
LLM_DISABLE_STREAMING workaround while the vendored gateway still ran
the old rejecting code would have restored the exact failure #1448
existed to route around.

Bumps the pin to 7944a3c (the #925 merge commit itself, not
contextual-orchestrator's later tip) in the three places this repo's
convention keeps in sync: the sidecar script's default, the pinned-SHA
contract test assertion, and ADR-0003's "today" reference. Recorded in
docs/product-technical-gap-baseline.md.

Copy link
Copy Markdown
Contributor Author

Confirmed and fixed, pushed (4924eff). Verified the exact concern with git merge-base --is-ancestor 30c6d71680e659f25a0a433d4726ad0d437f9757 7944a3cd98f7b60fba9272e7f89c3977a75af746 → true: the vendored pin genuinely predated contextual-orchestrator#925's merge, so removing the streaming workaround while the sidecar still ran the old rejecting code would have restored the exact failure #1448 existed to route around.

Bumped ORCHESTRATOR_PIN_SHA to 7944a3c (the #925 merge commit itself — deliberately not contextual-orchestrator's later tip, to keep this bump minimal and scoped to exactly the fix this revert depends on) in the three places this repo's convention keeps in sync: the sidecar script's default, the pinned-SHA contract test (test_sidecar_pins_the_vendored_orchestrator_revision), and ADR-0003's "today" reference. Recorded in docs/product-technical-gap-baseline.md.

On the suggested runtime contract proving a live tools+stream_options request against the exact vendored revision: reasonable in principle, but out of scope here — this repo's existing pin-bump pattern (see the #1422/#1430 history in the gap-baseline) validates pins via merge-base ancestry + the contract test's static assertions, not a live network call against the vendored gateway from this trusted-script test suite. Happy to revisit if the maintainers want that as a standing pattern.

Verification: PYTHONPATH=. coverage run -m pytest tests -q → 2023 passed, 1 skipped, 21 subtests. coverage report → 10250/10250 statements, 4084/4084 branches, 100%. interrogate → 100%.


Generated by Claude Code


Generated by Claude Code

devin-ai-integration[bot]

This comment was marked as resolved.

Copy link
Copy Markdown
Contributor Author

strix check failure — not this PR's, no fix exists yet

The strix check failed on this PR's current head (4924eff, run 33348306414). Root-caused via job logs:

openai.BadRequestError: Error code: 400 - {'error': {'code': 'invalid_request_error', 'message': "Model 'meta/llama-3.2-11b-vision-instruct' via agent 'nvidia_nim_meta_llama_3_2_11b_vision_instruct': This model only supports single tool-calls at once! ...

The sidecar itself started and became healthy normally (Provision contextual-orchestrator Strix sidecar step: success, ~5m41s including clone+install — well inside the 180s healthz watchdog, so Devin's discovery-timeout concern on this thread did not manifest in this run). The scan then failed mid-run because Strix's agent loop sent a multi-tool-call turn to meta/llama-3.2-11b-vision-instruct via nvidia_nim, and that model rejects multiple simultaneous tool calls.

Why this isn't this PR's failure: this PR's diff is scoped to (a) reverting #1448's LLM_DISABLE_STREAMING workaround and (b) bumping ORCHESTRATOR_PIN_SHA. Neither touches candidate selection. The catalog-selection behavior that reaches this specific model — CATALOG_FAMILY_CAP=8 and the alphabetical, reliability-blind selection documented in scripts/ci/contextual_orchestrator_review_sidecar.sh's own comments — was already present on main before this branch existed (confirmed: git show c10a557:scripts/ci/contextual_orchestrator_review_sidecar.sh already shows CATALOG_FAMILY_CAP="${ORCHESTRATOR_CATALOG_FAMILY_CAP:-8}", and c10a557/#1442 is this branch's own ancestor). Whether meta/llama-3.2-11b-vision-instruct gets selected into the pool depends only on live NVIDIA NIM catalog contents at scan time, not on anything in this diff.

No fix exists yet for models that reject multi-tool-call turns — there's no denylist/capability filter in scripts/ci/contextual_orchestrator_review_policy.py today. This is a real, separate gap (multi-tool-call capability isn't modeled in candidate selection) and I'll track/fix it separately rather than widening this PR's scope.

Re-running the failed strix job now (one retry, per this repo's CI-red protocol) since this is exactly the kind of transient candidate-selection outcome that can pick a different, working model on a fresh run.


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

@opencode-agent review

Please publish a substantive formal verdict for exact current head 8e4b1cca86d0ae1f802e6a1fe36fc49d54f4a5f2. Predecessor-head, COMMENTED, status-only, or model-status evidence must not satisfy the gate.

Copy link
Copy Markdown
Contributor Author

Status update for @opencode-agent's requested verdict on current head 8e4b1cca8:

Strix actually succeeded on this PR's previous head (4924eff, run 33348306414, 16.5 minutes, conclusion: success) — real, live proof the revert + ORCHESTRATOR_PIN_SHA bump work end-to-end through the contextual-orchestrator gateway. The org's own opencode-agent bot then auto-updated this branch from main (merge commit, no code conflict) to pick up unrelated concurrent work (contextual-orchestrator#941's independent-credential-account fix, etc.), which reset head to 8e4b1cca8 and started every check over, including Strix.

The first Strix attempt on this new head was cancelled almost immediately (~24s, before any real work — an infra hiccup); I re-ran it and it made real progress (~12 minutes) before being cancelled again, cause not yet clear from the API. Given the prior head's full success on materially the same code, re-running once more now.


Generated by Claude Code

@seonghobae

Copy link
Copy Markdown
Contributor Author

Cross-PR model-group correction: #1468 pins the shared Noema/Strix/OpenCode sidecar to contextual-orchestrator 0adca470 and removes the sidecar-local provider-family collapse. Each of BYTEZ_API_KEY, NVIDIA_NIM_API_KEY, NVIDIA_NIM_API_KEY_SUB, OPENROUTER_API_KEY, and OPENAI_API_KEY is an independent credential account; vendor/endpoint identity is not model equivalence. Only an explicit model_group in contextual-orchestrator may share routing evidence. Please rebase/stack on #1468 or treat its sidecar-policy portion as superseding overlapping family_cap/pin assumptions.

…streaming

# Conflicts:
#	docs/adr/0003-contextual-orchestrator-vendored-free-zdr.md
#	scripts/ci/contextual_orchestrator_review_sidecar.sh
#	tests/test_contextual_orchestrator_review_sidecar_contract.py

Copy link
Copy Markdown
Contributor Author

Confirmed both parts of your diagnosis and merged main to pick up the fixes — both landed directly on main since my last check (likely a concurrent session):

  • Sidecar pin: was still 7944a3c (my earlier bump, which predates contextual-orchestrator#941). main now carries c107e3e5 via .github#1469 — verified via git merge-base --is-ancestor that this SHA is a genuine descendant of both contextual-orchestrator#941 (removed the nvidia_nim/nvidia_nim_sub family-collapsing heuristic) and #945 (the follow-up test fix), plus #923's transient-retry fix. Merged into this branch (head now 414859e).
  • PROVIDER_FAMILIES: .github#1468 already removed contextual_orchestrator_review_policy.py's own separate provider_family()/PROVIDER_FAMILIES reimplementation, replacing it with provider_account(provider_name) -> provider_name (identity — no collapsing) and renaming family_capaccount_cap. Confirmed on current main.

Full validation on the merged head: PYTHONPATH=. coverage run -m pytest tests -q → 2092 passed, 1 skipped, 21 subtests; coverage report → 10454/10454 statements, 4164/4164 branches, 100%; interrogate → 100%.


Generated by Claude Code

coderabbitai[bot]

This comment was marked as resolved.

…n guards

exact-head-path-policy was failing on this PR's head with "opencode
required workflow bootstrap must not depend on required-workflow event
payload fields" -- reproduced against unmodified main and confirmed
unrelated to this PR's diff. Root cause and fix already exist in open
PR #1506: assert_opencode_review_uses_codegraph_and_contextual_orchestrator
extracted the required-workflow-bootstrap job block with an awk range
pattern (/^  required-workflow-bootstrap:$/,/^[^ ]/) whose end pattern
never matches (every job key is indented 2 spaces, never column 0), so
it swept an unrelated if: from the separate opencode-review-target job
(added legitimately by already-merged #1497) into the wrong job's
block. Ported #1506's fix by hand: a state-flag-bounded awk extraction,
a large-block regression test proving the bound survives a pipe-buffer-
sized job block, and the same grep -Fq -> grep -F fix for a second,
independently SIGPIPE-vulnerable assertion. Deliberately did not port
#1506's LLM_DISABLE_STREAMING test content, since that would reinstate
assertions for the exact workaround this PR reverts.

Also addresses a real CodeRabbit finding on this PR's own revert
commit: removing the LLM_DISABLE_STREAMING fake-log field/assertions
left no regression guard against the reverted workaround silently
reappearing in scripts/ci/strix_quick_gate.sh. Added
assert_file_not_contains checks for both removed code paths
(STRIX_CHILD_DISABLE_STREAMING threading and the child_env
LLM_DISABLE_STREAMING assignment).

test_strix_quick_gate.sh: PASS. Full suite: 2126 passed, 1 skipped,
21 subtests. Coverage and docstrings: 100%.

Copy link
Copy Markdown
Contributor Author

exact-head-path-policy failure fixed and pushed (85b764c): reproduced against unmodified main first, confirmed it's a pre-existing bug (broken awk job-block extraction in scripts/ci/test_strix_quick_gate.sh, root-caused and already fixed on open PR .github#1506) — not caused by this PR's own diff. Ported #1506's awk/SIGPIPE fix by hand, deliberately excluding its unrelated LLM_DISABLE_STREAMING test content since that would reinstate assertions for the exact workaround this PR reverts. Also added assert_file_not_contains regression guards against that same workaround silently reappearing (per a CodeRabbit nitpick on this PR's own revert commit).

Verified: bash scripts/ci/test_strix_quick_gate.shPASS. Full suite: 2126 passed, 1 skipped, 21 subtests. Coverage and docstrings: 100%.


Generated by Claude Code

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

Devin Review

Comment thread scripts/ci/test_strix_quick_gate.sh
Comment thread scripts/ci/test_strix_quick_gate.sh

Copy link
Copy Markdown
Contributor Author

opencode-review failed on head 85b764c after its full 90-minute polling window (180 × 30s) with no APPROVED/CHANGES_REQUESTED verdict from opencode-agent on this exact head SHA — this is the job's own designed fail-closed behavior ("this required check is not a review and must not succeed until the authenticated dispatch posts a current-head verdict"), not a code defect in this PR. The org's shared orchestrator/free review pipeline has been under heavy concurrent load all session (Strix runs taking 2.5+ hours, noema-review timing out on its own hardcoded budget elsewhere on this same PR), so the dispatch simply hasn't posted yet rather than having failed. Not forcing a manual re-run right now — that would just restart the same 90-minute wait against an already-loaded pipeline; letting the normal automated retry cycle catch up. Watching for the next dispatch attempt.


Generated by Claude Code

Copy link
Copy Markdown
Contributor Author

strix failed on head 85b764c (run 33445279374) — root-caused via job logs, not this PR's diff: Strix's own internal Caido proxy bootstrap (strix/runtime/caido_bootstrap.py::_login_as_guest) couldn't connect to its local 127.0.0.1:48080 service after 10 attempts (curl exit 7), so the scan itself never ran. This is a pre-recognized infra-unavailable class — scripts/ci/strix_quick_gate.sh's own backend_unavailable_signal regex already matches this exact loginAsGuest failed ... curl exit 7 ... port 48080 string and classifies it as STRIX_PROVIDER_UNAVAILABLE, which is why the run failed closed cleanly ("Strix could not complete authoritative vulnerability analysis because its provider/backend was unavailable") rather than reporting a false finding. The gate's own bounded internal retry (2 attempts) was already exhausted before this job-level failure. Nothing in this PR's diff touches Strix's browser/proxy runtime.

Re-running once now per this repo's CI-red protocol, since this is exactly the transient-infra class that classification exists for.


Generated by Claude Code

@seonghobae

Copy link
Copy Markdown
Contributor Author

Contextual-Orchestrator와 관계한 것들을 같이 손보든 어쩌든 해결하세요. Bypass merge 필요하면 가능 (chicken and eggs 상황이라면) + NVIDIA NIM 만 쓰는 건 허용하지 않아요. Contextual-Orchestrator를 쓰세요. Timeout은 적어도 3시간으로 잡으세요. 120초 같은 건 당황스럽군요. Opencode와 Noema 는 Coderabbitai 및 Devin 수준으로 실제로 리뷰를 하게 하시오. Strix도 보안 리뷰를 꼼꼼하게 하도록 하시오. 특히 보안 리뷰는 전체 코드로 수행하는 것입니다. Contextual-Orchestrator는 실시간으로 빠르면서 능력이 좋은 모델에 요청을 보내어 시간을 당기시오.

@seonghobae
seonghobae merged commit 1186a9f into main Sep 1, 2026
45 of 49 checks passed
@seonghobae
seonghobae deleted the revert-strix-disable-streaming branch September 1, 2026 00:55
seonghobae pushed a commit that referenced this pull request Sep 1, 2026
…revert

Main moved again mid-rebase: .github#1463 reverted #1448's
LLM_DISABLE_STREAMING workaround (which the previous commit on this
branch had just cited as the fix) now that the real root cause is
fixed upstream -- contextual-orchestrator#925 (merge commit 7944a3c,
"accept stream_options.include_usage=true for tools passthrough")
makes the gateway itself stop rejecting the combination, and #1463
bumped ORCHESTRATOR_PIN_SHA to 7944a3c so the vendored sidecar
actually carries that fix (verified: `git merge-base --is-ancestor
7944a3c <current pin>` is true).

The conclusion is unchanged (the gap is fixed) but the mechanism
is not what the previous commit said, so corrects both the
gap-baseline entry and this PR's own CHANGELOG bullet to point at the
current, real fix instead of the since-reverted workaround.

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants