fix: allow streamed usage on the tool-loop chat passthrough

[seonghobae]

Summary

• /v1/chat/completions rejected stream=true + stream_options.include_usage=true whenever tools were present, on the premise (from PR fix: accept Chat Completions streaming usage options #914) that this gateway "cannot emit that SSE contract" for tool calls. That premise is factually wrong for this specific path: a tools_list request routes through the single-agent tool-loop, which already resolves via proxy_completion (upstream stream is forced to False) before _chat_response_sse_chunks frames the complete response as SSE — its usage is already known when framing starts, not synthesized from partial output.
• Scoped the fail-closed guard to the case where the premise is actually true: the multi-step response_format-without-tools conduct path, whose usage would reflect only the final synthesizer step and undercount the real workflow cost.
• Removed _chat_response_sse_chunks's text-derived usage_source: "estimated" fallback for a provider response with no usage field — a token count guessed from response text is not this gateway's usage to report as fact. When the provider omits it, the usage chunk is now silently omitted from the stream, matching the same provider-reported-only honesty boundary already enforced on the other two streaming usage paths (ModelClient.stream_chat's relay-only usage, chat_completion_chunks' cost.measurement_status=="measured" gate).

This was root-caused via an 18-agent adversarially-verified investigation (gather → hypothesize → verify → synthesize) that read the actual code paths, the existing regression tests, the git history behind the original guard, and OpenAI's own API docs for stream_options+tools compatibility.

Developer experience

• tests/test_chat_tools_passthrough_controls_http_honesty.py and tests/test_stream_options_null_flags_noop_http_honesty.py split their combined tools+response_format 400 assertions into two tests each, one confirming the new 200 success path with a real usage_source: "reported" frame, one confirming response_format-only still fails closed.
• Removed _chat_response_sse_chunks's now-unused prompt_text parameter and the estimate_tokens import that only that removed branch used.

User experience

• A tool-calling client (e.g. an agent using Strix-style tool loops) that streams with stream_options.include_usage=true gets a 200 response with honest usage instead of an unconditional 400, while a client relying on a fabricated text-derived token estimate no longer receives one.

Test plan

• python3 -m pytest tests/test_chat_tools_passthrough_controls_http_honesty.py tests/test_stream_options_null_flags_noop_http_honesty.py tests/test_streaming.py tests/test_chat_stream_options_http_honesty.py tests/test_completions_stream_options_http_honesty.py tests/test_empty_stream_options_include_noop_http_honesty.py tests/test_responses_stream_options_http_honesty.py tests/test_routing_latency_stream_options_bool_coerce_http_honesty.py tests/test_sdk_null_object_optional_noop_http_honesty.py tests/test_stream_options_false_tool_choice_empty_noop_http_honesty.py tests/test_stream_options_unknown_null_reject_http_honesty.py -q — 78 passed
• python3 -m pytest tests -q (full suite) — 2770 passed, 1 skipped, 1 failed (pre-existing fast_mlsirm ModuleNotFoundError in this sandbox, unrelated and reproducible identically on unmerged main)

---

Generated by Claude Code

[coderabbitai]

Warning

Review limit reached

Next included review available in 37 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 2956db49-a845-40b0-8cd4-9ee729af9281

📥 Commits

Reviewing files that changed from the base of the PR and between 59bc2bd and 7a9faef.

📒 Files selected for processing (5)

• CHANGELOG.d/tool-loop-stream-usage-honest-reporting.md
• contextual_orchestrator/server.py
• tests/test_chat_tools_passthrough_controls_http_honesty.py
• tests/test_stream_options_null_flags_noop_http_honesty.py
• tests/test_streaming.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[seonghobae]

The failing opencode-review check (3s runtime, no opencode-agent review posted at this head) is the same "no current-head verdict dispatched yet" pipeline-timing gate seen on other PRs today — not a failure in this diff. noema-review (the actual LLM review that ran) completed normally in ~3 minutes and passed. Review dispatch for this PR is pending the scheduler's normal rotation across the org's PR queue; no action needed here.

---

Generated by Claude Code

[seonghobae]

Closing as superseded by #925 (merged), which fixed the same root cause — the stream_options.include_usage=true + tools rejection — independently and concurrently. Both PRs agree on the core diagnosis (the premise "cannot emit that SSE contract" was wrong for the tool-loop path, since proxy_completion already resolves with a known usage before SSE framing starts) and both narrow the rejection to the genuinely-unmeasured response_format-only case.

One real design difference, worth recording rather than silently dropping: this PR removes _chat_response_sse_chunks's text-derived usage_source: "estimated" fallback entirely, so the usage chunk is silently omitted when a provider's response has no usage field. #925 kept that fallback, honestly labeled (usage_source: "estimated", never fabricated as "reported") — and added a dedicated regression test (test_http_chat_tools_streams_estimated_usage_when_provider_omits_it, using a real loopback HTTP provider that omits usage, since mock:// agents always inject a zero-valued one) proving the fallback is honest, not guessed-and-passed-off-as-real. Reasoning for keeping it: stream_options.include_usage=true is an explicit request for a terminal usage frame; a client built against that contract may not handle its total absence gracefully, so an honestly-labeled estimate is more compatible than silence, provided it's never mislabeled as measured — which the added test now pins.

If there's a case where silent omission is actually the better contract (e.g. a client that treats absence as "not measured" and mislabeling as a live bug class), happy to revisit — but for now main already carries the honest-fallback version via #925, and this PR's own removal of it would conflict directly.

---

Generated by Claude Code

---

Generated by Claude Code