feat(api): SSE-proxy tools and response_format on chat completions by cursor[bot] · Pull Request #606 · ContextualWisdomLab/contextual-orchestrator

cursor · 2026-08-16T15:43:32Z

Summary

Successor to #596 at 7a07562. Keeps that head's fail-closed empty messages, spend/routing hoist, and request-knob honesty, and closes the buyer-visible gap those drafts still list as residual: tools / response_format + stream=true used to return 400 invalid_stream (or, before that, a billed JSON completion while the SDK waited for SSE).

TaskOrchestrator.proxy_completion_stream / ModelClient.proxy_stream_send SSE-proxy a single pool agent.
Mock frames are chat.completion.chunk whose concatenated content equals the non-stream JSON message.content.
Live providers are piped verbatim so delta.tool_calls survive (content-only parsers drop them).
Empty messages, seed / stop / n>1 / logprobs, batch routing, and stream_options.include_usage=true still fail closed before the first byte.
/v1/responses streaming stays rejected.

Buyer next action: send stream=true on tool-calling requests when the client reads SSE; omit stream_options.include_usage; still send a non-empty messages array of objects and known sync attribution.

Prefer this head over #582–#587, #589, #591–#597. Do not merge those in parallel. Independent non-author APPROVE + Full unit/Semgrep still required.

Test plan

python3 tests/test_passthrough_sse_tools_http_honesty.py
python3 tests/test_passthrough_stream_model_http_honesty.py
python3 tests/test_passthrough_attribution_routing_http_honesty.py
python3 tests/test_chat_messages_array_tools_passthrough_http_honesty.py
python3 tests/test_message_honesty_tools_passthrough_http_honesty.py
python3 tests/test_openai_passthrough.py test_true_streaming.py
python3 tests/test_paper_contracts.py test_self_check.py test_conventions.py test_api_contract.py
CI Full unit + Semgrep

Docs

docs/rest_api_design.md honesty contract (APA: OpenAI, 2024; WHATWG SSE).
docs/library_research.md Ponytail note: stdlib urllib, no httpx/sse-starlette.
README architecture bullet tells buyers to send stream=true on tool-calling requests.

…closed otherwise Chat history: message-level audio and legacy function_call are null/empty omit no-ops; non-empty fail closed with named errors (including tools passthrough). Tip substrate from #577 assistant refusal/annotations honesty. Local full unit: 940 passed.

…ed otherwise OpenAI fine-tune style message weight is not applied on this gateway. Accept null/0/1 as honest no-ops; reject other types and values with invalid_message_weight. Tip substrate from #578. Local full unit: 943 passed.

…ion role Reject unsupported message keys with named unknown_message_fields (not silent strip or tools-passthrough smuggle). Reject legacy function role with invalid_message_role migration to tool. Tip substrate from #579. Local full unit: 947 passed.

OpenAI partial-assistant prefix flag is not applied on this gateway. null/false are honest no-ops; true and non-booleans fail closed with invalid_message_prefix. Tip substrate from #580. Local full unit: 950 passed.

…therwise Named invalid_max_tool_calls on /v1/chat/completions instead of opaque unknown_fields. Aligns with Responses max_tool_calls honesty; gateway has no multi-step tool loop.

…losed otherwise Legacy /v1/completions treated max_tool_calls as unknown_fields. Accept the key for named invalid_max_tool_calls (null/empty/whitespace omit-equivalent), matching chat/Responses honesty so SDKs get a clear migration path.

SDK clients often send include_usage/include_obfuscation as JSON null. Drop null flag values before validation so null (and null+false mixes) match omit / all-false no-ops on chat, Completions, and Responses. True flags remain fail-closed with invalid_stream_options.

…age honesty Null include_usage/include_obfuscation stay omit-equivalent, but unknown stream_options keys no longer become no-ops just because their value is null. Weight, prefix, refusal, annotations, developer role, empty user/system content, and participant name now use the same named errors on the tools passthrough path as on orchestration. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Hoist stream, required model, stream_options, and temperature/top_p range checks before proxy_completion so a tools or response_format body cannot return a billed JSON completion when the SDK asked for SSE, or silently pick a pool model when model is omitted. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Hoist attribution and routing validation before proxy_completion so a tools or response_format body cannot bill a sync completion with an unknown spend dimension or a batch/latency_tolerant hint. Tools passthrough has no batch job plane. Buyer next action: send known sync attribution; omit routing.channel=batch and latency_tolerant=true on tool-calling requests. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Call the orchestration message, max_tokens, attribution, and routing validators before proxy_completion so a tools or response_format body cannot bill a completion with no prompt, crash on a non-object entry, or silently drop unknown spend/routing keys. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Match _validate_messages: tools + user content 123 must 400 invalid_message instead of billing a sync completion. Assistant/tool JSON null stays omit-equivalent. Buyer next action: send user/system content as a non-empty string or a content-parts array. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Match the tools-path honesty contract to the invalid_message behavior so SDK clients send a string or content-parts array. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Passthrough has no batch job plane. Reject routing.channel=batch and latency_tolerant=true before proxy_completion so a tool-calling body cannot bill a silent sync completion. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Hoist the remaining chat request knobs before proxy_completion so an OpenAI SDK tool-calling body cannot bill a sync completion for seed, stop, n>1, logprobs, logit_bias, out-of-range token/penalty values, unsupported reasoning_effort, or a non-default service_tier. Buyer next action: omit those fields on tool-calling requests; they are not applied on this gateway. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Call _validate_messages before proxy_completion so tools + [] / omitted / null / non-list / non-object messages raise invalid_message instead of billing a completion with no prompt. Buyer next action: always send a non-empty messages array of objects on tool-calling requests. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Hoist the orchestration request-knob validators before proxy_completion so a tools or response_format body cannot bill a completion that silently drops seed, stop, n>1, logprobs, logit_bias, or out-of-range penalties. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

stream_chat defaulted to 0.2 even after the HTTP path wrote default_temperature from the request. A streamed invoice summary at temperature=0.8 no longer silently falls back to 0.2. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

… proxy SDK optional defaults serialize omitted fields as JSON null. Accepting those keys without popping them is not omit-equivalent: proxy_completion forwards the body and several providers reject a null JSON Schema object. Pop the keys in place, keep non-null wrong types on invalid_tools, and assert omit-real via mock echo on chat and Responses. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

OpenAI SDK tool callers send stream=true. Returning 400 was honest but left every streaming tool client failing. Pipe a single pool agent as chat.completion.chunk frames so concatenated mock content matches the non-stream JSON body, and pipe live provider text so tool_calls deltas survive. Keep include_usage, empty messages, and unsupported knobs fail-closed. Buyer next action: send stream=true on tool-calling requests; omit stream_options.include_usage. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

Keep tools/response_format SSE proxy from this branch and take #601's null tool.function omit-pop, request-knob HTTP cases, and streamed route temperature. Conflict resolution keeps stream=true as SSE, not 400. CHANGELOG records the combined unreleased slice. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

cursor

Stale comment

Next slice already ported — cherry-pick, do not open a parallel stack

This head is the current honesty + SSE-tools tip. Do not merge #582–#605 in parallel.

Remaining fail-open named in the last review: tools + unknown tool_calls entry/function keys still 200. That hole is closed on #591 (b7f81a6) and already ported as cherry-pick source a3b7da0 on cursor/bc-b3f7c89f-c445-4d73-a0cd-16d6a8452e1d-1c23 (tests/test_tool_calls_entry_keys_http_honesty.py). Also port 4e3bc12 if this merge dropped thread-local ModelClient.request_sampling.

Do not merge that cherry-pick branch as a 140-file stack. Put a3b7da0 on this tip, then hoist include_orchestration_trace / mode type checks before proxy_completion / SSE proxy.

Buyer next action: send only id / type / function / optional index on assistant tool_calls. Omit stream only if you still need JSON; SSE tools passthrough is this tip's unique work. After the key port: rebase #598 a50911f KV allowlist onto this head.

Independent non-author APPROVE + Full unit / Semgrep still required. Do not self-approve.

_{Sent by Cursor Automation: Fix Issues}

cursor · 2026-08-16T15:59:12Z

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

cursor

Stale comment

Review

#606 at 0717921 still frames mock:// tools as delta.content + finish_reason=stop. A realistic invoice-lookup body (tools[0].function.name = lookup_balance, user text INV-9) must emit delta.tool_calls and finish as tool_calls so offline SDK clients match the live _stream_raw path this PR already preserves.

That helper change is on successor #617 63089e4 (this branch plus the mock tool_calls frames). Prefer #617 over merging #606. Do not merge #582–#606 in parallel.

Buyer next action: point stream clients at #617; send stream=true and include the invoice id in the user message.

Independent non-author APPROVE + Full unit/Semgrep still required. This automation does not self-approve.

View PR

_{Sent by Cursor Automation: Fix Issues}

cursor

Do not merge this tip. Live _stream_raw already preserves delta.tool_calls, but _mock_raw_sse still finishes every tools stream as stop with message.content chunks (contextual_orchestrator/orchestrator.py around the final _frame({}, finish="stop")). An invoice-lookup body (tools[0].function.name = lookup_balance) must emit delta.tool_calls and finish_reason=tool_calls so offline SDK clients match the billed provider stream.

The fix is already on successor #620 (8d7d534): JSON and SSE reconstruct to the same message.tool_calls, INV- binds from the user text (default INV-9), and tool_choice=none stays content/stop.

Buyer next action: review and land #620 instead of this head. Send stream=true on tool-calling requests and put the invoice id in the user message. Do not merge #606 / #609 / #610 / #613 in parallel. Independent non-author APPROVE is still required — this run cannot approve its own stack.

View PR

_{Sent by Cursor Automation: Fix Issues}

cursor

Do not merge this tip. _mock_raw_sse still finishes tools as stop with content chunks, so an invoice lookup_balance stream does not match the live _stream_raw delta.tool_calls shape.

Land #620 (baf2e84) instead. That tip already contains this stack plus:

mock JSON/SSE tool_calls (INV- binds from the user text, default INV-9; tool_choice=none stays content/stop)
streamed route top_p / penalties matching chat()
APA 7th citations (OpenAI function calling; Schick et al., 2023; Yao et al., 2023; Holtzman et al., 2020)

#617 is the same functional mock+sampling slice without those citations. Do not merge #606 / #609 / #610 / #613 / #617 in parallel with #620.

Buyer next action: review #620, send stream=true on tool-calling requests, and put the invoice id in the user message. Independent non-author APPROVE is still required.

View PR

_{Sent by Cursor Automation: Fix Issues}

opencode-agent

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.
Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.
Result: REQUEST_CHANGES
Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 0717921ca20a8bd532505b49896a30ebda556288.
Head SHA: 0717921ca20a8bd532505b49896a30ebda556288
Workflow run: 32103923479
Workflow attempt: 1

Coverage evidence

Coverage evidence job did not run or did not publish coverage evidence.

Changed-File Evidence Map

flowchart LR
  Evidence["OpenCode evidence"] --> Review["Current PR review path"]
  Review --> Verify["Required checks"]

opencode-agent · 2026-08-18T05:44:25Z

OpenCode Review Overview

Head SHA: 0717921ca20a8bd532505b49896a30ebda556288
Workflow run: 32103923479
Workflow attempt: 1
Gate result: REQUEST_CHANGES (approval step)

Pull request overview

OpenCode cannot approve yet because required coverage evidence did not pass.

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

Problem: The required coverage-evidence job result was failure, so OpenCode cannot establish approval sufficiency for this head.
Root cause: Automated approval is only valid when the same-head coverage-evidence job proves supported repository test suites passed and configured docstring gates passed or were advisory, or reports not applicable because no supported source files or package manifests exist. Missing, failed, skipped, unavailable, or unsupported-tooling test evidence is a blocker.
Fix: Install or configure the repository test/docstring evidence tooling when source files or package manifests exist, rerun the current-head coverage-evidence job, and approve only after it reports success with required evidence or explicit no-source not-applicable evidence.
Regression test: Keep the approval branch checking needs.coverage-evidence.result == success before posting APPROVE, and publish REQUEST_CHANGES when coverage-evidence blocker states such as cancelled, skipped, failed, unsupported-tooling, or below-100 evidence are present.
Result: REQUEST_CHANGES
Reason: coverage-evidence result was failure, so required test/docstring evidence was not proven for current head 0717921ca20a8bd532505b49896a30ebda556288.
Head SHA: 0717921ca20a8bd532505b49896a30ebda556288
Workflow run: 32103923479
Workflow attempt: 1

Coverage evidence

Coverage evidence job did not run or did not publish coverage evidence.

Changed-File Evidence Map

flowchart LR
  Evidence["OpenCode evidence"] --> Review["Current PR review path"]
  Review --> Verify["Required checks"]

seonghobae and others added 23 commits August 16, 2026 21:18

fix(api): treat message prefix null/false as omit; fail-closed on true

82038d3

OpenAI partial-assistant prefix flag is not applied on this gateway. null/false are honest no-ops; true and non-booleans fail closed with invalid_message_prefix. Tip substrate from #580. Local full unit: 950 passed.

fix(api): treat chat max_tool_calls null/empty as omit; fail-closed o…

1a196b0

…therwise Named invalid_max_tool_calls on /v1/chat/completions instead of opaque unknown_fields. Aligns with Responses max_tool_calls honesty; gateway has no multi-step tool loop.

docs(api): tell buyers non-string user content fails closed on tools

f6dfae5

Match the tools-path honesty contract to the invalid_message behavior so SDK clients send a string or content-parts array. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

docs: tell buyers to omit batch routing on tool-calling requests

dbfa0e5

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

docs: record passthrough knob and stream temperature fixes in CHANGELOG

7e9a339

Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>

cursor Bot requested a review from seonghobae August 16, 2026 15:43

cursor Bot mentioned this pull request Aug 16, 2026

fix(api): fail-closed empty messages, spend keys, and request knobs on tools passthrough #596

Closed

6 tasks

cursor Bot commented Aug 16, 2026

View reviewed changes

This was referenced Aug 16, 2026

fix(api): fail-closed seed, spend knobs, and batch hints on tools path #605

Closed

fix(api): fail-closed store, modalities, and surface knobs on tools path #609

Closed

seonghobae mentioned this pull request Aug 16, 2026

fix(api): cap tool.function.description at 1024 characters fail-closed #602

Closed

3 tasks

seonghobae marked this pull request as ready for review August 16, 2026 15:59

seonghobae enabled auto-merge (squash) August 16, 2026 15:59

cursor Bot mentioned this pull request Aug 16, 2026

fix(api): mock SSE tool_calls and streamed sampling knobs #617

Closed

8 tasks

cursor Bot commented Aug 16, 2026

View reviewed changes

This was referenced Aug 16, 2026

fix(security): read provider host allowlist from KV on the honesty tip #607

Closed

fix(api): mock SSE tool_calls and streamed route sampling #620

Closed

fix(api): accept Responses-native function tools on /v1/responses #604

Closed

cursor Bot commented Aug 16, 2026

View reviewed changes

opencode-agent Bot requested changes Aug 18, 2026

View reviewed changes

opencode-agent Bot disabled auto-merge August 18, 2026 09:59

seonghobae closed this Aug 20, 2026

Navigation Menu

Uh oh!

feat(api): SSE-proxy tools and response_format on chat completions - #606

feat(api): SSE-proxy tools and response_format on chat completions#606
cursor[bot] wants to merge 25 commits into
mainfrom
cursor/bc-99e57979-3f95-48ff-85c8-cb3ebb582df8-8170

cursor Bot commented Aug 16, 2026

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

cursor Bot commented Aug 16, 2026

Uh oh!

cursor Bot left a comment •

edited

Loading

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot left a comment

Uh oh!

opencode-agent Bot left a comment

Uh oh!

opencode-agent Bot commented Aug 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

cursor Bot commented Aug 16, 2026

Summary

Test plan

Docs

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Next slice already ported — cherry-pick, do not open a parallel stack

Uh oh!

cursor Bot commented Aug 16, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Review

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

opencode-agent Bot left a comment

Choose a reason for hiding this comment

Pull request overview

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

Coverage evidence

Changed-File Evidence Map

Uh oh!

opencode-agent Bot commented Aug 18, 2026

OpenCode Review Overview

Pull request overview

Review outcome

1. HIGH .github/workflows/opencode-review.yml:1 - Coverage evidence did not prove required test/docstring evidence

Coverage evidence

Changed-File Evidence Map

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading