You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Successor to #606 at 0717921. Keeps that head's SSE-proxy tools path, fail-closed tool_calls keys, and mode/trace honesty, and closes two buyer-visible gaps still listed on that stack:
mock:// tools streams finished as stop with message.content while live _stream_raw already preserves delta.tool_calls. Function-tool mocks now emit assistant tool_calls and stream delta.tool_calls with finish_reason=tool_calls. Invoice lookup_balance binds INV- identifiers from the user text (default INV-9).
Streamed route completions applied request temperature but dropped top_p / presence_penalty / frequency_penalty. stream_chat now uses the same request-scoped defaults as chat().
tool_choice=none and response_format streams stay content/stop. Empty messages, seed, and stream_options.include_usage=true still fail closed before the first byte.
Buyer next action: send stream=true on tool-calling requests; include the invoice id in the user message; send the nucleus and penalty values you want on streamed route requests; omit stream_options.include_usage.
Prefer this head over #606, #613, #601, and #582–#605. Do not merge those in parallel. Independent non-author APPROVE + Full unit/Semgrep still required.
…closed otherwise
Chat history: message-level audio and legacy function_call are null/empty omit
no-ops; non-empty fail closed with named errors (including tools passthrough).
Tip substrate from #577 assistant refusal/annotations honesty. Local full unit:
940 passed.
…ed otherwise
OpenAI fine-tune style message weight is not applied on this gateway.
Accept null/0/1 as honest no-ops; reject other types and values with
invalid_message_weight. Tip substrate from #578. Local full unit: 943 passed.
…ion role
Reject unsupported message keys with named unknown_message_fields (not silent
strip or tools-passthrough smuggle). Reject legacy function role with
invalid_message_role migration to tool. Tip substrate from #579. Local full
unit: 947 passed.
OpenAI partial-assistant prefix flag is not applied on this gateway.
null/false are honest no-ops; true and non-booleans fail closed with
invalid_message_prefix. Tip substrate from #580. Local full unit: 950 passed.
…therwise
Named invalid_max_tool_calls on /v1/chat/completions instead of opaque
unknown_fields. Aligns with Responses max_tool_calls honesty; gateway has
no multi-step tool loop.
…losed otherwise
Legacy /v1/completions treated max_tool_calls as unknown_fields. Accept the
key for named invalid_max_tool_calls (null/empty/whitespace omit-equivalent),
matching chat/Responses honesty so SDKs get a clear migration path.
SDK clients often send include_usage/include_obfuscation as JSON null.
Drop null flag values before validation so null (and null+false mixes)
match omit / all-false no-ops on chat, Completions, and Responses.
True flags remain fail-closed with invalid_stream_options.
…age honesty
Null include_usage/include_obfuscation stay omit-equivalent, but unknown
stream_options keys no longer become no-ops just because their value is
null. Weight, prefix, refusal, annotations, developer role, empty
user/system content, and participant name now use the same named errors
on the tools passthrough path as on orchestration.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist stream, required model, stream_options, and temperature/top_p range
checks before proxy_completion so a tools or response_format body cannot
return a billed JSON completion when the SDK asked for SSE, or silently
pick a pool model when model is omitted.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist stream, required model, stream_options, and temperature/top_p range
checks before proxy_completion so a tools or response_format body cannot
return a billed JSON completion when the SDK asked for SSE, or silently
pick a pool model when model is omitted.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist attribution and routing validation before proxy_completion so a
tools or response_format body cannot bill a sync completion with an
unknown spend dimension or a batch/latency_tolerant hint. Tools
passthrough has no batch job plane.
Buyer next action: send known sync attribution; omit routing.channel=batch
and latency_tolerant=true on tool-calling requests.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Call the orchestration message, max_tokens, attribution, and routing
validators before proxy_completion so a tools or response_format body
cannot bill a completion with no prompt, crash on a non-object entry,
or silently drop unknown spend/routing keys.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Match _validate_messages: tools + user content 123 must 400 invalid_message
instead of billing a sync completion. Assistant/tool JSON null stays
omit-equivalent.
Buyer next action: send user/system content as a non-empty string or a
content-parts array.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Match the tools-path honesty contract to the invalid_message behavior
so SDK clients send a string or content-parts array.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Passthrough has no batch job plane. Reject routing.channel=batch and
latency_tolerant=true before proxy_completion so a tool-calling body
cannot bill a silent sync completion.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist the remaining chat request knobs before proxy_completion so an
OpenAI SDK tool-calling body cannot bill a sync completion for seed,
stop, n>1, logprobs, logit_bias, out-of-range token/penalty values,
unsupported reasoning_effort, or a non-default service_tier.
Buyer next action: omit those fields on tool-calling requests; they are
not applied on this gateway.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Call _validate_messages before proxy_completion so tools + [] / omitted /
null / non-list / non-object messages raise invalid_message instead of
billing a completion with no prompt.
Buyer next action: always send a non-empty messages array of objects on
tool-calling requests.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist the orchestration request-knob validators before proxy_completion
so a tools or response_format body cannot bill a completion that silently
drops seed, stop, n>1, logprobs, logit_bias, or out-of-range penalties.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
stream_chat defaulted to 0.2 even after the HTTP path wrote
default_temperature from the request. A streamed invoice summary at
temperature=0.8 no longer silently falls back to 0.2.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
… proxy
SDK optional defaults serialize omitted fields as JSON null. Accepting those
keys without popping them is not omit-equivalent: proxy_completion forwards
the body and several providers reject a null JSON Schema object.
Pop the keys in place, keep non-null wrong types on invalid_tools, and assert
omit-real via mock echo on chat and Responses.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
OpenAI SDK tool callers send stream=true. Returning 400 was honest but
left every streaming tool client failing. Pipe a single pool agent as
chat.completion.chunk frames so concatenated mock content matches the
non-stream JSON body, and pipe live provider text so tool_calls deltas
survive. Keep include_usage, empty messages, and unsupported knobs
fail-closed.
Buyer next action: send stream=true on tool-calling requests; omit
stream_options.include_usage.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Keep tools/response_format SSE proxy from this branch and take #601's
null tool.function omit-pop, request-knob HTTP cases, and streamed
route temperature. Conflict resolution keeps stream=true as SSE, not
400. CHANGELOG records the combined unreleased slice.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Unknown assistant tool_calls entry/function fields, non-boolean
include_orchestration_trace, and unknown mode used to bill a JSON or SSE
completion on the tools proxy. They now return the same named 400s as the
orchestration path before the first byte.
Buyer next action: send only id/type/function/optional index on assistant
tool_calls; send a boolean or omit include_orchestration_trace; send
auto/route/conduct or omit mode.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
mock:// lookup_balance now returns assistant tool_calls and streams
delta.tool_calls with finish_reason=tool_calls so offline SDK clients
match the live _stream_raw path. Invoice identifiers bind from the
user text (default INV-9).
Buyer next action: send stream=true on tool-calling requests and include
the invoice id in the user message.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
stream_chat recorded temperature but omitted the request-scoped top_p,
presence_penalty, and frequency_penalty that chat() already forwards.
A streamed invoice summary at top_p=0.1 no longer drops those knobs.
Buyer next action: send the nucleus and penalty values you want on
stream=true route requests; streaming matches the JSON path.
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
seonghobae
changed the title
fix(api): mock SSE tool_calls for lookup_balance
fix(api): mock SSE tool_calls and streamed sampling knobs
Aug 16, 2026
_mock_raw / _mock_raw_sse now emit assistant tool_calls and delta.tool_calls with finish_reason=tool_calls for function tools. Invoice lookup_balance binds INV- from user text (default INV-9). tool_choice=none stays content/stop.
stream_chat records and forwards request-scoped top_p / presence_penalty / frequency_penalty the same way chat() already does. The HTTP invoice-summary case at top_p=0.1 no longer leaves last_top_p as None.
Docs and CHANGELOG tell the buyer to send stream=true, include the invoice id, and send nucleus/penalty knobs on streamed route requests.
No independent merge from this automation. Draft + REVIEW_REQUIRED. Independent non-author APPROVE + Full unit/Semgrep still required. Prefer this head over #606, #615, #613, and #601. Do not merge those in parallel.
Residual (do not block this slice)
/v1/responsesstream=true still 400s on this tip, and the Responses mock still returns content when tools are present. That is the next buyer-visible gap and is landing on the successor stacked on this head (feat(api): SSE-proxy /v1/responses with mock function_call events). KV host allowlist remains #619 — rebase/allowlist only there; do not open a third allowlist PR.
Buyer next action on this head: send stream=true on chat tool-calling requests; include the invoice id in the user text; send the nucleus and penalty values you want on streamed route requests; omit stream_options.include_usage.
Unique slice is sound. This head is now ready for review at a1823cc.
_mock_raw / _mock_raw_sse emit assistant tool_calls and delta.tool_calls with finish_reason=tool_calls for function tools. Invoice lookup_balance binds INV- from user text (default INV-9). Named tool_choice wins; tool_choice=none stays content/stop. Content-part prompts still bind.
stream_chat records and forwards request-scoped top_p / presence_penalty / frequency_penalty the same way chat() already does. The HTTP invoice-summary case at top_p=0.1 no longer leaves last_top_p as None.
Docs and CHANGELOG tell the buyer to send stream=true, include the invoice id, and send nucleus/penalty knobs on streamed route requests.
Re-verified on this exact tip: tests/test_passthrough_sse_tools_http_honesty.py, tests/test_stream_chat_request_sampling_http_honesty.py, tests/test_openai_passthrough.py, tests/test_true_streaming.py, tests/test_streaming.py, tests/test_paper_contracts.py, tests/test_self_check.py, tests/test_api_contract.py, tests/test_conventions.py all printed ok.
No independent merge from this automation. Independent non-author APPROVE + Full unit/Semgrep still required. Prefer this head as the chat-SSE-tools substrate for #628. Do not merge #606, #615, #613, #601, or #620 in parallel — #620 is the same functional slice plus Holtzman/Toolformer/ReAct citations; fold those APA 7th references into #628 rather than landing a second mock-tool_calls tip.
Residual (do not block this slice)
/v1/responsesstream=true still 400s here, and the Responses mock still returns content when tools are present. That buyer gap is landing on #628 (a5882c8) stacked on this head. Prefer #628 over #627 (content-only Responses SSE, no function_call events). KV host allowlist + token KV remain #621 (not #611/#616/#619). Mode/trace landing remains #629. Do not open a third mock-tool_calls, Responses-stream, or allowlist PR.
Buyer next action on this head: send stream=true on chat tool-calling requests; put the invoice id in the user text as INV-…; send the nucleus and penalty values you want on streamed route requests; omit stream_options.include_usage.
Deferring to tip #691 for product-gate focus (Full unit + Semgrep). Reopen if this delta is not on tip substrate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
seonghobae
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Summary
Successor to #606 at
0717921. Keeps that head's SSE-proxy tools path, fail-closedtool_callskeys, and mode/trace honesty, and closes two buyer-visible gaps still listed on that stack:mock://tools streams finished asstopwithmessage.contentwhile live_stream_rawalready preservesdelta.tool_calls. Function-tool mocks now emit assistanttool_callsand streamdelta.tool_callswithfinish_reason=tool_calls. Invoicelookup_balancebindsINV-identifiers from the user text (defaultINV-9).temperaturebut droppedtop_p/presence_penalty/frequency_penalty.stream_chatnow uses the same request-scoped defaults aschat().tool_choice=noneandresponse_formatstreams stay content/stop. Empty messages,seed, andstream_options.include_usage=truestill fail closed before the first byte.Buyer next action: send
stream=trueon tool-calling requests; include the invoice id in the user message; send the nucleus and penalty values you want on streamed route requests; omitstream_options.include_usage.Prefer this head over #606, #613, #601, and #582–#605. Do not merge those in parallel. Independent non-author APPROVE + Full unit/Semgrep still required.
Test plan
python3 tests/test_passthrough_sse_tools_http_honesty.pypython3 tests/test_stream_chat_request_sampling_http_honesty.pypython3 tests/test_openai_passthrough.pypython3 tests/test_passthrough_trace_mode_http_honesty.pypython3 tests/test_tool_calls_entry_keys_http_honesty.pypython3 tests/test_true_streaming.pytest_streaming.pypython3 tests/test_paper_contracts.pytest_self_check.pytest_conventions.pytest_api_contract.pyDocs
docs/rest_api_design.mdhonesty contract (APA: OpenAI, 2024; WHATWG SSE).docs/library_research.mdPonytail note: mock frames now matchproxy_completionincludingtool_calls.stream=true, include the invoice id, and send nucleus/penalty knobs on streamed route requests.