feat(api): stamp Responses SSE sequence_number and drop Chat [DONE] - #648
feat(api): stamp Responses SSE sequence_number and drop Chat [DONE]#648cursor[bot] wants to merge 31 commits into
Conversation
…closed otherwise Chat history: message-level audio and legacy function_call are null/empty omit no-ops; non-empty fail closed with named errors (including tools passthrough). Tip substrate from #577 assistant refusal/annotations honesty. Local full unit: 940 passed.
…ed otherwise OpenAI fine-tune style message weight is not applied on this gateway. Accept null/0/1 as honest no-ops; reject other types and values with invalid_message_weight. Tip substrate from #578. Local full unit: 943 passed.
…ion role Reject unsupported message keys with named unknown_message_fields (not silent strip or tools-passthrough smuggle). Reject legacy function role with invalid_message_role migration to tool. Tip substrate from #579. Local full unit: 947 passed.
OpenAI partial-assistant prefix flag is not applied on this gateway. null/false are honest no-ops; true and non-booleans fail closed with invalid_message_prefix. Tip substrate from #580. Local full unit: 950 passed.
…therwise Named invalid_max_tool_calls on /v1/chat/completions instead of opaque unknown_fields. Aligns with Responses max_tool_calls honesty; gateway has no multi-step tool loop.
…losed otherwise Legacy /v1/completions treated max_tool_calls as unknown_fields. Accept the key for named invalid_max_tool_calls (null/empty/whitespace omit-equivalent), matching chat/Responses honesty so SDKs get a clear migration path.
SDK clients often send include_usage/include_obfuscation as JSON null. Drop null flag values before validation so null (and null+false mixes) match omit / all-false no-ops on chat, Completions, and Responses. True flags remain fail-closed with invalid_stream_options.
…age honesty Null include_usage/include_obfuscation stay omit-equivalent, but unknown stream_options keys no longer become no-ops just because their value is null. Weight, prefix, refusal, annotations, developer role, empty user/system content, and participant name now use the same named errors on the tools passthrough path as on orchestration. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist stream, required model, stream_options, and temperature/top_p range checks before proxy_completion so a tools or response_format body cannot return a billed JSON completion when the SDK asked for SSE, or silently pick a pool model when model is omitted. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist stream, required model, stream_options, and temperature/top_p range checks before proxy_completion so a tools or response_format body cannot return a billed JSON completion when the SDK asked for SSE, or silently pick a pool model when model is omitted. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist attribution and routing validation before proxy_completion so a tools or response_format body cannot bill a sync completion with an unknown spend dimension or a batch/latency_tolerant hint. Tools passthrough has no batch job plane. Buyer next action: send known sync attribution; omit routing.channel=batch and latency_tolerant=true on tool-calling requests. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Call the orchestration message, max_tokens, attribution, and routing validators before proxy_completion so a tools or response_format body cannot bill a completion with no prompt, crash on a non-object entry, or silently drop unknown spend/routing keys. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Match _validate_messages: tools + user content 123 must 400 invalid_message instead of billing a sync completion. Assistant/tool JSON null stays omit-equivalent. Buyer next action: send user/system content as a non-empty string or a content-parts array. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Match the tools-path honesty contract to the invalid_message behavior so SDK clients send a string or content-parts array. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Passthrough has no batch job plane. Reject routing.channel=batch and latency_tolerant=true before proxy_completion so a tool-calling body cannot bill a silent sync completion. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist the remaining chat request knobs before proxy_completion so an OpenAI SDK tool-calling body cannot bill a sync completion for seed, stop, n>1, logprobs, logit_bias, out-of-range token/penalty values, unsupported reasoning_effort, or a non-default service_tier. Buyer next action: omit those fields on tool-calling requests; they are not applied on this gateway. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Call _validate_messages before proxy_completion so tools + [] / omitted / null / non-list / non-object messages raise invalid_message instead of billing a completion with no prompt. Buyer next action: always send a non-empty messages array of objects on tool-calling requests. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Hoist the orchestration request-knob validators before proxy_completion so a tools or response_format body cannot bill a completion that silently drops seed, stop, n>1, logprobs, logit_bias, or out-of-range penalties. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
stream_chat defaulted to 0.2 even after the HTTP path wrote default_temperature from the request. A streamed invoice summary at temperature=0.8 no longer silently falls back to 0.2. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
… proxy SDK optional defaults serialize omitted fields as JSON null. Accepting those keys without popping them is not omit-equivalent: proxy_completion forwards the body and several providers reject a null JSON Schema object. Pop the keys in place, keep non-null wrong types on invalid_tools, and assert omit-real via mock echo on chat and Responses. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
OpenAI SDK tool callers send stream=true. Returning 400 was honest but left every streaming tool client failing. Pipe a single pool agent as chat.completion.chunk frames so concatenated mock content matches the non-stream JSON body, and pipe live provider text so tool_calls deltas survive. Keep include_usage, empty messages, and unsupported knobs fail-closed. Buyer next action: send stream=true on tool-calling requests; omit stream_options.include_usage. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Keep tools/response_format SSE proxy from this branch and take #601's null tool.function omit-pop, request-knob HTTP cases, and streamed route temperature. Conflict resolution keeps stream=true as SSE, not 400. CHANGELOG records the combined unreleased slice. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Unknown assistant tool_calls entry/function fields, non-boolean include_orchestration_trace, and unknown mode used to bill a JSON or SSE completion on the tools proxy. They now return the same named 400s as the orchestration path before the first byte. Buyer next action: send only id/type/function/optional index on assistant tool_calls; send a boolean or omit include_orchestration_trace; send auto/route/conduct or omit mode. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
mock:// lookup_balance now returns assistant tool_calls and streams delta.tool_calls with finish_reason=tool_calls so offline SDK clients match the live _stream_raw path. Invoice identifiers bind from the user text (default INV-9). Buyer next action: send stream=true on tool-calling requests and include the invoice id in the user message. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
stream_chat recorded temperature but omitted the request-scoped top_p, presence_penalty, and frequency_penalty that chat() already forwards. A streamed invoice summary at top_p=0.1 no longer drops those knobs. Buyer next action: send the nucleus and penalty values you want on stream=true route requests; streaming matches the JSON path. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
OpenAI Responses SDKs send stream=true. Returning 400 left every streaming Responses client failing. Pipe a single pool agent as named response.* events so a streamed invoice lookup_balance reconstructs to the same function_call as the JSON body (INV-9 from the prompt). Content-only streams still match output_text. Live providers stay verbatim. include_usage=true and non-boolean stream still fail closed. Buyer next action: send stream=true on /v1/responses; include the invoice id in input; omit stream_options.include_usage. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
The script entrypoint still named the old rejects_empty_tools_array helper after the case became an omit-equivalent no-op. Direct python3 tests/test_responses_tools_shape_http_honesty.py now runs. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Official openai-python reconstructs argument chunks with item_id from output_item.added, and .done also carries name. The mock stream now emits those fields. HTTP tests POST the JSON twin, require two-plus argument deltas (INV-20260816009), and keep tool_choice=none on output_text. Buyer next action: send stream=true on /v1/responses; correlate function_call_arguments.delta with item_id; omit include_usage. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
Official response.* events now carry contiguous sequence_number values starting at 0, emit response.in_progress after response.created, and end on response.completed without a Chat Completions data: [DONE] trailer. Invoice lookup streams still reconstruct to the JSON twin. Co-authored-by: Seongho Bae <seonghobae@users.noreply.github.com>
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
| if is_responses: | ||
| error_payload = { | ||
| "type": "response.failed", | ||
| "sequence_number": 0, |
There was a problem hiding this comment.
Unique-slice bug: sequence_number is hardcoded to 0 on the synthetic response.failed event.
Official Responses streaming events require a monotonic sequence_number on response.failed as well. This handler runs after _begin_sse(), so a live (or mock) stream that already flushed response.created / in_progress / deltas has already used 0..N. Re-emitting 0 is a duplicate, not the next index — openai-python orders by sequence_number and will treat this as the first event or a gap.
Missing contract: if proxy_completion_stream yields official events (and/or a Chat data: [DONE] trailer) then raises, the body must contain event: response.failed with sequence_number == last_emitted + 1 (or omit the field honestly) and must not contain data: [DONE].
The [DONE] drop on this branch is correct; the stamped 0 is not.
| status, content_type, body = _post_raw(port, payload) | ||
| assert status == 200, body | ||
| assert content_type.startswith("text/event-stream") | ||
| assert "data: [DONE]" not in body |
There was a problem hiding this comment.
This locks the mock success envelope (contiguous sequence_number, in_progress after created, no [DONE], invoice reconstruct). It does not lock the PR body's other two claims:
- live provider
[DONE]trailer is dropped on/v1/responses - mid-stream
response.failedalso drops[DONE]and continues the sequence
_sse_events already skips [DONE], so a live-trailer regression would be invisible to reconstruction asserts. Add a fake proxy_completion_stream that yields data: [DONE]\n\n and then raises; assert the HTTP body has no data: [DONE] and that response.failed.sequence_number is not a reused 0.
| named errors on the tools path — do not send them on tool-calling requests. | ||
|
|
||
| `/v1/responses` with `stream=true` SSE-proxies a single pool agent as | ||
| named `response.*` events (OpenAI, 2024). Every event carries a |
There was a problem hiding this comment.
Buyer-facing overclaim for this unique slice: "Every event carries a contiguous sequence_number starting at 0."
That is true for mock _mock_responses_sse only. Live providers are still piped verbatim by _stream_raw (orchestrator.py docstring on this same commit). The gateway strips a Chat [DONE] trailer; it does not stamp sequence_number on forwarded frames.
Say: mock streams stamp sequence_number / response.in_progress; live /v1/responses is forwarded as the provider emitted it. Otherwise an SDK told to "order events by sequence_number" will mis-handle a non-OpenAI live hop.
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Unique-slice review (6de6eb0 vs 37d5cc3 only)
The mock happy path is real: contiguous sequence_number from 0, response.in_progress after response.created, no Chat [DONE], invoice lookup_balance still reconstructs to INV-20260816009. Do not merge this 155-file honesty stack onto main.
Do not land #648 as-is. Mid-stream response.failed hardcodes sequence_number: 0 after _begin_sse() has already flushed response.created / in_progress as 0 and 1. openai-python orders by sequence_number and will treat that failed event as the first event or a duplicate.
Landing vehicle for this envelope plus the failed-path contract is the successor that continues sequence_number from the last forwarded event and locks a live [DONE] drop. Prefer that head over #648. Do not open a third envelope PR. Do not fold #619 / #621 here. Do not merge #606 / #615 / #613 / #617 / #627 / #628 in parallel.
Buyer next action on a fixed head: send stream=true on /v1/responses; order mock events by sequence_number; close on response.completed or response.failed; do not wait for Chat data: [DONE].
Residual (not this slice): mock content path still skips response.content_part.done. Failed-stream analytics still record HTTP status_code: 200 because SSE headers are already sent.
Sent by Cursor Automation: Fix Issues
| if is_responses: | ||
| error_payload = { | ||
| "type": "response.failed", | ||
| "sequence_number": 0, |
There was a problem hiding this comment.
sequence_number: 0 is wrong once any official event has been flushed. This handler runs after _begin_sse(), so response.created (0) and response.in_progress (1) already occupy the start of the stream. Re-emitting 0 makes openai-python treat response.failed as the first event.
Continue from the last forwarded sequence_number (or omit the field honestly). Lock it with a raise-after-two-frames HTTP test: types created → in_progress → failed, numbers [0, 1, 2], and data: [DONE] absent. The success-path envelope test cannot catch this.
Pull request was closed
2 participants
Summary
Successor to #628 at
37d5cc3(Responsesstream=truenamedresponse.*events + mockfunction_callkeyed byitem_id). Unique commit closes the remaining official envelope gap on that tip:response.*event carries a contiguoussequence_numberstarting at 0.response.createdis followed byresponse.in_progressbefore output items.response.completed. Chat Completionsdata: [DONE]is dropped on/v1/responses(including a live provider trailer and the mid-streamresponse.failedpath).lookup_balancestill reconstructs to the JSON twin (INV-20260816009still exercises two-plus argument chunks)./v1/chat/completionsstreams still end withdata: [DONE].Buyer next action
Send
stream=trueon/v1/responses. Order events bysequence_number. Correlateresponse.function_call_arguments.deltawithitem_idfromresponse.output_item.added. Close the Responses stream onresponse.completed— do not wait for Chatdata: [DONE]. Include the invoice id ininput. Omitstream_options.include_usage.Prefer this head over #628 for the official envelope slice. Do not merge #606, #615, #613, #617, #627, or #628 in parallel with this head. #619 remains the KV host-allowlist landing — do not fold it here. Independent non-author APPROVE + Full unit/Semgrep are still required. This automation will not APPROVE or merge.
Test plan
python3 tests/test_passthrough_sse_responses_http_honesty.pypython3 tests/test_responses_stream_reject_http_honesty.pypython3 tests/test_stream_null_noop_http_honesty.pypython3 tests/test_responses_stream_options_http_honesty.pypython3 tests/test_passthrough_sse_tools_http_honesty.pypython3 tests/test_streaming.pytest_true_streaming.pytest_stream_chat_request_sampling_http_honesty.pypython3 tests/test_paper_contracts.pytest_self_check.pytest_api_contract.pytest_conventions.pyDocs
docs/rest_api_design.md,docs/architecture.md,docs/library_research.md,docs/papers/README.md,CHANGELOG.md, READMENext loop
After #619 / #621 land the KV host allowlist on this honesty tip: remaining
__main__.pybootstrap tokens (env only as KV seed). Failed-stream analytics still record HTTPstatus_code: 200because SSE headers are already sent — do not fold that into this envelope slice.