[Responses] Decouple SSE event helpers from Harmony context

[sfeng33]

Purpose

• Refactor SSE streaming event helpers to decouple shared event-building logic from Harmony-specific context
• Fix a bug for build-in python tool to generate code interpreter events instead of mcp events (according to https://community.openai.com/t/responses-api-streaming-the-simple-guide-to-events/1363122)
• Enhance unit test to validate streaming events

Architecture: Two-layer design in streaming_events.py

The core refactor splits streaming_events.py into two layers: dispatchers that understand Harmony context objects, and leaf helpers that only accept plain strings. Dispatchers extract values from StreamingHarmonyContext / HarmonyMessage and delegate to leaf helpers, which build SSE events from primitive types. This means the event-building logic is reusable by any future backend without depending on Harmony.

     serving.py                streaming_events.py
    ┌──────────┐       ┌────────────────────────────────────────────────┐
    │          │       │                                                │
    │  ctx ────┼──────▶│  DISPATCHERS (Harmony-specific)               │
    │          │       │  ┌──────────────────────────────────────┐      │
    │          │       │  │ emit_content_delta_events(ctx,state) │      │
    │          │       │  │ emit_previous_item_done_events(prev) │      │
    │          │       │  │ emit_tool_action_events(ctx,state,ts)│      │
    │          │       │  └──────────────┬───────────────────────┘      │
    │          │       │                 │ extract plain values         │
    │          │       │                 ▼                              │
    │          │       │  LEAF HELPERS (backend-agnostic)               │
    │          │       │  ┌──────────────────────────────────────┐      │
    │          │       │  │  Delta:                              │      │
    │          │       │  │    emit_text_delta_events(str,.)     │      │
    │          │       │  │    emit_reasoning_delta_events(str,.)│      │
    │          │       │  │    emit_function_call_delta_ev(str,.)│      │
    │          │       │  │    emit_mcp_delta_events(str,.)      │      │
    │          │       │  │    emit_code_interp_delta_ev(str,.)  │      │
    │          │       │  │                                      │      │
    │          │       │  │  Done:                               │      │
    │          │       │  │    emit_text_output_done_events(str) │      │
    │          │       │  │    emit_reasoning_done_events(str)   │      │
    │          │       │  │    emit_function_call_done_ev(str,.) │      │
    │          │       │  │    emit_mcp_completion_events(str,.) │      │
    │          │       │  │    emit_code_interp_completion(..)   │      │
    │          │       │  │    emit_browser_tool_events(..)      │      │
    │          │       │  └──────────────────────────────────────┘      │
    │          │       │                 │                              │
    │          │       │                 ▼                              │
    │  events◄─┼───────│  list[StreamingResponsesResponse]             │
    │          │       │                                                │
    └──────────┘       └────────────────────────────────────────────────┘

Test Plan

pytest tests/entrypoints/openai/responses/test_harmony.py

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @sfeng33.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[gemini-code-assist]

Code Review

This pull request is a significant and well-executed refactoring of the SSE event generation logic. The decoupling of event helpers from the Harmony-specific context into dispatchers and backend-agnostic leaf helpers is a great architectural improvement that enhances reusability and maintainability. The accompanying enhancements to the test suite, particularly the more robust validation of event stream pairing, ordering, and field consistency, are also excellent and will help ensure correctness going forward.

However, I've found one critical issue in the refactored logic for function calls. The call_id for a ResponseFunctionToolCall is not consistent between its in_progress and completed states in a streaming response. This breaks the tool-calling protocol for clients. Please see the detailed comment for more information.

[sfeng33]

PTAL: @qandrew @mgoin
cc @bbrowning

[chaunceyjiang]

Thanks~

[chaunceyjiang]

/cc @qandrew PTAL.

[chaunceyjiang]

https://buildkite.com/vllm/ci/builds/52856#019c8e44-d8e8-42f7-a2a8-b6b25c0767d9 @sfeng33 PTAL.

[qandrew]

thanks for the PR! lgtm, i trust Chauncey's review. is the eventual goal to have simpleContext/parsableContext use the logic in streaming_events too?

also cc @daniel-salib who is working on #35184 for streaming

[sfeng33]

thanks for the PR! lgtm, i trust Chauncey's review. is the eventual goal to have simpleContext/parsableContext use the logic in streaming_events too?

also cc @daniel-salib who is working on #35184 for streaming

Thanks for taking a look, yes the goal is for parsableContext to re-use streaming_events for sse events. And deprecate simpleContext eventually.

[bbrowning]

This looks like a reasonable cleanup that also fixes some bugs in our Responses streaming events. I don't think it necessarily fixes all the bugs (left a comment in one place about browser and container events), but I don't think fixing all bugs is necessarily the bar. Were you able to test this with a live model and a real client just to verify streaming behavior outside of what the unit test has?

[sfeng33]

This looks like a reasonable cleanup that also fixes some bugs in our Responses streaming events. I don't think it necessarily fixes all the bugs (left a comment in one place about browser and container events), but I don't think fixing all bugs is necessarily the bar. Were you able to test this with a live model and a real client just to verify streaming behavior outside of what the unit test has?

Totally agree there are remaining bugs, in this PR I tried to keep it the same functionality, while fixing the one obvious bug I list in the PR summary. In terms of manual testing, I tested with gpt oss 20b model in the basic text and reasoning case, and see the stream events are the same as main. For the tool call/mcp/function call events, I actually think the way the events are emitted aren't as expected as the openai response api should emit, e.g. we emit all browser related events once, where we should emit right after tool call is executed.