[Frontend] Add streaming tool-call support to Responses API (non-Harmony)

[sumitaryal]

Purpose

Fix for #29725,

Summary

This pull request fixes an issue where non harmony models using the Responses API with streaming and tools emit only ResponseTextDeltaEvent events, instead of ResponseFunctionCallArgumentsDeltaEvent when a tool call is selected. This prevents clients from reliably detecting and parsing tool call arguments from the stream.

Fix

This change updates the streaming path for non harmony models so that:

When the model selects a tool call, the arguments are surfaced as ResponseFunctionCallArgumentsDeltaEvent instead of plain text deltas. The event structure is now consistent with harmony models and with the non streaming Responses API behavior. With this, clients can treat harmony and non harmony models uniformly when handling tool calls during streaming.

Test Plan

Added a e2e test

Test Result:

For the example mentioned in the issue, the events emitted are as:

ResponseOutputItemAddedEvent(item=ResponseFunctionToolCall(arguments='', call_id='call_89716b1d95f08274', name='get_weather', type='function_call', id='chatcmpl-tool-a36680e5d0778655', status='in_progress'), output_index=0, sequence_number=2, type='response.output_item.added')
ResponseFunctionCallArgumentsDeltaEvent(delta='{"location": "', item_id='chatcmpl-tool-a36680e5d0778655', output_index=0, sequence_number=3, type='response.function_call_arguments.delta')
ResponseFunctionCallArgumentsDeltaEvent(delta='Paris', item_id='chatcmpl-tool-a36680e5d0778655', output_index=0, sequence_number=4, type='response.function_call_arguments.delta')
ResponseFunctionCallArgumentsDeltaEvent(delta='"}', item_id='chatcmpl-tool-a36680e5d0778655', output_index=0, sequence_number=5, type='response.function_call_arguments.delta')
ResponseFunctionCallArgumentsDoneEvent(arguments='{"location": "Paris"}', item_id='chatcmpl-tool-a36680e5d0778655', name='get_weather', output_index=0, sequence_number=6, type='response.function_call_arguments.done')
ResponseOutputItemDoneEvent(item=ResponseFunctionToolCall(arguments='{"location": "Paris"}', call_id='call_89716b1d95f08274', name='get_weather', type='function_call', id='chatcmpl-tool-a36680e5d0778655', status='completed'), output_index=0, sequence_number=7, type='response.output_item.done')

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

[gemini-code-assist]

Code Review

This pull request adds support for streaming tool calls in the Responses API for non-Harmony models, which is a great enhancement. The changes introduce new logic to parse tool calls from the streaming response and emit the correct events, aligning the behavior with Harmony models.

My review focuses on the new parsing logic. I've identified a critical issue in the manual JSON parsing implementation that could lead to incorrect behavior with certain tool call arguments. I've also pointed out an area where error handling could be improved to make debugging easier.

The addition of an end-to-end test for streaming tool calls is a good step towards ensuring correctness.

[gemini-code-assist]

The current implementation of _bracket_level does not account for JSON string contents. If a string within the JSON contains the opening ({) or closing (}) characters, the bracket level count will be incorrect. This will cause _filter_delta_text to fail, and ultimately _extract_tool_call_required_streaming will produce incorrect output or fail when streaming tool calls.

For example, a tool call with an argument like {"code": "function foo() { return 1; }"} would be misparsed.

This is a critical issue as it can lead to incorrect parsing of tool calls. For robustness, the parsing logic should be made aware of JSON string boundaries. Consider using partial_json_parser more extensively to handle JSON parsing instead of manual bracket counting, as it is already a dependency and is designed to handle such cases robustly.

[gemini-code-assist]

Catching a broad Exception and silently disabling tool_choice_auto can hide important errors from the user. If the tool parser fails to initialize due to a misconfiguration or a bug, the request will unexpectedly fall back to not using tools, which can be difficult to debug. It would be better to log this as an error and potentially fail the request with an informative message, rather than silently changing its behavior.

[ApostaC]

cc @DarkLight1337 @robertgshaw2-redhat

[chaunceyjiang]

I’m not quite sure I understand. It seems like you’re re-implementing a streaming parser for tool_call?

[sumitaryal]

This isn’t a new parser; it’s the same streaming tool-call handling that Chat already has. For tool_choice='auto' we invoke the existing ToolParser.extract_tool_calls_streaming; the manual code is only for tool_choice='required' which mirrors the Chat code path for tool_choice='required'

[chaunceyjiang]

Thanks @sumitaryal
I think this is overly complicated and doesn’t seem to reuse the existing tool call parser.

Currently, a community contributor has already implemented a simpler and more straightforward solution.

#20874 (comment)

[mergify]

Hi @sumitaryal, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.