Repository navigation
[Rust Frontend] Support namespace tools in the Responses API - #122
Closed
richard-for-bugen[bot] wants to merge 2 commits into
Closed
richard-for-bugen[bot] wants to merge 2 commits into
richard-for-bugen[bot] wants to merge 2 commits into
Conversation
Show each function of a Responses `namespace` tool to the model as `<namespace>__<name>`, like the Python frontend, and report calls of that flat name as a `function_call` with the function name and `namespace`. Named tool choices resolve by the flat name or a function name unique to one namespace, and `function_call` history items with a namespace replay under the flat name. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
The flat `<namespace>__<name>` spelling was built in four places, each walking the request tools on its own: the tools shown to the model, the named tool choice (one format! per namespaced function just to compare), the call-name map (three owned strings per function), and history calls. Identities travelled as bare strings and a `(name, namespace)` tuple. ToolIdentity now defines the spelling (`flat_name`), and ResponsesToolNames is built in the same single pass that converts the tools: it keys each flat name to its namespace length, so `decode` slices the identity out of the called name, and `resolve_choice` allocates only for candidate matches. Behavior is unchanged. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
BugenZhao
approved these changes
Nov 5, 2026
Author
|
Promoted as vllm-project#65853. |
Author
|
Merged upstream as vllm-project#65853. |
richard-for-bugen
Bot
deleted the
richard/parity-impl-responses-namespace-tools
branch
November 5, 2026 19:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The Responses API lets a client group function tools under a
namespacetool (vllm-project#47024, vllm-project#53763, vllm-project#59723). Codex-style clients send them, for example MCP servers exposed as{"type": "namespace", "name": "mcp__computer_use", "tools": [...]}.get_function_tools(vllm/tool_parsers/utils.py) shows each namespaced function to the model as one flat function tool named<namespace>__<name>, in request order, keeping its description, parameters andstrict. A call of a flat name is reported as afunction_callitem with the function's ownnameand itsnamespace(build_responses_tool_call_name_map), in the non-streaming output and in the streamedoutput_item.added,function_call_arguments.doneandoutput_item.doneevents. Afunction_callhistory item with anamespaceis replayed under the flat name. A namedtool_choiceresolves by exact name first (so a top-levelother__addnever stands in foradd), then by a function name defined in exactly one namespace, and an ambiguous local name is a 400 (_resolve_named_tool_choice,responses/protocol.py; documented indocs/features/tool_calling.md).ResponsesToolis a closed enum with onlyfunction, so any request carrying anamespacetool fails deserialization with a generic 400 (unknown variant 'namespace').What this PR changes
ResponsesTool::Namespace(NamespaceTool); the inner tools are function tools only, so acustomtool inside a namespace is rejected at deserialization, as a top-levelcustomtool already is (Python drops it silently from what the model sees).responses/tools.rsowns the namespace naming.ToolIdentity { namespace, name }is the client-side identity, and itsflat_nameis the only place the__spelling is built.ResponsesToolNames::convertwalks the request tools once, producing the flat tool list handed toResolvedToolContextand the request-local map from each flat name to its namespace length;decodeslices the identity out of a called name, andresolve_choiceresolves a namedtool_choice.OutputItemStreamertakesResponsesToolNamesat construction and reports namespaced calls withnameandnamespacein both modes (non-streaming consumes the same streamer).create_responsesnow builds the streamer once and hands it to the streaming and collecting paths, instead of threadinginclude_reasoningthrough both.function_callinput items acceptnamespaceand replay under the flat name.tool_choicecarries the resolved flat name andtoolsechoes the namespace tool, as in Python.Ownership and layering. Namespaces are a Responses wire concept: the chat layer and the tool parsers only ever see flat function tools, so flattening, tool-choice resolution and the reverse mapping all live in the Responses route (
vllm-server), next to the DTO lowering inconvert.rsand the output item state machine instreaming.rs. The map is fixed per request, so it is construction state ofOutputItemStreamer.ChatToolkeeps a plain stringname: the model boundary is a string for every consumer (templates, structural tags, tool parsers, history calls), Chat Completions never has namespaces, and the decode direction needs the request-local map either way. A structured name inChatToolbelongs with native namespace rendering (harmony, Muse Spark, DeepSeek V4.1ns::name), where the renderer rather than the route would choose the spelling.Deviations from Python. A function call without a namespace omits the
namespacekey, where Python serializes"namespace": null; this follows the existing Rust output items, which already omit Python's other null keys (async,caller). An innercustomtool is a 400 instead of being dropped.Python tests (§5a)
tests/entrypoints/openai/responses/test_namespace_tool_separator.pyis server-level (RemoteOpenAIServer,Qwen/Qwen3-1.7Bwithhermes) and fits the existing Rust Frontend Responses API step; it can be enabled in the e2e run PR once this lands. Its assertions (localname,namespace, flat name never exposed, in both modes) are ported intoresponses_namespace_tool_call_reports_function_name_and_namespace.Tests
routes::tests::responses::responses_namespace_tool_call_reports_function_name_and_namespace: HTTP route against the mock engine, the model emitting the flatmcp__computer_use__get_app_statecall through the Qwen tool parser; checks the non-streaming output, the streamedoutput_item.added/function_call_arguments.done/output_item.done, the terminal response, and the echoedtools.convert::tests::namespace_tools_reach_the_model_by_their_flat_names: the flat tool list, the resolved and echoedtool_choice, and a namespacedfunction_callhistory item replayed under the flat name.tools::tests: flattening order and fields, innercustomrejection, the reverse map, and tool-choice resolution (exact, flat, unique local name, ambiguous and missing errors with Python's messages).Red-green: on
mainevery request with anamespacetool is a 400. Disabling the reverse map fails the HTTP test and the map test; disabling the flattening fails the conversion and flattening tests.Validation
All on
da0b6ac221plus each commit. Second commit (c982cff1f4, theResponsesToolNamesrefactor, no behavior change): fmt, sort, deny bans and clippy clean; nextest 2555 passed, 13 skipped. First commit:The one timeout was
vllm-engine-core-clienttests::client::is_sleeping_returns_value_when_all_engines_agreewaiting for the mock engines' HELLO on a heavily loaded node; the crate is untouched here and the test passes on its own. The evidence reaches the in-process HTTP route with a scripted engine; no real model was run.Codex review
Second commit (
c982cff1f4): approve, no findings. Codex ran the 59 selected tests on both commits (all pass on each) and confirmed tool order and fields, unknown-name passthrough, embedded__, candidate order and error messages are unchanged.First commit: Approve with nits (
gpt-6.1-sol, high). Codex re-ran the selected server tests (59/59) and moved the new HTTP test ontomain, where it fails with 400.ChatRequest: rejected. The flat names reach the renderers as ordinaryChatToolandAssistantToolCallvalues on the existing data path, whose rendering the renderer tests already pin; this PR owns the lowering intoChatRequest, which the conversion test asserts.