Skip to content

[Rust Frontend] Support namespace tools in the Responses API - #122

Closed
richard-for-bugen[bot] wants to merge 2 commits into
mainfrom
richard/parity-impl-responses-namespace-tools
Closed

richard-for-bugen[bot] wants to merge 2 commits into
mainfrom
richard/parity-impl-responses-namespace-tools

Conversation

@richard-for-bugen

@richard-for-bugen richard-for-bugen Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Motivation

The Responses API lets a client group function tools under a namespace tool (vllm-project#47024, vllm-project#53763, vllm-project#59723). Codex-style clients send them, for example MCP servers exposed as {"type": "namespace", "name": "mcp__computer_use", "tools": [...]}.

  • Python: get_function_tools (vllm/tool_parsers/utils.py) shows each namespaced function to the model as one flat function tool named <namespace>__<name>, in request order, keeping its description, parameters and strict. A call of a flat name is reported as a function_call item with the function's own name and its namespace (build_responses_tool_call_name_map), in the non-streaming output and in the streamed output_item.added, function_call_arguments.done and output_item.done events. A function_call history item with a namespace is replayed under the flat name. A named tool_choice resolves by exact name first (so a top-level other__add never stands in for add), then by a function name defined in exactly one namespace, and an ambiguous local name is a 400 (_resolve_named_tool_choice, responses/protocol.py; documented in docs/features/tool_calling.md).
  • Rust today: ResponsesTool is a closed enum with only function, so any request carrying a namespace tool fails deserialization with a generic 400 (unknown variant 'namespace').

What this PR changes

  • ResponsesTool::Namespace(NamespaceTool); the inner tools are function tools only, so a custom tool inside a namespace is rejected at deserialization, as a top-level custom tool already is (Python drops it silently from what the model sees).
  • New responses/tools.rs owns the namespace naming. ToolIdentity { namespace, name } is the client-side identity, and its flat_name is the only place the __ spelling is built. ResponsesToolNames::convert walks the request tools once, producing the flat tool list handed to ResolvedToolContext and the request-local map from each flat name to its namespace length; decode slices the identity out of a called name, and resolve_choice resolves a named tool_choice.
  • OutputItemStreamer takes ResponsesToolNames at construction and reports namespaced calls with name and namespace in both modes (non-streaming consumes the same streamer). create_responses now builds the streamer once and hands it to the streaming and collecting paths, instead of threading include_reasoning through both.
  • function_call input items accept namespace and replay under the flat name.
  • The echoed tool_choice carries the resolved flat name and tools echoes the namespace tool, as in Python.

Ownership and layering. Namespaces are a Responses wire concept: the chat layer and the tool parsers only ever see flat function tools, so flattening, tool-choice resolution and the reverse mapping all live in the Responses route (vllm-server), next to the DTO lowering in convert.rs and the output item state machine in streaming.rs. The map is fixed per request, so it is construction state of OutputItemStreamer. ChatTool keeps a plain string name: the model boundary is a string for every consumer (templates, structural tags, tool parsers, history calls), Chat Completions never has namespaces, and the decode direction needs the request-local map either way. A structured name in ChatTool belongs with native namespace rendering (harmony, Muse Spark, DeepSeek V4.1 ns::name), where the renderer rather than the route would choose the spelling.

Deviations from Python. A function call without a namespace omits the namespace key, where Python serializes "namespace": null; this follows the existing Rust output items, which already omit Python's other null keys (async, caller). An inner custom tool is a 400 instead of being dropped.

Python tests (§5a)

tests/entrypoints/openai/responses/test_namespace_tool_separator.py is server-level (RemoteOpenAIServer, Qwen/Qwen3-1.7B with hermes) and fits the existing Rust Frontend Responses API step; it can be enabled in the e2e run PR once this lands. Its assertions (local name, namespace, flat name never exposed, in both modes) are ported into responses_namespace_tool_call_reports_function_name_and_namespace.

Tests

  • routes::tests::responses::responses_namespace_tool_call_reports_function_name_and_namespace: HTTP route against the mock engine, the model emitting the flat mcp__computer_use__get_app_state call through the Qwen tool parser; checks the non-streaming output, the streamed output_item.added / function_call_arguments.done / output_item.done, the terminal response, and the echoed tools.
  • convert::tests::namespace_tools_reach_the_model_by_their_flat_names: the flat tool list, the resolved and echoed tool_choice, and a namespaced function_call history item replayed under the flat name.
  • tools::tests: flattening order and fields, inner custom rejection, the reverse map, and tool-choice resolution (exact, flat, unique local name, ambiguous and missing errors with Python's messages).

Red-green: on main every request with a namespace tool is a 400. Disabling the reverse map fails the HTTP test and the map test; disabling the flattening fails the conversion and flattening tests.

Validation

All on da0b6ac221 plus each commit. Second commit (c982cff1f4, the ResponsesToolNames refactor, no behavior change): fmt, sort, deny bans and clippy clean; nextest 2555 passed, 13 skipped. First commit:

cargo fmt --manifest-path rust/Cargo.toml --all -- --check                    # ok
cargo sort --workspace --check rust                                            # ok
cargo deny --manifest-path rust/Cargo.toml --config rust/deny.toml check bans  # bans ok
cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --all-features --locked -- -D warnings  # ok
cargo nextest run --manifest-path rust/Cargo.toml --workspace --all-features --locked --no-fail-fast          # 2554 passed, 13 skipped, 1 timed out (see below)

The one timeout was vllm-engine-core-client tests::client::is_sleeping_returns_value_when_all_engines_agree waiting for the mock engines' HELLO on a heavily loaded node; the crate is untouched here and the test passes on its own. The evidence reaches the in-process HTTP route with a scripted engine; no real model was run.

Codex review

Second commit (c982cff1f4): approve, no findings. Codex ran the 59 selected tests on both commits (all pass on each) and confirmed tool order and fields, unknown-name passthrough, embedded __, candidate order and error messages are unchanged.

First commit: Approve with nits (gpt-6.1-sol, high). Codex re-ran the selected server tests (59/59) and moved the new HTTP test onto main, where it fails with 400.

  • P3, renderer coverage stops at ChatRequest: rejected. The flat names reach the renderers as ordinary ChatTool and AssistantToolCall values on the existing data path, whose rendering the renderer tests already pin; this PR owns the lowering into ChatRequest, which the conversion test asserts.

Show each function of a Responses `namespace` tool to the model as
`<namespace>__<name>`, like the Python frontend, and report calls of that
flat name as a `function_call` with the function name and `namespace`.
Named tool choices resolve by the flat name or a function name unique to
one namespace, and `function_call` history items with a namespace replay
under the flat name.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
The flat `<namespace>__<name>` spelling was built in four places, each
walking the request tools on its own: the tools shown to the model, the
named tool choice (one format! per namespaced function just to compare),
the call-name map (three owned strings per function), and history calls.
Identities travelled as bare strings and a `(name, namespace)` tuple.

ToolIdentity now defines the spelling (`flat_name`), and
ResponsesToolNames is built in the same single pass that converts the
tools: it keys each flat name to its namespace length, so `decode` slices
the identity out of the called name, and `resolve_choice` allocates only
for candidate matches. Behavior is unchanged.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
@richard-for-bugen

Copy link
Copy Markdown
Author

Promoted as vllm-project#65853.

@richard-for-bugen

Copy link
Copy Markdown
Author

Merged upstream as vllm-project#65853.

@richard-for-bugen richard-for-bugen Bot closed this Nov 5, 2026
@richard-for-bugen
richard-for-bugen Bot deleted the richard/parity-impl-responses-namespace-tools branch November 5, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant