Skip to content

[Rust Frontend] Support namespace tools in the Responses API - #65853

Merged
BugenZhao merged 2 commits into
vllm-project:mainfrom
BugenZhao:bz/rust-parity-responses-namespace-tools
Nov 5, 2026
Merged

BugenZhao merged 2 commits into
vllm-project:mainfrom
BugenZhao:bz/rust-parity-responses-namespace-tools

Conversation

@BugenZhao

Copy link
Copy Markdown
Member

Overview

The Rust frontend rejects every /v1/responses request that carries a namespace tool (unknown variant 'namespace', 400), so Codex-style clients that group MCP tools under namespaces cannot use it. This PR supports namespace tools the way the Python frontend does (#47024, #53763, #59723).

Claims

  • Each function of a namespace tool is shown to the model as a flat function tool named <namespace>__<name>, in request order, keeping its description, parameters and strict.
  • A call of a flat name is reported as a function_call item with the function's own name and its namespace, in the non-streaming output and in the streamed output_item.added, function_call_arguments.done and output_item.done events. The flat name is never exposed.
  • A function_call history item with a namespace is replayed under the flat name.
  • A named tool_choice resolves by exact name first (including the flat name), then by a function name defined in exactly one namespace; an ambiguous name is a 400 with Python's message.
  • Differences from Python: a call without a namespace omits the namespace key instead of serializing null, like the other optional keys Rust's output items already omit; a custom tool inside a namespace is a 400 instead of being dropped silently, as a top-level custom tool already is.

Validation

  • routes::tests::responses::responses_namespace_tool_call_reports_function_name_and_namespace: HTTP route against the mock engine, the model emitting the flat mcp__computer_use__get_app_state call through the Qwen tool parser; checks the non-streaming output, the streamed events, the terminal response and the echoed tools. Its assertions port those of tests/entrypoints/openai/responses/test_namespace_tool_separator.py.
  • namespace_tools_reach_the_model_by_their_flat_names: the flat tool list, the resolved and echoed tool_choice, and history replay.
  • tools::tests: flattening order and fields, inner custom rejection, the reverse map, and tool-choice resolution (exact, flat, unique local name, ambiguous and missing).

On main every request with a namespace tool is a 400; disabling the reverse map or the flattening fails the corresponding tests. test_namespace_tool_separator.py is server-level and fits the existing Rust Frontend Responses API CI step; it can be enabled there after this lands.

Rust CI set (.buildkite/scripts/run-rust-frontend-cargo-ci.sh) on this branch:

cargo fmt --manifest-path rust/Cargo.toml --all -- --check                    # clean
cargo sort --workspace --check rust                                           # clean
cargo deny --manifest-path rust/Cargo.toml --config rust/deny.toml check bans # clean
cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets --all-features --locked -- -D warnings  # clean
cargo nextest run --manifest-path rust/Cargo.toml --workspace --all-features --locked --no-fail-fast
# 2562 tests run: 2562 passed, 13 skipped

Details

Namespaces are a Responses wire concept: the chat layer and the tool parsers only ever see flat function tools. So the naming lives in a new responses/tools.rs in the Responses route:

  • ToolIdentity { namespace, name } is the client-side identity; its flat_name is the only place the __ spelling is built.
  • ResponsesToolNames::convert walks the request tools once, producing the flat tool list handed to ResolvedToolContext and a request-local map from each flat name to its namespace length. decode slices the identity out of a called name, and resolve_choice resolves a named tool_choice.
  • OutputItemStreamer takes the map at construction (non-streaming consumes the same streamer), and create_responses builds the streamer once for both paths.

Like Python, the flattening is a mechanical join without collision handling: a plain tool named crm__get next to namespace crm with function get both reach the model as crm__get. The __ join reproduces the flat names Codex itself used before namespaces (mcp__<server>__<tool>), and Codex already deduplicates on its side.

Not a duplicate. No open PR adds namespace tools to the Rust Responses API; #58783 (draft) predates the stateless Responses API on main and does not cover namespaces.

AI assistance. This change was developed with AI assistance (Claude Code). The human submitter reviewed every changed line and the validation results.


Pull Request Checklist
  • I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

  • AI assistance was used during the creation of this PR.

  • Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

  • Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

  • Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

  • Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CJckCByZpUCFtvUEyJQLsA

Show each function of a Responses `namespace` tool to the model as
`<namespace>__<name>`, like the Python frontend, and report calls of that
flat name as a `function_call` with the function name and `namespace`.
Named tool choices resolve by the flat name or a function name unique to
one namespace, and `function_call` history items with a namespace replay
under the flat name.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@BugenZhao
BugenZhao requested a review from njhill as a code owner November 5, 2026 07:56

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@BugenZhao BugenZhao added the ready ONLY add when PR is ready to merge/full CI is needed label Nov 5, 2026
@BugenZhao

Copy link
Copy Markdown
Member Author

/ci run

@github-actions

github-actions Bot commented Nov 5, 2026

Copy link
Copy Markdown

✅ Triggered Buildkite CI #100220 for commit d48ae4f61a3e.

ℹ️ This PR is 7 commits behind upstream main. Merged upstream main into this branch as d48ae4f61a3e before starting CI. Pull the branch before pushing more changes.

@BugenZhao
BugenZhao merged commit 29f7e70 into vllm-project:main Nov 5, 2026
29 checks passed
richard-for-bugen Bot added a commit to BugenZhao/vllm that referenced this pull request Nov 5, 2026
…st frontend

vllm-project#65853 added Responses namespace tools to the Rust
frontend, so run test_namespace_tool_separator.py in the Responses API step
(its directory is already a dependency of the step).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed rust

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants