fix: named tool choice should filter tool calls. by huitianbai · Pull Request #7808 · ai-dynamo/dynamo

huitianbai · 2026-04-02T09:02:21Z

As a supplement of #7589

When a request given multiple tool functions and a named tool choice, we should only consider the matched one.
For instance a request with :
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
...
}
},
{
"type": "function",
"function": {
"name": "get_location",
...
}
},
],
"tool_choice": {
"type": "function",
"function": {"name": "get_weather"}
},
....

the input prompt should only use the "get_weather function".

I implement a filter_tools function

closes: #7851

copy-pr-bot · 2026-04-02T09:02:26Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

github-actions · 2026-04-02T09:02:30Z

👋 Hi huitianbai! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

coderabbitai · 2026-04-02T09:09:24Z

Walkthrough

Modified tool visibility handling in OpenAI prompt template rendering. Added a new helper function to filter tools by function name and updated the render method to conditionally strip, keep, or filter tools based on the tool_choice parameter value and configuration settings.

Changes

Cohort / File(s)	Summary
Tool filtering logic `lib/llm/src/preprocessor/prompt/template/oai.rs`	Added `filter_tools_by_name` helper to filter JSON tools by function name. Updated `HfTokenizerConfigJsonFormatter::render` to handle tool visibility based on `tool_choice`: strips tools for `"none"`, keeps all for `"auto"`/`"required"`, and filters by function name for other values.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check	❓ Inconclusive	The PR description addresses the core issue and includes a code example, but lacks complete adherence to the template structure with missing sections.	Add an 'Overview' section summarizing the change, a 'Details' section explaining the implementation, and specify 'Where should the reviewer start' to guide code review.

✅ Passed checks (1 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'fix: named tool choice should filter tool calls' directly matches the main change: updating tool filtering logic to handle named tool choices.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

🧹 Nitpick comments (1)

lib/llm/src/preprocessor/prompt/template/oai.rs (1)
448-473: Add regression tests for named tool_choice filtering path.

The new named-choice branch is the core behavior change, but current tests near Line 1403-1437 only cover "none" and "auto". Please add cases for:

named function that exists (tools filtered to 1), and

named function that does not exist (tools omitted / expected fallback behavior).
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@lib/llm/src/preprocessor/prompt/template/oai.rs` around lines 448 - 473, Add
regression tests covering the named `tool_choice` branch so the new filtering
path is exercised: write tests that call the code path which uses
req.tool_choice() and the logic in the block that may call
filter_tools_by_name(name_str) — one test where the named function exists
(assert tools list is filtered to the single matching tool) and one where the
named function does not exist (assert expected fallback/omitted behavior, e.g.,
original tools or None per current logic). Target the behavior controlled by
exclude_tools_when_tool_choice_none and the variable tools, invoking the same
code paths currently covered for "none" and "auto" to ensure parity for the
named-case branch.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@lib/llm/src/preprocessor/prompt/template/oai.rs`:
- Around line 448-473: Add regression tests covering the named `tool_choice`
branch so the new filtering path is exercised: write tests that call the code
path which uses req.tool_choice() and the logic in the block that may call
filter_tools_by_name(name_str) — one test where the named function exists
(assert tools list is filtered to the single matching tool) and one where the
named function does not exist (assert expected fallback/omitted behavior, e.g.,
original tools or None per current logic). Target the behavior controlled by
exclude_tools_when_tool_choice_none and the variable tools, invoking the same
code paths currently covered for "none" and "auto" to ensure parity for the
named-case branch.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 441a411a-c4fa-4dd1-b807-96dd0318a735

📥 Commits

Reviewing files that changed from the base of the PR and between 9353005 and 9a896e6.

📒 Files selected for processing (1)

lib/llm/src/preprocessor/prompt/template/oai.rs

huitianbai · 2026-04-03T04:03:17Z

@ishandhanani Hi, please review if you have time.

lib/llm/src/preprocessor/prompt/template/oai.rs

grahamking · 2026-04-06T16:39:23Z

lib/llm/src/preprocessor/prompt/template/oai.rs

+
+fn filter_tools_by_name(origin_tools: Value, name: &str) -> Option<Value> {
+    let mut filtered_tools = Vec::new();
+    let tools = serde_json::to_value(origin_tools).unwrap();


We can't have .unwrap() as that would panic and stop the server.

If you're 100% certain that it will never panic, add // Safety: ...why it never panics ... above the line.

grahamking · 2026-04-06T16:41:26Z

lib/llm/src/preprocessor/prompt/template/oai.rs

+        let request: NvCreateChatCompletionRequest = serde_json::from_str(json_str).unwrap();
+        let name = "get_weather";
+        let tools = request.tools().unwrap();
+        let result = serde_json::to_value(filter_tools_by_name(tools, name)).unwrap();


Could you change the test to call filter_tools? That way both get tested.

Signed-off-by: baihuitian <baihuitian.bht@gmail.com>

vladnosiv · 2026-04-07T14:55:42Z

Hi !
As I wrote here: #7589 (comment) It seems that the named tool choice should work by forcing the JSON schema. In this case, it is not necessary to throw out other tools, because the model will be required to generate a specific call. But we can imagine that throwing away the rest of the tools will save input tokens, but at the same time it can reduce the level of cache hits if a named tool choice is applied for a specific call during the agent session.

huitianbai · 2026-04-08T01:54:44Z

Hi ! As I wrote here: #7589 (comment) It seems that the named tool choice should work by forcing the JSON schema. In this case, it is not necessary to throw out other tools, because the model will be required to generate a specific call. But we can imagine that throwing away the rest of the tools will save input tokens, but at the same time it can reduce the level of cache hits if a named tool choice is applied for a specific call during the agent session.

@vladnosiv Hi, I checked the OpenAI API, it seems having a special "allowed_tools" to maximize savings prompt caching. A named tool choice needs select one. I read this : https://developers.openai.com/api/docs/guides/function-calling#:~:text=When%20to%20use-,allowed_tools,-You%20might%20want

huitianbai requested a review from a team as a code owner April 2, 2026 09:02

pull-request-size bot added the size/M label Apr 2, 2026

github-actions bot added external-contribution Pull request is from an external contributor fix frontend `python -m dynamo.frontend` and `dynamo-run in=http|text|grpc` labels Apr 2, 2026

huitianbai mentioned this pull request Apr 2, 2026

fix: tool_choice=required bypasses format-specific parsers (#6821) #7589

Merged

coderabbitai bot reviewed Apr 2, 2026

View reviewed changes

huitianbai marked this pull request as draft April 2, 2026 09:58

huitianbai force-pushed the fix_named_tool_call branch 3 times, most recently from 22f6319 to f45f7a5 Compare April 2, 2026 16:24

pull-request-size bot added size/L and removed size/M labels Apr 2, 2026

huitianbai force-pushed the fix_named_tool_call branch from f45f7a5 to 7794aaa Compare April 2, 2026 16:26

huitianbai marked this pull request as ready for review April 2, 2026 16:27

huitianbai force-pushed the fix_named_tool_call branch 2 times, most recently from cbbe1f8 to f730fd4 Compare April 3, 2026 03:14

grahamking reviewed Apr 6, 2026

View reviewed changes

lib/llm/src/preprocessor/prompt/template/oai.rs Show resolved Hide resolved

grahamking reviewed Apr 6, 2026

View reviewed changes

fix: named tool choice should filter tool calls.

129fbfb

Signed-off-by: baihuitian <baihuitian.bht@gmail.com>

huitianbai force-pushed the fix_named_tool_call branch from f730fd4 to 129fbfb Compare April 7, 2026 04:41

Merge branch 'main' into fix_named_tool_call

0083ca3

Merge branch 'main' into fix_named_tool_call

8e997ef

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: named tool choice should filter tool calls.#7808

fix: named tool choice should filter tool calls.#7808
huitianbai wants to merge 3 commits intoai-dynamo:mainfrom
huitianbai:fix_named_tool_call

huitianbai commented Apr 2, 2026 •

edited

Loading

Uh oh!

copy-pr-bot bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

coderabbitai bot commented Apr 2, 2026 •

edited

Loading

❌ Failed checks (1 warning, 1 inconclusive)

Uh oh!

coderabbitai bot left a comment

Uh oh!

huitianbai commented Apr 3, 2026

Uh oh!

Uh oh!

grahamking Apr 6, 2026

Uh oh!

huitianbai Apr 7, 2026

Uh oh!

grahamking Apr 6, 2026

Uh oh!

huitianbai Apr 7, 2026

Uh oh!

vladnosiv commented Apr 7, 2026

Uh oh!

huitianbai commented Apr 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

huitianbai commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

copy-pr-bot bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

coderabbitai bot commented Apr 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

❌ Failed checks (1 warning, 1 inconclusive)

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

huitianbai commented Apr 3, 2026

Uh oh!

Uh oh!

grahamking Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

huitianbai Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

grahamking Apr 6, 2026

Choose a reason for hiding this comment

Uh oh!

huitianbai Apr 7, 2026

Choose a reason for hiding this comment

Uh oh!

vladnosiv commented Apr 7, 2026

Uh oh!

huitianbai commented Apr 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

huitianbai commented Apr 2, 2026 •

edited

Loading

coderabbitai bot commented Apr 2, 2026 •

edited

Loading