feat(llm): support root-level thinking_token_budget for chat and responses

[flpanbin]

Overview:

Implement root-level thinking_token_budget support for OpenAI-compatible chat completion and Responses requests, forwarding it to the backend's thinking_token_budget / max_thinking_tokens sampling parameter while preserving the legacy nvext.max_thinking_tokens passthrough.

Details:

• Added thinking_token_budget to NvCreateChatCompletionRequest and NvCreateResponse.
• Added get_thinking_token_budget() and implemented it for both request types.
• Updated the Python frontend vLLM pre/post processor to read the root-level field.
• Added unit tests for chat completion and Responses conversion.

Where should the reviewer start?

• lib/llm/src/protocols/openai/chat_completions.rs — new field and OpenAIStopConditionsProvider impl.
• lib/llm/src/protocols/openai/responses/mod.rs — Responses API request support and conversion.
• lib/llm/src/protocols/openai.rs — stop-conditions mapping precedence logic.
• lib/llm/src/protocols/openai/validate.rs — validation hook.
• components/src/dynamo/frontend/vllm_processor.py — frontend passthrough.
• tests/frontend/test_vllm_prepost_integration.py — Python integration tests.

Validation

Automated checks

• cargo fmt --package dynamo-llm
• cargo check -p dynamo-llm
• cargo test -p dynamo-llm thinking_token_budget (5 passed)
• cargo clippy -p dynamo-llm --all-targets
• python3 -m pytest tests/frontend/test_vllm_prepost_integration.py

Manual E2E (Frontend + vLLM worker, Qwen/Qwen3-0.6B)

1. Root-level thinking_token_budget is enforced — request with thinking_token_budget: 16 returns reasoning_tokens: 17):

curl -s http://<frontend>:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model "Qwen3-0.6B",
    "messages": [{"role": "user", "content": "What is 37 times 41? Please reason step by step."}],
    "chat_template_kwargs": {"enable_thinking": true},
    "max_tokens": 1000,
    "temperature": 0,
    "thinking_token_budget": 16
  }'

Response:

......
"usage": {
        "prompt_tokens": 24,
        "completion_tokens": 1000,
        "total_tokens": 1024,
        "completion_tokens_details": {
            "reasoning_tokens": 17
        }
    }

2. Omitted field keeps existing defaults — the same request without thinking_token_budget completes normally with no budget override applied.

Related Issues

⚠️ This section is required. Choose one path below and delete the other.

🔗 This PR is linked to an issue:

• Closes Pass through thinking_token_budget from OpenAI-compatible requests #12302

🚫 This PR is NOT linked to an issue:

• Confirmed — no related issue

---

Summary by CodeRabbit

• New Features

  • Added support for configuring a thinking token budget directly in chat and response requests.
  • The direct budget setting takes precedence over the legacy configuration when both are provided.
  • The setting is preserved across supported request formats and forwarded to generation controls.

• Tests

  • Added coverage for propagation, precedence, fallback behavior, and omission of the thinking token budget.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi flpanbin! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[coderabbitai]

Walkthrough

The change adds a root-level thinking_token_budget to OpenAI-compatible chat and Responses requests. The value is validated, preserved during conversion, resolved against the legacy extension field, and forwarded to vLLM preprocessing.

Changes

Thinking token budget passthrough

Layer / File(s)	Summary
Request contract and validation lib/llm/src/protocols/openai/chat_completions.rs, lib/llm/src/protocols/openai/responses/mod.rs, lib/llm/src/protocols/openai/validate.rs, lib/llm/src/http/service/openai.rs, lib/llm/src/entrypoint/input/text.rs, lib/llm/src/protocols/anthropic/types.rs, lib/llm/src/protocols/unified.rs, lib/llm/src/protocols/openai/chat_completions/delta.rs	Chat and Responses requests define the optional thinking_token_budget field. Validation and default-based test construction support the new field.
Stop-condition budget precedence lib/llm/src/protocols/openai.rs, lib/llm/src/protocols/openai/chat_completions.rs	Stop-condition extraction prefers the root-level budget and falls back to nvext.max_thinking_tokens. Tests cover both paths and omission.
Responses conversion and preservation lib/llm/src/protocols/openai/responses/mod.rs	Responses requests expose the budget and forward it to generated chat-completion requests. Tests cover preservation, precedence, and fallback.
vLLM preprocessing forwarding components/src/dynamo/frontend/vllm_processor.py, tests/frontend/test_vllm_prepost_integration.py	The selected budget is forwarded as stop_conditions.max_thinking_tokens. Integration tests cover forwarding and precedence.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The PR satisfies most requirements in [#12302], but validation does not enforce backend-supported upper bounds for thinking_token_budget.	Add validation for backend-supported upper bounds, or return a clear validation error when the requested budget exceeds them.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Out of Scope Changes check	✅ Passed	The changes remain within the linked issue scope and include only related request handling, conversion, validation, frontend forwarding, and tests.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding root-level thinking_token_budget support for chat and Responses requests.
Description check	✅ Passed	The description is complete, relevant, and covers the overview, implementation details, review starting points, validation steps, and linked issue. The Related Issues section includes the linked issue…

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

---

_{Comment @coderabbitai help to get the list of available commands.}

[devin-ai-integration]

Devin Review found 3 potential issues.

[pskiran1]

@flpanbin, please resolve conflicts and rebase with main.

[biswapanda]

Review feedback on the current head.

[biswapanda]

@flpanbin please take a look - some CI tests are failing

[biswapanda]

/ok to test 152bba5