fix(frontend): preserve chat_template_args through Responses conversion

[ayaangazali]

Summary

Fixes #12284, which @michaelfeil confirmed on the issue.

/v1/responses dropped chat-template arguments on the way to the chat request. NvCreateResponse had no field for them at all, and the conversion hard-coded chat_template_args: None, so template-driven behaviour such as reasoning and tool formatting could be controlled from /v1/chat/completions but not from /v1/responses, even for the model families that depend on it.

The field is added to the Responses request mirroring the chat spelling, including the chat_template_kwargs alias, so one client payload behaves the same on both endpoints, and the conversion forwards it.

The coupled change, and why it is in the same PR

The Responses handler left move_reasoning_to_content_when_empty unset, and a NOTE there explained why:

That is currently unreachable rather than wrong: the Responses-to-chat conversion hard-codes chat_template_args: None, so force_nonempty_content can never be set on this path in the first place. If that conversion ever forwards chat_template_args, [...] the aggregator backstop should be wired here too at that point.

Forwarding the args is exactly the condition that comment names, so the flag is wired here as well, mirroring the six lines the chat path already uses. The streaming stage gates on the request's own args through wants_reasoning_as_content_when_empty, but a non-streaming Responses request reaches the aggregating branch, which reads the flag instead. Shipping the forwarding alone would have made force_nonempty_content work when streaming and silently not when not, which is the failure mode the NOTE anticipated. The comment is rewritten to describe the wiring rather than the old exemption.

Adding a field to NvCreateResponse also required chat_template_args: None at 22 existing struct literals, all in test code and one test helper in input_trigger.rs. That is mechanical and separate from the behaviour change.

Validation

cargo test -p dynamo-llm --no-default-features --lib — 1935 passed. cargo clippy --no-default-features --all-targets clean, cargo fmt applied.

Three tests, following the repro shape in the issue:

• a Responses request carrying chat_template_args reaches the chat request with enable_thinking == true
• the chat_template_kwargs alias deserializes the same way, so the endpoints do not diverge on spelling
• a request without the field still produces None rather than an empty map, which would change downstream template rendering

The first two fail against the previous behaviour, verified by restoring chat_template_args: None in the conversion and keeping the tests:

FAILED protocols::openai::responses::tests::chat_template_args_survive_the_conversion
FAILED protocols::openai::responses::tests::chat_template_kwargs_alias_is_accepted

One caveat worth stating: http::service::service_v2::tests::test_oversized_body_returns_json_413 is flaky in my environment under the full parallel suite. It passed and failed on two consecutive runs of identical code, passes in isolation, and passes in isolation with this change stashed, so it is unrelated to this diff. It touches service_v2.rs, which this PR does not.

Not verified here: no live model, so I have not observed a template actually consuming a forwarded argument end to end. The change is at the conversion and wiring layer and the tests cover those.

---

Summary by CodeRabbit

• New Features

  • Added support for optional chat template arguments in the Responses API.
  • Supports both chat_template_args and the chat_template_kwargs alias.
  • Preserves these settings when converting Responses API requests to chat completions.

• Bug Fixes

  • Configured reasoning-only output can now be included in message content during non-streaming response aggregation.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

👋 Hi ayaangazali! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8cdb4e2e-8d3f-4138-a67b-3de29726ee56

📥 Commits

Reviewing files that changed from the base of the PR and between d023c6a and e58f28a.

📒 Files selected for processing (3)

• lib/llm/src/http/service/openai.rs
• lib/llm/src/protocols/common/input_trigger.rs
• lib/llm/src/protocols/openai/responses/mod.rs

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

---

Walkthrough

The Responses API now accepts optional chat_template_args, preserves them during conversion to Chat Completions, and uses them during non-streaming aggregation to move reasoning-only output into content when configured. Tests cover field aliases, preservation, omission, and updated fixtures.

Changes

Responses chat-template argument propagation

Layer / File(s)	Summary
Request contract and conversion lib/llm/src/protocols/openai/responses/mod.rs, lib/llm/src/http/service/openai.rs, lib/llm/src/protocols/common/input_trigger.rs	NvCreateResponse accepts chat_template_args and the chat_template_kwargs alias. Conversion forwards the value to Chat Completions requests. Tests cover preservation, omission, alias deserialization, and updated fixtures.
Response aggregation behavior lib/llm/src/http/service/openai.rs	Non-streaming Responses aggregation derives move_reasoning_to_content_when_empty from chat_template_args and applies it to parser options.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to e58f2

This PR forwards chat-template arguments through Responses conversion and preserves the existing unset behavior when omitted; no actionable merge-blocking risk remains beyond normal checks and review.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main change: preserving chat_template_args during Responses conversion.
Description check	✅ Passed	The description explains the change, rationale, validation, related issue, and known test caveat, but omits the template's reviewer-start section.
Linked Issues check	✅ Passed	The changes satisfy issue #12284 by adding, aliasing, preserving, and forwarding chat_template_args during Responses-to-Chat conversion.
Out of Scope Changes check	✅ Passed	The changes remain within scope, including the required aggregation wiring and mechanical test-fixture updates.
Docstring Coverage	✅ Passed	Docstring coverage is 86.67% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 30 functions across 3 files.

---

_{Comment @coderabbitai help to get the list of available commands.}

[rmccorm4]

Thanks for the contribution @ayaangazali 🙏

Adding chat_template_args to NvCreateResponse object instead of upstream on CreateResponse object in frontend-crates repo matches existing customization of chat_template_args on NvCreateChatCompletionRequest instead of frontend-crates CreateChatCompletionRequest object

[rmccorm4]

/ok to test e58f28a

[ayaangazali]

Correcting something I said in this PR's validation notes, now that this has merged.

I described test_oversized_body_returns_json_413 as a flake that "passes in isolation". I have since measured it properly on clean main with nothing else running, and that was wrong. Run on its own it fails about half the time:

cargo test -p dynamo-llm --lib --no-default-features test_oversized_body_returns_json_413
run 1 ok | run 2 ok | run 3 FAILED | run 4 FAILED | run 5 FAILED | run 6 ok

The failure is the same every time, and it is not the status assertion:

panicked at lib/llm/src/http/service/service_v2.rs:1883:
body must be JSON: reqwest::Error { kind: Decode, source: hyper::Error(Body, "connection error") }

So the 413 status arrives, and the JSON body behind it is what gets lost. The test posts a 2 MB payload against a 1 MB limit; DefaultBodyLimit rejects it without reading the rest, and the connection closes while the client is still writing, which can take the buffered response with it.

I am not proposing a change. The server behaviour is ordinary HTTP, a server may respond and close before a request body is fully sent, and making the response reliably readable would mean draining some bounded amount of a body that was rejected for being too large, which has obvious cost implications and is a call for whoever owns that path. The test, added in #13102, asserts a slightly stronger guarantee than HTTP gives it.

Flagging it rather than fixing it because a coin-flip test in the merge gate is worth knowing about, and because my earlier note here understated it. Happy to open a proper issue if that is the preferred home.