[maheshrbapatu]

Overview

Forward output_config.effort and enabled-thinking budgets through Dynamo’s existing reasoning controls. This preserves acceptance of Claude Code’s adaptive/omitted/effort-high request shape while delivering the requested controls to workers.

Key Changes

Approximately 58 added production-code lines beyond the merged admission validation, excluding tests, comments, and whitespace.

• anthropic.rs normalizes reasoning controls before dispatch.
• types.rs maps effort to reasoning_effort and budgets to thinking_token_budget, preserving explicit adaptive and between-tools behavior.
• Documentation describes backend/model enforcement requirements; forwarding is not a guarantee that every worker enforces a budget.

Validation

Case	Before	After / regression
Supported effort and enabled budgets	Discarded	Five representative HTTP cases cover all effort grades, two budgets and both response modes, checking canonical fields and stop conditions
Claude Code adaptive/omitted/high	Effort ignored	Accepted and forwarded without forcing up-front thinking
between_tools plus effort	Effort ignored	Effort forwarded while the explicit mode prevents up-front thinking
Invalid controls	Ambiguous handling	Rejected before backend dispatch

After rebasing onto main 1732fd7, all 28 real-HTTP admission, replay, and reasoning-control tests pass. These use scripted CPU backends and do not qualify live inference or Anthropic hosted behavior.

Issues and Ongoing Work

• Closes fix(anthropic): forward effort and thinking budgets through backend controls #17375. Uses the budget admission and interleaved-beta handling merged in fix(anthropic): align Messages framing, errors, and validation #17367.
• fix(anthropic): preserve tool failures and thinking display #17381 owns display projection; fix: gate Anthropic thinking blocks on request #8485 separately concerns whether reasoning is emitted as thinking blocks.
• Effort requires a supporting model/template. Budget enforcement depends on the worker configuration; no live-model or hosted reasoning lifecycle qualification is claimed.

Pull Request Checklist

• I used Dynamo's /pr-checklist skill.

Summary by CodeRabbit

• New Features
  • Anthropic requests now support reasoning effort and thinking options, with supported settings forwarded to the backend.
• Bug Fixes
  • Invalid Anthropic requests receive structured JSON errors with matching request IDs; malformed fields and unsupported values are rejected before generation.
  • Anthropic streaming responses now use protocol-specific terminal events instead of the [DONE] sentinel.
• Documentation
  • Updated the documented default HTTP body limit from 192 MB to 45 MB and expanded Anthropic API compatibility details.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[github-actions]

👋 Hi maheshrbapatu! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

[coderabbitai]

No actionable comments were generated in the recent review.

Walkthrough

The Anthropic Messages service now validates request fields, returns request-ID-linked Anthropic errors, and forwards supported reasoning controls into backend requests. The change also updates streamed-response termination handling and documents request limits, validation rules, and control mappings.

Changes

Anthropic Messages compatibility

Layer / File(s)	Summary
Request admission and error responses docs/fern/pages/reference/components/frontend-configuration.mdx, lib/llm/Cargo.toml, lib/llm/src/http/service/anthropic.rs, lib/llm/src/http/service/anthropic/validation.rs, lib/llm/tests/anthropic_http_admission.rs, lib/llm/tests/frontend_protocol_validation_http.rs	The handlers validate raw request data and return Anthropic-formatted errors with request IDs. Validation covers system content, tool choices, metadata, thinking options, and numeric bounds. Documentation describes these rules and the 45 MiB body limit.
Reasoning-control conversion docs/fern/pages/reference/components/frontend-configuration.mdx, lib/llm/src/http/service/anthropic.rs, lib/llm/src/protocols/anthropic/types.rs, lib/llm/tests/anthropic_reasoning_controls_http.rs, lib/llm/tests/anthropic_http_replay.rs	Request conversion forwards supported effort values and thinking budgets, maps between_tools to disabled thinking, and normalizes reasoning-template arguments. Tests cover accepted and rejected controls.
Protocol-specific stream termination lib/llm/src/http/service/disconnect.rs, lib/llm/src/http/service/openai.rs, lib/llm/src/http/service/anthropic.rs, lib/llm/tests/anthropic_http_replay.rs, lib/llm/tests/common/http_harness.rs	The disconnect monitor receives an explicit done-sentinel setting. Anthropic streams are checked for a final message_stop or error event, while Chat Completions retains its done sentinel.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Severity of issue fixed: Medium

Merge Risk: Minimal · up to 64911

The checked reasoning controls and stream termination behavior do not present a known merge blocker.

Pre-merge checks: 3 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Out of Scope Changes check		The PR contains changes outside #17375's stated HTTP/conversion/adapter reasoning-control scope. The documentation changes DYN_HTTP_BODY_LIMIT_MB from 192 MB to 45 MB, which is unrelated to reasonin…	Revert the unrelated DYN_HTTP_BODY_LIMIT_MB default change and its dependent 45 MiB documentation claim. Move the budget shape/range validation to #17362/#17367, or exclude it from this PR if that dependency is merged separately. Retain t…

Passed checks: 3 passed

Check name	Status	Explanation
Title check		The title clearly identifies the main change: forwarding Anthropic effort and thinking budgets.
Linked Issues check		The PR meets the coding requirements in #17375. AnthropicCreateMessageRequest conversion forwards supported output_config.effort values to reasoning_effort and enabled budget_tokens to `thinki…
Description check		The pull request description includes all required sections: Overview, Key Changes, Validation, and Issues and Ongoing Work. It provides implementation details, test coverage, limitations, linked issu…

Full details: Linked Issues check

Explanation

The PR meets the coding requirements in #17375. AnthropicCreateMessageRequest conversion forwards supported output_config.effort values to reasoning_effort and enabled budget_tokens to thinking_token_budget. The handler normalizes adaptive and between_tools without forcing up-front thinking, then normalizes template arguments before dispatch. Existing stop-condition extraction receives the forwarded budget. Real HTTP tests cover distinct effort and budget values, omission, the Claude Code adaptive/high shape, streaming and non-streaming requests, and between_tools with effort. The documentation states backend enforcement limits and does not claim live-model behavior.

Full details: Description check

Explanation

The pull request description includes all required sections: Overview, Key Changes, Validation, and Issues and Ongoing Work. It provides implementation details, test coverage, limitations, linked issues, and a completed checklist.

Full details: Out of Scope Changes check

Explanation

The PR contains changes outside #17375's stated HTTP/conversion/adapter reasoning-control scope. The documentation changes DYN_HTTP_BODY_LIMIT_MB from 192 MB to 45 MB, which is unrelated to reasoning-control forwarding. The PR also adds budget shape/range admission validation in validation.rs, although #17375 assigns that work to #17362/#17367. The broader error-envelope, request-validation, and SSE changes may support the dependency work, but the unrelated body-limit change is demonstrated out of scope.

Resolution

Revert the unrelated DYN_HTTP_BODY_LIMIT_MB default change and its dependent 45 MiB documentation claim. Move the budget shape/range validation to #17362/#17367, or exclude it from this PR if that dependency is merged separately. Retain the forwarding, normalization, stop-condition, focused tests, and compatibility documentation required by #17375.

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

CodeRabbit Verify

• Run CodeRabbit Verify to test the changes in this pull request.

Recent review info

Run configuration

• Source mode: Auto
• Repository: ai-dynamo/dynamo
• Branch: main
• File: .coderabbit.yaml
• Commit: 95f1b12
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Enterprise
• Run ID: b62bfed2-9146-4726-a5e7-5120d9670e1f

Commits

Reviewing files that changed from the base of the PR and between 95f1b12 and 6491165.

Files selected for processing: 12

• docs/fern/pages/reference/components/frontend-configuration.mdx
• lib/llm/Cargo.toml
• lib/llm/src/http/service/anthropic.rs
• lib/llm/src/http/service/anthropic/validation.rs
• lib/llm/src/http/service/disconnect.rs
• lib/llm/src/http/service/openai.rs
• lib/llm/src/protocols/anthropic/types.rs
• lib/llm/tests/anthropic_http_admission.rs
• lib/llm/tests/anthropic_http_replay.rs
• lib/llm/tests/anthropic_reasoning_controls_http.rs
• lib/llm/tests/common/http_harness.rs
• lib/llm/tests/frontend_protocol_validation_http.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

[dynamo-review-agent]

Every /v1/messages request now incurs multiple whole-body JSON scans before generation. identity() has already called fields(), then into_request() calls it again here, and Json::<T>::from_bytes parses the complete payload a third time; additionally, Box<RawValue> owns a full copy of the body. For supported requests near the 45 MiB ingress limit, this adds substantial per-request CPU and memory pressure directly on the inference admission path. Parse the request once while preserving the few raw fields needed for header-aware validation, and derive model/stream identity from that result instead of reparsing the body.

🤖 AI Fix

Replace the RawValue reparse pipeline with a single-pass extractor that produces the typed request, validation inputs, and model/stream identity without copying and rescanning the complete body.

[maheshrbapatu]

Rebased onto #17367’s admission fix: ingress Bytes replace the owned body copy, and model/stream labels come from the single borrowed-field scan. Original typed parsing still provides duplicate-key rejection and field paths. Two linear passes remain; the measurements and their limits are recorded in #17367’s updated description.

[dynamo-review-agent]

This creates 20 scripted HTTP exchanges by crossing five efforts, two budgets, and both response modes, but the production conversion handles these dimensions independently: effort is validated/copied separately, the budget is copied directly, and the backend request is always internally streaming. Repeating every effort for both budgets and both client stream values therefore protects no additional backend-field contract. Five cases can retain every effort while distributing both budget values and both client response modes, still detecting a hard-coded budget and stream-specific admission failure.

🤖 AI Fix

Replace the Cartesian product with five explicit cases that cover each effort once while distributing 1024/2048 budgets and false/true client streaming across them.

[maheshrbapatu]

Replaced the 20-request cross-product with five explicit representative cases. Together they cover every effort grade, both budgets, and unary/streaming requests while asserting the canonical effort and thinking-budget stop condition.

[rmccorm4]

Don't include these types of notes, it is assumed

[maheshrbapatu]

Removed the build note. The documentation now describes the behavior directly.

[rmccorm4]

Don't include these types of notes, it is assumed

[maheshrbapatu]

Removed the issue-dependent wording and described the reasoning controls in the present tense. Docs checks and the strict link check pass.