[MatejKosec]

Summary

• gate Anthropic thinking block emission on the original Messages API thinking.type == "enabled" request setting
• preserve backend reasoning_content as ordinary text when thinking is omitted or explicitly disabled, avoiding silent data loss while keeping the Anthropic block shape compatible
• add streaming and non-streaming regression tests for omitted, disabled, and enabled thinking contexts

Root cause

Dynamo already preserved the Anthropic request context, but the response converters always serialized parsed reasoning_content as Anthropic thinking blocks. Official Anthropic docs tie thinking content blocks to requests that enable extended thinking. Returning that block type when the client did not opt in breaks clients expecting Anthropic-compatible response shapes.

The revised behavior only gates the block type. If a backend produces reasoning anyway, Dynamo returns it as text unless the request explicitly enabled Anthropic thinking.

Validation

• cargo fmt -p dynamo-llm
• git diff --check
• cargo check -p dynamo-llm --no-default-features --lib
• cargo test -p dynamo-protocols anthropic

cargo test -p dynamo-llm protocols::anthropic was attempted locally but macOS cannot complete this package test target: the default build hits Linux-only NUMA/fallocate code, and --no-default-features gets to link then fails on an existing native -lstdc++ dependency. Linux CI should execute the added tests.

Reviewers requested: @rmccorm4 and @ishandhanani.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Walkthrough

Introduced conditional gating for Anthropic thinking content block emission based on context configuration. Modified AnthropicStreamConverter with an emit_thinking field and new constructor, and updated response conversion logic to selectively emit Thinking blocks when enabled.

Changes

Cohort / File(s)

Summary

Anthropic Thinking Content Gating lib/llm/src/protocols/anthropic/stream_converter.rs, lib/llm/src/protocols/anthropic/types.rs

Added emit_thinking bool field to AnthropicStreamConverter with new constructor new_with_thinking. Updated process_chunk logic to conditionally emit thinking content blocks based on context.thinking.thinking_type == "enabled". Modified chat_completion_to_anthropic_response to apply the same conditional gating when converting responses. Added unit tests covering reasoning suppression and emission scenarios with various context configurations.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: gating Anthropic thinking block emission based on the request setting.
Docstring Coverage	✅ Passed	Docstring coverage is 84.21% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description check	✅ Passed	The PR description is well-structured and comprehensive, covering summary, root cause, validation steps, and reviewer context.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

lib/llm/src/protocols/anthropic/stream_converter.rs (1)

63-109: ⚠️ Potential issue | 🟠 Major

Confirm fallback path silently enables thinking blocks.

Production code at lib/llm/src/http/service/anthropic.rs:371 falls back to AnthropicStreamConverter::new() when anthropic_ctx is None. This hardcodes emit_thinking = true regardless of the client's actual thinking setting, silently bypassing the gating mechanism when request context is unavailable.

Verify whether this fallback is intentional or if a safeguard is needed to respect the client's thinking preference even when context is missing.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@lib/llm/src/protocols/anthropic/stream_converter.rs` around lines 63 - 109,
The constructor AnthropicStreamConverter::new currently hardcodes emit_thinking
= true which causes the fallback in lib/llm/src/http/service/anthropic.rs to
silently enable thinking blocks when anthropic_ctx is None; change the behavior
so the fallback does not assume the client's preference: either make
AnthropicStreamConverter::new default emit_thinking = false (by delegating to
new_with_thinking with false) or update the caller in anthropic.rs to call
new_with_thinking(model, estimated_input_tokens, false) (or pass an explicit
client-controlled flag) so thinking blocks are only enabled when explicitly
requested; adjust with_context usage if you change the default to ensure
context-driven thinking still works.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@lib/llm/src/protocols/anthropic/stream_converter.rs`:
- Around line 63-109: The constructor AnthropicStreamConverter::new currently
hardcodes emit_thinking = true which causes the fallback in
lib/llm/src/http/service/anthropic.rs to silently enable thinking blocks when
anthropic_ctx is None; change the behavior so the fallback does not assume the
client's preference: either make AnthropicStreamConverter::new default
emit_thinking = false (by delegating to new_with_thinking with false) or update
the caller in anthropic.rs to call new_with_thinking(model,
estimated_input_tokens, false) (or pass an explicit client-controlled flag) so
thinking blocks are only enabled when explicitly requested; adjust with_context
usage if you change the default to ensure context-driven thinking still works.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: bc0a23ea-a719-4293-81a4-199dfd961e77

📥 Commits

Reviewing files that changed from the base of the PR and between 77a0051 and af23b77.

📒 Files selected for processing (2)

• lib/llm/src/protocols/anthropic/stream_converter.rs
• lib/llm/src/protocols/anthropic/types.rs

[rmccorm4]

/ok to test d00c3ff

[MatejKosec]

/rerun

[github-actions]

This PR is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.