fix(mistral): clamp reasoning_effort, drop prompt_mode, replay thinking traces - #1241
Merged
Merged
Conversation
Mistral accepts a reasoning_effort of only "high" or "none", so sending any_llm's finer-grained levels verbatim failed with "reasoning_effort low is not supported for this model". Collapse every explicit level onto "high" instead, matching how other gateways clamp restricted effort scales. Also stop sending prompt_mode="reasoning". Mistral documents that param only for the deprecated native-reasoning magistral models and now rejects it on every model, magistral included, with "Reasoning prompt mode is not enabled for this model". An explicit reasoning_effort is sufficient to get a thinking block on every current model, verified against magistral-medium-latest, magistral-small-latest, mistral-small-latest and mistral-medium-3-5. Point the reasoning test model at mistral-medium-3-5, since the magistral models are deprecated. Separately, preserve reasoning traces across turns. Mistral requires the full assistant message, thinking trace included, to be replayed on subsequent turns, and warns that dropping it degrades output quality. any_llm normalizes the trace onto message.reasoning, so fold it back into a ThinkChunk when converting messages, carrying the replay signature through extra_content the way the Anthropic provider already does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
njbrake
temporarily deployed
to
integration-tests
August 5, 2026 14:27 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
njbrake
added
the
run-integration-tests
github-actions
Bot
removed
the
run-integration-tests
WalkthroughChangesMistral reasoning parameters now collapse explicit efforts to Mistral reasoning support
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codecov Report❌ Patch coverage is
... and 1 file with indirect coverage changes 🚀 New features to boost your workflow:
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/any_llm/providers/mistral/mistral.py`:
- Around line 107-115: Update the comments around _convert_completion_params in
src/any_llm/providers/mistral/mistral.py#L107-L115 to state only that the
provider does not add prompt_mode automatically, without claiming
caller-supplied prompt_mode is never forwarded. Update the corresponding test
description/comments in tests/unit/providers/test_mistral_provider.py#L390-L395
to state that default conversion does not add prompt_mode; no behavioral test
change is required.
In `@src/any_llm/providers/mistral/utils.py`:
- Around line 501-506: Update the message-processing loop around
_build_mistral_assistant_content so it never rebinds the loop variable msg. Keep
msg unchanged and assign any modified assistant message to a separate
patched_msg before appending it to processed_msg; otherwise append the original
message. Run Ruff lint/format and strict mypy afterward.
In `@tests/unit/providers/test_mistral_provider.py`:
- Around line 104-109: Replace the ANN401-violating Any annotation on the
reasoning parameter of test_patch_messages_rebuilds_thinking_chunk_for_replay
with str | dict[str, str], matching the two parametrized input shapes. Run Ruff
formatting/linting and strict mypy to verify the test passes all checks.
- Around line 1263-1288: Add a test alongside
test_create_openai_chunk_captures_thinking_signature covering a ThinkChunk with
thinking content but no signature. Build the same streaming event without the
signature field, call _create_openai_chunk_from_mistral_chunk, and assert the
resulting delta.extra_content is None while preserving the reasoning content
assertion.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: 3819a32b-571b-483c-8286-763dd5e87c32
📒 Files selected for processing (4)
src/any_llm/providers/mistral/mistral.pysrc/any_llm/providers/mistral/utils.pytests/conftest.pytests/unit/providers/test_mistral_provider.py
Comment on lines
+107
to
+115
| # An explicit reasoning_effort is all Mistral needs to emit a `thinking` block; every | ||
| # model defaults to not reasoning, so "auto" (leave the choice to Mistral) and "none" | ||
| # both mean send nothing. Note that prompt_mode="reasoning" must not be sent alongside | ||
| # it: Mistral documents that param only for the deprecated native-reasoning magistral | ||
| # models and now rejects it on every model, magistral included, with "Reasoning prompt | ||
| # mode is not enabled for this model". | ||
| reasoning_effort = converted_params.pop("reasoning_effort", None) | ||
| mistral_effort = _REASONING_EFFORT_TO_MISTRAL.get(reasoning_effort) if reasoning_effort else None | ||
| if mistral_effort is not None: | ||
| converted_params["reasoning_effort"] = mistral_effort | ||
| converted_params["prompt_mode"] = "reasoning" | ||
| if reasoning_effort is not None and reasoning_effort not in ("auto", "none"): | ||
| converted_params["reasoning_effort"] = _MISTRAL_REASONING_EFFORT |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Describe automatic prompt_mode removal accurately.
The implementation forwards caller-supplied prompt_mode through converted_params.update(kwargs). The current comments state that prompt_mode must never accompany reasoning_effort. Limit the wording to the provider not adding prompt_mode automatically.
src/any_llm/providers/mistral/mistral.py#L107-L115: state that_convert_completion_paramsdoes not addprompt_mode.tests/unit/providers/test_mistral_provider.py#L390-L395: state that the default conversion does not addprompt_mode.
📍 Affects 2 files
src/any_llm/providers/mistral/mistral.py#L107-L115(this comment)tests/unit/providers/test_mistral_provider.py#L390-L395
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/any_llm/providers/mistral/mistral.py` around lines 107 - 115, Update the
comments around _convert_completion_params in
src/any_llm/providers/mistral/mistral.py#L107-L115 to state only that the
provider does not add prompt_mode automatically, without claiming
caller-supplied prompt_mode is never forwarded. Update the corresponding test
description/comments in tests/unit/providers/test_mistral_provider.py#L390-L395
to state that default conversion does not add prompt_mode; no behavioral test
change is required.
Comment on lines
501
to
506
| for i, msg in enumerate(messages): | ||
| if msg.get("role") == "assistant": | ||
| rebuilt_content = _build_mistral_assistant_content(msg) | ||
| if rebuilt_content is not None: | ||
| msg = {**msg, "content": rebuilt_content} | ||
| processed_msg.append(msg) |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Do not rebind the loop variable.
Ruff reports PLW2901 because line 505 reassigns msg. Keep msg as the input message and append a separate patched_msg.
As per coding guidelines, ruff lint and format, plus strict mypy, must pass before review.
🧰 Tools
🪛 Ruff (0.16.1)
[warning] 505-505: for loop variable msg overwritten by assignment target
(PLW2901)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/any_llm/providers/mistral/utils.py` around lines 501 - 506, Update the
message-processing loop around _build_mistral_assistant_content so it never
rebinds the loop variable msg. Keep msg unchanged and assign any modified
assistant message to a separate patched_msg before appending it to
processed_msg; otherwise append the original message. Run Ruff lint/format and
strict mypy afterward.
Sources: Coding guidelines, Linters/SAST tools
Comment on lines
+104
to
+109
| @pytest.mark.parametrize( | ||
| "reasoning", | ||
| ["thought hard", {"content": "thought hard"}], | ||
| ids=["serialized-string", "object-form"], | ||
| ) | ||
| def test_patch_messages_rebuilds_thinking_chunk_for_replay(reasoning: Any) -> None: |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Replace Any in the test parameter.
Ruff reports ANN401 on line 109. Use str | dict[str, str] for reasoning, because these are the only parametrised shapes.
As per coding guidelines, ruff lint and format, plus strict mypy, must pass before review.
🧰 Tools
🪛 Ruff (0.16.1)
[warning] 109-109: Dynamically typed expressions (typing.Any) are disallowed in reasoning
(ANN401)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/unit/providers/test_mistral_provider.py` around lines 104 - 109,
Replace the ANN401-violating Any annotation on the reasoning parameter of
test_patch_messages_rebuilds_thinking_chunk_for_replay with str | dict[str,
str], matching the two parametrized input shapes. Run Ruff formatting/linting
and strict mypy to verify the test passes all checks.
Sources: Coding guidelines, Linters/SAST tools
Comment on lines
+1263
to
+1288
| def test_create_openai_chunk_captures_thinking_signature() -> None: | ||
| """Streaming deltas surface the thinking signature the same way as non-streaming.""" | ||
| pytest.importorskip("mistralai") | ||
| from mistralai.client.models import TextChunk, ThinkChunk | ||
|
|
||
| from any_llm.providers.mistral.utils import _create_openai_chunk_from_mistral_chunk | ||
|
|
||
| choice = Mock() | ||
| choice.index = 0 | ||
| choice.delta.content = [ThinkChunk(thinking=[TextChunk(text="step one")], signature="sig-abc")] | ||
| choice.delta.role = "assistant" | ||
| choice.delta.tool_calls = None | ||
| choice.finish_reason = None | ||
|
|
||
| event = Mock() | ||
| event.data.id = "chatcmpl-abc" | ||
| event.data.created = 1_700_000_000 | ||
| event.data.model = "mistral-medium-3-5" | ||
| event.data.choices = [choice] | ||
| event.data.usage = None | ||
|
|
||
| chunk = _create_openai_chunk_from_mistral_chunk(event) | ||
|
|
||
| assert chunk.choices[0].delta.reasoning is not None | ||
| assert chunk.choices[0].delta.reasoning.content == "step one" | ||
| assert chunk.choices[0].delta.extra_content == {"mistral": {"signature": "sig-abc"}} |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Test the unsigned streaming path.
Lines 283-284 add a signature-present branch. Lines 311-316 must omit extra_content when Mistral does not send a signature. Add a streaming ThinkChunk without a signature and assert that delta.extra_content is None.
As per coding guidelines, test every new branch, including edge paths.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/unit/providers/test_mistral_provider.py` around lines 1263 - 1288, Add
a test alongside test_create_openai_chunk_captures_thinking_signature covering a
ThinkChunk with thinking content but no signature. Build the same streaming
event without the signature field, call _create_openai_chunk_from_mistral_chunk,
and assert the resulting delta.extra_content is None while preserving the
reasoning content assertion.
Source: Coding guidelines
8 tasks
This was referenced Aug 16, 2026
Merged
9 tasks
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes the two
test_reasoning.py::*[mistral]failures onmain(run 31011944484), and one related bug found while verifying.reasoning_effortclamping. Mistral accepts only"high"or"none", so passing any_llm's finer-grained levels through returned400 reasoning_effort low is not supported for this model. Every explicit level now collapses onto"high". Bifrost does the same thing for Mistral, and both bifrost and litellm clamp rather than propagate a predictable 400 on other restricted providers (Gemini Pro, Bedrock Nova).prompt_moderemoved. #1236 addedprompt_mode="reasoning", which Mistral documents only for the deprecated magistral models. It now returns400 Reasoning prompt mode is not enabled for this modelon every model, magistral included. An explicitreasoning_effortis sufficient to get a thinking block; I verified that againstmistral-medium-3-5,mistral-small-latest,magistral-medium-latestandmagistral-small-latest. The reasoning test model moves tomistral-medium-3-5since magistral is deprecated.Thinking replay. Mistral requires the full assistant message, thinking trace included, on later turns and warns that dropping it degrades quality. We parsed traces out but never folded them back in, so multi-turn reasoning silently lost them.
_patch_messagesnow rebuilds theThinkChunk, carrying the replaysignaturethroughextra_contentas the Anthropic provider already does.PR Type
Relevant issues
None filed; caught by the post-merge integration run above.
Checklist
Verified locally with a real
MISTRAL_API_KEY: both previously failing tests pass, the full Mistral integration selection is green (21 passed, 6 skipped),tests/unitis green (1881 passed), and multi-turn replay was confirmed end to end to carry the trace into turn 2.AI Usage Information
AI Model used: Claude Opus 5
AI Developer Tool used: Claude Code
Any other info you'd like to share: Behavior was established by probing the live Mistral API rather than from docs alone, which is what showed
prompt_modenow failing on magistral too. Cross-checked the clamping approach against litellm and bifrost before landing it.I am an AI Agent filling out this form (check box if true)
Summary by CodeRabbit
New Features
Bug Fixes