fix(mistral): trim reasoning at the opening response tag by tonycoder-hub · Pull Request #1302 · mozilla-ai/any-llm

tonycoder-hub · 2026-08-17T10:40:56Z

Description

Some Mistral reasoning models hide the answer in a response block inside the thinking trace. The converter already extracted the answer but trimmed reasoning at the closing tag, so the opening tag and a duplicate of the answer stayed in reasoning.content.

Split on the opening tag instead. Tests: uv run pytest tests/unit/providers/test_mistral_provider.py -> 62 passed; tests/unit 2157 passed.

Distinct from #1296-#1301.

PR Type

Bug Fix

Relevant issues

No open issue.

Checklist

I understand the code I am submitting.
I have added unit tests that prove my fix/feature works
I have run this code locally and verified it fixes the issue.
New and existing tests pass locally
Documentation was updated where necessary
I have read and followed the contribution guidelines
AI Usage:
- This is fully AI-generated.

AI Usage Information

AI Model used: Claude Opus 5
AI Developer Tool used: Cursor cloud agent
I am an AI Agent filling out this form (check box if true)

When a Mistral model returns empty content and hides the answer in a <response> block inside the thinking trace, the converter recovered the answer but trimmed the reasoning at </response>, so the reported reasoning still contained the opening tag and a duplicate of the answer. Trim at the opening tag instead. Co-authored-by: Tony Coder <407243179@qq.com>

coderabbitai · 2026-08-17T10:41:13Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: b1580d39-a63e-4254-90da-9e0c1eed88a6

📥 Commits

Reviewing files that changed from the base of the PR and between c0f3ffb and 2133910.

📒 Files selected for processing (2)

src/any_llm/providers/mistral/utils.py
tests/unit/providers/test_mistral_provider.py

Included review availability: Your plan includes up to 8 reviews per rolling hour; 4 remain after this review.

Walkthrough

Changes

Mistral response extraction

Layer / File(s)	Summary
Response marker handling `src/any_llm/providers/mistral/utils.py`, `tests/unit/providers/test_mistral_provider.py`	Mistral conversion retains reasoning text before the opening `<response>` marker. Tests cover answer extraction, tag removal, and responses with no remaining reasoning.

Possibly related PRs

mozilla-ai/any-llm#1241: Both changes update Mistral reasoning and response conversion and its tests.

Suggested reviewers: njbrake, peteski22

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the Mistral reasoning fix.
Description check	✅ Passed	The description covers the change, type, tests, checklist, and AI usage information.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

## Description Non-streaming _convert_response mapped only text, tool_use, and thinking, then raised ValueError on anything else. Anthropic returns redacted_thinking when safety flags part of an extended-thinking trace, so completion() crashed even though the rest of the answer was fine. The streaming converter already ignores unmapped block types. Skip redacted_thinking quietly; warn and skip other unmapped types. Shared by anthropic, azureanthropic, vertexaianthropic, and meta. Tests: uv run pytest tests/unit -> 2158 passed. Distinct from #1292 and #1296-#1302. ## PR Type - Bug Fix ## Relevant issues No open issue. ## Checklist - [x] I understand the code I am submitting. - [x] I have added unit tests that prove my fix/feature works - [x] I have run this code locally and verified it fixes the issue. - [x] New and existing tests pass locally - [x] Documentation was updated where necessary - [x] I have read and followed the contribution guidelines - [x] **AI Usage:** - [x] This is fully AI-generated. ## AI Usage Information - AI Model used: Claude Opus 5 - AI Developer Tool used: Cursor cloud agent - [x] I am an AI Agent filling out this form (check box if true) Co-authored-by: Cursor Agent <cursoragent@cursor.com>

codecov · 2026-08-18T09:40:45Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

Files with missing lines	Coverage Δ
src/any_llm/providers/mistral/utils.py	`66.98% <100.00%> (-10.70%)`	⬇️

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Brings in upstream's merges of our mozilla-ai#1291/mozilla-ai#1292/mozilla-ai#1310 plus mozilla-ai#1297, mozilla-ai#1299, mozilla-ai#1301, mozilla-ai#1302, mozilla-ai#1303, mozilla-ai#1305. Carried-until-merged fork work stays: mozilla-ai#1294 (gemini reasoning_effort=none), mozilla-ai#1308 (aresponses timeout), mozilla-ai#1309 (gemini native tool dicts), and the mozilla-ai#1300 carry. One conflict in tests/unit/test_responses.py: kept our mozilla-ai#1308 timeout test next to upstream's flatten test. Unit suite: 2234 passed. Claude-Session: https://claude.ai/code/session_018D3FGNvb1hRZQmsXFoA44J

…ming path too 🔧 mozilla-ai#1302 fixed this for the non-streaming converter (_create_mistral_completion_from_response) but the streaming converter (_create_openai_chunk_from_mistral_chunk) never got the same treatment. With stream=True, a Magistral response that wraps its answer in <response>...</response> inside the thinking delta comes out with delta.content=None on every chunk - the whole answer, tags included, sits only in delta.reasoning.content. A caller following the standard OpenAI streaming contract (accumulate delta.content) gets an empty response with no error. Factored the split/trim logic out of the non-streaming function into a shared _split_response_tag_from_reasoning() helper and call it from both converters, so the two paths can't drift apart a third time. Streaming semantics note: this only catches the case where both <response> and </response> land in the same chunk's reasoning text; a marker split across two chunks is not reassembled, and that limitation is called out in the docstring.

…ming path too (#1336) ## What #1302 fixed a bug where some Mistral reasoning models (Magistral) sometimes return the real answer wrapped in `<response>...</response>` inside the reasoning/thinking content instead of in `content`. That fix only landed in `_create_mistral_completion_from_response` (the non-streaming converter), `src/any_llm/providers/mistral/utils.py:209-218` on main before this PR. `_create_openai_chunk_from_mistral_chunk` (the streaming converter, same file, ~256-334) never got the equivalent change. ## The bug With `stream=True`, when a `ThinkChunk` delta's thinking text contains the `<response>` markers, `choice.delta.content` stays `None` for that chunk and the whole answer, tags included, ends up only in `delta.reasoning.content`. A caller following the standard OpenAI streaming contract (accumulate `delta.content` across chunks) gets an empty response. No exception, no error - it just silently returns nothing. The identical request with `stream=False` returns the answer correctly today (because of #1302), which is what made this easy to miss. ## The fix Pulled the split/trim logic out of the non-streaming function into a shared `_split_response_tag_from_reasoning()` helper and call it from both converters. That's the part that actually matters here - the two converters had this same logic and one of them fell behind once already; sharing it is what stops that from happening a second time. ## Streaming semantics limitation Deltas arrive in fragments, so in principle `<response>` or `</response>` could be split across two separate chunks. This fix only handles the case where a single chunk's reasoning text contains both the opening and closing tag, mirroring what #1302 already does for a single non-streaming response body. It does not buffer or reassemble text across chunks to catch a tag boundary that lands mid-chunk-break. Noting this here rather than silently leaving it unhandled; happy to follow up with a stateful/buffered version if that's wanted, but wanted to keep this diff minimal and match #1302's scope. ## Testing Added to `tests/unit/providers/test_mistral_provider.py`: - `test_create_openai_chunk_strips_response_block_from_reasoning` - a streaming chunk whose thinking text contains `<response>...</response>` yields `delta.content` with the answer and `reasoning` trimmed to what came before the opening tag - `test_create_openai_chunk_leaves_plain_reasoning_unchanged` - regression: a normal streaming chunk with plain reasoning and no tags is unaffected Ran: - `uv run pytest tests/unit/providers/test_mistral_provider.py -p no:rerunfailures` - 64 passed (62 previously existing + 2 new) - Reverted the source change only (kept the new tests) and reran the two new tests to confirm they fail against the old code: `test_create_openai_chunk_strips_response_block_from_reasoning` failed with `AssertionError: assert None == 'The answer is 42.'` - `delta.content` stayed `None`, matching the reported bug exactly. Restored the fix afterward and reran the full file to confirm 64 passed again. - `uv run ruff check` / `uv run ruff format --check` on both changed files - clean - `uv run mypy src/any_llm/providers/mistral/utils.py` - no issues ## PR Type  - 🐛 Bug Fix ## Relevant issues  Follow-up to #1302, which fixed the same bug in the non-streaming converter but missed the streaming one. ## Checklist  - [x] I understand the code I am submitting. - [x] I have added unit tests that prove my fix/feature works - [x] I have run this code locally and verified it fixes the issue. - [x] New and existing tests pass locally - [ ] Documentation was updated where necessary - [x] I have read and followed the [contribution guidelines](https://github.com/mozilla-ai/any-llm/blob/main/CONTRIBUTING.md) - [x] **AI Usage:** - [ ] No AI was used. - [x] AI was used for drafting/refactoring. - [ ] This is fully AI-generated. ## AI Usage Information  - AI Model used: Claude Opus 5 - AI Developer Tool used: Claude Code - Any other info you'd like to share: Written in conjunction with my pair programmer Claude. The finding was verified against upstream before any code was written, and the fix is covered by tests that fail against the unpatched source. When answering questions by the reviewer, please respond yourself, do not copy/paste the reviewer comments into an AI system and paste back its answer. We want to discuss with you, not your AI :) - [x] I am an AI Agent filling out this form (check box if true)  ## Summary by CodeRabbit * **Bug Fixes** * Improved Mistral response handling when answers are embedded within reasoning content. * Recovered wrapped answers consistently in both standard and streaming responses. * Prevented response markers from appearing in displayed reasoning. * Preserved existing behaviour for ordinary reasoning without response markers.

github-actions Bot added the missing-template PR is missing required template checklist label Aug 17, 2026

tonycoder-hub mentioned this pull request Aug 17, 2026

fix(anthropic): skip redacted_thinking instead of crashing #1303

Merged

9 tasks

github-actions Bot removed the missing-template PR is missing required template checklist label Aug 17, 2026

tbille approved these changes Aug 18, 2026

View reviewed changes

tonycoder-hub temporarily deployed to integration-tests August 18, 2026 09:39 — with GitHub Actions Inactive

tbille merged commit 31fbc52 into mozilla-ai:main Aug 18, 2026
17 of 18 checks passed

shoemoney mentioned this pull request Aug 23, 2026

fix(mistral): trim reasoning at the opening response tag in the streaming path too #1336

Merged

11 tasks

github-actions Bot added the 1.27.0 Included in release 1.27.0 label Sep 3, 2026

Navigation Menu

fix(mistral): trim reasoning at the opening response tag - #1302

fix(mistral): trim reasoning at the opening response tag#1302
tbille merged 1 commit into
mozilla-ai:mainfrom
tonycoder-hub:cursor/fix-mistral-response-reasoning-trim-a4f4

tonycoder-hub commented Aug 17, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Aug 17, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Aug 18, 2026

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

tonycoder-hub commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

PR Type

Relevant issues

Checklist

AI Usage Information

Uh oh!

coderabbitai Bot commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

codecov Bot commented Aug 18, 2026

Codecov Report

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tonycoder-hub commented Aug 17, 2026 •

edited

Loading

coderabbitai Bot commented Aug 17, 2026 •

edited

Loading