Skip to content

[Mistral Grammar] Fix tool and reasoning parsing.#2

Open
juliendenize wants to merge 16 commits intoimprove_mistral_parsingfrom
fix_parsing_on_top_of_grammar
Open

[Mistral Grammar] Fix tool and reasoning parsing.#2
juliendenize wants to merge 16 commits intoimprove_mistral_parsingfrom
fix_parsing_on_top_of_grammar

Conversation

@juliendenize
Copy link
Copy Markdown
Owner

@juliendenize juliendenize commented Mar 27, 2026

Purpose

When Mistral models are served with --tool-call-parser mistral and a mistral_common-compatible tokenizer (tekken/v11+), since this PR, tool parsers has a grammar-based approach: adjust_request injects a Lark grammar from mistral-common's grammar factory into structured_outputs. This grammar constrains the model output to follow valid Mistral tool-call formatting at the decoding level.

However, the PR only handled the grammar injection which broke the vLLM tool-parsing. This PR makes the serving layer actually use the Mistral tool parser and reasoning parser to parse the grammar-constrained output, rather than falling through to the generic vLLM tool-call parsing paths which don't understand Mistral's format.

Test Plan

The branch adds:

  • Unit tests in tests/tool_parsers/test_mistral_tool_parser.py for is_mistral_grammar_path, build_non_streaming_tool_calls, and the extract_maybe_reasoning_and_tool_streaming state machine.
  • E2E tests in tests/tool_use/mistral/test_mistral_tool_calls.py covering auto, required, none, no-tools, and tool-response-followup scenarios for ministral-3b — both streaming and non-streaming.
# Unit tests
pytest tests/tool_parsers/test_mistral_tool_parser.py -v
# E2E tests (requires GPU + model download)
pytest tests/tool_use/mistral/test_mistral_tool_calls.py -v

Test Result

The tests pass


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

Copy link
Copy Markdown
Owner Author

@juliendenize juliendenize left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a bunch of comments for context

Comment on lines -784 to -789
@model_validator(mode="before")
@classmethod
def set_include_reasoning_for_none_effort(cls, data: Any) -> Any:
if data.get("reasoning_effort") == "none":
data["include_reasoning"] = False
return data
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was introduce by me when adding reasoning_effort="none", it was a bad idea as sometimes model would fail to not generate or include_reasoning can have side effects on the parsers.

Comment on lines +55 to +58
# Override to ensure both image_processor and tokenizer are used.
# The base get_attributes() introspects __init__ parameters and
# misses image_processor since it is created internally rather
# than passed as an __init__ argument.
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is due to a regression that was recently introduced, i'd need to check if this fix is sound on Monday and if so i'll open another pr just for that

Comment on lines +53 to +54
_from_tool_parser: bool = field(default=False, init=False)
"""CAUTION: Should only be set by ToolParser.adjust_request"""
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

needed to know if the grammar is active without ambiguity

Comment on lines +140 to +144
_is_mistral_tool_parser = self.tool_parser is not None and issubclass(
self.tool_parser, MistralToolParser
)
if _is_mistral_tool_parser and self.reasoning_parser_cls is not None:
MistralToolParser.model_can_reason = True
Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is not the cleanest maybe but this is the best i found to have a non-intrusive way of knowing if reasoning should be expected by the grammar.

@juliendenize juliendenize changed the title [Mistral Grammar] Use Mistral Grammar path for tool and reasoning parser [Mistral Grammar] Fix tool and reasoning parsing. Mar 27, 2026
@juliendenize juliendenize force-pushed the improve_mistral_parsing branch from e90a7a1 to 955532c Compare March 30, 2026 11:36
@juliendenize juliendenize force-pushed the fix_parsing_on_top_of_grammar branch from c46ff15 to 40f5d9b Compare March 30, 2026 11:38
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
@juliendenize juliendenize force-pushed the improve_mistral_parsing branch from 955532c to 3772f0c Compare April 1, 2026 14:39
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
Signed-off-by: juliendenize <julien.denize@mistral.ai>
@juliendenize juliendenize force-pushed the fix_parsing_on_top_of_grammar branch from 40f5d9b to 5b4589a Compare April 1, 2026 14:55
@juliendenize juliendenize force-pushed the improve_mistral_parsing branch from e747e48 to 5c0f3b2 Compare April 2, 2026 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant