fix(platform): reimplement MiniMax usage accounting patch#7835
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request improves the stability and reliability of the MiniMax usage accounting mechanism by moving away from source-code manipulation. By implementing a runtime wrapper, the system is now less susceptible to upstream source drift in vllm. Additionally, it refines the reasoning token counting logic to ensure accurate reporting and adds necessary test coverage to validate these changes. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
Suggested PR Title:
[Ops][BugFix] Refactor MiniMax usage accounting to wrapper-based patching and fix reasoning token countSuggested PR Summary:
### What this PR does / why we need it?
This PR refactors the MiniMax usage accounting implementation in `vllm_ascend`. It replaces the previous AST-based method patching with a more maintainable wrapper-based approach for `chat_completion_stream_generator` and `chat_completion_full_generator`. Additionally, it fixes a bug in `_count_minimax_reasoning_tokens` where the absence of an end token incorrectly counted all tokens as reasoning tokens; it now correctly returns 0.
### Does this PR introduce _any_ user-facing change?
Yes, it improves the accuracy of token usage reporting for MiniMax models when the reasoning block is not properly terminated.
### How was this patch tested?
The patch was tested using updated and new unit tests in `tests/ut/patch/platform/test_patch_minimax_usage_accounting.py` covering usage tracking state, SSE payload handling, and final usage summation.I have no further feedback to provide.
87dc85a to
9a97ff1
Compare
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
a88818b to
098a120
Compare
…ct#7835) ## Summary - replace the MiniMax usage accounting monkey patch with a runtime wrapper implementation instead of source-text rewriting - preserve MiniMax reasoning-token semantics when `</think>` is missing by counting the emitted output as reasoning tokens - add unit coverage for usage tracking helpers and MiniMax reasoning-token counting ## Why The previous implementation rewrote `OpenAIServingChat` by matching exact source blocks. That was brittle against `vllm` source drift and could crash during early plugin initialization with: `RuntimeError: Failed to locate expected block while patching OpenAIServingChat usage accounting.` This change keeps the usage-accounting backport, but applies it by wrapping the original stream/full generators and tracking output token ids at runtime. For MiniMax reasoning counting, a missing `</think>` should not be treated as zero reasoning tokens. It can mean the whole output is still in thinking mode, or that generation stopped before the closing token was produced. In that case, the emitted output should still be counted as reasoning. ## Validation - `pytest -q tests/ut/patch/platform/test_patch_minimax_usage_accounting.py` - `vllm serve --help` Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
…pty content (#8400) ### What this PR does / why we need it? This backports the forced-tool-choice `content=None` guard to the `releases/v0.18.0` compatibility layer. Upstream vLLM still has forced named tool-choice branches that assert `content is not None` after reasoning extraction. Some reasoning parsers can legally consume the full output and return `(reasoning, None)`, which makes the assert reachable and can surface as a server-side failure. This PR follows the same compatibility-patch pattern used by: - `7314bbe2` fix(platform): reimplement MiniMax usage accounting patch (#7835) - `f83cb0e6` [Bugfix][Platform] Fix GLM47 tool-call finish backfill (#7710) The patch is intentionally narrow: - normalize `content=None` to `""` only for forced named tool choice - patch both chat-completions and responses parser entry points - keep the rest of upstream behavior unchanged Upstream tracking: - issue: vllm-project/vllm#40147 - PR: vllm-project/vllm#40148 ### Does this PR introduce _any_ user-facing change? Yes. Forced named tool choice becomes robust when the reasoning parser returns no post-reasoning content, avoiding an internal assertion failure and emitting an empty-argument function call instead. ### How was this patch tested? Unit tests: ```bash pytest -sv tests/ut/patch/platform/test_patch_tool_choice_none_content.py \ tests/ut/patch/platform/test_patch_glm_tool_call_parser.py \ tests/ut/patch/platform/test_patch_minimax_usage_accounting.py ``` Result: 22 passed. --------- Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: #7700 - vLLM Ascend PR: #7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
…project#8831) ## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: vllm-project#7700 - vLLM Ascend PR: vllm-project#7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
…project#8831) ## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: vllm-project#7700 - vLLM Ascend PR: vllm-project#7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: PiratePai <416932041@qq.com>
…project#8831) ## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: vllm-project#7700 - vLLM Ascend PR: vllm-project#7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: yangzhe-2026 <yangzhe@isrc.iscas.ac.cn>
…project#8831) ## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: vllm-project#7700 - vLLM Ascend PR: vllm-project#7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: ZhuQi-seu <zhuqi12@huawei.com>
…project#8831) ## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: vllm-project#7700 - vLLM Ascend PR: vllm-project#7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: nanxing <1014662416@qq.com>
…pty content (vllm-project#8400) ### What this PR does / why we need it? This backports the forced-tool-choice `content=None` guard to the `releases/v0.18.0` compatibility layer. Upstream vLLM still has forced named tool-choice branches that assert `content is not None` after reasoning extraction. Some reasoning parsers can legally consume the full output and return `(reasoning, None)`, which makes the assert reachable and can surface as a server-side failure. This PR follows the same compatibility-patch pattern used by: - `7314bbe2` fix(platform): reimplement MiniMax usage accounting patch (vllm-project#7835) - `f83cb0e6` [Bugfix][Platform] Fix GLM47 tool-call finish backfill (vllm-project#7710) The patch is intentionally narrow: - normalize `content=None` to `""` only for forced named tool choice - patch both chat-completions and responses parser entry points - keep the rest of upstream behavior unchanged Upstream tracking: - issue: vllm-project/vllm#40147 - PR: vllm-project/vllm#40148 ### Does this PR introduce _any_ user-facing change? Yes. Forced named tool choice becomes robust when the reasoning parser returns no post-reasoning content, avoiding an internal assertion failure and emitting an empty-argument function call instead. ### How was this patch tested? Unit tests: ```bash pytest -sv tests/ut/patch/platform/test_patch_tool_choice_none_content.py \ tests/ut/patch/platform/test_patch_glm_tool_call_parser.py \ tests/ut/patch/platform/test_patch_minimax_usage_accounting.py ``` Result: 22 passed. --------- Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Summary
</think>is missing by counting the emitted output as reasoning tokensWhy
The previous implementation rewrote
OpenAIServingChatby matching exact source blocks. That was brittle againstvllmsource drift and could crash during early plugin initialization with:RuntimeError: Failed to locate expected block while patching OpenAIServingChat usage accounting.This change keeps the usage-accounting backport, but applies it by wrapping the original stream/full generators and tracking output token ids at runtime.
For MiniMax reasoning counting, a missing
</think>should not be treated as zero reasoning tokens. It can mean the whole output is still in thinking mode, or that generation stopped before the closing token was produced. In that case, the emitted output should still be counted as reasoning.Validation
pytest -q tests/ut/patch/platform/test_patch_minimax_usage_accounting.pyvllm serve --help