Skip to content

fix(platform): reimplement MiniMax usage accounting patch#7835

Merged
yiz-liu merged 1 commit into
vllm-project:releases/v0.18.0from
QwertyJack:pr/minimax-usage-accounting-reimpl
Mar 31, 2026
Merged

fix(platform): reimplement MiniMax usage accounting patch#7835
yiz-liu merged 1 commit into
vllm-project:releases/v0.18.0from
QwertyJack:pr/minimax-usage-accounting-reimpl

Conversation

@QwertyJack

@QwertyJack QwertyJack commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

Summary

  • replace the MiniMax usage accounting monkey patch with a runtime wrapper implementation instead of source-text rewriting
  • preserve MiniMax reasoning-token semantics when </think> is missing by counting the emitted output as reasoning tokens
  • add unit coverage for usage tracking helpers and MiniMax reasoning-token counting

Why

The previous implementation rewrote OpenAIServingChat by matching exact source blocks. That was brittle against vllm source drift and could crash during early plugin initialization with:
RuntimeError: Failed to locate expected block while patching OpenAIServingChat usage accounting.

This change keeps the usage-accounting backport, but applies it by wrapping the original stream/full generators and tracking output token ids at runtime.

For MiniMax reasoning counting, a missing </think> should not be treated as zero reasoning tokens. It can mean the whole output is still in thinking mode, or that generation stopped before the closing token was produced. In that case, the emitted output should still be counted as reasoning.

Validation

  • pytest -q tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  • vllm serve --help

@QwertyJack QwertyJack requested a review from wangxiyuan as a code owner March 30, 2026 13:58
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request improves the stability and reliability of the MiniMax usage accounting mechanism by moving away from source-code manipulation. By implementing a runtime wrapper, the system is now less susceptible to upstream source drift in vllm. Additionally, it refines the reasoning token counting logic to ensure accurate reporting and adds necessary test coverage to validate these changes.

Highlights

  • Refactored Usage Accounting: Replaced the brittle source-text rewriting monkey patch for MiniMax usage accounting with a more robust runtime wrapper implementation.
  • Reasoning Token Logic: Corrected reasoning token counting to return 0 when the tag is absent, preventing inaccurate token reporting.
  • Improved Test Coverage: Added comprehensive unit tests for usage tracking helpers and verified the corrected reasoning-token behavior.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Suggested PR Title:

[Ops][BugFix] Refactor MiniMax usage accounting to wrapper-based patching and fix reasoning token count

Suggested PR Summary:

### What this PR does / why we need it?
This PR refactors the MiniMax usage accounting implementation in `vllm_ascend`. It replaces the previous AST-based method patching with a more maintainable wrapper-based approach for `chat_completion_stream_generator` and `chat_completion_full_generator`. Additionally, it fixes a bug in `_count_minimax_reasoning_tokens` where the absence of an end token incorrectly counted all tokens as reasoning tokens; it now correctly returns 0.

### Does this PR introduce _any_ user-facing change?
Yes, it improves the accuracy of token usage reporting for MiniMax models when the reasoning block is not properly terminated.

### How was this patch tested?
The patch was tested using updated and new unit tests in `tests/ut/patch/platform/test_patch_minimax_usage_accounting.py` covering usage tracking state, SSE payload handling, and final usage summation.

I have no further feedback to provide.

@QwertyJack QwertyJack force-pushed the pr/minimax-usage-accounting-reimpl branch 3 times, most recently from 87dc85a to 9a97ff1 Compare March 30, 2026 14:57
@yiz-liu yiz-liu added this to the v0.18.0rc1 milestone Mar 31, 2026
@yiz-liu yiz-liu added ready enable e2e test for PR ready-for-test labels Mar 31, 2026
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
@yiz-liu yiz-liu merged commit 7314bbe into vllm-project:releases/v0.18.0 Mar 31, 2026
28 checks passed
@QwertyJack QwertyJack deleted the pr/minimax-usage-accounting-reimpl branch March 31, 2026 11:03
1kzk pushed a commit to 1kzk/vllm-ascend that referenced this pull request Apr 20, 2026
…ct#7835)

## Summary
- replace the MiniMax usage accounting monkey patch with a runtime
wrapper implementation instead of source-text rewriting
- preserve MiniMax reasoning-token semantics when `</think>` is missing
by counting the emitted output as reasoning tokens
- add unit coverage for usage tracking helpers and MiniMax
reasoning-token counting

## Why
The previous implementation rewrote `OpenAIServingChat` by matching
exact source blocks. That was brittle against `vllm` source drift and
could crash during early plugin initialization with:
`RuntimeError: Failed to locate expected block while patching
OpenAIServingChat usage accounting.`

This change keeps the usage-accounting backport, but applies it by
wrapping the original stream/full generators and tracking output token
ids at runtime.

For MiniMax reasoning counting, a missing `</think>` should not be
treated as zero reasoning tokens. It can mean the whole output is still
in thinking mode, or that generation stopped before the closing token
was produced. In that case, the emitted output should still be counted
as reasoning.

## Validation
- `pytest -q
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py`
- `vllm serve --help`

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
zzzzwwjj pushed a commit that referenced this pull request Apr 23, 2026
…pty content (#8400)

### What this PR does / why we need it?

This backports the forced-tool-choice `content=None` guard to the
`releases/v0.18.0` compatibility layer.

Upstream vLLM still has forced named tool-choice branches that assert
`content is not None` after reasoning extraction. Some reasoning parsers
can legally consume the full output and return `(reasoning, None)`,
which makes the assert reachable and can surface as a server-side
failure.

This PR follows the same compatibility-patch pattern used by:
- `7314bbe2` fix(platform): reimplement MiniMax usage accounting patch
(#7835)
- `f83cb0e6` [Bugfix][Platform] Fix GLM47 tool-call finish backfill
(#7710)

The patch is intentionally narrow:
- normalize `content=None` to `""` only for forced named tool choice
- patch both chat-completions and responses parser entry points
- keep the rest of upstream behavior unchanged

Upstream tracking:
- issue: vllm-project/vllm#40147
- PR: vllm-project/vllm#40148

### Does this PR introduce _any_ user-facing change?

Yes.

Forced named tool choice becomes robust when the reasoning parser
returns no post-reasoning content, avoiding an internal assertion
failure and emitting an empty-argument function call instead.

### How was this patch tested?

Unit tests:
```bash
pytest -sv tests/ut/patch/platform/test_patch_tool_choice_none_content.py \
  tests/ut/patch/platform/test_patch_glm_tool_call_parser.py \
  tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
```

Result: 22 passed.

---------

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
wangxiyuan pushed a commit that referenced this pull request Apr 30, 2026
## What this PR does / why we need it?

Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM
Ascend platform patch layer for the vLLM 0.19.1 runtime.

The patch:
- adds completion_tokens_details.reasoning_tokens to UsageInfo
- fixes MiniMax-M2 reasoning token counting before the first </think>
token
- wraps chat streaming/non-streaming generators to track raw output
token ids and inject reasoning usage details
- avoids source-code extraction/replacement of OpenAIServingChat methods

References:
- vLLM upstream PR: vllm-project/vllm#37955
- vLLM Ascend PR: #7700
- vLLM Ascend PR: #7835

## Does this PR introduce _any_ user-facing change?

Yes. MiniMax-M2 OpenAI-compatible chat responses can now include
accurate completion_tokens_details.reasoning_tokens usage accounting.

## How was this patch tested?

- PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - 10 passed
- ruff check
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
vllm_ascend/patch/platform/__init__.py
  - passed
- python -m py_compile
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - passed

Note: running the test without --confcutdir currently loads the
repo-wide UT conftest, which imports worker patches and fails before
test collection because ../vllm v0.19.1 does not contain
vllm.model_executor.models.qwen3_dflash.
- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
yangzhe-2026 pushed a commit to yangzhe-2026/vllm-ascend that referenced this pull request May 6, 2026
…project#8831)

## What this PR does / why we need it?

Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM
Ascend platform patch layer for the vLLM 0.19.1 runtime.

The patch:
- adds completion_tokens_details.reasoning_tokens to UsageInfo
- fixes MiniMax-M2 reasoning token counting before the first </think>
token
- wraps chat streaming/non-streaming generators to track raw output
token ids and inject reasoning usage details
- avoids source-code extraction/replacement of OpenAIServingChat methods

References:
- vLLM upstream PR: vllm-project/vllm#37955
- vLLM Ascend PR: vllm-project#7700
- vLLM Ascend PR: vllm-project#7835

## Does this PR introduce _any_ user-facing change?

Yes. MiniMax-M2 OpenAI-compatible chat responses can now include
accurate completion_tokens_details.reasoning_tokens usage accounting.

## How was this patch tested?

- PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - 10 passed
- ruff check
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
vllm_ascend/patch/platform/__init__.py
  - passed
- python -m py_compile
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - passed

Note: running the test without --confcutdir currently loads the
repo-wide UT conftest, which imports worker patches and fails before
test collection because ../vllm v0.19.1 does not contain
vllm.model_executor.models.qwen3_dflash.
- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
PiratePai pushed a commit to PiratePai/vllm-ascend that referenced this pull request May 7, 2026
…project#8831)

## What this PR does / why we need it?

Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM
Ascend platform patch layer for the vLLM 0.19.1 runtime.

The patch:
- adds completion_tokens_details.reasoning_tokens to UsageInfo
- fixes MiniMax-M2 reasoning token counting before the first </think>
token
- wraps chat streaming/non-streaming generators to track raw output
token ids and inject reasoning usage details
- avoids source-code extraction/replacement of OpenAIServingChat methods

References:
- vLLM upstream PR: vllm-project/vllm#37955
- vLLM Ascend PR: vllm-project#7700
- vLLM Ascend PR: vllm-project#7835

## Does this PR introduce _any_ user-facing change?

Yes. MiniMax-M2 OpenAI-compatible chat responses can now include
accurate completion_tokens_details.reasoning_tokens usage accounting.

## How was this patch tested?

- PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - 10 passed
- ruff check
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
vllm_ascend/patch/platform/__init__.py
  - passed
- python -m py_compile
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - passed

Note: running the test without --confcutdir currently loads the
repo-wide UT conftest, which imports worker patches and fails before
test collection because ../vllm v0.19.1 does not contain
vllm.model_executor.models.qwen3_dflash.
- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: PiratePai <416932041@qq.com>
yangzhe-2026 pushed a commit to yangzhe-2026/vllm-ascend that referenced this pull request May 10, 2026
…project#8831)

## What this PR does / why we need it?

Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM
Ascend platform patch layer for the vLLM 0.19.1 runtime.

The patch:
- adds completion_tokens_details.reasoning_tokens to UsageInfo
- fixes MiniMax-M2 reasoning token counting before the first </think>
token
- wraps chat streaming/non-streaming generators to track raw output
token ids and inject reasoning usage details
- avoids source-code extraction/replacement of OpenAIServingChat methods

References:
- vLLM upstream PR: vllm-project/vllm#37955
- vLLM Ascend PR: vllm-project#7700
- vLLM Ascend PR: vllm-project#7835

## Does this PR introduce _any_ user-facing change?

Yes. MiniMax-M2 OpenAI-compatible chat responses can now include
accurate completion_tokens_details.reasoning_tokens usage accounting.

## How was this patch tested?

- PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - 10 passed
- ruff check
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
vllm_ascend/patch/platform/__init__.py
  - passed
- python -m py_compile
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - passed

Note: running the test without --confcutdir currently loads the
repo-wide UT conftest, which imports worker patches and fails before
test collection because ../vllm v0.19.1 does not contain
vllm.model_executor.models.qwen3_dflash.
- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: yangzhe-2026 <yangzhe@isrc.iscas.ac.cn>
ZhuQi-seu pushed a commit to ZhuQi-seu/vllm-ascend that referenced this pull request May 12, 2026
…project#8831)

## What this PR does / why we need it?

Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM
Ascend platform patch layer for the vLLM 0.19.1 runtime.

The patch:
- adds completion_tokens_details.reasoning_tokens to UsageInfo
- fixes MiniMax-M2 reasoning token counting before the first </think>
token
- wraps chat streaming/non-streaming generators to track raw output
token ids and inject reasoning usage details
- avoids source-code extraction/replacement of OpenAIServingChat methods

References:
- vLLM upstream PR: vllm-project/vllm#37955
- vLLM Ascend PR: vllm-project#7700
- vLLM Ascend PR: vllm-project#7835

## Does this PR introduce _any_ user-facing change?

Yes. MiniMax-M2 OpenAI-compatible chat responses can now include
accurate completion_tokens_details.reasoning_tokens usage accounting.

## How was this patch tested?

- PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - 10 passed
- ruff check
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
vllm_ascend/patch/platform/__init__.py
  - passed
- python -m py_compile
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - passed

Note: running the test without --confcutdir currently loads the
repo-wide UT conftest, which imports worker patches and fails before
test collection because ../vllm v0.19.1 does not contain
vllm.model_executor.models.qwen3_dflash.
- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: ZhuQi-seu <zhuqi12@huawei.com>
nanxingMy pushed a commit to nanxingMy/vllm-ascend that referenced this pull request May 15, 2026
…project#8831)

## What this PR does / why we need it?

Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM
Ascend platform patch layer for the vLLM 0.19.1 runtime.

The patch:
- adds completion_tokens_details.reasoning_tokens to UsageInfo
- fixes MiniMax-M2 reasoning token counting before the first </think>
token
- wraps chat streaming/non-streaming generators to track raw output
token ids and inject reasoning usage details
- avoids source-code extraction/replacement of OpenAIServingChat methods

References:
- vLLM upstream PR: vllm-project/vllm#37955
- vLLM Ascend PR: vllm-project#7700
- vLLM Ascend PR: vllm-project#7835

## Does this PR introduce _any_ user-facing change?

Yes. MiniMax-M2 OpenAI-compatible chat responses can now include
accurate completion_tokens_details.reasoning_tokens usage accounting.

## How was this patch tested?

- PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - 10 passed
- ruff check
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
vllm_ascend/patch/platform/__init__.py
  - passed
- python -m py_compile
vllm_ascend/patch/platform/patch_minimax_usage_accounting.py
tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
  - passed

Note: running the test without --confcutdir currently loads the
repo-wide UT conftest, which imports worker patches and fails before
test collection because ../vllm v0.19.1 does not contain
vllm.model_executor.models.qwen3_dflash.
- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: nanxing <1014662416@qq.com>
immengzi pushed a commit to immengzi/vllm-ascend that referenced this pull request May 21, 2026
…pty content (vllm-project#8400)

### What this PR does / why we need it?

This backports the forced-tool-choice `content=None` guard to the
`releases/v0.18.0` compatibility layer.

Upstream vLLM still has forced named tool-choice branches that assert
`content is not None` after reasoning extraction. Some reasoning parsers
can legally consume the full output and return `(reasoning, None)`,
which makes the assert reachable and can surface as a server-side
failure.

This PR follows the same compatibility-patch pattern used by:
- `7314bbe2` fix(platform): reimplement MiniMax usage accounting patch
(vllm-project#7835)
- `f83cb0e6` [Bugfix][Platform] Fix GLM47 tool-call finish backfill
(vllm-project#7710)

The patch is intentionally narrow:
- normalize `content=None` to `""` only for forced named tool choice
- patch both chat-completions and responses parser entry points
- keep the rest of upstream behavior unchanged

Upstream tracking:
- issue: vllm-project/vllm#40147
- PR: vllm-project/vllm#40148

### Does this PR introduce _any_ user-facing change?

Yes.

Forced named tool choice becomes robust when the reasoning parser
returns no post-reasoning content, avoiding an internal assertion
failure and emitting an empty-argument function call instead.

### How was this patch tested?

Unit tests:
```bash
pytest -sv tests/ut/patch/platform/test_patch_tool_choice_none_content.py \
  tests/ut/patch/platform/test_patch_glm_tool_call_parser.py \
  tests/ut/patch/platform/test_patch_minimax_usage_accounting.py
```

Result: 22 passed.

---------

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready enable e2e test for PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants