[OpenAI] Fix reasoning_tokens accounting for MiniMax M2 chat completions#37955
[OpenAI] Fix reasoning_tokens accounting for MiniMax M2 chat completions#37955QwertyJack wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces the capability to track and report 'reasoning tokens' as part of the usage information for chat completions, specifically for Minimax M2 models. This involves defining a new CompletionTokenUsageInfo data structure, implementing the logic to count reasoning tokens within the MiniMaxM2ReasoningParser and MiniMaxM2AppendThinkReasoningParser classes, and integrating this counting into the OpenAIServingChat class for both full and streaming chat completions. New unit tests were added to verify the correct calculation and reporting of reasoning tokens for these parsers.
03be0f6 to
2598da4
Compare
## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: #7700 - vLLM Ascend PR: #7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
…project#8831) ## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: vllm-project#7700 - vLLM Ascend PR: vllm-project#7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
…project#8831) ## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: vllm-project#7700 - vLLM Ascend PR: vllm-project#7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: PiratePai <416932041@qq.com>
…project#8831) ## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: vllm-project#7700 - vLLM Ascend PR: vllm-project#7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: yangzhe-2026 <yangzhe@isrc.iscas.ac.cn>
…project#8831) ## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: vllm-project#7700 - vLLM Ascend PR: vllm-project#7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: ZhuQi-seu <zhuqi12@huawei.com>
…project#8831) ## What this PR does / why we need it? Backports the MiniMax-M2 reasoning usage accounting fix into the vLLM Ascend platform patch layer for the vLLM 0.19.1 runtime. The patch: - adds completion_tokens_details.reasoning_tokens to UsageInfo - fixes MiniMax-M2 reasoning token counting before the first </think> token - wraps chat streaming/non-streaming generators to track raw output token ids and inject reasoning usage details - avoids source-code extraction/replacement of OpenAIServingChat methods References: - vLLM upstream PR: vllm-project/vllm#37955 - vLLM Ascend PR: vllm-project#7700 - vLLM Ascend PR: vllm-project#7835 ## Does this PR introduce _any_ user-facing change? Yes. MiniMax-M2 OpenAI-compatible chat responses can now include accurate completion_tokens_details.reasoning_tokens usage accounting. ## How was this patch tested? - PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - 10 passed - ruff check vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py vllm_ascend/patch/platform/__init__.py - passed - python -m py_compile vllm_ascend/patch/platform/patch_minimax_usage_accounting.py tests/ut/patch/platform/test_patch_minimax_usage_accounting.py - passed Note: running the test without --confcutdir currently loads the repo-wide UT conftest, which imports worker patches and fails before test collection because ../vllm v0.19.1 does not contain vllm.model_executor.models.qwen3_dflash. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: nanxing <1014662416@qq.com>
2598da4 to
f232eaa
Compare
f232eaa to
f976977
Compare
|
Hi @QwertyJack, the pre-commit checks have failed. Please run: uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
f976977 to
f9d1af8
Compare
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
f9d1af8 to
3232393
Compare
Summary
completion_tokens_details.reasoning_tokensto OpenAI chat-completions usage</think>token as the reasoning boundary for bothminimax_m2andminimax_m2_append_thinkskip_global_cleanupbecause they do not allocate accelerator state and do not need global cleanupTesting
python -m pytest -q tests/entrypoints/openai/chat_completion/test_serving_chat.py -k 'test_chat_usage_includes_reasoning_tokens_for_minimax_parser or test_chat_stream_usage_includes_reasoning_tokens_for_minimax_parser'python -m pytest -q tests/reasoning/test_minimax_m2_reasoning_parser.py tests/reasoning/test_minimax_m2_append_reasoning_parser.py -k count_reasoning_tokens