[BugFix][Patch] Backport forced tool choice none-content handling#8833
Conversation
Backport the vLLM forced tool-choice handling for cases where reasoning extraction leaves content as None. Return no tool calls instead of synthesizing empty arguments, and keep normal forced tool behavior when content is present. Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request backports a fix from the upstream vLLM project to address an issue where forced tool choice requests would trigger an assertion error when reasoning extraction resulted in empty content. By patching the tool call parsing logic, the system now correctly returns an empty tool-call list instead of failing, ensuring stability when generation stops prematurely or content is otherwise unavailable. Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. |
There was a problem hiding this comment.
Code Review
Suggested PR Title:\n\nmarkdown\n[vllm-ascend][Platform][BugFix] Patch forced tool choice to handle None content\n\n\nSuggested PR Summary:\n\nmarkdown\n### What this PR does / why we need it?\nThis PR introduces a patch to handle scenarios where forced tool choice receives `None` content, which typically occurs when reasoning extraction consumes the entire generated text (e.g., when generation stops at `max_tokens`). This prevents assertions in upstream vLLM 0.19.1 by returning an empty tool-call list and marking the tool-choice result to avoid downstream errors.\n\n### Does this PR introduce _any_ user-facing change?\nNo.\n\n### How was this patch tested?\nCI passed with new unit tests added in `tests/ut/patch/platform/test_patch_tool_choice_none_content.py` that verify the behavior for both `OpenAIServing` and `DelegatingParser`.\n\n\nI have no feedback to provide as there were no review comments to assess.
…lm-project#8833) ### What this PR does / why we need it? Backports the forced tool-choice `content=None` handling from vLLM Ascend PR vllm-project#8400 for the vLLM 0.19.1 patch layer. Related issue and PR: - vLLM issue: vllm-project/vllm#40147 - vLLM PR: vllm-project/vllm#40148 - vLLM Ascend PR: vllm-project#8400 Per the latest PR #40148 discussion, this patch does not synthesize an empty function call or empty `{}` arguments. When forced tool choice sees `content=None`, it returns an empty tool-call list and preserves normal forced tool-call behavior when content is present. ### Does this PR introduce _any_ user-facing change? Yes. Forced tool-choice requests no longer assert when reasoning extraction leaves `content=None`; they return no tool calls for that empty-content result instead. ### How was this patch tested? - `ruff format --check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py` - `ruff check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py` - `python -m py_compile vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py` - `PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_tool_choice_none_content.py` Also ran `bash format.sh ci`; it passed the available hooks but could not complete `shellcheck` because `shellcheck` is not installed in this environment. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
…lm-project#8833) ### What this PR does / why we need it? Backports the forced tool-choice `content=None` handling from vLLM Ascend PR vllm-project#8400 for the vLLM 0.19.1 patch layer. Related issue and PR: - vLLM issue: vllm-project/vllm#40147 - vLLM PR: vllm-project/vllm#40148 - vLLM Ascend PR: vllm-project#8400 Per the latest PR #40148 discussion, this patch does not synthesize an empty function call or empty `{}` arguments. When forced tool choice sees `content=None`, it returns an empty tool-call list and preserves normal forced tool-call behavior when content is present. ### Does this PR introduce _any_ user-facing change? Yes. Forced tool-choice requests no longer assert when reasoning extraction leaves `content=None`; they return no tool calls for that empty-content result instead. ### How was this patch tested? - `ruff format --check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py` - `ruff check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py` - `python -m py_compile vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py` - `PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_tool_choice_none_content.py` Also ran `bash format.sh ci`; it passed the available hooks but could not complete `shellcheck` because `shellcheck` is not installed in this environment. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: PiratePai <416932041@qq.com>
…lm-project#8833) ### What this PR does / why we need it? Backports the forced tool-choice `content=None` handling from vLLM Ascend PR vllm-project#8400 for the vLLM 0.19.1 patch layer. Related issue and PR: - vLLM issue: vllm-project/vllm#40147 - vLLM PR: vllm-project/vllm#40148 - vLLM Ascend PR: vllm-project#8400 Per the latest PR #40148 discussion, this patch does not synthesize an empty function call or empty `{}` arguments. When forced tool choice sees `content=None`, it returns an empty tool-call list and preserves normal forced tool-call behavior when content is present. ### Does this PR introduce _any_ user-facing change? Yes. Forced tool-choice requests no longer assert when reasoning extraction leaves `content=None`; they return no tool calls for that empty-content result instead. ### How was this patch tested? - `ruff format --check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py` - `ruff check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py` - `python -m py_compile vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py` - `PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_tool_choice_none_content.py` Also ran `bash format.sh ci`; it passed the available hooks but could not complete `shellcheck` because `shellcheck` is not installed in this environment. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: yangzhe-2026 <yangzhe@isrc.iscas.ac.cn>
…lm-project#8833) ### What this PR does / why we need it? Backports the forced tool-choice `content=None` handling from vLLM Ascend PR vllm-project#8400 for the vLLM 0.19.1 patch layer. Related issue and PR: - vLLM issue: vllm-project/vllm#40147 - vLLM PR: vllm-project/vllm#40148 - vLLM Ascend PR: vllm-project#8400 Per the latest PR #40148 discussion, this patch does not synthesize an empty function call or empty `{}` arguments. When forced tool choice sees `content=None`, it returns an empty tool-call list and preserves normal forced tool-call behavior when content is present. ### Does this PR introduce _any_ user-facing change? Yes. Forced tool-choice requests no longer assert when reasoning extraction leaves `content=None`; they return no tool calls for that empty-content result instead. ### How was this patch tested? - `ruff format --check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py` - `ruff check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py` - `python -m py_compile vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py` - `PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_tool_choice_none_content.py` Also ran `bash format.sh ci`; it passed the available hooks but could not complete `shellcheck` because `shellcheck` is not installed in this environment. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: ZhuQi-seu <zhuqi12@huawei.com>
…lm-project#8833) ### What this PR does / why we need it? Backports the forced tool-choice `content=None` handling from vLLM Ascend PR vllm-project#8400 for the vLLM 0.19.1 patch layer. Related issue and PR: - vLLM issue: vllm-project/vllm#40147 - vLLM PR: vllm-project/vllm#40148 - vLLM Ascend PR: vllm-project#8400 Per the latest PR #40148 discussion, this patch does not synthesize an empty function call or empty `{}` arguments. When forced tool choice sees `content=None`, it returns an empty tool-call list and preserves normal forced tool-call behavior when content is present. ### Does this PR introduce _any_ user-facing change? Yes. Forced tool-choice requests no longer assert when reasoning extraction leaves `content=None`; they return no tool calls for that empty-content result instead. ### How was this patch tested? - `ruff format --check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py` - `ruff check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py` - `python -m py_compile vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py` - `PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_tool_choice_none_content.py` Also ran `bash format.sh ci`; it passed the available hooks but could not complete `shellcheck` because `shellcheck` is not installed in this environment. - vLLM version: v0.19.1 - vLLM main: vllm-project/vllm@d886c26 Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com> Signed-off-by: nanxing <1014662416@qq.com>
What this PR does / why we need it?
Backports the forced tool-choice
content=Nonehandling from vLLM Ascend PR #8400 for the vLLM 0.19.1 patch layer.Related issue and PR:
Per the latest PR #40148 discussion, this patch does not synthesize an empty function call or empty
{}arguments. When forced tool choice seescontent=None, it returns an empty tool-call list and preserves normal forced tool-call behavior when content is present.Does this PR introduce any user-facing change?
Yes. Forced tool-choice requests no longer assert when reasoning extraction leaves
content=None; they return no tool calls for that empty-content result instead.How was this patch tested?
ruff format --check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.pyruff check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.pypython -m py_compile vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.pyPYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_tool_choice_none_content.pyAlso ran
bash format.sh ci; it passed the available hooks but could not completeshellcheckbecauseshellcheckis not installed in this environment.