Skip to content

[BugFix][Patch] Backport forced tool choice none-content handling#8833

Merged
wangxiyuan merged 1 commit into
vllm-project:mainfrom
QwertyJack:backport/tool-choice-none-content
Apr 30, 2026
Merged

[BugFix][Patch] Backport forced tool choice none-content handling#8833
wangxiyuan merged 1 commit into
vllm-project:mainfrom
QwertyJack:backport/tool-choice-none-content

Conversation

@QwertyJack

@QwertyJack QwertyJack commented Apr 30, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it?

Backports the forced tool-choice content=None handling from vLLM Ascend PR #8400 for the vLLM 0.19.1 patch layer.

Related issue and PR:

Per the latest PR #40148 discussion, this patch does not synthesize an empty function call or empty {} arguments. When forced tool choice sees content=None, it returns an empty tool-call list and preserves normal forced tool-call behavior when content is present.

Does this PR introduce any user-facing change?

Yes. Forced tool-choice requests no longer assert when reasoning extraction leaves content=None; they return no tool calls for that empty-content result instead.

How was this patch tested?

  • ruff format --check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py
  • ruff check vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py
  • python -m py_compile vllm_ascend/patch/platform/patch_tool_choice_none_content.py tests/ut/patch/platform/test_patch_tool_choice_none_content.py
  • PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform tests/ut/patch/platform/test_patch_tool_choice_none_content.py

Also ran bash format.sh ci; it passed the available hooks but could not complete shellcheck because shellcheck is not installed in this environment.

Backport the vLLM forced tool-choice handling for cases where reasoning extraction leaves content as None. Return no tool calls instead of synthesizing empty arguments, and keep normal forced tool behavior when content is present.

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
@QwertyJack QwertyJack requested a review from wangxiyuan as a code owner April 30, 2026 05:52
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request backports a fix from the upstream vLLM project to address an issue where forced tool choice requests would trigger an assertion error when reasoning extraction resulted in empty content. By patching the tool call parsing logic, the system now correctly returns an empty tool-call list instead of failing, ensuring stability when generation stops prematurely or content is otherwise unavailable.

Highlights

  • Bugfix for Forced Tool Choice: Implemented a patch to handle cases where forced tool choice receives 'None' content, preventing assertion errors during reasoning extraction.
  • Monkey-patching Implementation: Added monkey-patches to 'OpenAIServing._parse_tool_calls_from_content' and 'DelegatingParser._parse_tool_calls' to gracefully return empty tool-call lists when content is missing.
  • Testing: Added a new unit test file 'tests/ut/patch/platform/test_patch_tool_choice_none_content.py' to verify the fix for both standard and delegating parsers.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

  • A PR should do only one thing, smaller PRs enable faster reviews.
  • Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
  • Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

@QwertyJack QwertyJack changed the title [Bugfix][Patch] Backport forced tool choice none-content handling [BugFix][Patch] Backport forced tool choice none-content handling Apr 30, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

Suggested PR Title:\n\nmarkdown\n[vllm-ascend][Platform][BugFix] Patch forced tool choice to handle None content\n\n\nSuggested PR Summary:\n\nmarkdown\n### What this PR does / why we need it?\nThis PR introduces a patch to handle scenarios where forced tool choice receives `None` content, which typically occurs when reasoning extraction consumes the entire generated text (e.g., when generation stops at `max_tokens`). This prevents assertions in upstream vLLM 0.19.1 by returning an empty tool-call list and marking the tool-choice result to avoid downstream errors.\n\n### Does this PR introduce _any_ user-facing change?\nNo.\n\n### How was this patch tested?\nCI passed with new unit tests added in `tests/ut/patch/platform/test_patch_tool_choice_none_content.py` that verify the behavior for both `OpenAIServing` and `DelegatingParser`.\n\n\nI have no feedback to provide as there were no review comments to assess.

@QwertyJack QwertyJack closed this Apr 30, 2026
@QwertyJack QwertyJack reopened this Apr 30, 2026
@wangxiyuan wangxiyuan merged commit f171285 into vllm-project:main Apr 30, 2026
30 of 31 checks passed
@QwertyJack QwertyJack deleted the backport/tool-choice-none-content branch April 30, 2026 06:46
yangzhe-2026 pushed a commit to yangzhe-2026/vllm-ascend that referenced this pull request May 6, 2026
…lm-project#8833)

### What this PR does / why we need it?
Backports the forced tool-choice `content=None` handling from vLLM
Ascend PR vllm-project#8400 for the
vLLM 0.19.1 patch layer.

Related issue and PR:
- vLLM issue: vllm-project/vllm#40147
- vLLM PR: vllm-project/vllm#40148
- vLLM Ascend PR: vllm-project#8400

Per the latest PR #40148 discussion, this patch does not synthesize an
empty function call or empty `{}` arguments. When forced tool choice
sees `content=None`, it returns an empty tool-call list and preserves
normal forced tool-call behavior when content is present.

### Does this PR introduce _any_ user-facing change?
Yes. Forced tool-choice requests no longer assert when reasoning
extraction leaves `content=None`; they return no tool calls for that
empty-content result instead.

### How was this patch tested?
- `ruff format --check
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py
vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py`
- `ruff check
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py
vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py`
- `python -m py_compile
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py`
- `PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_tool_choice_none_content.py`

Also ran `bash format.sh ci`; it passed the available hooks but could
not complete `shellcheck` because `shellcheck` is not installed in this
environment.

- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
PiratePai pushed a commit to PiratePai/vllm-ascend that referenced this pull request May 7, 2026
…lm-project#8833)

### What this PR does / why we need it?
Backports the forced tool-choice `content=None` handling from vLLM
Ascend PR vllm-project#8400 for the
vLLM 0.19.1 patch layer.

Related issue and PR:
- vLLM issue: vllm-project/vllm#40147
- vLLM PR: vllm-project/vllm#40148
- vLLM Ascend PR: vllm-project#8400

Per the latest PR #40148 discussion, this patch does not synthesize an
empty function call or empty `{}` arguments. When forced tool choice
sees `content=None`, it returns an empty tool-call list and preserves
normal forced tool-call behavior when content is present.

### Does this PR introduce _any_ user-facing change?
Yes. Forced tool-choice requests no longer assert when reasoning
extraction leaves `content=None`; they return no tool calls for that
empty-content result instead.

### How was this patch tested?
- `ruff format --check
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py
vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py`
- `ruff check
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py
vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py`
- `python -m py_compile
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py`
- `PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_tool_choice_none_content.py`

Also ran `bash format.sh ci`; it passed the available hooks but could
not complete `shellcheck` because `shellcheck` is not installed in this
environment.

- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: PiratePai <416932041@qq.com>
yangzhe-2026 pushed a commit to yangzhe-2026/vllm-ascend that referenced this pull request May 10, 2026
…lm-project#8833)

### What this PR does / why we need it?
Backports the forced tool-choice `content=None` handling from vLLM
Ascend PR vllm-project#8400 for the
vLLM 0.19.1 patch layer.

Related issue and PR:
- vLLM issue: vllm-project/vllm#40147
- vLLM PR: vllm-project/vllm#40148
- vLLM Ascend PR: vllm-project#8400

Per the latest PR #40148 discussion, this patch does not synthesize an
empty function call or empty `{}` arguments. When forced tool choice
sees `content=None`, it returns an empty tool-call list and preserves
normal forced tool-call behavior when content is present.

### Does this PR introduce _any_ user-facing change?
Yes. Forced tool-choice requests no longer assert when reasoning
extraction leaves `content=None`; they return no tool calls for that
empty-content result instead.

### How was this patch tested?
- `ruff format --check
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py
vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py`
- `ruff check
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py
vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py`
- `python -m py_compile
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py`
- `PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_tool_choice_none_content.py`

Also ran `bash format.sh ci`; it passed the available hooks but could
not complete `shellcheck` because `shellcheck` is not installed in this
environment.

- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: yangzhe-2026 <yangzhe@isrc.iscas.ac.cn>
ZhuQi-seu pushed a commit to ZhuQi-seu/vllm-ascend that referenced this pull request May 12, 2026
…lm-project#8833)

### What this PR does / why we need it?
Backports the forced tool-choice `content=None` handling from vLLM
Ascend PR vllm-project#8400 for the
vLLM 0.19.1 patch layer.

Related issue and PR:
- vLLM issue: vllm-project/vllm#40147
- vLLM PR: vllm-project/vllm#40148
- vLLM Ascend PR: vllm-project#8400

Per the latest PR #40148 discussion, this patch does not synthesize an
empty function call or empty `{}` arguments. When forced tool choice
sees `content=None`, it returns an empty tool-call list and preserves
normal forced tool-call behavior when content is present.

### Does this PR introduce _any_ user-facing change?
Yes. Forced tool-choice requests no longer assert when reasoning
extraction leaves `content=None`; they return no tool calls for that
empty-content result instead.

### How was this patch tested?
- `ruff format --check
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py
vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py`
- `ruff check
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py
vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py`
- `python -m py_compile
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py`
- `PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_tool_choice_none_content.py`

Also ran `bash format.sh ci`; it passed the available hooks but could
not complete `shellcheck` because `shellcheck` is not installed in this
environment.

- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: ZhuQi-seu <zhuqi12@huawei.com>
nanxingMy pushed a commit to nanxingMy/vllm-ascend that referenced this pull request May 15, 2026
…lm-project#8833)

### What this PR does / why we need it?
Backports the forced tool-choice `content=None` handling from vLLM
Ascend PR vllm-project#8400 for the
vLLM 0.19.1 patch layer.

Related issue and PR:
- vLLM issue: vllm-project/vllm#40147
- vLLM PR: vllm-project/vllm#40148
- vLLM Ascend PR: vllm-project#8400

Per the latest PR #40148 discussion, this patch does not synthesize an
empty function call or empty `{}` arguments. When forced tool choice
sees `content=None`, it returns an empty tool-call list and preserves
normal forced tool-call behavior when content is present.

### Does this PR introduce _any_ user-facing change?
Yes. Forced tool-choice requests no longer assert when reasoning
extraction leaves `content=None`; they return no tool calls for that
empty-content result instead.

### How was this patch tested?
- `ruff format --check
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py
vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py`
- `ruff check
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py
vllm_ascend/patch/platform/__init__.py vllm_ascend/patch/__init__.py`
- `python -m py_compile
vllm_ascend/patch/platform/patch_tool_choice_none_content.py
tests/ut/patch/platform/test_patch_tool_choice_none_content.py`
- `PYTHONPATH=../vllm:. pytest -q --confcutdir=tests/ut/patch/platform
tests/ut/patch/platform/test_patch_tool_choice_none_content.py`

Also ran `bash format.sh ci`; it passed the available hooks but could
not complete `shellcheck` because `shellcheck` is not installed in this
environment.

- vLLM version: v0.19.1
- vLLM main:
vllm-project/vllm@d886c26

Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Co-authored-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
Signed-off-by: nanxing <1014662416@qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants