Repository navigation
[BugFix][Frontend] Load reasoning parser plugins in headless mode - #14651
QwertyJack wants to merge 1 commit into
Conversation
|
👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:
If CI fails, you can run linting and testing checks locally according Contributing and Testing. Tip 💡 Consider Linking a Related Issue or RFCYour PR title contains the [BugFix] tag, indicating a bug fix or new feature. Linking a related issue or RFC in the PR description is strongly encouraged — it gives reviewers helpful context and speeds up the review. You can use any of these keywords:
🙏 Thanks for helping us keep the project well-organized! |
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses a Highlights
New Features🧠 You can now enable Memory (public preview) to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
Suggested PR Title:\n\nmarkdown\n[Ops][BugFix] Patch reasoning parser plugin import in VllmConfig post-init\n\n\nSuggested PR Summary:\n\nmarkdown\n### What this PR does / why we need it?\nThis PR adds a patch to `VllmConfig.__post_init__` to import the configured reasoning parser plugin before the original initialization resolves reasoning token IDs. This fixes an issue where `vllm serve --headless` fails because the custom reasoning parser is not registered in time.\n\nFeedback: A potential `AttributeError` was identified in `patch_reasoning_parser_plugin.py` where `vllm_config.structured_outputs_config` is accessed directly without checking if it is `None`. A defensive check should be added to prevent server startup crashes.\n\n### Does this PR introduce _any_ user-facing change?\nNo.\n\n### How was this patch tested?\nNew unit tests were added in `tests/ut/patch/platform/test_patch_reasoning_parser_plugin.py` to cover import behavior, skip conditions, and contract drift validation.\n
e750559 to
7b987e3
Compare
Import missing custom reasoning parser plugins before VllmConfig post-initialization so headless DP ranks can resolve parser token IDs. Signed-off-by: QwertyJack <7554089+QwertyJack@users.noreply.github.com>
7b987e3 to
284147a
Compare
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
Closed by #18229 |
What this PR does / why we need it?
This PR adds a temporary platform monkey patch for custom reasoning parser plugins in headless mode. Upstream vLLM creates
VllmConfigbefore importing--reasoning-parser-pluginon the headless path, so reasoning token initialization raisesKeyErrorbefore workers start.The patch imports the configured plugin before the original
VllmConfig.__post_init__only when the requested parser is not already registered. This keeps the regular API server and programmatic parser registration behavior unchanged. It also fails explicitly if the privateVllmConfig.__post_init__(self)contract changes.Fixes #14573.
Upstream issue: vllm-project/vllm#53072
Does this PR introduce any user-facing change?
Yes. Ascend headless DP ranks can start with a custom
--reasoning-parser-plugininstead of failing withKeyErrorduring engine config creation.How was this patch tested?
python -m pytest -q tests/ut/patch/platform/test_patch_reasoning_parser_plugin.py tests/ut/test_platform.py: 72 passed, 1 skippedbash format.sh ci: passedManual headless startup with Qwen3-0.6B, DP size 2, local DP size 1, and start rank 1 reached
Launching 1 data parallel engine(s) in headless modewithout the former parserKeyError. The process was then stopped because the DP head was intentionally absent.vLLM main: vllm-project/vllm@ba07e4a