feat: Support vLLM v0.15.1 or newer#94
Merged
walterbm-cohere merged 1 commit intomainfrom Apr 14, 2026
Merged
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 9e65a48. Configure here.
- Branch OpenAI entrypoint imports on vllm version: vllm > 0.14.1 uses the reorganized paths introduced in vllm-project/vllm#32240 - Add ty: ignore[unresolved-import] suppressions for version-gated imports that may not exist in the installed vllm - Matrix py-check CI job across vllm 0.14.1 and 0.15.1 - Fix is_reasoning_end signature: list[int] -> Sequence[int] to match the abstract base class
d3b7eed to
e3aa4f8
Compare
shun-cohere
commented
Apr 14, 2026
| return content_ids | ||
|
|
||
| def is_reasoning_end(self, input_ids: list[int]) -> bool: | ||
| def is_reasoning_end(self, input_ids: Sequence[int]) -> bool: |
Contributor
Author
There was a problem hiding this comment.
To make ty happy, otherwise the following error occurs
error[invalid-method-override]: Invalid override of method `is_reasoning_end`
--> cohere_melody_vllm/parser.py:144:9
|
142 | return content_ids
143 |
144 | def is_reasoning_end(self, input_ids: list[int]) -> bool:
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Definition is incompatible with `ReasoningParser.is_reasoning_end`
145 | end_token_id = self.model_tokenizer.convert_tokens_to_ids("<|END_THINKING|>")
146 | return any(input_id == end_token_id for input_id in reversed(input_ids))
|
::: .venv/lib/python3.12/site-packages/vllm/reasoning/abs_reasoning_parsers.py:54:9
|
53 | @abstractmethod
54 | def is_reasoning_end(self, input_ids: Sequence[int]) -> bool:
| -------------------------------------------------------- `ReasoningParser.is_reasoning_end` defined here
55 | """
56 | Check if the reasoning content ends in the input_ids.
|
walterbm-cohere
approved these changes
Apr 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Description
This PR aims to support vLLM v0.15.1 or newer verions.
To do this, we introduce a conditional import logic at the top of
parser.pyRelated Issue
vllm-project/vllm#32240 introduces new structure, which is a breaking change for melody
Motivation and Context
Melody does not support vLLM v0.15 or newer verions.
How Has This Been Tested?
Check1: Confirm that import works with both vLLM versions
Check2: Tool Calling works with vLLM v0.15.1
Start server
and then send a tool calling query
Note
Medium Risk
Adds version-dependent imports for vLLM OpenAI protocol types, so a mistake in version detection or module paths could cause runtime import failures across supported vLLM versions.
Overview
Adds vLLM version-aware import logic in
cohere_melody_vllm/parser.pyto handle the OpenAI entrypoint protocol module reorganization introduced after vLLM0.14.1, enabling the plugin to run against both old and new layouts.Updates the Python bindings CI
py-checkjob to runty checkin a matrix against vLLM0.14.1and0.15.1to continuously validate compatibility.Reviewed by Cursor Bugbot for commit e3aa4f8. Bugbot is set up for automated code reviews on this repo. Configure here.