[V0 Deprecation] Remove AsyncLLMEngine#25025
Conversation
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai>
WoosukKwon
requested review from
aarnphm,
chaunceyjiang,
njhill,
robertgshaw2-redhat and
simon-mo
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request effectively removes the deprecated V0 AsyncLLMEngine and its related components, which is a great cleanup. The changes are mostly straightforward removals of files and code blocks. However, I've identified a critical issue in vllm/entrypoints/openai/api_server.py where the removal of the V0 engine fallback path now leads to an unhelpful AssertionError if a configuration unsupported by the V1 engine is used. My review includes a suggestion to replace this with a more informative error to improve user experience.
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
This pull request has merge conflicts that must be resolved before it can be |
WoosukKwon
requested review from
DarkLight1337,
NickLucche and
youkaichao
as code owners
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
|
Is there a migration guide for existing code which uses the |
|
@cadedaniel Please correct me if I'm wrong, but no action should be needed because from vllm import AsyncLLMEngine
e = AsyncLLMEngine.from_engine_args(...)will just work (and create a V1 engine). |
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai> Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai> Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai> Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu> Signed-off-by: charlifu <charlifu@amd.com>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai> Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Signed-off-by: Woosuk Kwon <woosuk@thinkingmachines.ai> Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
3 participants
No description provided.