[RFC]: Path to vLLM 1.0 Release

[robertgshaw2-redhat]

Motivation.

vLLM's V1 architecture has been in production for a while now, and we have seen good results with the transition from V0 to V1. We are now in progress of deprecating the V0 code path and removing the technical debt from vLLM.

As we complete this effort, it is now time to begin planning for the 1.0 release which will occur sometime this fall.

With the V1 engine update focused on refactoring and eliminating the technical debt in vLLM's internal subsystems, with the major version update for 1.0, we will now take the opportunity to clean up vLLM's external interfaces and adhere to a semantic versioning scheme.

This is increasingly important as more interfaces are becoming pluggable in vLLM V1

• V0 Deprecation Issue: [RFC]: Deprecating vLLM V0 #18571

Proposed Change.

We are developing a work plan towards this 1.0 release.

Major items we have targeted are:

• Stabilize and document the existing Pluggable Interfaces (Scheduler, KVConnector, ModelRunner, Model)
• Clarify and document which subsystems are intended to be pluggable (public apis) vs not (private apis)
• Stabilize and document AsyncLLM and LLM interfaces - what is public vs private
• Redo and simplify the vLLM configuration setup (EngineArgs, Env variables)
• Redo and simplify the vLLM logging setup
• Simplify vLLM Prometheus metrics
• Overhaul docs and recipes
• Overhaul performance tracking system
• Website

Feedback Period.

End of August

CC List.

@simon-mo @WoosukKwon @youkaichao @njhill @tlrmchlsmth @mgoin @ywang96 @yeqcharlotte @houseroad

Any Other Things.

What else are we missing

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[ProExpertProg]

I think we should also stabilize the CLI and clarify what are "user" and what are "developer"-facing flags.

Overhaul performance tracking system

++, we should also include startup time breakdown in this.

[qiaoning]

vllm-plugin is a great mechanism to extend other cuda-like device platforms, and I'd like to see that make vllm plugin more easy to intergrated with OOT GPU platforms.

[gaby]

Having proper support for gpt-oss should be at the top of the list, there's several bugs and issues still. Lots of issues complaining about tools.

[robertgshaw2-redhat]

Having proper support for gpt-oss should be at the top of the list, there's several bugs and issues still. Lots of issues complaining about tools.

Thanks, this is unrelated to the 1.0 release planning

[hmellor]

For the config/CLI we could:

• remove EngineArgs and just stick with VllmConfig
  • that's where all the typing/checking/documentation is best
  • users configuring vLLM with a class called EngineArgs seems counterintuitive as it requires some knowledge of vLLM's internals (i.e. What's an engine?) and there is already a class called VllmConfig
• use typing.Annotated or pydantic.PrivateAttr to indicate if a particular field user facing or not
• automatically add a CLI argument for anything that's not been marked as private

[shajrawi]

For ROCm backend we should also consider AITER planning. Our performant kernel library.
For it to be part of the release making it into a solid pip installable package would be a prerequisite.
Then we should look into getting rid of most/all operator specific enjoinment variables to make the use of AITER transparent to the users. cc @gshtras

[robertgshaw2-redhat]

For ROCm backend we should also consider AITER planning. Our performant kernel library. For it to be part of the release making it into a solid pip installable package would be a prerequisite. Then we should look into getting rid of most/all operator specific enjoinment variables to make the use of AITER transparent to the users. cc @gshtras

Do you think a pip install for vllm on ROCm is worthwhile too? Or should we focus on docker?

[robertgshaw2-redhat]

• remove EngineArgs and just stick with VllmConfig

I actually disagree. There is way too many public flags for vllm. We should only have a very small number of these publicly exposed.

[shajrawi]

Do you think a pip install for vllm on ROCm is worthwhile too? Or should we focus on docker?

Totally worthwhile. Had more asks for it recently. Our main focus will remain docker releases, but we had customer interest in running baremetal

[hmellor]

I actually disagree. There is way too many public flags for vllm. We should only have a very small number of these publicly exposed.

You make a good point, even if we annotated to opt in to public config, an LLM user would still see all the private stuff from their IDE.

[yeqcharlotte]

Overhaul performance tracking system

cc: @Jialin @linzebing @huydhn @yangw-dev are planning to iterate https://hud.pytorch.org/benchmark/llms?repoName=vllm-project%2Fvllm to add more fine grained metrics, and add alerting and bisection functionalities.

[yeqcharlotte]

• remove EngineArgs and just stick with VllmConfig

I actually disagree. There is way too many public flags for vllm. We should only have a very small number of these publicly exposed.

I also agree that we VllmConfig should be internal only andEngineArgs should be gradually simplified to offer fewer and less confusing args for command lines.

[yeqcharlotte]

Major items we have targeted are:

@robertgshaw2-redhat would like to add one item: inventory and contract of plug-ins.

As we expect many people to build on top of vLLM, I think we should call out those components and iterate them into stable interface and continuously test interface behavior.

As of now I'm aware of the following components can be swapped out with different implementations (which is probably incomplete) -- model, hardware, kv connector, tool/reasoning parser, metrics, model loader, video backend.

I think we should start going throw all components and have an inventory of pluggable components and components we explicitly choose not to allow extension. Then document, share implementation for plugin management and have CI against them.