vllm main updates

[prashantgupta24]

Description

This branch has a fix for:

• Caching the token_ids (now the new tokens are cached in execute_model instead of update_states. This is because of [Optimization] Cache sampled token ids in model runner vllm#20291. )
• Changes from the CachedRequestData (✨ Use shared CachedRequestData as vllm:main #273)

Related Issues

Fix for #271

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[yannicks1]

quick question before reviewing this one thoroughly:
Do we want to ensure backwards compatibility (I see a couple of copied files from upstream, but also failing test with missing make_empty attribute for CachedRequestData). If yes down to which version do we ensure compatibility, if not we should enforce a minimum version which does not fail the test...

[prashantgupta24]

quick question before reviewing this one thoroughly: Do we want to ensure backwards compatibility (I see a couple of copied files from upstream, but also failing test with missing make_empty attribute for CachedRequestData). If yes down to which version do we ensure compatibility, if not we should enforce a minimum version which does not fail the test...

I think the general consensus is that we want to pin to vllm>=0.9.2, we were just waiting for vllm to release the version

[joerunde]

@yannicks1 Yeah we talked very briefly about it last Thursday, our assertion here is that our current delivery mechanism is images that bundle vllm-spyre with vllm, so we're the ones that have to worry about building images with compatible versions.

The small risk here is that there could be a regression in vllm 0.9.2 with some model or models that we then can't easily roll back, but we haven't matured this stack yet to GA support anywhere so there's currently no chance of that causing a product regression

[joerunde]

🍎🍎🍎