New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Upgrade to vllm 0.6.0 #4

Closed

stikkireddy opened this issue Sep 5, 2024 · 2 comments

Owner

stikkireddy commented Sep 5, 2024

https://github.com/vllm-project/vllm/releases

incorporate --num-scheduler-steps 8
test openai tool spec
test multi multi modal input support
test tensor parallel for ViT models in multi modal llms
supported tensor-parallel ViT models. [(also test the new xformers ]([VLM][Model] TP support for ViTs vllm-project/vllm#7186)
- Siglip
- Clip
- Blip
- Intern ViT

Owner Author

stikkireddy commented Sep 6, 2024

upgraded to 0.6.0 new feature support remaining

Owner Author

stikkireddy commented Sep 8, 2024

will not support --num-scheduler-steps 8, too many limitations
test openai tool spec (misunderstanding of release)
test tensor parallel for ViT models in multi modal llms (tensor parallel issues in azure databricks)

stikkireddy closed this as completed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment