-
Notifications
You must be signed in to change notification settings - Fork 3.6k
[env, sglang] feat: Bump new sglang version to fix vlm OOM #3183
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
fe734cc
15aa1fb
8ebbea9
924045a
0134c01
f9e6973
692158b
2220fa7
efc675b
2512a9d
33250fc
a306bbb
e877188
cc045c7
e256309
009e65b
0333425
0deb807
63747a5
3744a35
52eaf98
72518a6
488f571
6e4e645
65a92c0
ba4c4ce
67f01d4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -36,11 +36,11 @@ RUN pip config set global.index-url "${PIP_INDEX}" && \ | |
| pip config set global.extra-index-url "${PIP_INDEX}" && \ | ||
| python -m pip install --upgrade pip | ||
|
|
||
| # Install sglang-0.4.6.post5 and torch-memory-saver | ||
| RUN pip uninstall -y cuda-python && pip install "sglang[all]==0.4.6.post5" --no-cache-dir --find-links https://flashinfer.ai/whl/cu124/torch2.6/flashinfer-python && pip install torch-memory-saver --no-cache-dir | ||
| # Install sglang-0.4.10.post2 and torch-memory-saver | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think we can deprecate this docker file, and please create a new dockerfile under this folder: cc @ocss884 @ETOgaosion for awareness
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think that we can keep sglang community docker file here, since your update progress can be faster than the official image. Or we rename it to @PopSoda2002 This PR can only modify this for faster merging. |
||
| RUN pip uninstall -y cuda-python && pip install "sglang[all]==0.4.10.post2" --no-cache-dir --find-links https://flashinfer.ai/whl/cu124/torch2.7/flashinfer-python && pip install torch-memory-saver --no-cache-dir | ||
|
|
||
| # Install torch-2.6.0 | ||
| RUN pip install --no-cache-dir torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 tensordict torchdata \ | ||
| # Install torch-2.7.1 | ||
| RUN pip install --no-cache-dir torch==2.7.1 torchvision==0.22.1 torchaudio==2.7.1 tensordict torchdata \ | ||
| transformers>=4.49.0 accelerate datasets peft hf_transfer \ | ||
| ray[default] codetiming hydra-core pandas pyarrow>=15.0.0 pylatexenc qwen-vl-utils wandb liger-kernel \ | ||
| pytest pre-commit py-spy pyext | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@PopSoda2002 Could you skip changing these CI for now? Currently might have some conflicts and due to our security issue we shall use our own image for CI.
If this PR only has relation with SGLang's community image, it's faster to be merged, and the official support will come afterwards.
This way can be a norm for future sglang's upgrade, community first and official support later soon following the way.