Skip to content

Add large RAM GPU runner for vllm #1596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
h-vetinari merged 2 commits into from
Jul 29, 2025
Merged

Add large RAM GPU runner for vllm #1596

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
eed4c3a
Add large RAM GPU runner for `vllm`
shermansiu Jul 29, 2025
40566d7
Merge branch 'main' into feat/add_runner_for_vllm
h-vetinari Jul 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions requests/vllm.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
action: cirun
feedstocks:
- vllm
resources:
- cirun-openstack-gpu-2xlarge
Copy link
Copy Markdown
Member

@h-vetinari h-vetinari Jul 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine to start off like this, but if vllm can be compiled with CPU-agents (compiling CUDA doesn't need a GPU), then we should do that.

There'll be one more PR to https://github.com/conda-forge/.cirun, which is also where we can then add further resource policies in the future

Copy link
Copy Markdown
Member

@h-vetinari h-vetinari Jul 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

conda-forge/.cirun#109

Copy link
Copy Markdown
Contributor Author

@shermansiu shermansiu Jul 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for creating the cirun PR! Do we need any additional PRs to use the larger CPU runners like ci_2xlarge?

Copy link
Copy Markdown
Member

@h-vetinari h-vetinari Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[...] https://github.com/conda-forge/.cirun, which is also where we can then add further resource policies in the future

☝️

Check out the pull request I linked. It creates a policy for vllm and adds it to a list of policies that are enabled for cirun-openstack-gpu-2xlarge runners. To be able to use other (e.g. smaller and/or CPU-only) runners, you need to add the vllm policy to the respective list.

Copy link
Copy Markdown
Contributor Author

@shermansiu shermansiu Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, so as I understand it, subsequent PRs will be made directly to https://github.com/conda-forge/.cirun and not https://github.com/conda-forge/admin-requests? Because now, the CI policy has been created?

Copy link
Copy Markdown
Member

@h-vetinari h-vetinari Jul 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. Please don't make me repeat myself N times

# switch to true if you need it on pull-requests too
pull_request: true
# revoke access to the gpu runner
revoke: false
# Send an automatic PR to the feedstock to enable cirun defaults
send_pr: true
Loading