Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[hardware][rocm] allow rocm to override default env var #7926

Merged
merged 4 commits into from
Aug 28, 2024
Merged

[hardware][rocm] allow rocm to override default env var #7926

merged 4 commits into from
Aug 28, 2024

Conversation

youkaichao
Copy link
Member

fixes #7791

@github-actions GitHub Actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which consists a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of default ones by unblocking the steps in your fast-check build on Buildkite UI.

Once the PR is approved and ready to go, please make sure to run full CI as it is required to merge (or just use auto-merge).

To run full CI, you can do one of these:

  • Comment /ready on the PR
  • Add ready label to the PR
  • Enable auto-merge.

🚀

@youkaichao youkaichao mentioned this pull request Aug 27, 2024
Closed
@hongxiayang
Copy link
Collaborator

Thanks for the fix. @youkaichao

clintg6
clintg6 suggested changes Aug 28, 2024
View reviewed changes
Copy link

@clintg6 clintg6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not work. The if statement will never execute here because it is initially None not fork. All that's needed is

os.environ["VLLM_WORKER_MULTIPROC_METHOD"] = "spawn"
logger.warning("VLLM_WORKER_MULTIPROC_METHOD is overridden to `spawn`.")

@youkaichao
Copy link
Member Author

how about now?

clintg6
clintg6 approved these changes Aug 28, 2024
View reviewed changes
Copy link

@clintg6 clintg6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested successfully runs and handles inference requests. Fixes issue in #7791.

@youkaichao youkaichao merged commit bc6e42a into vllm-project:main Aug 28, 2024
28 checks passed
@youkaichao youkaichao deleted the rocm_override branch August 28, 2024 02:50
kushanam pushed a commit to kushanam/vllm that referenced this pull request Aug 28, 2024
@youkaichao
[hardware][rocm] allow rocm to override default env var (vllm-project…
5100309 
…#7926)
hbikki pushed a commit to hbikki/vllm that referenced this pull request Aug 28, 2024
@youkaichao
[hardware][rocm] allow rocm to override default env var (vllm-project…
ab4bb24 
…#7926)
kushanam pushed a commit to kushanam/vllm that referenced this pull request Aug 28, 2024
@youkaichao
[hardware][rocm] allow rocm to override default env var (vllm-project…
05f2b20 
…#7926)
triple-Mu pushed a commit to triple-Mu/vllm_official that referenced this pull request Sep 4, 2024
@youkaichao @triple-Mu
[hardware][rocm] allow rocm to override default env var (vllm-project…
a01454e 
…#7926)
Jeffwan pushed a commit to aibrix/vllm that referenced this pull request Sep 19, 2024
@youkaichao @Jeffwan
[hardware][rocm] allow rocm to override default env var (vllm-project…
37c3ed1 
…#7926)
Alvant pushed a commit to compressa-ai/vllm that referenced this pull request Oct 26, 2024
@youkaichao @Alvant
[hardware][rocm] allow rocm to override default env var (vllm-project…
520e576 
…#7926)

Signed-off-by: Alvant <[email protected]>
KuntaiDu pushed a commit to KuntaiDu/vllm that referenced this pull request Nov 20, 2024
@youkaichao
[hardware][rocm] allow rocm to override default env var (vllm-project…
4d6d5ad 
…#7926)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@clintg6 clintg6 clintg6 approved these changes

Assignees
No one assigned
Labels
rocm
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Bug]: Critical distributed executor bug
4 participants
@youkaichao @hongxiayang @clintg6 @WoosukKwon