[Core][Env Cache] Enable environment variable cache for Worker as well#27632
[Core][Env Cache] Enable environment variable cache for Worker as well#27632Jialin wants to merge 3 commits intovllm-project:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request enables environment variable caching within the Worker process by calling enable_envs_cache() at the end of the init_device method. This change aligns with a performance optimization already implemented in other components like EngineCore and Executor. The implementation is straightforward, correctly placed in the worker's initialization lifecycle, and should effectively reduce overhead from environment variable lookups in worker processes. The change appears correct and safe.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Head branch was pushed to by a user without write access
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
|
#27896 should be a better replacement of this PR, so close it for now. |
Purpose
Enable environment variable caching in Worker at the end of model_load.
This should be helpful for use cases who use ray of torchrun to spawn processes.
We also avoid duplicated functools.cache wrappings and enhanced unit tests in this PR.
Context
We firstly introduced and enabled environment variable caching in #26146 in EngineCore and Executor, this PR is a followup PR suggested by @youkaichao (#26146 (review))
Test Plan / Test Result
CI
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.