Skip to content

[Core][Env Cache] Enable environment variable cache for Worker as well#27632

Closed
Jialin wants to merge 3 commits intovllm-project:mainfrom
Jialin:env
Closed

[Core][Env Cache] Enable environment variable cache for Worker as well#27632
Jialin wants to merge 3 commits intovllm-project:mainfrom
Jialin:env

Conversation

@Jialin
Copy link
Copy Markdown
Collaborator

@Jialin Jialin commented Oct 28, 2025

Purpose

Enable environment variable caching in Worker at the end of model_load.

This should be helpful for use cases who use ray of torchrun to spawn processes.

We also avoid duplicated functools.cache wrappings and enhanced unit tests in this PR.

Context
We firstly introduced and enabled environment variable caching in #26146 in EngineCore and Executor, this PR is a followup PR suggested by @youkaichao (#26146 (review))

Test Plan / Test Result

CI


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

@Jialin Jialin requested review from 22quinn and youkaichao October 28, 2025 06:03
@mergify mergify bot added the v1 label Oct 28, 2025
@Jialin Jialin requested a review from mgoin October 28, 2025 06:03
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enables environment variable caching within the Worker process by calling enable_envs_cache() at the end of the init_device method. This change aligns with a performance optimization already implemented in other components like EngineCore and Executor. The implementation is straightforward, correctly placed in the worker's initialization lifecycle, and should effectively reduce overhead from environment variable lookups in worker processes. The change appears correct and safe.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copy link
Copy Markdown
Collaborator

@22quinn 22quinn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@22quinn 22quinn enabled auto-merge (squash) October 28, 2025 17:49
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 28, 2025
auto-merge was automatically disabled October 29, 2025 18:35

Head branch was pushed to by a user without write access

Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
@Jialin
Copy link
Copy Markdown
Collaborator Author

Jialin commented Nov 2, 2025

#27896 should be a better replacement of this PR, so close it for now.

@Jialin Jialin closed this Nov 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed v1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants