[Perf] Enable environment cache in EngineCore to enable the feature for UniProcExecutor as well#29289
[Perf] Enable environment cache in EngineCore to enable the feature for UniProcExecutor as well#29289mgoin merged 4 commits intovllm-project:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request refactors the environment variable caching by moving the enable_envs_cache() call from EngineCoreProc to the base class EngineCore. This correctly extends the performance optimization to UniProcExecutor and other non-distributed execution paths. I have reviewed the change and its impact on different executors and initialization sequences. The new placement of the call is safe, occurring after environment-dependent setup and before performance-critical code paths. The change is sound and provides a good performance improvement.
|
Also CC @mgoin |
80a49b3 to
1b9858f
Compare
|
Failed to repro CI test failure locally, so rebase against to confirm if it's a transient issue. |
|
The CI failure is due to multiple unit tests are running in the same process, so the later test is misusing the environment variable To fix this, introduce disable_envs_cache and invoke in cleanup_dist_env_and_memory which should fully isolate environment variables across 2 tests. |
…ons to UniProcExecutor Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
…or UniProcExecutor as well (vllm-project#29289) Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com> Signed-off-by: Ubuntu <mjtaheri68@gmail.com>
…or UniProcExecutor as well (vllm-project#29289) Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com> Signed-off-by: dsuhinin <suhinin.dmitriy@gmail.com>
Purpose
Expand the coverage of the feature. Similar to #26146, we found the environment variable is beneficial to output processing as well.
Test Plan & Test Result
CI Signals
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.