[torch.compile] caching of config fields should be opt-out by default by vnadathur · Pull Request #26468 · vllm-project/vllm

vnadathur · 2025-10-09T06:45:03Z

Referring: #23107

Implements opt-out system of hashing, suggested by #16501

def compute_hash(self):
   factors = list(self.__dict__.values())
   factors.remove("enforce_eager")
   factors.remove("tokenizer_config")
   ...

github-actions · 2025-10-09T06:53:47Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

ProExpertProg

This is much cleaner than I last remember! Nice work

tests/config/test_config_utils.py

vllm/compilation/backends.py

vllm/envs.py

vllm/config/parallel.py

vllm/config/model.py

vllm/compilation/backends.py

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

vllm/config/parallel.py

vllm/config/compilation.py

reposting this from old pr. implements an opt-out system of hashing for env vars. Signed-off-by: vnadathur <glvikramn@gmail.com>

Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com> Signed-off-by: vnadathur <glvikramn@gmail.com> Co-Authored-By: Srreyansh Sethi <107075589+worldexplored@users.noreply.github.com> Co-Authored-By: vnadathur <236933696+vnadathur@users.noreply.github.com>

- Concated the lazily. - refactored comments. - updated ignored factors - logged erors - persists if files don't already exist Srreyansh Sethi <srreyansh.sethi@gmail.com> Co-Authored-By: vnadathur <236933696+vnadathur@users.noreply.github.com> Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>

Added lazy logging util, called in backends.py Signed-off-by: vnadathur <glvikramn@gmail.com>

Fixed config hashing and dataclass hashing Signed-off-by: Srreyansh Sethi <srreyansh.sethi@gmail.com> Co-Authored-By: vnadathur <236933696+vnadathur@users.noreply.github.com> Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>

Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>

Signed-off-by: vnadathur <glvikramn@gmail.com>

Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com> Co-Authored-By: vnadathur <236933696+vnadathur@users.noreply.github.com>

Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>

tests/config/test_config_utils.py

zou3519

LGTM, just some minor nits

Signed-off-by: vnadathur <glvikramn@gmail.com> Co-Authored-By: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>

Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com> Co-Authored-By: vnadathur <236933696+vnadathur@users.noreply.github.com>

Signed-off-by: vnadathur <glvikramn@gmail.com>

vllm/config/compilation.py

hmellor · 2025-11-21T10:16:40Z

vllm/compilation/backends.py

+            factors = [env_hash, config_hash, code_hash, compiler_hash]
+            # Use SHA-256 for cache key hashing to be consistent across
+            # compute_hash functions. Truncate for a short cache dir name.
+            hash_key = hashlib.sha256(str(factors).encode()).hexdigest()[:10]


Should we have used the hash_factors method here?

will add to this pr: #29117

…vllm-project#26468) Signed-off-by: vnadathur <glvikramn@gmail.com> Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com> Signed-off-by: Srreyansh Sethi <srreyansh.sethi@gmail.com> Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com> Co-authored-by: WorldExplored <srreyansh.sethi@gmail.com> Co-authored-by: Srreyansh Sethi <107075589+worldexplored@users.noreply.github.com> Co-authored-by: vnadathur <236933696+vnadathur@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

ProExpertProg linked an issue Oct 9, 2025 that may be closed by this pull request

[RFC]: vLLM x torch.compile caching should be opt-out by default #16501

Closed

1 task

ProExpertProg reviewed Oct 9, 2025

View reviewed changes

vllm/compilation/backends.py Outdated Show resolved Hide resolved

WorldExplored force-pushed the envhashing branch from 1a1a820 to 6dfdf90 Compare October 9, 2025 21:47

ProExpertProg reviewed Oct 9, 2025

View reviewed changes

vllm/compilation/backends.py Outdated Show resolved Hide resolved

vnadathur marked this pull request as ready for review October 10, 2025 07:18

vnadathur requested review from WoosukKwon, heheda12345, hmellor, houseroad, mgoin, robertgshaw2-redhat, simon-mo, tlrmchlsmth, yewentao256, youkaichao and zou3519 as code owners October 10, 2025 07:19

chatgpt-codex-connector bot reviewed Oct 10, 2025

View reviewed changes

vllm/config/parallel.py Show resolved Hide resolved

vllm/config/compilation.py Show resolved Hide resolved

vnadathur and others added 5 commits October 10, 2025 13:04

Opt-out hashing for torch.compile cache keys (ModelConfig + envs)

97a6194

reposting this from old pr. implements an opt-out system of hashing for env vars. Signed-off-by: vnadathur <glvikramn@gmail.com>

added lazy logging to logging utils

d6dccaf

Added lazy logging util, called in backends.py Signed-off-by: vnadathur <glvikramn@gmail.com>

Addresed Codex Problems

1359256

Fixed config hashing and dataclass hashing Signed-off-by: Srreyansh Sethi <srreyansh.sethi@gmail.com> Co-Authored-By: vnadathur <236933696+vnadathur@users.noreply.github.com> Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>

WorldExplored force-pushed the envhashing branch from ea8cf77 to 1359256 Compare October 10, 2025 17:04

WorldExplored and others added 5 commits October 10, 2025 16:06

Merge branch 'main' into envhashing

3a3af9b

Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>

solve merge conflict

1485803

Signed-off-by: vnadathur <glvikramn@gmail.com>

fixed ignore list in model.py

e4db7f4

Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com> Co-Authored-By: vnadathur <236933696+vnadathur@users.noreply.github.com>

Merge branch 'main' into envhashing

0648852

Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>

Merge branch 'main' into envhashing

226a4ae

Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>

zou3519 reviewed Nov 18, 2025

View reviewed changes

tests/config/test_config_utils.py Show resolved Hide resolved

zou3519 approved these changes Nov 18, 2025

View reviewed changes

vnadathur and others added 2 commits November 18, 2025 14:22

nits and feedback

e8e3b2f

Signed-off-by: vnadathur <glvikramn@gmail.com> Co-Authored-By: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>

Merge branch 'main' into envhashing

4fed317

Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com> Co-Authored-By: vnadathur <236933696+vnadathur@users.noreply.github.com>

ProExpertProg enabled auto-merge (squash) November 18, 2025 22:41

ProExpertProg approved these changes Nov 18, 2025

View reviewed changes

fixing CI

a14319b

Signed-off-by: vnadathur <glvikramn@gmail.com>

auto-merge was automatically disabled November 19, 2025 00:24
Head branch was pushed to by a user without write access

ProExpertProg approved these changes Nov 19, 2025

View reviewed changes

ProExpertProg enabled auto-merge (squash) November 19, 2025 01:19

vllm-bot merged commit 1ffe934 into vllm-project:main Nov 19, 2025
126 of 131 checks passed

github-project-automation bot moved this from In review to Done in torch.compile integration Nov 19, 2025

vnadathur deleted the envhashing branch November 19, 2025 21:45

vnadathur mentioned this pull request Nov 20, 2025

[torch.compile] split shared compilation hash helpers #29057

Closed

hmellor reviewed Nov 20, 2025

View reviewed changes

vllm/config/compilation.py Show resolved Hide resolved

This was referenced Nov 20, 2025

[torch.compile] add condition to normalize_value #29096

Closed

[torch.compile] refactor config hashing to compile_factors and unify factor collection #29117

Open

hmellor reviewed Nov 21, 2025

View reviewed changes

hjjq mentioned this pull request Nov 21, 2025

[Spec Decode] Add support for EAGLE3 heads that do not use_aux_hidden_states #27688

Merged

hmellor mentioned this pull request Nov 24, 2025

[Misc] Add backup hash algorithm for FIPS constrained environments #28795

Merged

5 tasks

mickaelseznec mentioned this pull request Dec 1, 2025

Add Mistral Large 3 and Ministral 3 #29757

Merged

5 tasks

Uh oh!

Conversation

vnadathur commented Oct 9, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Oct 9, 2025

Uh oh!

ProExpertProg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zou3519 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hmellor Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

vnadathur Nov 21, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

vnadathur commented Oct 9, 2025 •

edited by github-actions bot

Loading