[torch.compile] refactor config hashing to compile_factors and unify factor collection#29117
Open
vnadathur wants to merge 88 commits intovllm-project:mainfrom
Open
[torch.compile] refactor config hashing to compile_factors and unify factor collection#29117vnadathur wants to merge 88 commits intovllm-project:mainfrom
vnadathur wants to merge 88 commits intovllm-project:mainfrom
Conversation
Updated all config classes to support an optional 'return_factors' argument in their compute_hash methods, allowing retrieval of hash factors instead of just the hash string. Signed-off-by: vnadathur <glvikramn@gmail.com> Co-Authored-By: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com> Co-Authored-By: vnadathur <236933696+vnadathur@users.noreply.github.com>
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com> Co-Authored-By: vnadathur <236933696+vnadathur@users.noreply.github.com>
Signed-off-by: vnadathur <glvikramn@gmail.com> Co-Authored-By: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
allows us to stop handling passconfig specially. Signed-off-by: vnadathur <glvikramn@gmail.com> Co-Authored-By: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
vnadathur
requested review from
ProExpertProg,
WoosukKwon,
heheda12345,
hmellor,
houseroad,
markmc,
mgoin,
noooop,
robertgshaw2-redhat,
tlrmchlsmth,
yewentao256,
youkaichao and
zou3519
as code owners
Signed-off-by: vnadathur <glvikramn@gmail.com>
|
This pull request has merge conflicts that must be resolved before it can be |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request is a large-scale refactoring that renames compute_hash to compile_factors across many configuration classes, aiming for a more uniform and consistent hashing mechanism. The changes introduce a standardized way to handle nested configurations and allow for either returning the hash factors or the final hash string. Overall, the refactoring improves code consistency. However, I've identified a critical issue in vllm/envs.py that will cause an infinite recursion and needs to be addressed.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Signed-off-by: vnadathur <glvikramn@gmail.com>
Signed-off-by: vnadathur <glvikramn@gmail.com>
vnadathur
changed the title
compute_hash & fixing small bugs
Signed-off-by: vnadathur <glvikramn@gmail.com>
Signed-off-by: vnadathur <glvikramn@gmail.com>
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>
Contributor
New Generated Cache |
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
|
Hi @vnadathur, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>
|
Hi @vnadathur, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
|
This pull request has merge conflicts that must be resolved before it can be |
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Signed-off-by: Srreyansh Sethi <107075589+WorldExplored@users.noreply.github.com>
Contributor
|
Vllm serve: |
Signed-off-by: Vikram Nadathur <glvikramn@gmail.com>
|
Hi @vnadathur, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
Signed-off-by: WorldExplored <srreyansh.sethi@gmail.com>
|
Hi @vnadathur, the pre-commit checks have failed. Please run: uv pip install pre-commit
pre-commit install
pre-commit run --all-filesThen, commit the changes and push to your branch. For future commits, Tip Is
|
|
This pull request has merge conflicts that must be resolved before it can be |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
After this PR: #26468 there were a couple follow-ups to add.
In this PR, the following are added/changed:
normalize_value&hash_factorsacross configs with a compute_hash functioncompute_hashtocompile_factorsthis is the reason for so many files changedcompute_hashisn’t the right name because real computation of the hash is actually done byutils.hash_factors.Additions:
normalize valueinget_compile_factors, it checks if .compile_factors exists on the subobject - based on Luka's suggestionPassConfigspecially as in the case for compilation.py_compute_code_hashandcompilation_config_hash_factorscache_key_factors.json
cc @ProExpertProg @hmellor @zou3519
Note
Unifies torch.compile cache key generation and factor collection.
CompileFactorsandget_compile_factors, replacingcompute_hashwithcompile_factorsacross config classes and compiler adaptors;VllmConfig.compile_factorsaggregates nested config factorsnormalize_valueandhash_factors, including nestedcompile_factorsfor subobjectscompute_env_and_config_hashesandget_code_factors; cache keys now combineenv/config/compiler/codefactors and persist tocache_key_factors.jsonget_compile_factorsand validates multimodal backend factor changes; minor enum import path fixWritten by Cursor Bugbot for commit 7fbc6a6. This will update automatically on new commits. Configure here.
Note
Unifies torch.compile cache key derivation and factor collection across the codebase.
CompileFactorsandget_compile_factors; migrates config classes and compiler adaptors fromcompute_hashtocompile_factors(opt-out factor collection, nested configs supported)normalize_valueandhash_factors;VllmConfig.compile_factorsaggregates nested config factorscompute_env_and_config_hashesandget_code_factors; backends now derive cache dirs from hashed{env, config, compiler, code}factors and persist them incache_key_factors.jsonget_compile_factorsand validate factor changes; fixes enum import forAttentionBackendEnum{}where configs don’t affect graphs; usePathfor traced files; consistent DP hash validation)Written by Cursor Bugbot for commit 929d2f7. This will update automatically on new commits. Configure here.
Note
Cursor Bugbot is generating a summary for commit a0b60a4. Configure here.
Note
Unifies torch.compile cache key derivation and factor collection across the codebase.
CompileFactorsandget_compile_factors; migrates configs and compiler adaptors fromcompute_hashtocompile_factors(opt‑out, nested configs supported)normalize_valueandhash_factors;VllmConfig.compile_factorsaggregates nested factorscompute_env_and_config_hashesandget_code_factors; backends now hash{env, config, compiler, code}to derive cache dirs and persistcache_key_factors.jsonPathusage, typing tweaks, pass manager UUID includes post‑cleanup)Written by Cursor Bugbot for commit a0b60a4. This will update automatically on new commits. Configure here.
Note
Standardizes torch.compile cache keys and factor collection across the codebase.
CompileFactorsandget_compile_factors; replacescompute_hashwithcompile_factorsin configs and compiler adaptors, and aggregates inVllmConfig.compile_factorscompute_env_and_config_hashesandget_code_factors; backends now build cache dirs from hashed{env, config, compiler, code}and persistcache_key_factors.jsonWritten by Cursor Bugbot for commit 8f3d1af. This will update automatically on new commits. Configure here.