Skip to content

[Offload] Remove Accelerate#530

Merged
kylesayrs merged 12 commits intomainfrom
kylesayrs/remove-accelerate
Jan 27, 2026
Merged

[Offload] Remove Accelerate#530
kylesayrs merged 12 commits intomainfrom
kylesayrs/remove-accelerate

Conversation

@kylesayrs
Copy link
Copy Markdown
Collaborator

@kylesayrs kylesayrs commented Dec 18, 2025 •
edited
Loading

@kylesayrs kylesayrs force-pushed the kylesayrs/remove-accelerate branch 3 times, most recently from 42dc5c7 to 355dd6c Compare December 18, 2025 04:33
@kylesayrs kylesayrs changed the base branch from main to kylesayrs/torch_offloader December 18, 2025 04:34
@kylesayrs kylesayrs force-pushed the kylesayrs/remove-accelerate branch 4 times, most recently from e30978b to 3511775 Compare December 18, 2025 05:58
@kylesayrs kylesayrs mentioned this pull request Dec 18, 2025
Merged
@kylesayrs kylesayrs force-pushed the kylesayrs/remove-accelerate branch 5 times, most recently from 89fb74e to 0a7f261 Compare December 19, 2025 07:09
@kylesayrs kylesayrs changed the title [TorchOffloader] Remove Accelerate [Offload] Remove Accelerate Dec 23, 2025
@kylesayrs kylesayrs mentioned this pull request Dec 31, 2025
Closed
17 tasks
@kylesayrs kylesayrs force-pushed the kylesayrs/remove-accelerate branch 4 times, most recently from 81159f7 to 1748ba5 Compare January 4, 2026 21:58
@dsikka dsikka mentioned this pull request Jan 12, 2026
Closed
5 tasks
@kylesayrs kylesayrs force-pushed the kylesayrs/remove-accelerate branch from 1748ba5 to 3c8e73b Compare January 13, 2026 17:21
@kylesayrs kylesayrs marked this pull request as ready for review January 14, 2026 06:17
@dsikka
Copy link
Copy Markdown
Collaborator

dsikka commented Jan 19, 2026

@Mergifyio refresh

@mergify
Copy link
Copy Markdown

mergify Bot commented Jan 19, 2026

refresh

✅ Pull request refreshed

Base automatically changed from kylesayrs/torch_offloader to main January 21, 2026 20:43
@mergify
Copy link
Copy Markdown

mergify Bot commented Jan 21, 2026

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @kylesayrs.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Jan 21, 2026
@kylesayrs kylesayrs force-pushed the kylesayrs/remove-accelerate branch from 3c8e73b to c57ad93 Compare January 22, 2026 00:43
@mergify mergify Bot removed the needs-rebase label Jan 22, 2026
@mergify
Copy link
Copy Markdown

mergify Bot commented Jan 27, 2026

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @kylesayrs.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label Jan 27, 2026
@kylesayrs
remove accelerate (mostly)
5225268 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs
fix rebase
fd57237 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs
remove accelerate
0bbc5c2 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs
fix transform tests
32338be 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs
fix rebase
10c84c4 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs
fix bug in align_module_device
93113ce 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs
add onloaded_values
fb68184 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs
disallow none default
ef6346b 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs
fix args
0bba8ac 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs
revert weakref dict
74aaf94 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs kylesayrs dismissed stale reviews from HDCharles and brian-dellabetta via 74aaf94 January 27, 2026 18:57
@kylesayrs kylesayrs force-pushed the kylesayrs/remove-accelerate branch from d6438b5 to 74aaf94 Compare January 27, 2026 18:57
@mergify mergify Bot removed the needs-rebase label Jan 27, 2026
HDCharles
HDCharles previously approved these changes Jan 27, 2026
View reviewed changes
@kylesayrs
add back accelerate dep
ea21652 
Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
@kylesayrs kylesayrs dismissed stale reviews from brian-dellabetta and HDCharles via ea21652 January 27, 2026 20:33
@kylesayrs kylesayrs enabled auto-merge (squash) January 27, 2026 22:05
@kylesayrs kylesayrs merged commit dafa0d6 into main Jan 27, 2026
4 checks passed
@kylesayrs kylesayrs deleted the kylesayrs/remove-accelerate branch January 27, 2026 22:09
kylesayrs added a commit to vllm-project/llm-compressor that referenced this pull request Jan 28, 2026
@kylesayrs
[Offload] Switch to torch offloader from accelerate (#2148)
37de437 
## Co-requisites ##
* Compressed Tensors: [[TorchOffloader] Remove
Accelerate](vllm-project/compressed-tensors#530)

## Changes ##
* Perform tracing in the `disable_onloading` context (any tensors that
are mentioned will be referenced as meta tensors to avoid excess
onloading)
* Dispatch model after tracing (not really necessary)
* `dispatch_for_sequential` is now an alias for `offload_model`
* `dispatch_for_generation` is now an alias for `dispatch_model`
* Add a `get_main_device` utility to help with centralized device
getting
* `untie_word_embeddings` is now simpler
* Fix fusing test by initializing tensors without gradients

## Testing ##
*
https://github.com/neuralmagic/llm-compressor-testing/actions/runs/21348973738
* Optional TODO: add more tracing tests

---------

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
cajeonrh pushed a commit to cajeonrh/llm-compressor that referenced this pull request Feb 10, 2026
@kylesayrs @cajeonrh
[Offload] Switch to torch offloader from accelerate (vllm-project#2148)
795cb6b 
## Co-requisites ##
* Compressed Tensors: [[TorchOffloader] Remove
Accelerate](vllm-project/compressed-tensors#530)

## Changes ##
* Perform tracing in the `disable_onloading` context (any tensors that
are mentioned will be referenced as meta tensors to avoid excess
onloading)
* Dispatch model after tracing (not really necessary)
* `dispatch_for_sequential` is now an alias for `offload_model`
* `dispatch_for_generation` is now an alias for `dispatch_model`
* Add a `get_main_device` utility to help with centralized device
getting
* `untie_word_embeddings` is now simpler
* Fix fusing test by initializing tensors without gradients

## Testing ##
*
https://github.com/neuralmagic/llm-compressor-testing/actions/runs/21348973738
* Optional TODO: add more tracing tests

---------

Signed-off-by: Kyle Sayers <kylesayrs@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@brian-dellabetta brian-dellabetta brian-dellabetta approved these changes

@HDCharles HDCharles HDCharles approved these changes

@dsikka dsikka Awaiting requested review from dsikka

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@kylesayrs @dsikka @brian-dellabetta @HDCharles