Skip to content

revert: Discard weight when finish generation in the main loop (#2495) - #2658

Merged
terrykong merged 1 commit into
mainfrom
terryk/rl-781-revert-vllm-weight-discard-on-sleep
Jun 1, 2026
Merged

revert: Discard weight when finish generation in the main loop (#2495)#2658
terrykong merged 1 commit into
mainfrom
terryk/rl-781-revert-vllm-weight-discard-on-sleep

Conversation

@terrykong

@terrykong terrykong commented Jun 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

What does this PR do?

Reverts #2495 which introduced vLLM sleep level 2 (weight discarding) for colocated inference. The optimization caused silent failures with DAPO and speculative decoding (#2646). Reverting until the feature is fully tested against these cases.

Changes

  • grpo.py: Remove discard_weights=colocated_inference from finish_generation() call and remove POLICY_GENERATION_STALE = True flag
  • vllm_generation.py: Remove kwargs = {} and **kwargs passthrough in finish_generation
  • vllm_worker.py: Remove discard_weights param from sleep(), revert to level=1
  • vllm_worker_async.py: Remove discard_weights param from sleep_async(), revert to level=1
  • test_grpo.py: Remove discard_weights=True assertion
  • test_vllm_generation.py: Remove three test functions for discard weight routing/levels

Issues

Related to #2646

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally?
  • Did you add or update any necessary documentation?

@terrykong
revert: Discard weight when finish generation in the main loop (#2495)
fd42793 
This reverts commit 2a2e7c1 which introduced vLLM sleep level 2
(weight discarding) for colocated inference. The optimization caused
silent failures with DAPO and speculative decoding (issue #2646).
Reverting until the feature is fully tested against these cases.

Closes #2646

Signed-off-by: Terry Kong <terryk@nvidia.com>
@terrykong
terrykong requested review from a team as code owners June 1, 2026 22:32
@copy-pr-bot

copy-pr-bot Bot commented Jun 1, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@terrykong terrykong added the CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version) label Jun 1, 2026
@terrykong

terrykong commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator Author

/ok to test fd42793

@terrykong
terrykong enabled auto-merge (squash) June 1, 2026 22:40
guyueh1
guyueh1 approved these changes Jun 1, 2026
View reviewed changes

@guyueh1 guyueh1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no problem with the revert

@terrykong
terrykong merged commit e94d33c into main Jun 1, 2026
77 checks passed
@terrykong
terrykong deleted the terryk/rl-781-revert-vllm-weight-discard-on-sleep branch June 1, 2026 23:39
pengdurice pushed a commit to pengdurice/RL that referenced this pull request Jun 12, 2026
@terrykong @pengdurice
revert: Discard weight when finish generation in the main loop (NVIDI…
88005d5 
…A-NeMo#2495) (NVIDIA-NeMo#2658)

Signed-off-by: Terry Kong <terryk@nvidia.com>
@Alvorecer721 Alvorecer721 mentioned this pull request Aug 11, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@guyueh1 guyueh1 guyueh1 approved these changes

Assignees

No one assigned

Labels

CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@terrykong @guyueh1