Repository navigation
verl abstract remote backend - #7
Merged
sfc-gh-kganesan merged 4 commits intoJun 26, 2026
Merged
sfc-gh-kganesan merged 4 commits into
sfc-gh-kganesan merged 4 commits into
Conversation
sfc-gh-truwase
commented
Jun 25, 2026
sfc-gh-truwase
commented
Collaborator
- vllm sleep/wake
- arctic yaml schema
…Arctic Platform (verl-project#39) Ported from recipe/rl-correctness d826d5ab (Mert Hidayetoglu) + the follow-on rename 6178d5d9 (inference -> rollout). Upstream touches verl/trainer/ppo/arctic_rl_client.py + verl/trainer/config/ppo_trainer.yaml (arctic_rl.rollout block); on this PR the equivalents are verl/workers/remote_client/arctic_rl.py + verl/trainer/config/remote_backend/arctic.yaml. Add a `rollout` block (zorro_inference.enable / speculative_decoding.model) to the per-backend yaml and pass it straight through to the Arctic server as `arctic_inference_config`. Newshape adaptation: rather than reproducing the historical verl-project#39 {use_fca, spec_model} translation, the adapter forwards the raw rollout sub-config (OmegaConf.to_container) to match the current arctic_platform.rl parse_arctic_inference_rollout contract, which keys on zorro_inference.enable / speculative_decoding.model. An all-disabled block is treated as "Arctic inference off" server-side. Co-authored-by: Cursor <cursoragent@cursor.com>
Ported from recipe/rl-correctness c7ed9201 (Olatunji Ruwase). Upstream
touches verl/trainer/ppo/arctic_rl_client.py + verl/trainer/config/ppo_trainer.yaml
(arctic_rl block); on this PR the equivalents are
verl/workers/remote_client/arctic_rl.py + verl/trainer/config/remote_backend/arctic.yaml.
Group the two flat flags (cuda_ipc_weight_sync / low_memory_weight_sync) under a
single `weight_sync` block (cuda_ipc / low_memory). Adapter reads
self._backend_config.weight_sync.{cuda_ipc,low_memory}.
Co-authored-by: Cursor <cursoragent@cursor.com>
Ported from recipe/rl-correctness 8e7e747f (Olatunji Ruwase). Drop the per-call `shift` plumbing through no_padding_2_padding and move the zorro response-alignment correction into the forwarder worker. - verl/workers/utils/padding.py: remove the `shift` arg; restore the single legacy "predict-next" slice (shift=-1) for all callers. - verl/trainer/ppo/ray_trainer.py: remove `_model_output_shift()` and its call sites (same path as upstream; pure revert). - newshape: upstream applied the response-aligned->predict-next shift in verl/workers/arctic_workers.py::compute_any_log_prob (which doesn't exist on this PR). The equivalent njt construction lives in the Arctic forwarder worker, so the shift helper moves to verl/remote_backend/worker_utils.py and is applied in ArcticRLActorRolloutRefWorker._run_log_prob when remote_backend.zorro_train.enable is set. Co-authored-by: Cursor <cursoragent@cursor.com>
sfc-gh-sbekman
approved these changes
Jun 25, 2026
sfc-gh-kganesan
merged commit Jun 26, 2026
39c911b
into
karthik/rl-correctness-deltas-on-share-v0.7.1
1 check passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.