Repository navigation
[trainer, remote_backend] refactor: make RemoteBackend adapters plugin-only; strip Arctic from verl core - #10
Merged
sfc-gh-kganesan merged 1 commit intoJul 15, 2026
Conversation
…n-only; strip Arctic from verl core Follow-up to the RemoteBackend abstraction (verl-project#6422). Turns the existing Arctic-aware verl core into a truly generic + plugin-only shape so adapters can live entirely out-of-tree, loaded via the existing VERL_USE_EXTERNAL_MODULES hook. Zero behavioural change for verl's built-in rollout backends (vllm / sglang / trtllm) and for non-remote PPO paths; the trainer registers the plugin adapter on first `fit()`-time lookup exactly the way it registered the (previously hardcoded) Arctic worker. ## Why Today, `verl/trainer/main_ppo.py` hardcodes: if backend_name == "arctic": from verl.remote_backend.workers.arctic_rl import ( ArcticRLActorRolloutRefWorker, ) from verl.workers.remote_client import arctic_rl # noqa: F401 actor_rollout_cls = ArcticRLActorRolloutRefWorker else: raise ValueError(f"Unknown trainer.remote_backend={backend_name!r}. Known: 'arctic'. ...") and `verl/workers/rollout/replica.py` eagerly does: def _load_arctic(): from verl.workers.rollout.remote_rollout.arctic_rollout.arctic_rollout import ArcticReplica return ArcticReplica RolloutReplicaRegistry.register("arctic", _load_arctic) Both branches pull Arctic-specific modules (and their transitive deps: DeepSpeed, arctic_training, arctic_inference, vLLM) into every verl process the moment a user selects `trainer.remote_backend=arctic`, even though verl core has no need for the code and out-of-tree adapters can't slot in without patching main_ppo. ## What lands 1. `RemoteBackendRegistry` grows two symmetrical worker-side classmethods (`base.py`, ~85 lines): * `register_worker(name, loader)` -- plugin calls this to register a lazy loader (a zero-arg callable) for the `ActorRollout(Ref)` forwarder worker class. * `get_worker(name)` -- `main_ppo` calls this to resolve the worker class, running the loader once and caching the result. 2. `main_ppo.py` drops the hardcoded `if backend_name == "arctic"` branch and instead does: worker_cls = RemoteBackendRegistry.get_worker(backend_name) if worker_cls is None: raise ValueError(...) # names the plugin contract in the error actor_rollout_cls = worker_cls Error message points the user at `RemoteBackendRegistry.register_worker(...)` and `VERL_USE_EXTERNAL_MODULES` so the wiring failure mode is self-explanatory. 3. `workers/rollout/replica.py` drops `_load_arctic` and its eager `RolloutReplicaRegistry.register("arctic", _load_arctic)`; comment in place directs readers to the plugin path. 4. `worker_utils.py` docstring updated to note that per-backend forwarder workers now ship in adapter packages, not verl core. 5. Deletes 10 Arctic-specific files from verl core (all functionality moves to the out-of-tree plugin under `arctic_platform/integrations/verl/`): ``` verl/remote_backend/workers/__init__.py verl/remote_backend/workers/arctic_rl/__init__.py verl/remote_backend/workers/arctic_rl/worker.py verl/workers/remote_client/__init__.py verl/workers/remote_client/arctic_rl.py verl/workers/rollout/remote_rollout/__init__.py verl/workers/rollout/remote_rollout/arctic_rollout/__init__.py verl/workers/rollout/remote_rollout/arctic_rollout/arctic_rollout.py verl/trainer/config/remote_backend/arctic.yaml examples/arctic_rl/run_gsm8k_grpo_arl_zorro_yes.sh ``` Net: +144 / -1682 across 15 files. ## Companion Arctic-Platform PR Paired plugin package landed as Arctic-Platform commit `89bd4f7` (feat/integrations-verl) -- `arctic_platform/integrations/verl/` subpackage with a `register.py` that: * decorator-registers `ArcticRLClientWrapper` (via `@RemoteBackendRegistry.register("arctic")` on the class), * calls `RemoteBackendRegistry.register_worker("arctic", ...)` with a lazy loader for its `ActorRollout(Ref)` forwarder, * calls `RolloutReplicaRegistry.register("arctic", ...)` with a lazy loader for its `RolloutReplica`. Everything except the backend-class decorator is lazy; a plain `import arctic_platform.integrations.verl.register` stays cheap and takes on no DeepSpeed / vLLM / arctic_inference cost unless the selected backend at `fit()` is `"arctic"`. ## Smoke test Full 4-step GSM8K GRPO run (Qwen3-1.7B, single H200) against a verl tree at this commit + Arctic-Platform's plugin package loaded via `VERL_USE_EXTERNAL_MODULES=arctic_platform.integrations.verl.register` + `hydra.searchpath=[file://.../integrations/verl/config]`: | step | MFU (actor) | throughput (tok/s) | step time (s) | |---:|---:|---:|---:| | 1 | 0.261 | 5164 | 48.4 | | 2 | 0.256 | 5860 | 42.4 | | 3 | 0.260 | 6283 | 39.0 | | 4 | 0.264 | 6478 | 38.4 | `val-core/openai/gsm8k/acc/mean@1=0.00152` at step 4 (expected: near-zero for a 4-step GRPO run from a stock baseline; the goal is exercising the eval path). Weight sync runs over CUDA-IPC and the receiver logs `[weight-sync names validated] context=cuda_ipc sender=310 expected=310` on every step. No tracebacks; clean vLLM engine shutdown. Co-authored-by: Cursor <cursoragent@cursor.com>
sfc-gh-truwase
approved these changes
Jul 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to the RemoteBackend abstraction (verl-project/verl#6422). Turns the Arctic-aware verl core into a plugin-only shape so adapters live entirely out-of-tree via the existing
VERL_USE_EXTERNAL_MODULEShook. Zero behavioural change for verl's built-in vllm / sglang / trtllm paths.Why
verl/trainer/main_ppo.pycurrently hardcodesif backend_name == "arctic": import ...andverl/workers/rollout/replica.pyeagerly registers_load_arctic. Both pull Arctic-specific modules and their transitive deps (DeepSpeed, arctic_training, arctic_inference, vLLM) into every verl process the moment a user selectsremote_backend=arctic, and there is no way for an out-of-tree adapter to slot in without patchingmain_ppo.What lands
RemoteBackendRegistry(base.py) gains two symmetrical classmethods:register_worker(name, loader)— plugin registers a lazy loader for theActorRollout(Ref)forwarder class.get_worker(name)—main_pporesolves the class, running the loader once and caching.main_ppo.pydrops the hardcodedif backend_name == "arctic"branch forRemoteBackendRegistry.get_worker(backend_name). Error message names the plugin contract so misconfiguration is self-explanatory.workers/rollout/replica.pydrops_load_arcticand its eagerregister("arctic", ...); comment points at the plugin path.worker_utils.pydocstring notes that per-backend forwarder workers now ship in adapter packages.Deletes 10 Arctic-specific files from verl core; functionality moves to the out-of-tree plugin under
arctic_platform/integrations/verl/.Net: +144 / −1682 across 15 files.
Companion Arctic-Platform PR
Paired plugin ships as Snowflake-AI-Research/Arctic-Platform#36 —
arctic_platform/integrations/verl/register.pyfires@RemoteBackendRegistry.register("arctic")on adapter import, then registers lazy loaders for the worker + replica.import registerstays cheap; DeepSpeed / vLLM / arctic_inference only load if the backend is selected atfit().Pre-commit
Full verl pre-commit runs clean:
ruff,ruff-format,mypy,check-license,check-docstrings,compileall,autogen-trainer-cfg.Smoke tests
Both Arctic-Platform Golden Runs pass end-to-end against this tree + the paired plugin, using the shipped recipe launchers unchanged.
Golden Run 1 — GSM8K (Qwen3-1.7B, 1×H200, 4 steps)
Weight sync validated every step (
[weight-sync names validated] context=cuda_ipc).Golden Run 2 — BIRD text-to-SQL (Qwen3-0.6B, 1×H200, 20 steps)
Training reward (
critic/rewards/mean, format-bonus floor 0.1):Reward 0.335 → 0.415 while response length drops 1116 → 920. Validation @ step 20 vs pre-plugin reference log:
val-core/bird/reward/mean@1val-aux/bird/execution_success/mean@1val-aux/bird/format_correct/mean@1Follow-ups (not in this PR)
main; the V1 trainer path doesn't exist onrelease/v0.7.1.