Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 2 additions & 6 deletions docs/workers/fsdp_workers.rst
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@
PyTorch FSDP Backend
======================

Last updated: 02/12/2025.
Last updated: 12/01/2025.

We support PyTorch FSDP Backend by implementing various workers for
actor, critic, reference, rollout and reward models. We also implement
the ``FSDPVLLMShardingManager`` that reshard weight between FSDP and
vLLM in `fsdp_vllm.py <https://github.com/volcengine/verl/blob/main/verl/workers/sharding_manager/fsdp_vllm.py>`_.
actor, critic, reference, rollout and reward models.

**Pros**

Expand Down Expand Up @@ -58,8 +56,6 @@ highlighted below:
2. ``vLLMRollout`` support generation with vLLM. We modify the vLLM
Engine and make it executed under SPMD to fit into our
``WorkerGroup`` design.
3. ``FSDPVLLMShardingManager`` a context manager to perform actual
resharding between actor and rollout.

See `source code <https://github.com/volcengine/verl/blob/main/verl/workers/fsdp_workers.py>`_. for more information.

Expand Down
9 changes: 1 addition & 8 deletions docs/workers/megatron_workers.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
Megatron-LM Backend
===================

Last updated: 06/24/2025.
Last updated: 12/01/2025.

We support Megatron Backend by implementing various workers for actor,
critic, reference, rollout and reward models. We also implement the
Expand Down Expand Up @@ -121,8 +121,6 @@ highlighted below:
2. ``vLLMRollout`` support generation with vLLM. We modify the vLLM
Engine and make it executed under SPMD to fit into our
``WorkerGroup`` design.
3. ``MegatronVLLMShardingManager`` a context manager to perform actual
resharding between actor and rollout.

See `source code <https://github.com/volcengine/verl/blob/main/verl/workers/megatron_workers.py#L63>`_ for more information.

Expand All @@ -143,11 +141,6 @@ See `source code <https://github.com/volcengine/verl/blob/main/verl/workers/mega
tokenizer=self.tokenizer,
model_hf_config=self.actor_model_config,
train_tp=mpu.get_tensor_model_parallel_world_size())
# perform weight resharding between actor and rollout
sharding_manager = MegatronVLLMShardingManager(module=self.hybrid_engine,
inference_engine=rollout.inference_engine,
model_config=self.actor_model_config,
layer_name_mapping=layer_name_mapping)
...

1. Generate sequence and recompute log prob
Expand Down
2 changes: 1 addition & 1 deletion verl/trainer/main_ppo.py
Original file line number Diff line number Diff line change
Expand Up @@ -248,7 +248,7 @@ def add_ref_policy_worker(self, config, ref_policy_cls):
from verl.trainer.ppo.ray_trainer import Role

# Ref policy has been fused into ActorRolloutRefWorker in new model engine,
# we don't need to add a separate ref policy worker goup.
# we don't need to add a separate ref policy worker group.
use_legacy_worker_impl = config.trainer.get("use_legacy_worker_impl", "auto")
if use_legacy_worker_impl == "disable":
return
Expand Down
Loading