[Dev] Fix for rope when enabling THD + Dynamic-CP; and use the naming Dynamic-CP. - #3405
Merged
Merged
Conversation
yanring
marked this pull request as draft
February 24, 2026 15:04
Contributor
|
please provide the mirror PR |
xiaoyao0115
force-pushed
the
dynamic-cp-rename-and-fix
branch
from
March 5, 2026 08:06
fc91c8f to
6c7f81a
Compare
xiaoyao0115
force-pushed
the
dynamic-cp-rename-and-fix
branch
5 times, most recently
from
March 9, 2026 06:52
a5ca19d to
a94f922
Compare
Contributor
|
/ok to test a94f922 |
Contributor
Author
|
/ok to test cad019e |
Contributor
Author
|
/ok to test da32801 |
xiaoyao0115
force-pushed
the
dynamic-cp-rename-and-fix
branch
from
March 9, 2026 15:01
da32801 to
537a3ac
Compare
Contributor
Author
|
/ok to test 537a3ac |
xiaoyao0115
force-pushed
the
dynamic-cp-rename-and-fix
branch
from
March 10, 2026 03:30
537a3ac to
458e7af
Compare
Contributor
Author
|
/ok to test 458e7af |
kunlunl
approved these changes
Mar 10, 2026
yanring
changed the title
xiaoyao0115
force-pushed
the
dynamic-cp-rename-and-fix
branch
2 times, most recently
from
March 10, 2026 09:57
6fd9ef5 to
da5b9b7
Compare
Contributor
Author
|
/ok to test da5b9b7 |
xiaoyao0115
force-pushed
the
dynamic-cp-rename-and-fix
branch
from
March 10, 2026 13:32
da5b9b7 to
d1e1d63
Compare
Contributor
Author
|
/ok to test d1e1d63 |
…amic-cp Signed-off-by: xiaoyao0115 <1804647152@qq.com>
Signed-off-by: xiaoyao0115 <1804647152@qq.com>
Signed-off-by: xiaoyao0115 <1804647152@qq.com>
Signed-off-by: xiaoyao0115 <1804647152@qq.com>
xiaoyao0115
force-pushed
the
dynamic-cp-rename-and-fix
branch
from
March 11, 2026 05:18
d1e1d63 to
88733ab
Compare
Contributor
Author
|
/ok to test 88733ab |
yanring
approved these changes
Mar 11, 2026
Contributor
|
🔄 Merge queue validation started! You can track the progress here: https://github.com/NVIDIA/Megatron-LM/actions/runs/22941770675 |
wuxibin89
pushed a commit
to verl-project/verl
that referenced
this pull request
Mar 31, 2026
### What does this PR do? This pull request introduces **Dynamic Context Parallelism (Dynamic CP)** into verl’s Megatron engine. Dynamic CP allows each micro-batch to adaptively select an effective context-parallel size based on the longest sequence in the batch, avoiding the overhead of a fixed, over-provisioned CP size. Compared with the previous static CP approach this: 1. Keeps memory usage under control for the few extremely long sequences. 2. Eliminates unnecessary CP overhead for the majority of short or medium sequences. 3. Preserves training stability while providing better throughput in RL fine-tuning. ### Test Functional validation is automated by the script below, which launches a small-scale SFT experiment that exercises all supported CP sizes (1, 2) on a 8-GPU node. tested on docker image `verlai/verl:vllm017.latest` Note: tested on the latest(20260318) `dev` branch of megatron(NVIDIA/Megatron-LM@7c3eea6) or any branch with PR NVIDIA/Megatron-LM#3405. ### API and Usage Example No user-visible Python API changes are required. Dynamic CP is enabled purely through the YAML/JSON config: ```yaml engine: dynamic_context_parallel: true # Turn Dynamic CP on max_seqlen_per_dp_cp_rank: 2048 # Upper-bound of tokens handled by a single CP rank ``` Constraints: - `dp_size * max_seqlen_per_dp_cp_rank ≥ max_source_len + max_target_len` Runtime selection of `local_cp_size` follows this rule set: 1. Compute `max_seq_len_in_batch` for the current minibatch. 2. If `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank` then `local_cp_size = 1` (no context parallelism). 3. Otherwise pick the smallest `n` such that: - `n` is a power of two, - `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank × n`, - `n ≤ dp_size`. 4. The resulting `n` is the `local_cp_size` used for this micro-batch example: ```bash #!/usr/bin/env bash set -xeuo pipefail export CC=gcc # for jit compile on verlai/verl:vllm017.latest ENTRYPOINT=${ENTRYPOINT:-"-m verl.trainer.sft_trainer"} TRAIN_FILES=/data/cot_dataset.parquet backend=${BACKEND:-megatron} NOTES=${NOTES:-""} project_name=verl_sft_dynamic_cp RESUME_MODE=auto MODEL_NAME=${MODEL_NAME:-Qwen3-8B-Base} MODEL_PATH=/models/${MODEL_NAME} TP_SIZE=${TP_SIZE:-4} PP_SIZE=${PP_SIZE:-1} VPP_SIZE=${VPP_SIZE:-null} CP_SIZE=${CP_SIZE:-1} PAD_MODE=${PAD_MODE:-no_padding} USE_REMOVE_PADDING=${USE_REMOVE_PADDING:-True} DTYPE=${DTYPE:-"bfloat16"} MEGATRON_ENGINE_CONFIG="\ engine=${backend} \ optim=${backend} \ optim.lr=2e-5 \ optim.lr_warmup_steps=5 \ optim.weight_decay=0.1 \ optim.betas="[0.9,0.95]" \ optim.clip_grad=1.0 \ optim.lr_warmup_init=0 \ optim.lr_decay_style=cosine \ optim.min_lr=2e-6 \ engine.tensor_model_parallel_size=${TP_SIZE} \ engine.pipeline_model_parallel_size=${PP_SIZE} \ engine.virtual_pipeline_model_parallel_size=${VPP_SIZE} \ engine.context_parallel_size=${CP_SIZE} \ engine.use_mbridge=True \ engine.dtype=${DTYPE} \ engine.dynamic_context_parallel=True \ engine.max_seqlen_per_dp_cp_rank=2000" ENGINE_CONFIG="$MEGATRON_ENGINE_CONFIG" echo "Using megatron engine" exp_name=${MODEL_NAME}-${backend}-tp${TP_SIZE}-pp${PP_SIZE}-vpp${VPP_SIZE}-cp${CP_SIZE}-megatron-1103a1 ckpts_home=${ckpts_home:-${exp_name}} mkdir -p "${ckpts_home}" torchrun --nnodes=1 --nproc_per_node=8 ${ENTRYPOINT} \ data.train_files="${TRAIN_FILES}" \ data.train_batch_size=96 \ data.max_length=8192 \ data.pad_mode=${PAD_MODE} \ data.truncation=error \ data.use_dynamic_bsz=True \ data.max_token_len_per_gpu=8192 \ data.messages_key=messages \ data.ignore_input_ids_mismatch=True \ model.path=$MODEL_PATH \ model.use_remove_padding=${USE_REMOVE_PADDING} \ ${ENGINE_CONFIG} \ trainer.test_freq=-1 \ trainer.save_freq=500 \ trainer.logger=['console','wandb'] \ trainer.project_name="${project_name}" \ trainer.experiment_name="${exp_name}" \ trainer.total_epochs=1 \ trainer.default_local_dir="${ckpts_home}" \ trainer.resume_mode=${RESUME_MODE} \ trainer.max_ckpt_to_keep=10 \ checkpoint.save_contents=[model,optimizer,extra] ``` ### Design & Code Changes 1. **Megatron Engine Only** – dynamic CP is implemented for the new Megatron engine; the legacy Megatron worker is not supported. 2. **Logical DP = 1** – when Dynamic CP is active `engine.get_data_parallel_size()` forcibly returns 1, while the real DP groups are obtained through `engine.get_data_parallel_group()`. 3. **Lightweight Data Scheduler** – we reuse verl’s existing dynamic-batching pipeline. Each minibatch is split by `dynamic_cp_split_batch()` before forward and merged back by `dynamic_cp_merge_output()` afterwards. 4. **Supported CP Combinations** – with `dp_size = 4` we currently allow `[1,1,1,1]`, `[2,2]`, and `[4]`. The `[2,1,1]` pattern supported upstream is disabled for the first iteration. ### TODO After Merge 1. Publish a best-practice guide for RL training with Dynamic CP. 2. Refactor verl’s Dynamic CP data-scheduler to support re-batching and ordering for higher utilisation.
sijyang
pushed a commit
to sijyang/verl
that referenced
this pull request
Apr 1, 2026
### What does this PR do? This pull request introduces **Dynamic Context Parallelism (Dynamic CP)** into verl’s Megatron engine. Dynamic CP allows each micro-batch to adaptively select an effective context-parallel size based on the longest sequence in the batch, avoiding the overhead of a fixed, over-provisioned CP size. Compared with the previous static CP approach this: 1. Keeps memory usage under control for the few extremely long sequences. 2. Eliminates unnecessary CP overhead for the majority of short or medium sequences. 3. Preserves training stability while providing better throughput in RL fine-tuning. ### Test Functional validation is automated by the script below, which launches a small-scale SFT experiment that exercises all supported CP sizes (1, 2) on a 8-GPU node. tested on docker image `verlai/verl:vllm017.latest` Note: tested on the latest(20260318) `dev` branch of megatron(NVIDIA/Megatron-LM@7c3eea6) or any branch with PR NVIDIA/Megatron-LM#3405. ### API and Usage Example No user-visible Python API changes are required. Dynamic CP is enabled purely through the YAML/JSON config: ```yaml engine: dynamic_context_parallel: true # Turn Dynamic CP on max_seqlen_per_dp_cp_rank: 2048 # Upper-bound of tokens handled by a single CP rank ``` Constraints: - `dp_size * max_seqlen_per_dp_cp_rank ≥ max_source_len + max_target_len` Runtime selection of `local_cp_size` follows this rule set: 1. Compute `max_seq_len_in_batch` for the current minibatch. 2. If `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank` then `local_cp_size = 1` (no context parallelism). 3. Otherwise pick the smallest `n` such that: - `n` is a power of two, - `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank × n`, - `n ≤ dp_size`. 4. The resulting `n` is the `local_cp_size` used for this micro-batch example: ```bash #!/usr/bin/env bash set -xeuo pipefail export CC=gcc # for jit compile on verlai/verl:vllm017.latest ENTRYPOINT=${ENTRYPOINT:-"-m verl.trainer.sft_trainer"} TRAIN_FILES=/data/cot_dataset.parquet backend=${BACKEND:-megatron} NOTES=${NOTES:-""} project_name=verl_sft_dynamic_cp RESUME_MODE=auto MODEL_NAME=${MODEL_NAME:-Qwen3-8B-Base} MODEL_PATH=/models/${MODEL_NAME} TP_SIZE=${TP_SIZE:-4} PP_SIZE=${PP_SIZE:-1} VPP_SIZE=${VPP_SIZE:-null} CP_SIZE=${CP_SIZE:-1} PAD_MODE=${PAD_MODE:-no_padding} USE_REMOVE_PADDING=${USE_REMOVE_PADDING:-True} DTYPE=${DTYPE:-"bfloat16"} MEGATRON_ENGINE_CONFIG="\ engine=${backend} \ optim=${backend} \ optim.lr=2e-5 \ optim.lr_warmup_steps=5 \ optim.weight_decay=0.1 \ optim.betas="[0.9,0.95]" \ optim.clip_grad=1.0 \ optim.lr_warmup_init=0 \ optim.lr_decay_style=cosine \ optim.min_lr=2e-6 \ engine.tensor_model_parallel_size=${TP_SIZE} \ engine.pipeline_model_parallel_size=${PP_SIZE} \ engine.virtual_pipeline_model_parallel_size=${VPP_SIZE} \ engine.context_parallel_size=${CP_SIZE} \ engine.use_mbridge=True \ engine.dtype=${DTYPE} \ engine.dynamic_context_parallel=True \ engine.max_seqlen_per_dp_cp_rank=2000" ENGINE_CONFIG="$MEGATRON_ENGINE_CONFIG" echo "Using megatron engine" exp_name=${MODEL_NAME}-${backend}-tp${TP_SIZE}-pp${PP_SIZE}-vpp${VPP_SIZE}-cp${CP_SIZE}-megatron-1103a1 ckpts_home=${ckpts_home:-${exp_name}} mkdir -p "${ckpts_home}" torchrun --nnodes=1 --nproc_per_node=8 ${ENTRYPOINT} \ data.train_files="${TRAIN_FILES}" \ data.train_batch_size=96 \ data.max_length=8192 \ data.pad_mode=${PAD_MODE} \ data.truncation=error \ data.use_dynamic_bsz=True \ data.max_token_len_per_gpu=8192 \ data.messages_key=messages \ data.ignore_input_ids_mismatch=True \ model.path=$MODEL_PATH \ model.use_remove_padding=${USE_REMOVE_PADDING} \ ${ENGINE_CONFIG} \ trainer.test_freq=-1 \ trainer.save_freq=500 \ trainer.logger=['console','wandb'] \ trainer.project_name="${project_name}" \ trainer.experiment_name="${exp_name}" \ trainer.total_epochs=1 \ trainer.default_local_dir="${ckpts_home}" \ trainer.resume_mode=${RESUME_MODE} \ trainer.max_ckpt_to_keep=10 \ checkpoint.save_contents=[model,optimizer,extra] ``` ### Design & Code Changes 1. **Megatron Engine Only** – dynamic CP is implemented for the new Megatron engine; the legacy Megatron worker is not supported. 2. **Logical DP = 1** – when Dynamic CP is active `engine.get_data_parallel_size()` forcibly returns 1, while the real DP groups are obtained through `engine.get_data_parallel_group()`. 3. **Lightweight Data Scheduler** – we reuse verl’s existing dynamic-batching pipeline. Each minibatch is split by `dynamic_cp_split_batch()` before forward and merged back by `dynamic_cp_merge_output()` afterwards. 4. **Supported CP Combinations** – with `dp_size = 4` we currently allow `[1,1,1,1]`, `[2,2]`, and `[4]`. The `[2,1,1]` pattern supported upstream is disabled for the first iteration. ### TODO After Merge 1. Publish a best-practice guide for RL training with Dynamic CP. 2. Refactor verl’s Dynamic CP data-scheduler to support re-batching and ordering for higher utilisation.
ZouKexin-522
pushed a commit
to ZouKexin-522/verl
that referenced
this pull request
Apr 8, 2026
### What does this PR do? This pull request introduces **Dynamic Context Parallelism (Dynamic CP)** into verl’s Megatron engine. Dynamic CP allows each micro-batch to adaptively select an effective context-parallel size based on the longest sequence in the batch, avoiding the overhead of a fixed, over-provisioned CP size. Compared with the previous static CP approach this: 1. Keeps memory usage under control for the few extremely long sequences. 2. Eliminates unnecessary CP overhead for the majority of short or medium sequences. 3. Preserves training stability while providing better throughput in RL fine-tuning. ### Test Functional validation is automated by the script below, which launches a small-scale SFT experiment that exercises all supported CP sizes (1, 2) on a 8-GPU node. tested on docker image `verlai/verl:vllm017.latest` Note: tested on the latest(20260318) `dev` branch of megatron(NVIDIA/Megatron-LM@7c3eea6) or any branch with PR NVIDIA/Megatron-LM#3405. ### API and Usage Example No user-visible Python API changes are required. Dynamic CP is enabled purely through the YAML/JSON config: ```yaml engine: dynamic_context_parallel: true # Turn Dynamic CP on max_seqlen_per_dp_cp_rank: 2048 # Upper-bound of tokens handled by a single CP rank ``` Constraints: - `dp_size * max_seqlen_per_dp_cp_rank ≥ max_source_len + max_target_len` Runtime selection of `local_cp_size` follows this rule set: 1. Compute `max_seq_len_in_batch` for the current minibatch. 2. If `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank` then `local_cp_size = 1` (no context parallelism). 3. Otherwise pick the smallest `n` such that: - `n` is a power of two, - `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank × n`, - `n ≤ dp_size`. 4. The resulting `n` is the `local_cp_size` used for this micro-batch example: ```bash #!/usr/bin/env bash set -xeuo pipefail export CC=gcc # for jit compile on verlai/verl:vllm017.latest ENTRYPOINT=${ENTRYPOINT:-"-m verl.trainer.sft_trainer"} TRAIN_FILES=/data/cot_dataset.parquet backend=${BACKEND:-megatron} NOTES=${NOTES:-""} project_name=verl_sft_dynamic_cp RESUME_MODE=auto MODEL_NAME=${MODEL_NAME:-Qwen3-8B-Base} MODEL_PATH=/models/${MODEL_NAME} TP_SIZE=${TP_SIZE:-4} PP_SIZE=${PP_SIZE:-1} VPP_SIZE=${VPP_SIZE:-null} CP_SIZE=${CP_SIZE:-1} PAD_MODE=${PAD_MODE:-no_padding} USE_REMOVE_PADDING=${USE_REMOVE_PADDING:-True} DTYPE=${DTYPE:-"bfloat16"} MEGATRON_ENGINE_CONFIG="\ engine=${backend} \ optim=${backend} \ optim.lr=2e-5 \ optim.lr_warmup_steps=5 \ optim.weight_decay=0.1 \ optim.betas="[0.9,0.95]" \ optim.clip_grad=1.0 \ optim.lr_warmup_init=0 \ optim.lr_decay_style=cosine \ optim.min_lr=2e-6 \ engine.tensor_model_parallel_size=${TP_SIZE} \ engine.pipeline_model_parallel_size=${PP_SIZE} \ engine.virtual_pipeline_model_parallel_size=${VPP_SIZE} \ engine.context_parallel_size=${CP_SIZE} \ engine.use_mbridge=True \ engine.dtype=${DTYPE} \ engine.dynamic_context_parallel=True \ engine.max_seqlen_per_dp_cp_rank=2000" ENGINE_CONFIG="$MEGATRON_ENGINE_CONFIG" echo "Using megatron engine" exp_name=${MODEL_NAME}-${backend}-tp${TP_SIZE}-pp${PP_SIZE}-vpp${VPP_SIZE}-cp${CP_SIZE}-megatron-1103a1 ckpts_home=${ckpts_home:-${exp_name}} mkdir -p "${ckpts_home}" torchrun --nnodes=1 --nproc_per_node=8 ${ENTRYPOINT} \ data.train_files="${TRAIN_FILES}" \ data.train_batch_size=96 \ data.max_length=8192 \ data.pad_mode=${PAD_MODE} \ data.truncation=error \ data.use_dynamic_bsz=True \ data.max_token_len_per_gpu=8192 \ data.messages_key=messages \ data.ignore_input_ids_mismatch=True \ model.path=$MODEL_PATH \ model.use_remove_padding=${USE_REMOVE_PADDING} \ ${ENGINE_CONFIG} \ trainer.test_freq=-1 \ trainer.save_freq=500 \ trainer.logger=['console','wandb'] \ trainer.project_name="${project_name}" \ trainer.experiment_name="${exp_name}" \ trainer.total_epochs=1 \ trainer.default_local_dir="${ckpts_home}" \ trainer.resume_mode=${RESUME_MODE} \ trainer.max_ckpt_to_keep=10 \ checkpoint.save_contents=[model,optimizer,extra] ``` ### Design & Code Changes 1. **Megatron Engine Only** – dynamic CP is implemented for the new Megatron engine; the legacy Megatron worker is not supported. 2. **Logical DP = 1** – when Dynamic CP is active `engine.get_data_parallel_size()` forcibly returns 1, while the real DP groups are obtained through `engine.get_data_parallel_group()`. 3. **Lightweight Data Scheduler** – we reuse verl’s existing dynamic-batching pipeline. Each minibatch is split by `dynamic_cp_split_batch()` before forward and merged back by `dynamic_cp_merge_output()` afterwards. 4. **Supported CP Combinations** – with `dp_size = 4` we currently allow `[1,1,1,1]`, `[2,2]`, and `[4]`. The `[2,1,1]` pattern supported upstream is disabled for the first iteration. ### TODO After Merge 1. Publish a best-practice guide for RL training with Dynamic CP. 2. Refactor verl’s Dynamic CP data-scheduler to support re-batching and ordering for higher utilisation.
DaizeDong
pushed a commit
to DaizeDong/verl
that referenced
this pull request
Apr 19, 2026
### What does this PR do? This pull request introduces **Dynamic Context Parallelism (Dynamic CP)** into verl’s Megatron engine. Dynamic CP allows each micro-batch to adaptively select an effective context-parallel size based on the longest sequence in the batch, avoiding the overhead of a fixed, over-provisioned CP size. Compared with the previous static CP approach this: 1. Keeps memory usage under control for the few extremely long sequences. 2. Eliminates unnecessary CP overhead for the majority of short or medium sequences. 3. Preserves training stability while providing better throughput in RL fine-tuning. ### Test Functional validation is automated by the script below, which launches a small-scale SFT experiment that exercises all supported CP sizes (1, 2) on a 8-GPU node. tested on docker image `verlai/verl:vllm017.latest` Note: tested on the latest(20260318) `dev` branch of megatron(NVIDIA/Megatron-LM@7c3eea6) or any branch with PR NVIDIA/Megatron-LM#3405. ### API and Usage Example No user-visible Python API changes are required. Dynamic CP is enabled purely through the YAML/JSON config: ```yaml engine: dynamic_context_parallel: true # Turn Dynamic CP on max_seqlen_per_dp_cp_rank: 2048 # Upper-bound of tokens handled by a single CP rank ``` Constraints: - `dp_size * max_seqlen_per_dp_cp_rank ≥ max_source_len + max_target_len` Runtime selection of `local_cp_size` follows this rule set: 1. Compute `max_seq_len_in_batch` for the current minibatch. 2. If `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank` then `local_cp_size = 1` (no context parallelism). 3. Otherwise pick the smallest `n` such that: - `n` is a power of two, - `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank × n`, - `n ≤ dp_size`. 4. The resulting `n` is the `local_cp_size` used for this micro-batch example: ```bash #!/usr/bin/env bash set -xeuo pipefail export CC=gcc # for jit compile on verlai/verl:vllm017.latest ENTRYPOINT=${ENTRYPOINT:-"-m verl.trainer.sft_trainer"} TRAIN_FILES=/data/cot_dataset.parquet backend=${BACKEND:-megatron} NOTES=${NOTES:-""} project_name=verl_sft_dynamic_cp RESUME_MODE=auto MODEL_NAME=${MODEL_NAME:-Qwen3-8B-Base} MODEL_PATH=/models/${MODEL_NAME} TP_SIZE=${TP_SIZE:-4} PP_SIZE=${PP_SIZE:-1} VPP_SIZE=${VPP_SIZE:-null} CP_SIZE=${CP_SIZE:-1} PAD_MODE=${PAD_MODE:-no_padding} USE_REMOVE_PADDING=${USE_REMOVE_PADDING:-True} DTYPE=${DTYPE:-"bfloat16"} MEGATRON_ENGINE_CONFIG="\ engine=${backend} \ optim=${backend} \ optim.lr=2e-5 \ optim.lr_warmup_steps=5 \ optim.weight_decay=0.1 \ optim.betas="[0.9,0.95]" \ optim.clip_grad=1.0 \ optim.lr_warmup_init=0 \ optim.lr_decay_style=cosine \ optim.min_lr=2e-6 \ engine.tensor_model_parallel_size=${TP_SIZE} \ engine.pipeline_model_parallel_size=${PP_SIZE} \ engine.virtual_pipeline_model_parallel_size=${VPP_SIZE} \ engine.context_parallel_size=${CP_SIZE} \ engine.use_mbridge=True \ engine.dtype=${DTYPE} \ engine.dynamic_context_parallel=True \ engine.max_seqlen_per_dp_cp_rank=2000" ENGINE_CONFIG="$MEGATRON_ENGINE_CONFIG" echo "Using megatron engine" exp_name=${MODEL_NAME}-${backend}-tp${TP_SIZE}-pp${PP_SIZE}-vpp${VPP_SIZE}-cp${CP_SIZE}-megatron-1103a1 ckpts_home=${ckpts_home:-${exp_name}} mkdir -p "${ckpts_home}" torchrun --nnodes=1 --nproc_per_node=8 ${ENTRYPOINT} \ data.train_files="${TRAIN_FILES}" \ data.train_batch_size=96 \ data.max_length=8192 \ data.pad_mode=${PAD_MODE} \ data.truncation=error \ data.use_dynamic_bsz=True \ data.max_token_len_per_gpu=8192 \ data.messages_key=messages \ data.ignore_input_ids_mismatch=True \ model.path=$MODEL_PATH \ model.use_remove_padding=${USE_REMOVE_PADDING} \ ${ENGINE_CONFIG} \ trainer.test_freq=-1 \ trainer.save_freq=500 \ trainer.logger=['console','wandb'] \ trainer.project_name="${project_name}" \ trainer.experiment_name="${exp_name}" \ trainer.total_epochs=1 \ trainer.default_local_dir="${ckpts_home}" \ trainer.resume_mode=${RESUME_MODE} \ trainer.max_ckpt_to_keep=10 \ checkpoint.save_contents=[model,optimizer,extra] ``` ### Design & Code Changes 1. **Megatron Engine Only** – dynamic CP is implemented for the new Megatron engine; the legacy Megatron worker is not supported. 2. **Logical DP = 1** – when Dynamic CP is active `engine.get_data_parallel_size()` forcibly returns 1, while the real DP groups are obtained through `engine.get_data_parallel_group()`. 3. **Lightweight Data Scheduler** – we reuse verl’s existing dynamic-batching pipeline. Each minibatch is split by `dynamic_cp_split_batch()` before forward and merged back by `dynamic_cp_merge_output()` afterwards. 4. **Supported CP Combinations** – with `dp_size = 4` we currently allow `[1,1,1,1]`, `[2,2]`, and `[4]`. The `[2,1,1]` pattern supported upstream is disabled for the first iteration. ### TODO After Merge 1. Publish a best-practice guide for RL training with Dynamic CP. 2. Refactor verl’s Dynamic CP data-scheduler to support re-batching and ordering for higher utilisation.
zwluestc
pushed a commit
to zwluestc/verl
that referenced
this pull request
May 12, 2026
### What does this PR do? This pull request introduces **Dynamic Context Parallelism (Dynamic CP)** into verl’s Megatron engine. Dynamic CP allows each micro-batch to adaptively select an effective context-parallel size based on the longest sequence in the batch, avoiding the overhead of a fixed, over-provisioned CP size. Compared with the previous static CP approach this: 1. Keeps memory usage under control for the few extremely long sequences. 2. Eliminates unnecessary CP overhead for the majority of short or medium sequences. 3. Preserves training stability while providing better throughput in RL fine-tuning. ### Test Functional validation is automated by the script below, which launches a small-scale SFT experiment that exercises all supported CP sizes (1, 2) on a 8-GPU node. tested on docker image `verlai/verl:vllm017.latest` Note: tested on the latest(20260318) `dev` branch of megatron(NVIDIA/Megatron-LM@7c3eea6) or any branch with PR NVIDIA/Megatron-LM#3405. ### API and Usage Example No user-visible Python API changes are required. Dynamic CP is enabled purely through the YAML/JSON config: ```yaml engine: dynamic_context_parallel: true # Turn Dynamic CP on max_seqlen_per_dp_cp_rank: 2048 # Upper-bound of tokens handled by a single CP rank ``` Constraints: - `dp_size * max_seqlen_per_dp_cp_rank ≥ max_source_len + max_target_len` Runtime selection of `local_cp_size` follows this rule set: 1. Compute `max_seq_len_in_batch` for the current minibatch. 2. If `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank` then `local_cp_size = 1` (no context parallelism). 3. Otherwise pick the smallest `n` such that: - `n` is a power of two, - `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank × n`, - `n ≤ dp_size`. 4. The resulting `n` is the `local_cp_size` used for this micro-batch example: ```bash #!/usr/bin/env bash set -xeuo pipefail export CC=gcc # for jit compile on verlai/verl:vllm017.latest ENTRYPOINT=${ENTRYPOINT:-"-m verl.trainer.sft_trainer"} TRAIN_FILES=/data/cot_dataset.parquet backend=${BACKEND:-megatron} NOTES=${NOTES:-""} project_name=verl_sft_dynamic_cp RESUME_MODE=auto MODEL_NAME=${MODEL_NAME:-Qwen3-8B-Base} MODEL_PATH=/models/${MODEL_NAME} TP_SIZE=${TP_SIZE:-4} PP_SIZE=${PP_SIZE:-1} VPP_SIZE=${VPP_SIZE:-null} CP_SIZE=${CP_SIZE:-1} PAD_MODE=${PAD_MODE:-no_padding} USE_REMOVE_PADDING=${USE_REMOVE_PADDING:-True} DTYPE=${DTYPE:-"bfloat16"} MEGATRON_ENGINE_CONFIG="\ engine=${backend} \ optim=${backend} \ optim.lr=2e-5 \ optim.lr_warmup_steps=5 \ optim.weight_decay=0.1 \ optim.betas="[0.9,0.95]" \ optim.clip_grad=1.0 \ optim.lr_warmup_init=0 \ optim.lr_decay_style=cosine \ optim.min_lr=2e-6 \ engine.tensor_model_parallel_size=${TP_SIZE} \ engine.pipeline_model_parallel_size=${PP_SIZE} \ engine.virtual_pipeline_model_parallel_size=${VPP_SIZE} \ engine.context_parallel_size=${CP_SIZE} \ engine.use_mbridge=True \ engine.dtype=${DTYPE} \ engine.dynamic_context_parallel=True \ engine.max_seqlen_per_dp_cp_rank=2000" ENGINE_CONFIG="$MEGATRON_ENGINE_CONFIG" echo "Using megatron engine" exp_name=${MODEL_NAME}-${backend}-tp${TP_SIZE}-pp${PP_SIZE}-vpp${VPP_SIZE}-cp${CP_SIZE}-megatron-1103a1 ckpts_home=${ckpts_home:-${exp_name}} mkdir -p "${ckpts_home}" torchrun --nnodes=1 --nproc_per_node=8 ${ENTRYPOINT} \ data.train_files="${TRAIN_FILES}" \ data.train_batch_size=96 \ data.max_length=8192 \ data.pad_mode=${PAD_MODE} \ data.truncation=error \ data.use_dynamic_bsz=True \ data.max_token_len_per_gpu=8192 \ data.messages_key=messages \ data.ignore_input_ids_mismatch=True \ model.path=$MODEL_PATH \ model.use_remove_padding=${USE_REMOVE_PADDING} \ ${ENGINE_CONFIG} \ trainer.test_freq=-1 \ trainer.save_freq=500 \ trainer.logger=['console','wandb'] \ trainer.project_name="${project_name}" \ trainer.experiment_name="${exp_name}" \ trainer.total_epochs=1 \ trainer.default_local_dir="${ckpts_home}" \ trainer.resume_mode=${RESUME_MODE} \ trainer.max_ckpt_to_keep=10 \ checkpoint.save_contents=[model,optimizer,extra] ``` ### Design & Code Changes 1. **Megatron Engine Only** – dynamic CP is implemented for the new Megatron engine; the legacy Megatron worker is not supported. 2. **Logical DP = 1** – when Dynamic CP is active `engine.get_data_parallel_size()` forcibly returns 1, while the real DP groups are obtained through `engine.get_data_parallel_group()`. 3. **Lightweight Data Scheduler** – we reuse verl’s existing dynamic-batching pipeline. Each minibatch is split by `dynamic_cp_split_batch()` before forward and merged back by `dynamic_cp_merge_output()` afterwards. 4. **Supported CP Combinations** – with `dp_size = 4` we currently allow `[1,1,1,1]`, `[2,2]`, and `[4]`. The `[2,1,1]` pattern supported upstream is disabled for the first iteration. ### TODO After Merge 1. Publish a best-practice guide for RL training with Dynamic CP. 2. Refactor verl’s Dynamic CP data-scheduler to support re-batching and ordering for higher utilisation.
xvlincaigou
pushed a commit
to xvlincaigou/verl
that referenced
this pull request
May 19, 2026
### What does this PR do? This pull request introduces **Dynamic Context Parallelism (Dynamic CP)** into verl’s Megatron engine. Dynamic CP allows each micro-batch to adaptively select an effective context-parallel size based on the longest sequence in the batch, avoiding the overhead of a fixed, over-provisioned CP size. Compared with the previous static CP approach this: 1. Keeps memory usage under control for the few extremely long sequences. 2. Eliminates unnecessary CP overhead for the majority of short or medium sequences. 3. Preserves training stability while providing better throughput in RL fine-tuning. ### Test Functional validation is automated by the script below, which launches a small-scale SFT experiment that exercises all supported CP sizes (1, 2) on a 8-GPU node. tested on docker image `verlai/verl:vllm017.latest` Note: tested on the latest(20260318) `dev` branch of megatron(NVIDIA/Megatron-LM@7c3eea6) or any branch with PR NVIDIA/Megatron-LM#3405. ### API and Usage Example No user-visible Python API changes are required. Dynamic CP is enabled purely through the YAML/JSON config: ```yaml engine: dynamic_context_parallel: true # Turn Dynamic CP on max_seqlen_per_dp_cp_rank: 2048 # Upper-bound of tokens handled by a single CP rank ``` Constraints: - `dp_size * max_seqlen_per_dp_cp_rank ≥ max_source_len + max_target_len` Runtime selection of `local_cp_size` follows this rule set: 1. Compute `max_seq_len_in_batch` for the current minibatch. 2. If `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank` then `local_cp_size = 1` (no context parallelism). 3. Otherwise pick the smallest `n` such that: - `n` is a power of two, - `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank × n`, - `n ≤ dp_size`. 4. The resulting `n` is the `local_cp_size` used for this micro-batch example: ```bash #!/usr/bin/env bash set -xeuo pipefail export CC=gcc # for jit compile on verlai/verl:vllm017.latest ENTRYPOINT=${ENTRYPOINT:-"-m verl.trainer.sft_trainer"} TRAIN_FILES=/data/cot_dataset.parquet backend=${BACKEND:-megatron} NOTES=${NOTES:-""} project_name=verl_sft_dynamic_cp RESUME_MODE=auto MODEL_NAME=${MODEL_NAME:-Qwen3-8B-Base} MODEL_PATH=/models/${MODEL_NAME} TP_SIZE=${TP_SIZE:-4} PP_SIZE=${PP_SIZE:-1} VPP_SIZE=${VPP_SIZE:-null} CP_SIZE=${CP_SIZE:-1} PAD_MODE=${PAD_MODE:-no_padding} USE_REMOVE_PADDING=${USE_REMOVE_PADDING:-True} DTYPE=${DTYPE:-"bfloat16"} MEGATRON_ENGINE_CONFIG="\ engine=${backend} \ optim=${backend} \ optim.lr=2e-5 \ optim.lr_warmup_steps=5 \ optim.weight_decay=0.1 \ optim.betas="[0.9,0.95]" \ optim.clip_grad=1.0 \ optim.lr_warmup_init=0 \ optim.lr_decay_style=cosine \ optim.min_lr=2e-6 \ engine.tensor_model_parallel_size=${TP_SIZE} \ engine.pipeline_model_parallel_size=${PP_SIZE} \ engine.virtual_pipeline_model_parallel_size=${VPP_SIZE} \ engine.context_parallel_size=${CP_SIZE} \ engine.use_mbridge=True \ engine.dtype=${DTYPE} \ engine.dynamic_context_parallel=True \ engine.max_seqlen_per_dp_cp_rank=2000" ENGINE_CONFIG="$MEGATRON_ENGINE_CONFIG" echo "Using megatron engine" exp_name=${MODEL_NAME}-${backend}-tp${TP_SIZE}-pp${PP_SIZE}-vpp${VPP_SIZE}-cp${CP_SIZE}-megatron-1103a1 ckpts_home=${ckpts_home:-${exp_name}} mkdir -p "${ckpts_home}" torchrun --nnodes=1 --nproc_per_node=8 ${ENTRYPOINT} \ data.train_files="${TRAIN_FILES}" \ data.train_batch_size=96 \ data.max_length=8192 \ data.pad_mode=${PAD_MODE} \ data.truncation=error \ data.use_dynamic_bsz=True \ data.max_token_len_per_gpu=8192 \ data.messages_key=messages \ data.ignore_input_ids_mismatch=True \ model.path=$MODEL_PATH \ model.use_remove_padding=${USE_REMOVE_PADDING} \ ${ENGINE_CONFIG} \ trainer.test_freq=-1 \ trainer.save_freq=500 \ trainer.logger=['console','wandb'] \ trainer.project_name="${project_name}" \ trainer.experiment_name="${exp_name}" \ trainer.total_epochs=1 \ trainer.default_local_dir="${ckpts_home}" \ trainer.resume_mode=${RESUME_MODE} \ trainer.max_ckpt_to_keep=10 \ checkpoint.save_contents=[model,optimizer,extra] ``` ### Design & Code Changes 1. **Megatron Engine Only** – dynamic CP is implemented for the new Megatron engine; the legacy Megatron worker is not supported. 2. **Logical DP = 1** – when Dynamic CP is active `engine.get_data_parallel_size()` forcibly returns 1, while the real DP groups are obtained through `engine.get_data_parallel_group()`. 3. **Lightweight Data Scheduler** – we reuse verl’s existing dynamic-batching pipeline. Each minibatch is split by `dynamic_cp_split_batch()` before forward and merged back by `dynamic_cp_merge_output()` afterwards. 4. **Supported CP Combinations** – with `dp_size = 4` we currently allow `[1,1,1,1]`, `[2,2]`, and `[4]`. The `[2,1,1]` pattern supported upstream is disabled for the first iteration. ### TODO After Merge 1. Publish a best-practice guide for RL training with Dynamic CP. 2. Refactor verl’s Dynamic CP data-scheduler to support re-batching and ordering for higher utilisation.
This was referenced Jun 10, 2026
sunjie279
pushed a commit
to lds-ustc/EasyOPD
that referenced
this pull request
Jun 17, 2026
### What does this PR do? This pull request introduces **Dynamic Context Parallelism (Dynamic CP)** into verl’s Megatron engine. Dynamic CP allows each micro-batch to adaptively select an effective context-parallel size based on the longest sequence in the batch, avoiding the overhead of a fixed, over-provisioned CP size. Compared with the previous static CP approach this: 1. Keeps memory usage under control for the few extremely long sequences. 2. Eliminates unnecessary CP overhead for the majority of short or medium sequences. 3. Preserves training stability while providing better throughput in RL fine-tuning. ### Test Functional validation is automated by the script below, which launches a small-scale SFT experiment that exercises all supported CP sizes (1, 2) on a 8-GPU node. tested on docker image `verlai/verl:vllm017.latest` Note: tested on the latest(20260318) `dev` branch of megatron(NVIDIA/Megatron-LM@7c3eea6) or any branch with PR NVIDIA/Megatron-LM#3405. ### API and Usage Example No user-visible Python API changes are required. Dynamic CP is enabled purely through the YAML/JSON config: ```yaml engine: dynamic_context_parallel: true # Turn Dynamic CP on max_seqlen_per_dp_cp_rank: 2048 # Upper-bound of tokens handled by a single CP rank ``` Constraints: - `dp_size * max_seqlen_per_dp_cp_rank ≥ max_source_len + max_target_len` Runtime selection of `local_cp_size` follows this rule set: 1. Compute `max_seq_len_in_batch` for the current minibatch. 2. If `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank` then `local_cp_size = 1` (no context parallelism). 3. Otherwise pick the smallest `n` such that: - `n` is a power of two, - `max_seq_len_in_batch ≤ max_seqlen_per_dp_cp_rank × n`, - `n ≤ dp_size`. 4. The resulting `n` is the `local_cp_size` used for this micro-batch example: ```bash #!/usr/bin/env bash set -xeuo pipefail export CC=gcc # for jit compile on verlai/verl:vllm017.latest ENTRYPOINT=${ENTRYPOINT:-"-m verl.trainer.sft_trainer"} TRAIN_FILES=/data/cot_dataset.parquet backend=${BACKEND:-megatron} NOTES=${NOTES:-""} project_name=verl_sft_dynamic_cp RESUME_MODE=auto MODEL_NAME=${MODEL_NAME:-Qwen3-8B-Base} MODEL_PATH=/models/${MODEL_NAME} TP_SIZE=${TP_SIZE:-4} PP_SIZE=${PP_SIZE:-1} VPP_SIZE=${VPP_SIZE:-null} CP_SIZE=${CP_SIZE:-1} PAD_MODE=${PAD_MODE:-no_padding} USE_REMOVE_PADDING=${USE_REMOVE_PADDING:-True} DTYPE=${DTYPE:-"bfloat16"} MEGATRON_ENGINE_CONFIG="\ engine=${backend} \ optim=${backend} \ optim.lr=2e-5 \ optim.lr_warmup_steps=5 \ optim.weight_decay=0.1 \ optim.betas="[0.9,0.95]" \ optim.clip_grad=1.0 \ optim.lr_warmup_init=0 \ optim.lr_decay_style=cosine \ optim.min_lr=2e-6 \ engine.tensor_model_parallel_size=${TP_SIZE} \ engine.pipeline_model_parallel_size=${PP_SIZE} \ engine.virtual_pipeline_model_parallel_size=${VPP_SIZE} \ engine.context_parallel_size=${CP_SIZE} \ engine.use_mbridge=True \ engine.dtype=${DTYPE} \ engine.dynamic_context_parallel=True \ engine.max_seqlen_per_dp_cp_rank=2000" ENGINE_CONFIG="$MEGATRON_ENGINE_CONFIG" echo "Using megatron engine" exp_name=${MODEL_NAME}-${backend}-tp${TP_SIZE}-pp${PP_SIZE}-vpp${VPP_SIZE}-cp${CP_SIZE}-megatron-1103a1 ckpts_home=${ckpts_home:-${exp_name}} mkdir -p "${ckpts_home}" torchrun --nnodes=1 --nproc_per_node=8 ${ENTRYPOINT} \ data.train_files="${TRAIN_FILES}" \ data.train_batch_size=96 \ data.max_length=8192 \ data.pad_mode=${PAD_MODE} \ data.truncation=error \ data.use_dynamic_bsz=True \ data.max_token_len_per_gpu=8192 \ data.messages_key=messages \ data.ignore_input_ids_mismatch=True \ model.path=$MODEL_PATH \ model.use_remove_padding=${USE_REMOVE_PADDING} \ ${ENGINE_CONFIG} \ trainer.test_freq=-1 \ trainer.save_freq=500 \ trainer.logger=['console','wandb'] \ trainer.project_name="${project_name}" \ trainer.experiment_name="${exp_name}" \ trainer.total_epochs=1 \ trainer.default_local_dir="${ckpts_home}" \ trainer.resume_mode=${RESUME_MODE} \ trainer.max_ckpt_to_keep=10 \ checkpoint.save_contents=[model,optimizer,extra] ``` ### Design & Code Changes 1. **Megatron Engine Only** – dynamic CP is implemented for the new Megatron engine; the legacy Megatron worker is not supported. 2. **Logical DP = 1** – when Dynamic CP is active `engine.get_data_parallel_size()` forcibly returns 1, while the real DP groups are obtained through `engine.get_data_parallel_group()`. 3. **Lightweight Data Scheduler** – we reuse verl’s existing dynamic-batching pipeline. Each minibatch is split by `dynamic_cp_split_batch()` before forward and merged back by `dynamic_cp_merge_output()` afterwards. 4. **Supported CP Combinations** – with `dp_size = 4` we currently allow `[1,1,1,1]`, `[2,2]`, and `[4]`. The `[2,1,1]` pattern supported upstream is disabled for the first iteration. ### TODO After Merge 1. Publish a best-practice guide for RL training with Dynamic CP. 2. Refactor verl’s Dynamic CP data-scheduler to support re-batching and ordering for higher utilisation.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR does four things:
1. Rename
hybrid-cp→dynamic-cpAfter discussion with @parthmannan , we decided to rename "hybrid context parallel" to "dynamic context parallel". The term "hybrid" was ambiguous — it could be confused with hybrid parallelism strategies (e.g., TP+PP hybrid) or hierarchical CP. "Dynamic" more accurately describes the feature's behavior: dynamically assigning different CP sub-group sizes to different microbatches based on sequence length at runtime.
The rename is applied consistently across 21 files, including:
hybrid_context_parallel→dynamic_context_parallelinModelParallelConfigandGPTDatasetConfigHybridCPDataLoaderWrapper→DynamicCPDataLoaderWrapper,HybridCPMegatronPretrainingSampler→DynamicCPMegatronPretrainingSamplerget_batch_on_this_hybrid_cp_rank→get_batch_on_this_dynamic_cp_rank,hybrid_context_parallel_forward_backward→dynamic_context_parallel_forward_backward_HYBRID_DP_CP_GROUPS→_DYNAMIC_DP_CP_GROUPShybrid_cp_schedule.py→dynamic_cp_schedule.py2. Fix RoPE when enabling THD + Dynamic CP
When using Dynamic CP with THD (packed sequence) format, RoPE was computing incorrect position offsets. The root cause:
apply_rotary_pos_embderivescp_rankandcp_sizefrompg_collection.cpto determine each token's position in the zigzag-sharded sequence. In Dynamic CP mode, different microbatches use different CP sub-groups, butpg_collection.cpwas still pointing to the original (global) CP group — causing position mismatch.Changes:
megatron/core/transformer/attention.py: At the start ofAttention.forward(), overrideself.pg_collection.cpwith the dynamic CP sub-group frompacked_seq_params.cp_groupwhenlocal_cp_sizeis set. This ensures downstream RoPE calls use the correctcp_rank/cp_size.megatron/core/extensions/transformer_engine.py: Refactored the Dynamic CP group handling inTEDotProductAttention.forward()to uselocal_cp_sizeas the primary dispatch key (instead of checkingcp_groupfirst), and added an assert to ensurecp_groupis set whenlocal_cp_size > 1.megatron/core/parallel_state.py: Increate_dynamic_dp_cp_groups, removed the[1:]slice fromgroup_sizesso that size=1 groups are now created. This is needed because whenlocal_cp_size == 1(a rank processes a sequence independently),get_dynamic_data_context_parallel_groups(group_size=1)must return a valid group.megatron/core/utils.py: Simplifiedget_batch_on_this_dynamic_cp_rank— always fetch the dynamic group (including size=1) instead of special-casinglocal_cp_size > 1.3. Eager initialization of NCCL communicator groups in
parallel_state.pyPyTorch performs lazy initialization of NCCL communicator groups. In Dynamic CP, sub-groups of varying sizes are created during
initialize_model_parallelbut may not be actually initialized until first use during training — which can cause hangs if different ranks reach different communicators at different times.Added explicit
torch.distributed.barrier()+torch.cuda.synchronize()for all dynamic DP×CP groups right after creation to force NCCL communicator initialization eagerly. Also included size=1 groups (removed the[1:]slice fromgroup_sizes) to support the case where a rank processes a sequence independently without CP communication.4. Added Dynamic CP correctness test
Added
test_dynamic_cp_formatintests/unit_tests/transformer/test_thd_correctness.pythat compares fixed CP THD baseline against Dynamic CP THD with RoPE enabled. Covers:PR for main branch: #3717
Code review
The following process is enforced via the CODEOWNERS file for changes into
megatron/core. For changes outside ofmegatron/core, it is up to the PR author whether or not to tag the Final Reviewer team.For MRs into `dev` branch
The proposed review process for `dev` branch is under active discussion.MRs are mergable after one approval by either
eharper@nvidia.comorzijiey@nvidia.com.Merging your PR
Any member of core-adlr and
core-nemowill be able to merge your PR.