Support GLM-5.2

CONTRIBUTING.md

@@ -6,7 +6,7 @@ Thank you for your interest in contributing to slime! We deeply appreciate every
 
 ## Collaboration Scope
 
-slime is the RL training infrastructure behind [GLM-4.5 through GLM-5.1](https://z.ai) and a large number of internal experiments at Z.ai. We open-sourced slime because we believe the training scenarios used internally cover the majority of cutting-edge RL algorithm requirements, and we hope to provide the community with a correct and efficient large-scale RL training infrastructure.
+slime is the RL training infrastructure behind [GLM-4.5 through GLM-5.2](https://z.ai) and a large number of internal experiments at Z.ai. We open-sourced slime because we believe the training scenarios used internally cover the majority of cutting-edge RL algorithm requirements, and we hope to provide the community with a correct and efficient large-scale RL training infrastructure.
 
 Our goal for open-source collaboration is focused on **bug fixes** and **general-purpose large-scale RL optimizations**. We have had several successful collaborations with the community in this area, including:
 

README.md

@@ -17,7 +17,7 @@ This makes slime one of the most battle-tested open RL post-training frameworks:
 
 ## Why This Design Matters
 
-- **Battle-tested by frontier model training**: slime is the RL framework behind [GLM-5.1](https://z.ai/blog/glm-5.1), [GLM-5](https://z.ai/blog/glm-5), [GLM-4.7](https://z.ai/blog/glm-4.7), [GLM-4.6](https://z.ai/blog/glm-4.6), and [GLM-4.5](https://z.ai/blog/glm-4.5). This validates the full post-training loop, not only isolated examples.
+- **Battle-tested by frontier model training**: slime is the RL framework behind [GLM-5.2](https://z.ai/blog/glm-5.2), [GLM-5.1](https://z.ai/blog/glm-5.1), [GLM-5](https://z.ai/blog/glm-5), [GLM-4.7](https://z.ai/blog/glm-4.7), [GLM-4.6](https://z.ai/blog/glm-4.6), and [GLM-4.5](https://z.ai/blog/glm-4.5). This validates the full post-training loop, not only isolated examples.
 - **Correctness-first infrastructure**: RL bugs are often silent. slime keeps the dataflow explicit, supports separate rollout-only and train-only debugging paths, and documents reproducibility, fault tolerance, tracing, profiling, and CI as first-class engineering concerns.
 - **Native by design**: slime passes Megatron arguments through directly and exposes installed SGLang arguments with a `--sglang-` prefix. New upstream training and serving optimizations can be used without adding another abstraction layer inside slime.
 - **Maximum data-generation freedom**: math, code, search, tools, sandboxes, verifiers, environments, multi-agent systems, and long-horizon agentic workflows plug in as data generation or reward workflows. They do not fork the training kernel.

README_zh.md

@@ -17,7 +17,7 @@ slime 的设计目标，是让这两大能力彼此强化，同时避免把系
 
 ## 为什么这个设计重要
 
-- **经过 frontier model 训练验证**：slime 是 [GLM-5.1](https://z.ai/blog/glm-5.1)、[GLM-5](https://z.ai/blog/glm-5)、[GLM-4.7](https://z.ai/blog/glm-4.7)、[GLM-4.6](https://z.ai/blog/glm-4.6)、[GLM-4.5](https://z.ai/blog/glm-4.5) 背后的 RL 训练框架。这验证的是完整 post-training loop，而不是孤立 example。
+- **经过 frontier model 训练验证**：slime 是 [GLM-5.2](https://z.ai/blog/glm-5.2)、[GLM-5.1](https://z.ai/blog/glm-5.1)、[GLM-5](https://z.ai/blog/glm-5)、[GLM-4.7](https://z.ai/blog/glm-4.7)、[GLM-4.6](https://z.ai/blog/glm-4.6)、[GLM-4.5](https://z.ai/blog/glm-4.5) 背后的 RL 训练框架。这验证的是完整 post-training loop，而不是孤立 example。
 - **以正确性为先的基础设施**：RL bug 往往不会立刻报错。slime 保持显式的数据流，支持 rollout-only 和 train-only 分离调试，并把可复现性、容错、trace、profiling 和 CI 作为一等工程问题来维护。
 - **从设计开始就是 native**：slime 直接透传 Megatron 参数，并通过 `--sglang-` 前缀暴露当前安装版本 SGLang 支持的参数。新的上游训练和 serving 优化可以直接使用，不需要在 slime 里再加一层抽象。
 - **最大化的数据生成自由度**：math、code、search、tool、sandbox、verifier、environment、multi-agent system 以及 long-horizon agentic workflow 都可以作为 data generation 或 reward workflow 接入，而不需要 fork training kernel。

docs/en/examples/glm5.2-744B-A40B.md

@@ -0,0 +1,177 @@
+# GLM-5.2 744B-A40B with 256xH100
+
+This is the recommended 32-node, 256-H100 training example for [GLM-5.2](https://z.ai/blog/glm-5.2).
+
+The recipe uses the GLM-5.2 BF16 checkpoint for Megatron training and the FP8 checkpoint for SGLang rollout. It assumes two Hugging Face repositories will be available:
+
+- BF16: `zai-org/GLM-5.2`
+- FP8: `zai-org/GLM-5.2-FP8`
+
+## Environment Setup
+
+For environment setup and dataset download, see [Example: Qwen3-4B](qwen3-4B.md). For multi-node training, make sure every node can access the same `$BASE_DIR` path.
+
+### Download Model
+
+```bash
+hf download zai-org/GLM-5.2 --local-dir $BASE_DIR/GLM-5.2
+hf download zai-org/GLM-5.2-FP8 --local-dir $BASE_DIR/GLM-5.2-FP8
+```
+
+The open-source GLM-5.2 config uses `model_type: glm_moe_dsa`, which slime maps onto
+the DeepSeek-V3.2 bridge (`slime_plugins.mbridge.deepseek_v32`) since the two share the
+same DSA weight layout.
+
+### Convert Checkpoint
+
+The training side needs the BF16 Hugging Face checkpoint converted to the Megatron torch_dist format. The torch_dist format is reshardable, so the conversion parallel layout does **not** need to match training; we use a layout that satisfies Megatron's expert-group constraint on the conversion node count.
+
+Run the following on 4 nodes / 32 GPUs:
+
+```bash
+cd /root/slime
+pip install -e . --no-deps
+source scripts/models/glm5.2-744B-A40B.sh
+PYTHONPATH=/root/Megatron-LM/ torchrun \
+   --nproc-per-node 8 \
+   --master-addr ${MASTER_ADDR} --master-port 12345 \
+   --nnodes=4 --node-rank ${NODE_RANK} \
+   tools/convert_hf_to_torch_dist.py \
+   ${MODEL_ARGS[@]} \
+   --tensor-model-parallel-size 8 \
+   --pipeline-model-parallel-size 2 \
+   --decoder-last-pipeline-num-layers 40 \
+   --expert-model-parallel-size 16 \
+   --expert-tensor-parallel-size 1 \
+   --hf-checkpoint $BASE_DIR/GLM-5.2/ \
+   --save $BASE_DIR/GLM-5.2_torch_dist/
+```
+
+Here, `MASTER_ADDR` is the IP of node0, and `NODE_RANK` is the current node index.
+
+`MODEL_ARGS` includes `--allgather-cp`, a slime-only flag, so `tools/convert_hf_to_torch_dist.py` registers it too (it is a no-op for conversion). On 32 GPUs, Megatron requires `expert_tp(1) * expert_model_parallel * pp` to divide the world size, so we convert with `EP=16` (`1*16*2=32`). The resulting checkpoint still loads at training-time `EP=32` because torch_dist is reshardable.
+
+## Run Training
+
+From node0:
+
+```bash
+cd /root/slime
+export BASE_DIR=/shared/path
+export MASTER_ADDR=<node0-ip>
+export HOSTFILE=$BASE_DIR/hostfile  # one worker IP per line, all 32 nodes
+bash scripts/run-glm5.2-744B-A40B.sh
+```
+
+If `HOSTFILE` is not set, join the other nodes to the Ray cluster manually.
+
+### Parameter Introduction
+
+#### Model Configuration
+
+`scripts/models/glm5.2-744B-A40B.sh` contains the GLM-5.2 DSA + cross-layer index sharing configuration: 256 routed experts, top-8 activation, 1 shared expert, and 78 layers total (3 dense + 75 MoE).
+
+The DSA index sharing schedule, such as `index_topk_freq=4` and `index_skip_topk_offset=3`, is read from the Hugging Face config. The Megatron side uses the shared `slime_plugins.models.glm5.glm5:get_glm5_spec` provider and enables:
+
+```bash
+--allgather-cp
+```
+
+This makes DSA + context parallel use the allgather-CP layout, and the index-share provider gathers index K/V across the CP group.
+
+#### Training Parallelism
+
+The default script targets 32 nodes and 256 GPUs:
+
+```bash
+PERF_ARGS=(
+   --tensor-model-parallel-size 4
+   --pipeline-model-parallel-size 8
+   --decoder-first-pipeline-num-layers 14
+   --decoder-last-pipeline-num-layers 16
+   --context-parallel-size 8
+   --expert-model-parallel-size 32
+   --expert-tensor-parallel-size 1
+   ...
+)
+```
+
+`TP=4 * PP=8 * CP=8 = 256` GPUs form one training group (`DP=1`). The expert group constraint `expert_tp(1) * EP(32) * PP(8) = 256` divides the world size exactly (`expert_dp=1`).
+
+DSA cross-layer index sharing requires every pipeline stage to **start** on a "computing" layer. With `index_topk_freq=4` / `index_skip_topk_offset=3`, the computing layers are 1, 2, 3, 7, 11, ..., 75. A uniform `78/8` split would start stages on skip layers and fail the index-share assertion in `get_glm5_spec`. We therefore use `--decoder-first-pipeline-num-layers 14` and `--decoder-last-pipeline-num-layers 16`, leaving 6 middle stages of `(78-14-16)/6 = 8` layers each. The stage starts land on global layers 1, 15, 23, 31, 39, 47, 55, 63 — all computing layers.
+
+#### BF16 Training + FP8 Rollout
+
+The launcher writes the default paths directly in `CKPT_ARGS` and `ROLLOUT_ARGS`, matching the style of the other example scripts:
+
+```bash
+CKPT_ARGS=(
+   --hf-checkpoint $BASE_DIR/GLM-5.2-FP8
+   --ref-load $BASE_DIR/GLM-5.2_torch_dist
+   --load $BASE_DIR/GLM-5.2_slime
+   --save $BASE_DIR/GLM-5.2_slime
+   --save-interval 20
+)
+
+ROLLOUT_ARGS=(
+   --prompt-data $BASE_DIR/dapo-math-17k/dapo-math-17k.jsonl
+   ...
+)
+```
+
+`--hf-checkpoint` provides FP8 weights and the tokenizer for SGLang rollout; `--ref-load` is the Megatron torch_dist checkpoint converted from BF16. To debug BF16 rollout, change `--hf-checkpoint` in the script to `$BASE_DIR/GLM-5.2`.
+
+#### SGLang Configuration
+
+The rollout side runs with **prefill/decode (PD) disaggregation**: 1 prefill engine (64 GPU) + 3 decode engines (192 GPU) = 256 GPUs total (which must equal the colocated `rollout_num_gpus`). Each engine spans 64 GPUs with DP attention and `EP=64` (DeepEP's dispatch config map supports up to 160 EP ranks, so a single 256-GPU engine would be invalid). Prefill uses the `auto` DeepEP path; decode uses `low_latency` + `deep_gemm`. The split is configured via the `--sglang-config` YAML:
+
+```yaml
+sglang:
+  - name: default
+    server_groups:
+      - worker_type: prefill
+        num_gpus: 64
+        num_gpus_per_engine: 64
+        overrides: { deepep_mode: auto, ... }
+      - worker_type: decode
+        num_gpus: 192
+        num_gpus_per_engine: 64
+        overrides: { deepep_mode: low_latency, moe_runner_backend: deep_gemm, ... }
+```
+
+PD transfer runs over RDMA/IB with the mooncake backend:
+
+```bash
+--sglang-disaggregation-transfer-backend mooncake
+--sglang-disaggregation-ib-device mlx5_100,...,mlx5_107
+```
+
+The rest of the rollout uses FP8 KV cache and the NSA + DeepEP backends:
+
+```bash
+SGLANG_ARGS=(
+   --sglang-enable-dp-attention
+   --sglang-ep-size 64
+   --sglang-dp-size 64
+   --sglang-kv-cache-dtype fp8_e4m3
+   --sglang-nsa-decode-backend flashmla_kv
+   --sglang-nsa-prefill-backend flashmla_sparse
+   --sglang-attention-backend nsa
+   ...
+)
+```
+
+MTP / EAGLE speculative decoding is enabled using the model's own next-token-prediction layer (the GLM-5.2 checkpoint ships an MTP layer), so no separate draft model is needed:
+
+```bash
+--sglang-speculative-algorithm EAGLE
+--sglang-speculative-num-steps 4
+--sglang-speculative-eagle-topk 1
+--sglang-speculative-num-draft-tokens 5
+```
+
+`SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK` must cover the largest decode batch: `max cuda_graph_max_bs (decode group = 12) * speculative_num_draft_tokens (5) = 60`, rounded up to `64`. A value below this trips the DeepEP low-latency dispatch buffer assertion during the decode group's CUDA-graph capture.
+
+#### Networking
+
+DeepEP/NVSHMEM communication across nodes needs the IB-aware NCCL settings in the Ray runtime env (`NCCL_SOCKET_IFNAME`, `NCCL_IB_*`, `NCCL_NET_GDR_LEVEL`, `NCCL_P2P_LEVEL=NVL`, `NCCL_NVLS_ENABLE=0`, `MC_IB_PCI_RELAXED_ORDERING`, ...). The script defaults to `SOCKET_IFNAME=eth0`; set `SOCKET_IFNAME` before launch if your environment differs, and it will be written to `GLOO_SOCKET_IFNAME`, `TP_SOCKET_IFNAME`, and `NCCL_SOCKET_IFNAME`. DeepEP also requires `NVSHMEM_DISABLE_NCCL=1`.

docs/en/get_started/quick_start.md

@@ -587,6 +587,7 @@ export NVSHMEM_BOOTSTRAP_UID_SOCK_IFNAME=$(ip -o -4 addr show | awk '$4 ~ /^10\.
 slime has been deeply optimized for distributed training of large-scale Mixture of Experts (MoE) models. We provide some end-to-end training cases for reference:
 
 - [Example: 8xH100 Training GLM-4.7-Flash](../examples/glm4.7-30B-A3B.md)
+- [Example: 32xH100 Training GLM-5.2](../examples/glm5.2-744B-A40B.md)
 - [Example: 64xH100 Training GLM-4.7](../examples/glm4.7-355B-A32B.md)
 - [Example: 128xH100 Training DeepSeek-R1](../examples/deepseek-r1.md)
 - Scripts such as `scripts/run_qwen3_30b_a3b.py` and `scripts/run_glm45_355b_a32b.py` also support multi-node training. Their documentation is still being expanded.

docs/en/index.rst

@@ -13,7 +13,7 @@ This makes slime one of the most battle-tested open RL post-training frameworks:
 Why This Design Matters
 -----------------------
 
-- **Battle-tested by frontier model training**: slime is the RL framework behind `GLM-5.1 <https://z.ai/blog/glm-5.1>`_, `GLM-5 <https://z.ai/blog/glm-5>`_, `GLM-4.7 <https://z.ai/blog/glm-4.7>`_, `GLM-4.6 <https://z.ai/blog/glm-4.6>`_, and `GLM-4.5 <https://z.ai/blog/glm-4.5>`_.
+- **Battle-tested by frontier model training**: slime is the RL framework behind `GLM-5.2 <https://z.ai/blog/glm-5.2>`_, `GLM-5.1 <https://z.ai/blog/glm-5.1>`_, `GLM-5 <https://z.ai/blog/glm-5>`_, `GLM-4.7 <https://z.ai/blog/glm-4.7>`_, `GLM-4.6 <https://z.ai/blog/glm-4.6>`_, and `GLM-4.5 <https://z.ai/blog/glm-4.5>`_.
 - **Native by design**: slime passes Megatron arguments through directly and exposes installed SGLang arguments with a ``--sglang-`` prefix, so upstream training and serving optimizations remain available without adding another wrapper layer.
 - **SGLang-focused rollout**: slime chooses one rollout backend intentionally. This avoids flattening multiple inference engines into a lowest-common-denominator abstraction and lets RL workloads use SGLang-specific serving, routing, caching, disaggregation, and weight-sync behavior directly.
 - **Agentic workflows as data generation**: tool use, sandbox interaction, verifier rewards, environment feedback, multi-agent loops, and long-horizon agentic workflows plug into the same training / rollout / Data Buffer path instead of forking the training kernel.
@@ -67,6 +67,7 @@ Start by Use Case
 
    examples/glm4.7-30B-A3B.md
    examples/qwen3-30B-A3B.md
+   examples/glm5.2-744B-A40B.md
    examples/glm4.7-355B-A32B.md
    examples/deepseek-r1.md