lora: GLM-5.1 + 5.2 (MoE+MLA+DSA) LoRA training — bridge support, launcher, registries, CI - #1373

Closed

yushengsu-thu wants to merge 54 commits into

radixark:mainfrom

yushengsu-thu:miles-dev-2026-06-16

Commits on Jun 18, 2026

lora: map DSA indexer modules across sglang↔megatron-bridge names
yushengsu-thu
committed
12e4e88
lora: include MLA + DSA indexer modules in all-linear expansion
yushengsu-thu
committed
7e5de94
lora: register dsa experimental-attention-variant spec for bridge LoRA (GLM-5.1)
yushengsu-thu
committed
cb84455

Commits on Jun 19, 2026

Commits on Jun 20, 2026

Commits on Jun 22, 2026

Commits on Jun 23, 2026

Commits on Jun 24, 2026

Commits on Jun 25, 2026

Commits on Jun 26, 2026

lora rl: add periodic held-out eval (interval default 5)

yushengsu-thu
and
claude
committed
c5051fe
fix: window eval submission to avoid dp-attention colocate hang

yushengsu-thu
and
claude
committed
c378269
Revert "fix: window eval submission to avoid dp-attention colocate hang"

yushengsu-thu
and
claude
committed
d4f0431
add NVSHMEM_DISABLE_NCCL to avoid cuda graph hang

yueming-yuan
authored and
yushengsu-thu
committed
ff8b4fe
lora rl: enable MoE/MLP LoRA by default + select per-expert vs shared-outer grouped-expert adapter
yushengsu-thu
committed
d79e27c

Commits on Jun 27, 2026

Commits on Jun 28, 2026

Commits on Jun 29, 2026

Revert "lora rl: add periodic held-out eval (interval default 5)"
yushengsu-thu
committed
f402c4d
glm5 lora: add launch_glm_rl.sh (full GLM-5.2 attention-only LoRA-RL launcher)

yushengsu-thu
and
claude
committed
d888950
glm5 lora: one-command launch_glm_rl.sh + pre-run cleanup + 4x16 batch

yushengsu-thu
and
claude
committed
7d75076
glm5 lora: enable MoE-expert LoRA (gate/up/down) + lora-rank 4

yushengsu-thu
and
claude
committed
b14acdc
glm5 lora: lora-rank 8 (rank 4 invalid under TP=8 / moe_router_topk=8)

yushengsu-thu
and
claude
committed
1d93245
glm5 lora: fix step-0 OOM — MoE LoRA last-N layers + NVSHMEM_DISABLE_NCCL=0

yushengsu-thu
and
claude
committed
60b746f
glm5 lora: bump --num-rollout to 10000 for a long MoE-LoRA RL run

yushengsu-thu
and
claude
committed
1ce9f53

Commits on Jun 30, 2026

Commits on Jul 1, 2026

[launch] split launcher: rename to att_fused_moe; add att_unfused_moe (per-expert MoE LoRA, TOY=1 smoke)

yushengsu-thu
and
claude
committed
815b691