Skip to content

Add GLM-5.3-Flash Miles FFT recipe - #645

Draft
zhouhelena1 wants to merge 3 commits into
mainfrom
helena/glm53flash-fft
Draft

zhouhelena1 wants to merge 3 commits into
mainfrom
helena/glm53flash-fft

Conversation

@zhouhelena1

Copy link
Copy Markdown
Contributor

Adds a GLM-5.3-Flash full-parameter GRPO recipe for Miles, isolated from the bridge LoRA onboarding in #639. This follows upstream Miles (radixark/miles#2786), using its 32-GPU full-model configuration:

Recipe

  • Model: zai-org/GLM-5.3-Flash, full-parameter training through Megatron and raw HF-to-torch_dist conversion.
  • Cluster: 32 B300 GPUs on four nodes of eight; training TP8 / PP4 / EP8 / CP1, sequence parallelism, first/last pipeline stages with 11/12 layers. Rollout engines use eight GPUs each with TP8 / EP8.
  • Sampling: four prompts with eight responses each, global batch 32, temperature 0.8, math reward, and a 4,096-token response cap. The dataset is supplied separately through TrainConfig; upstream uses DAPO math.
  • Training: GRPO, Adam at 1e-6, constant learning rate, weight decay 0.1, betas 0.9/0.98, clipping 0.2/0.28, and zero KL/entropy coefficients. Microbatch 1, static batching, full activation recomputation, and an 8,192-token per-GPU training budget.
  • Memory/serving: colocated training and rollout, trainer disk offload, SGLang memory fraction 0.7, Triton MoE, TileLang DSA, and BF16 KV cache. This recipe does not include the LoRA recipe's 32K configuration.

Differences from upstream Miles

  • The logical 32-GPU parallelism matches, but Modal uses 4 × 8 B300s, while upstream's example uses 8 × 4 GPUs. Physical communication placement differs.
  • Pins the glm53next image by digest and Miles to cc76e23915b2132ecfff65b97331fd83b02637e6.
  • Adds Modal checkpoint/data paths, automatic conversion at TP8 / PP1 / EP8, local disk reservations, and a longer initial rollout health-check grace period.
  • Applies three model-specific patches: KDA/Triton compatibility, FP8 checkpoint loading onto each rank's GPU, and Dojo timing instrumentation.
  • Explicitly enables the distributed optimizer. The recipe defaults to one rollout rather than upstream's five; callers choose the training horizon. Checkpoint flags omit optimizer/RNG state, so saved weights do not provide an exact optimizer-state resume.

Validation

No completed FFT training validation run yet. This PR is a draft.

The attempted 10-step FFT run (chewy-bag) failed while writing the converted checkpoint, before any training steps started (inline_container.cc: unexpected pos). End-to-end training correctness, stability, and performance remain unvalidated. Successful LoRA runs do not validate this FFT recipe.

Local checks previously passed: 128 existing registry/Miles tests, recipe import and topology checks, compilation, lint, and models-table freshness. No new tests or model Markdown files are included.

@github-actions

github-actions Bot commented Oct 3, 2026

Copy link
Copy Markdown

This branch is waiting to be deployed

1 waiting deployment
modal — cd4c543a Waiting Oct 3, 2026 by zhouhelena1 via validate (Moonlight-16B-A3B-Instruct) #2709
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant