Skip to content

cookbook: add NVFP4 4/6 recipe using upstream Miles - #90

Merged
jvmncs merged 1 commit into
mainfrom
nvfp4-46-recipe-upstream-miles
Jul 30, 2026
Merged

cookbook: add NVFP4 4/6 recipe using upstream Miles#90
jvmncs merged 1 commit into
mainfrom
nvfp4-46-recipe-upstream-miles

Conversation

@jvmncs

@jvmncs jvmncs commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Supersedes #22 now that the upstream Miles NVFP4 work has merged and is included in Stitch's shared Miles pin.

  • add the Qwen3-30B-A3B NVFP4 4/6 cookbook recipe on current main
  • rely on the shared Miles, TransformerEngine, SGLang, and FlashInfer versions instead of recipe-specific repository pins, installs, and patches
  • propagate the recipe's PREP_ENV into checkpoint conversion so the served base and trainer export use the same quantizer contract
  • retain FlashAttention for trainer attention after reproducing CUDNN_STATUS_BAD_PARAM in the cuDNN fused-attention backward path

The shared Miles pin 1eb7520018446cb94b7406715f66dff1a271b53b was verified to descend from the upstream NVFP4 merge commit 1fca8f8b6f023a76f984f0fdc7d8c2c44bb09d4c (radixark/miles#1261).

Verification

  • uv run pytest — 84 passed
  • Ruff checks pass for all changed Python files
  • Modal stack probe confirmed the pinned Miles ancestry, TransformerEngine 2.17 dequantized-backward fix, FlashInfer 0.6.14, and usable NVFP4 conversion CLI
  • Modal checkpoint prep completed for Qwen3-30B-A3B: 61 GB BF16 masters and all 16 NVFP4 shards
  • one-rollout Modal smoke completed on 8x B200: rollout generation, reference/current logprobs, backward, training step 0, disk-delta v1 publication, and serving reconciliation to IDLE

The short smoke's samples all received zero reward, so v1 contained an empty delta and the serving side correctly skipped the physical reload after reconciling the version pointer.

@jvmncs
jvmncs force-pushed the nvfp4-46-recipe-upstream-miles branch from ac88656 to bea24ef Compare July 30, 2026 22:08
@jvmncs
jvmncs merged commit 19ec2db into main Jul 30, 2026
@nanjiangwill
nanjiangwill deleted the nvfp4-46-recipe-upstream-miles branch August 2, 2026 03:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant