Migrate Qwen3 dense and MoE models to HybridModel

examples/model_verification_cards/qwen3-30b-a3b/card.yaml

@@ -98,33 +98,34 @@ items:
   manual_forward_pass:
     status: verified
     precision: bf16
-    bridge_commit: 24f3dcde1fe27a1c3f1097b94d69b38e37a57415  # pragma: allowlist secret
+    bridge_commit: 83caabba49ac32f76d8e365a36926366c6940001  # pragma: allowlist secret
     command: >
       uv run python -m torch.distributed.run --standalone --nproc_per_node=8
       examples/conversion/compare_hf_and_megatron/compare.py
       --hf_model_path Qwen/Qwen3-30B-A3B
       --hf-revision ad44e777bcd18fa416d9da3bd8f70d33ebb85d39
       --megatron_model_path work/model-verification/qwen3-30b-a3b/imported-megatron/iter_0000000
       --tp 1 --pp 8 --prompt "The capital of France is the city of"
-    last_verified: 2026-07-20
+    last_verified: 2026-07-26
     expected_result: >
       The pinned-revision one-step comparison exits successfully; the Hugging
       Face and Megatron next-token predictions match at token ID 12095
-      (" Paris"), and cosine similarity is 0.999317, above the 0.99 correlation
-      gate. The maximum and mean absolute logit differences are 0.593750 and
-      0.130064, respectively; both are report-only diagnostic observations.
+      (" Paris"), and cosine similarity is 0.999725, above the 0.99 correlation
+      gate. The maximum and mean absolute logit differences are 0.390625 and
+      0.067908, respectively; both are report-only diagnostic observations.
 
   inference:
     status: verified
     precision: bf16
+    bridge_commit: 83caabba49ac32f76d8e365a36926366c6940001  # pragma: allowlist secret
     command: >
       uv run python -m torch.distributed.run --standalone --nproc_per_node=8
       examples/conversion/hf_to_megatron_generate_text.py
       --hf_model_path Qwen/Qwen3-30B-A3B
       --megatron_model_path work/model-verification/qwen3-30b-a3b/imported-megatron/iter_0000000
       --tp 4 --pp 2 --ep 4
       --prompt "The capital of France is" --max_new_tokens 32
-    last_verified: 2026-07-17
+    last_verified: 2026-07-26
     expected_result: >
       Two independent executions exit successfully after exactly 32 new tokens
       and print this byte-identical completion, including its leading
@@ -136,7 +137,7 @@ items:
     H100:
       status: verified
       precision: bf16
-      bridge_commit: 5b9d9cf501193277e1ca47a99b97c640b8f39f90  # pragma: allowlist secret
+      bridge_commit: 83caabba49ac32f76d8e365a36926366c6940001  # pragma: allowlist secret
       enabled_features:
         cuda_graph:
           implementation: transformer_engine
@@ -161,18 +162,18 @@ items:
         --save_dir work/model-verification/qwen3-30b-a3b/pretrain-convergence-v1-reference-checkpoints
         --save_interval 50 logger.log_interval=1 logger.log_throughput=true
         logger.tensorboard_dir=null
-      last_verified: 2026-07-19
+      last_verified: 2026-07-26
       metrics:
-        initial_loss: 12.41145
-        final_loss: 6.139116
-        last_10_steps_step_time_ms_avg: 30289.550
-        last_10_steps_model_tflops_per_gpu_avg: 199.120
+        initial_loss: 12.40963
+        final_loss: 6.063886
+        last_10_steps_step_time_ms_avg: 31601.840
+        last_10_steps_model_tflops_per_gpu_avg: 190.840
       expected_result: >
         On 16x H100, the public alias resolves to the 16-GPU recipe and completes
         exactly 100 bounded RP2 optimizer steps with TP1/PP1/CP1/EP16/ETP1,
         DP16, SP off, GBS/MBS 1024/1, and 64-way gradient accumulation. Natural
         routing, HybridEP, and Transformer Engine CUDA graphs for moe_router and
-        moe_preprocess remain active. Loss is finite from 12.41145 to 6.139116
+        moe_preprocess remain active. Loss is finite from 12.40963 to 6.063886
         with no skipped or NaN iterations, all four metrics are recorded, and
         complete iter_0000050 and iter_0000100 checkpoints are saved.
 
@@ -259,7 +260,7 @@ items:
     H100:
       status: verified
       precision: bf16
-      bridge_commit: f3ae2767b5e18aeb67b726cd8d5f1db58216dcc9  # pragma: allowlist secret
+      bridge_commit: 83caabba49ac32f76d8e365a36926366c6940001  # pragma: allowlist secret
       enabled_features:
         sequence_packing: offline
         context_parallel_size: 2
@@ -285,26 +286,26 @@ items:
         scheduler.lr_decay_iters=20 validation.eval_iters=0 validation.eval_interval=0
         checkpoint.load=null checkpoint.save=null
         logger.log_interval=1 logger.log_throughput=true
-      last_verified: 2026-07-20
+      last_verified: 2026-07-26
       metrics:
-        initial_loss: 1.645009
-        final_loss: 1.468103
-        last_10_steps_step_time_ms_avg: 142663.710
-        last_10_steps_model_tflops_per_gpu_avg: 26.120
+        initial_loss: 1.645200
+        final_loss: 1.468164
+        last_10_steps_step_time_ms_avg: 37263.530
+        last_10_steps_model_tflops_per_gpu_avg: 100.040
       expected_result: >
         The immutable-revision 16-GPU run completes exactly 20 Tulu3 SFT steps at
         sequence length 32768 with TP8/PP1/CP2/EP8/SP-on, DeepEP, and explicit
-        pad-16 offline packing. LM loss is 1.645009 to 1.468103; skipped/NaN
+        pad-16 offline packing. LM loss is 1.645200 to 1.468164; skipped/NaN
         totals are 0/0. The persisted post-setup runtime config matches the
         command, packing is 99.28%, and the sampled training window contains
-        13,573,663 actual supervised tokens. PP=1 keeps tokens, labels, loss
+        13,644,614 actual supervised tokens. PP=1 keeps tokens, labels, loss
         masks, and packed-sequence boundaries on one pipeline stage.
 
   peft:
     H100:
       status: verified
       precision: bf16
-      bridge_commit: 619cc20bd3c7eca1dc84e8ea0f822307dfbd5cc3  # pragma: allowlist secret
+      bridge_commit: 83caabba49ac32f76d8e365a36926366c6940001  # pragma: allowlist secret
       enabled_features:
         sequence_packing: offline
         moe_dispatcher: deepep
@@ -330,28 +331,28 @@ items:
         --save_dir work/model-verification/qwen3-30b-a3b/peft-tp4-ep4-pad4-checkpoints
         --save_interval 100
         logger.log_interval=1 logger.log_throughput=true
-      last_verified: 2026-07-21
+      last_verified: 2026-07-26
       metrics:
-        initial_loss: 1.575987
-        final_loss: 1.113119
-        last_10_steps_step_time_ms_avg: 22347.640
-        last_10_steps_model_tflops_per_gpu_avg: 13.840
+        initial_loss: 1.575093
+        final_loss: 1.114007
+        last_10_steps_step_time_ms_avg: 16181.250
+        last_10_steps_model_tflops_per_gpu_avg: 19.120
       expected_result: >
         The immutable-revision 4-GPU run completes exactly 100 PEFT steps at
         TP4/PP1/CP1/EP4/ETP1, DP1, SP on, GBS/MBS 32/1, and 32-way gradient
         accumulation with natural routing and DeepEP. Pad-4 offline packing is
         99.39% efficient, and the sampled 6,553,600 token slots contain
         4,332,480 supervised tokens after label masking. Only rank-8, alpha-16,
         zero-dropout LoRA on linear_qkv and linear_proj is trainable. LM loss is
-        finite from 1.575987 to 1.113119 with no skipped or NaN iterations, all
+        finite from 1.575093 to 1.114007 with no skipped or NaN iterations, all
         four metrics are recorded, and the complete four-shard iter_0000100
         adapter checkpoint covers all 192 expected adapter entries.
 
   checkpoint_resume:
     H100:
       status: verified
       precision: bf16
-      bridge_commit: 5b9d9cf501193277e1ca47a99b97c640b8f39f90  # pragma: allowlist secret
+      bridge_commit: 83caabba49ac32f76d8e365a36926366c6940001  # pragma: allowlist secret
       depends_on: pretrain
       command: >
         ./scripts/training/train.sh --nodes 2 --gpus-per-node 8
@@ -375,12 +376,12 @@ items:
         train.empty_unused_memory_level=2
         logger.log_interval=1 logger.log_throughput=true
         logger.tensorboard_dir=null
-      last_verified: 2026-07-19
+      last_verified: 2026-07-26
       metrics:
-        initial_loss: 6.989006
-        final_loss: 6.145390
-        last_10_steps_step_time_ms_avg: 31142.650
-        last_10_steps_model_tflops_per_gpu_avg: 193.640
+        initial_loss: 6.704744
+        final_loss: 6.060962
+        last_10_steps_step_time_ms_avg: 34594.660
+        last_10_steps_model_tflops_per_gpu_avg: 174.340
       resume_comparison:
         reference_item: pretrain
         sentinel_steps: [51, 100]
@@ -392,8 +393,8 @@ items:
         iter_0000050, begins at step 51, and finishes at step 100 in the distinct
         resumed root with finite losses and no skipped or NaN iterations. Releasing
         unused cached memory after optimizer steps is execution-only. Step-51 loss
-        6.989006 matches the uninterrupted reference exactly; step-100 loss
-        6.145390 differs from reference 6.139116 by 0.102197%, within the declared
+        6.704744 matches the uninterrupted reference exactly; step-100 loss
+        6.060962 differs from reference 6.063886 by 0.048220%, within the declared
         one-percent gate, so both sentinels match and all four metrics are recorded.
 
   pretrain_performance: