Arm launch step timeout from the arms' readiness deadline; NVFP4 skips the rank-asymmetric fused_moe autotune

[gunbai-bot]

Follow-up to #13040 (GLM Group A staircase), for the NVFP4 diagnostic launch.

Launch-step timeout tied to the arms' readiness deadline

• The arm launch step's timeout-minutes was a literal 60, below the 3600 s front-door wait it encloses. Run 37190290418 was killed between apply and verdict, leaving neither a commit nor a rollback.
• arm_launch_step_needed_minutes() computes the needed time: the longest arm VLLM_ENGINE_READY_TIMEOUT_S, read by the front door's own engine_ready_timeout_for, plus 30 minutes for apply and 20 for rollback.
• The workflow renders the Duration literal arm_launch_step_timeout_minutes = 110. Witness w_the_launch_step_timeout_covers_every_arms_readiness checks it is at least the derivation.
• It is a literal only because the refinement-cast wall refuses a computed Int-to-Duration cast. The named trigger is a declared conversion.

NVFP4 arm: VLLM_FLASHINFER_AUTOTUNE_SKIP_OPS=trtllm::fused_moe::gemm1,trtllm::fused_moe::gemm2

• New extdeps.vllm.environment row, citing the pinned image's sources. MoERunner.get_cache_key_extras keys tactics by tp/ep/cluster rank, and the sparse-MLA pre-tune runs on the leader only.
• Without the variable, the synchronized autotune deadlocks the non-leader ranks.
• Cost, stated on the row: rank 0 runs its tuned tactic and ranks 1-3 run the heuristic one, so NVFP4 throughput from this arm is a lower bound.
• Failure mode filed: a_leader_only_pre_tune_seeds_a_rank_keyed_cache.

Witnesses: group_arm_launch 20/20, workflow_dispatch_input 42/42, run remotely at an exact SHA.

🤖 Generated with Claude Code

[gunbai-bot]

Review 76352, string-assembled shell. Two parts, ruled by valiant-crab-775:

1. Deliberate start: fixed in 0c72892. The start now has one realization, the typed HostOperation legs in gunbc.spark.deliberate_unit_start deliberate_unit_start_leg. The apply and rollback no longer spell it in shell. Each is split into a write/prepare script, the typed start leg, and a wait/finish script (spark_native_apply_sequence, spark_native_rollback_sequence), and the shell rendering is deleted. deliberate_unit_start_witness_test w_no_stage_script_spells_a_start pins it.

2. The incarnation poll loop stays under the existing feature:remote-shell-body-emission marker. Its trigger is General orchestration intent to Bash emit fold over grammar rows #5828.

   • Its missing carrier is a real language capability: there is no modeled sleep or poll-until step, which is exactly the bounded remote operation sequence General orchestration intent to Bash emit fold over grammar rows #5828 tracks.
   • Moving the loop executor-side, so each poll is a typed leg, is not a substitute. It would put the deadline on the executor's clock. That undoes the side-chat ruling's finding 1, which anchors the one readiness deadline to the incarnation's own StartedAt on the host that runs it, so no clock is compared across hosts.
   • When General orchestration intent to Bash emit fold over grammar rows #5828 lands, the loop becomes a projection of that sequence with the same anchored deadline.

— sent from cool-carp-342