[pranavraja99]

Summary

Runs gpu02's long-context r2 at --prefill-decode-interval 2 (two decode steps between prefill chunks). r1 stays exactly #294 at pdi 1 and serves as a live control. The two replicas' config_variant differs only in pdi1/pdi2, so prod dashboards can compare them directly.

Why: under long prefills, W4AFP8's larger KV pool keeps about 3 sequences decoding while the next ~700K-token prefill runs. At pdi 1, each of their decode steps waits behind a prefill chunk. That is most of the long tier's loaded-TPOT penalty.

Evidence (gpu31, stored long-tier traffic, 2026-09-24)

• pdi 2 vs pdi 1, paired, both island orders: TPOT p50 0.53–0.72×, paired TTFT ≤ 1.02×, 100% completion.

• Ship qualification of the full candidate (pdi 2 + the DSA indexer split from the companion PR + chunk 16384) vs GLM-5.3 Flash: W4AFP8 + HiCache long-context file for gpu02 (both replicas = campaign-2 L2) #294, on the same GPUs:

  Workload	TTFT ratio	TPOT ratio
  Burst, 8 × ~720K	0.76	0.72–0.76
  Burst + short-request hammer	0.77–0.78	0.78–0.85
  long-a2	0.61–0.69	0.53–0.56

  Both arms: 0 restarts, 0 OOMs, 0 fatal log lines. Quality at the noise floor (GSM8K, MMLU, passkey, teacher-forced agreement).

Write-up: Notion

Changes

• Generator: sets the interval per replica; emits per-replica config_variant.
• Validator: pins each replica's argv and variant.
• Tests: a new case rejects either replica taking the other's pdi.
• Rollout doc: new "Canary" section covering the r2-only compose/down then compose/up, a dry-run gate that must plan exactly r2, the r1-vs-r2 readout over 24 h+, and rollback.

Validation

• prepare_glm53_w4afp8_long_context.py --check passes.
• test_glm53_w4afp8_long_context.py: 10/10 pass under Ruby 3.3. Local Ruby 2.6 lacks filter_map.
• All scripts/validate_*.rb pass.

Notes

• Deployment: operators do it. It needs a recreate of r2 only; r1, the proxy and the registrar are untouched.
• Conflicts with Arm opt-in SGLang tracing on GLM-5.3 long tier #296 (tracing): both edit the variant string and the validator argv. Whichever lands second needs a simple rebase.

🤖 Generated with Claude Code

[github-actions]

✅ OpenCodeReview: Review failed: 0 finding(s); 3 of 3 selected item(s) failed.

[lloydmak99]

Canary move of r2 to --prefill-decode-interval 2 while keeping r1 at 1 as a live control. The generator, generated YAML, validator, and tests are internally consistent: r1's command stays byte-identical to the pre-canary config (pdi 1), r2 correctly overrides to pdi 2, and the per-replica divergence in the command/labels/deploy fields is properly excluded from the validator's shared-runtime check, so nothing new trips. No blocking issues found.

Checks: git diff --check passed; prepare_glm53_w4afp8_long_context.py --check exited 0 (generated file not stale); GeneratedFileTest 4/4 passed; grep confirmed r1=pdi1 / r2=pdi2 anchors and 3×pdi1 + 3×pdi2 config_variant labels; worktree clean. The Ruby validator and ValidatorContractTest couldn't run (no ruby in this environment) and were verified by inspection — author reports 10/10 pass under Ruby 3.3.