[pranavraja99]

Third and final PR of the long-tier chain: #299 (pdi 2) and #300 (the opt-in split patch) are merged, the glm53-hicache-w4afp8-v2 image is published, and this pins it on r2 only.

What changes, and what deliberately does not

	r1 (control)	r2 (canary)
image	fde25985aea3 (#294) — unchanged	8ff1a487b98a (v2, from #300)
chunk	8192 — unchanged	16384
SGLANG_DSA_INDEXER_QSPLIT	unset — unchanged	1
pdi	1 — unchanged	2 (from #299)

r1's service definition and the shared engine anchor are byte-identical to origin/main; the only other changed path is r2's own scrape job. A targeted compose up therefore recreates r2 alone, and r1 keeps serving throughout.

The structural bit worth reviewing

r2 now needs its own image and environment. A YAML merge key replaces a list wholesale rather than extending it, so r2 cannot inherit the anchor's environment and add one variable. The generator derives r2's block from the anchor at generation time instead of duplicating it as a literal, so the two cannot drift apart.

The chunk and the split are one change

At 16384 without the split, a concurrent long burst left 0.04–0.65 GB free per GPU — the condition that preceded the gpu02 crash. With it, 9.3–9.5 GB, clearing the ≥3 GB gate.

The validator now enforces the pairing in both directions:

• --chunked-prefill-size 16384 on a replica without SGLANG_DSA_INDEXER_QSPLIT=1 → rejected
• SGLANG_DSA_INDEXER_QSPLIT=1 on an image that does not carry the patch → rejected

Both were verified by mutation, including the case where the expected spec itself is edited so every equality check passes and only the gate can catch it. That test mattered: my first placement of the chunk gate sat in validate_w4afp8_base, guarding the c4096 file where it could never fire. It was dead code until the isolation test caught it.

Evidence (gpu31, candidate vs #294)

Workload	TTFT ratio	TPOT ratio
Burst, 8 × ~720K	0.76	0.72–0.76
Burst + short-request hammer	0.77–0.78	0.78–0.85
long-a2	0.61–0.69	0.53–0.56

Minimum free device memory 3.7 → 9.3 GB. Quality at the noise floor: GSM8K 97.41 vs 97.37, MMLU 87.66 vs 87.66, passkey to ~450K 12/12 both, teacher-forced agreement matching control-vs-control. 0 restarts, 0 OOMs, 0 fatal log lines.

Write-up: Notion

Validation

• prepare_glm53_w4afp8_long_context.py --check passes.
• validate_glm53_prod_config.rb passes on all five prod files.
• test_glm53_w4afp8_long_context.py: 12 tests, 2 added. The 2 failures under local Ruby 2.6 are the pre-existing filter_map gap — identical on origin/main (verified), and CI runs 3.3.
• Structural diff proves r1 and the anchor are unchanged.

Untested

The all-gather's cost under CC/PPCIe inside a CVM. gpu31 is bare metal with CC off, so this canary is the first measurement of it. If r2's TTFT regresses against r1 rather than improving, suspect the all-gather and roll back.

Deployment and rollback

r2-only compose/down then compose/up, with a dry_run that must plan exactly model-sg-glm53-w4afp8-tp4-r2. Rollback is a single step documented in the rollout doc: redeploy the preceding tag for r2 alone, which restores its image, chunk and pdi together while r1 keeps serving.

🤖 Generated with Claude Code

[github-actions]

🔍 OpenCodeReview found 2 issue(s) in this PR.

• ✅ 2 posted as inline comment(s)
• 📝 0 posted as summary

⚠️ 1 warning(s) occurred during review.

[lloydmak99]

Canary changes are confined to r2's block — image bumped to v2 (8ff1a487b98a), chunk 8192→16384, its own full environment: block, and r2-specific telemetry labels; r1 and the shared anchor are untouched. r2 correctly inherits all of r1's 18 env vars plus SGLANG_DSA_INDEXER_QSPLIT=1, engine_image placeholders resolve to the right per-replica shas across all three sinks, and image shas are consistent across the compose file, generator, and validator. Looks good to merge.

Optional follow-up (non-blocking):

• Two compiled-bytecode artifacts were accidentally committed under scripts/__pycache__/ (prepare_glm53_w4afp8_long_context.cpython-314.pyc, test_glm53_w4afp8_long_context.cpython-314.pyc); no runtime impact, just clutter — git rm --cached them and add __pycache__/ / *.pyc to .gitignore.

Checks: generator --check passed (target not stale); PyYAML parse + structural inspection confirmed r2's image/chunk/env, placeholder resolution, and r1/anchor identity; 4/4 Python GeneratedFileTest cases passed incl. committed-file byte-match; git diff origin/main confirmed changes confined to r2; both validate CI runs green (Ruby validator + Ruby-dependent tests run there, as Ruby/Docker were unavailable locally).