Skip to content

Qualify GG r25 and native mixed Trellis - #18

Merged
malaiwah merged 1 commit into
mainfrom
codex/r25-qualification
Aug 3, 2026
Merged

malaiwah merged 1 commit into
mainfrom
codex/r25-qualification

Conversation

@malaiwah

@malaiwah malaiwah commented Aug 3, 2026

Copy link
Copy Markdown
Owner

What changed

  • pin the immutable GG v20-r25 image and fail closed on its exact vLLM,
    SparkInfer, LMCache, EXL3, and #117 source hashes;
  • remove the superseded r20 mixed-tier cache-key overlay now that r25 ships
    SparkInfer #117's runtime-dynamic mixed-Trellis ABI;
  • suppress vLLM's misleading low-scheduler-budget warning only when the
    calculated speculative slot delta is zero (serial GLM MTP);
  • isolate persistent compilation artifacts in an r25 namespace;
  • add CI coverage and the complete AIBeast r25, 521K retrieval, performance,
    memory, KLD, and FP8-KV qualification record.

Why

The 3.36-bpw checkpoint has two real expert partitions: 206 K3 + 50 K4 at
layer 3, and 160 K3 + 96 K4 at layers 4–77. #117 passes those counts at launch
time so one compiled kernel is correct for both. The scheduler warning was
also false for serial MTP because it consumes no extra scheduler slots.

Validation

  • full repository lint/direct-test suite and all three config-smoke profiles;
  • exact final Dockerfile built successfully on AIBeast and passed container
    config smoke;
  • four RTX PRO 6000 Blackwell GPUs, driver 595.71.05, CUDA 13.2, 280 W/GPU;
  • TP4/DCP4/MTP3, online K6, dynamic NVFP4, exactly 524,288 active KV tokens;
  • PP 2362.8/2284.9/2143.8 tok/s at 3K/32K/128K;
  • TG C1/C2/C4/C8 100.7/162.2/240.1/297.0 tok/s, zero failures/preemptions;
  • strict structured output passed; 5/5 needles in 521,275 actual tokens;
  • promoted on AIBeast :8000, both aliases passed, live-agent soak healthy
    with zero restarts/OOM/errors.

Independent upstream evidence:
local-inference-lab/b12x#117 (comment)

@malaiwah
malaiwah merged commit 70a53c5 into main Aug 3, 2026
2 checks passed
@malaiwah
malaiwah deleted the codex/r25-qualification branch August 3, 2026 13:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant