[None][test] Switch MiniMax-M3 Eagle3 accuracy test to the GQA draft head - #17406
Closed
zheyuf wants to merge 1 commit into
Closed
[None][test] Switch MiniMax-M3 Eagle3 accuracy test to the GQA draft head#17406zheyuf wants to merge 1 commit into
zheyuf wants to merge 1 commit into
Conversation
…head The AgentX MiniMax-M3 submission moves its EAGLE3 draft head from the MHA checkpoint (Inferact/MiniMax-M3-EAGLE3) to the GQA retrain (Inferact/MiniMax-M3-EAGLE3-GQA): same training data, attention changed from MHA to GQA (num_key_value_heads 64 -> 4), giving a 16x smaller draft KV cache and a lighter draft share in the disagg KV transfer. test_nvfp4_eagle3 now loads MiniMax-M3-EAGLE3-GQA from LLM_MODELS_ROOT. No runtime change is needed: the draft KV cache manager reads num_key_value_heads from the drafter config, the same path the GQA Llama EAGLE3 heads use. The GQA card publishes no chat-GSM8K figures, so the MHA-derived acceptance thresholds (rate > 0.78, length > 3.3) are carried over and marked provisional pending calibration on the first CI run. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com>
zheyuf
added a commit
to zheyuf/TensorRT-LLM
that referenced
this pull request
Aug 11, 2026
…t disagg acceptance Supersedes NVIDIA#17406. The drafter becomes Inferact/MiniMax-M3-EAGLE3-GQA in both arms, with thresholds recalibrated from the first GQA run: disagg chat-GSM8K measures length 2.773 / rate 0.591 (consistent with the GQA card's MT-Bench 2.668; the MHA head measured 3.44-3.48 here), so the MHA-derived floors do not carry over. The disagg arm now also asserts acceptance through the generation worker's /metrics iteration stats (enable_iter_perf_stats; the fixture's DuckLLM gains a router_url field to locate the worker via /cluster_info) — closing the gap where a drafter-KV transfer regression was invisible to the accuracy assert. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Switches
TestMiniMaxM3::test_nvfp4_eagle3from the MHA EAGLE3 draft head(
Inferact/MiniMax-M3-EAGLE3) to the GQA retrain(
Inferact/MiniMax-M3-EAGLE3-GQA).Per the model card the GQA head is trained on the same data (kimi-mtp, OpenCodeInstruct, SWE-bench,
SWE-bench-Pro); the only architectural change is the draft attention,
num_key_value_heads64 → 4.Why
The EAGLE3 draft is a single dense attention layer, so its KV cache is disproportionately expensive
next to MiniMax-M3's sparse-attention target. From the published configs (
head_dim128, BF16 KV),draft KV per token per rank at TP4:
num_key_value_heads64)num_key_value_heads4)A 16× smaller draft cache leaves substantially more of the KV pool for the target, and in
disaggregated serving it proportionally shrinks the draft share of the KV transfer.
Acceptance is comparable: the GQA card reports MT-Bench 2.668 / 55.62% and SPEED-Bench (qualitative)
2.561 / 52.04% mean accepted length / accept rate at
num_speculative_tokens=3.Why no runtime change is needed
The draft KV cache manager and the drafter's attention both read
num_key_value_headsfrom thedrafter's own config, not the target's:
SpecDecOneEngineForCausalLMbuilds the draft config viaModelConfig.from_pretrained(spec_config.speculative_model)(modeling_speculative.py)Eagle3Attentionpassesconfig.num_key_value_headsstraight intoAttention(
modeling_speculative.py)KvCacheCreator._get_effective_draft_config()returns that draft config, and the V2 KV cachemanager derives per-rank KV heads from it (
_util.py,kv_cache_manager_v2.py)This is the same path the existing GQA Llama EAGLE3 heads already exercise (e.g.
EAGLE3-LLaMA3.1-Instruct-8Bis 32 Q / 8 KV heads) — the MiniMax-M3 MHA head was the unusual case,not this one.
Test Coverage
accuracy/test_llm_api_pytorch.py::TestMiniMaxM3::test_nvfp4_eagle3— bothl0_dgx_b200_m3variants(
attention_dpFalse/True). Test ids are unchanged, so no test-list edits are required.Verification done
tests/,docs/,examples/,jenkins/): this test is the only reference tothe MHA head, on both the original and current
feat/m3_with_msa.num_key_value_heads, and that theGQA weights match it (
k_projis[512, 12288]= 4 × 128, all BF16).changes and no errors.
Acceptance thresholds
The GQA card publishes MT-Bench / SPEED-Bench but no chat-GSM8K figures, so the MHA-derived
references (rate 0.839, length 3.518) and thresholds (rate > 0.78, length > 3.3) are carried over and
labelled provisional in the test comments. They should be recalibrated from the first green CI run on
this head.
Dependencies
MiniMax-M3-EAGLE3-GQAadded to the CI model store (ftp/llm-models!583, merged) — allow theusual sync delay to the CI scratch before running the test