[Spec] Consolidate the verify step into eagle_worker_common.run_eagle_verify - #31380
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
/rerun-test registered/spec/eagle/test_spec_eagle.py registered/spec/eagle/test_spec_eagle_topk_page.py registered/spec/eagle/test_eagle_constrained_decoding.py registered/spec/eagle/test_eagle_reject_sampling.py registered/spec/test_gemma4_mtp_31b_extra.py registered/spec/test_spec_standalone.py registered/spec/test_frozen_kv_mtp.py registered/unit/spec/test_eagle_worker_v2_topk1_fastpath.py |
|
Results for 🚀 🚀 🚀 |
# Conflicts: # python/sglang/srt/speculative/eagle_worker_v2.py
|
/tag-and-rerun-ci |
|
/rerun-test registered/models_e2e/test_mimo_v2.py registered/models_e2e/test_deepseek_v3_mtp.py registered/models_e2e/test_dsa_glm52_tp_mtp.py registered/models_e2e/test_qwen3_next_models_mtp.py registered/spec/eagle/test_eagle_dp_attention.py registered/spec/eagle/test_adaptive_speculative.py |
|
Results for 🚀 🚀 🚀 |
Consolidate the verify step of
EAGLEWorkerV2andMultiLayerEagleWorkerV2into oneeagle_worker_common.run_eagle_verifyfunction. The single-layer eagleverify()(the superset) is the source of truth; both workers'verify()become thin argument-binding wrappers, and multi-layer's stale partial copy is deleted. The two compaction helpers (_finalize_accept_tree_path,_compact_accept_to_front) move along as module-private functions.The two real differences (preserved verbatim behind switches)
metadata_ready_pre_padFalse— relies oneagle_prepare_for_verifymarking metadata ready only when the cuda-graph load_batch path ran; the non-graph path re-inits post-padTrue— marks ready pre-pad unconditionally (its pre-existing behavior, kept verbatim)finalize_tree_pathTrue— compacts the accepted tree path to the front of each per-req block for topk > 1False— has never run this compaction (kept verbatim; whether topk > 1 multi-layer should adopt it is left as a follow-up)What multi-layer gains (each a gated no-op for its configurations)
generate_token_bitmask+vocab_mask)batch.has_grammar;eagle_sample'svocab_maskdefaults toNone, so the old 3-arg call was already equivalentcommit_mamba_states_after_verifyearly-returns unless the model is mambaish and the backend has the commit hookgetattr(..., "clear_unaccepted_c128_draft_states", None)isNonefor other KV caches_is_npu and model_config.model_is_mropeVerification
Verification script: https://gist.github.com/hnyls2002/ee26d74ec93ddcd45aa4c485a7c0b996 — runs against the pinned base/head commits:
run_eagle_verifyis AST-equivalent to the base commit's eagleverifybody plus exactly three declared edits — thefinalize_tree_pathgate prefix, the gated pre-pad metadata block, and the deletion of thenum_tokens_per_reqre-stamp (mirroring [Spec] Converge DP-attention spec width scaling ontonum_tokens_per_req#31244, which removed it from the base copy after this branch was cut). No other line differs beyond the declared parameter renames.Cross-checks: no test patches any of the moved names;
_finalize_accept_tree_path/_compact_accept_to_fronthave no external callers;StandaloneWorkerV2/FrozenKVMTPWorkerV2inherit the eagle wrapper (same switch values as the body they inherited before). compile + ruff clean acrossspeculative/; black reflow AST-equivalence checked.GPU accuracy validation (eagle + MTP configs) to follow before merge, in addition to the CI reruns.
CI States
Latest PR Test (Base): 🚫 Run #29472859914
Latest PR Test (Extra): ❌ Run #29472859780