Repository navigation
Conversation
perf(beam): fuse the attention Q/K norm, RoPE and SSMax prologue
Use FP32 math attention in the Beam test oracle
Fix Docker Dynamo dependency resolution
…leanup Fix unused request states after parallel sampling
…20260928 fix: retain page-aligned SWA history for prefix reuse
…60928 fix: preserve token-ID logprobs in PD streaming
Use a literal analysis opener whose closer also consumes the next message's anchor, give calls the same closer between parallel calls, require the XML constraint marker and newline value framing, accept any recipient-style tool name, and let answers end on <|call|>. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…y-swa-20261014 [Cherry-pick] Bring unified-memory and SWA fixes into dev
…-20261014 [Speculative] Allow DFlash with FlashInfer MegaMoE under attention DP
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
github-actions
Bot
added
documentation
This was referenced Oct 19, 2026
This was referenced Oct 20, 2026
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Add native serving support for Beam, a 501B-parameter MoE model with 23B active parameters and a 1M-token context configuration.
Modifications
Accuracy Tests
Recorded results at head
f7473673ddf6175f301782af57e684d33d3e3f96, using the original Beam verifier ate698bb518d5117a3f1ba93357bfb72999a586020and its full GSM8K 10-shot preset. All runs use actual DFlash decoding with eight draft tokens and no acceptance simulation.Startup and the configured 1M context limit were checked. Actual 1M-token prompts and maximum-capacity operation were not exercised. Upstream CI and comparison against current
mainremain pending for this draft.Speed Tests and Profiling
Recorded OpenHands multi-turn replay on four GB300 GPUs, NVFP4 TP4/EP1: 127,588.4 total tokens/s/GPU at 185.87 tokens/s/user (p50), concurrency 32. Total throughput includes cached input. This performance replay uses eight draft tokens, simulated acceptance length 4 with real draft tokens, and 88 GB HiCache per rank. It is a single measurement and does not measure natural draft acceptance or model accuracy.
No fresh performance comparison against current
mainhas been run for this draft.Checklist
CI States
Latest PR Test (Base): ❌ Run #38909662145
Latest PR Test (Extra): ❌ Run #38909662423
Latest PR Test (AMD ROCm 10): ❌ Run #38909662232