Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
302 commits
Select commit Hold shift + click to select a range
a2cafa5
[Fix] sampling: deterministic top-p / top-k renorm when tp_size > 1 (…
gilfordting Oct 31, 2026
4df3895
[Refactor] disaggregation: Commit the PD handoff token through the sh…
hnyls2002 Oct 31, 2026
1f152be
[Doc] [AMD] cookbook: Add Qwen3.8-Flash-Next MI355X AgentX recipe (#4…
chunfangamd Oct 31, 2026
bf1ece1
[Test] rust-server: Drop request-fake members the output streamer no …
hnyls2002 Oct 31, 2026
98a7f7e
[CI] codespell: Ignore miniTE blog name (#47522)
zijiexia Oct 31, 2026
a810654
[Doc] blogs: Sync LMSYS SGLang blog cards (#44825)
sglang-bot Oct 31, 2026
1a50817
[Doc] [NPU] docs: Rename NPU hardware to A2/A3 Series products (#47380)
axx-ty911 Oct 31, 2026
c729000
[Perf] [NPU] mimo: fold attention_value_scale into o_proj weights (#4…
iridiumine Oct 31, 2026
ab86251
[Perf] diffusion: Replay Qwen-Image-2.1's text-only encoder from CUDA…
mickqian Oct 31, 2026
96a76f5
[Perf] diffusion: Hold a Qwen-Image-2.1 Q/K norm + RoPE row with a ha…
mickqian Oct 31, 2026
006ad25
[Fix] Minimax-M3: Deterministic tie-breaking in the CUDA radix decode…
Godmook Oct 31, 2026
7bc9210
[Fix] diffusion: Check cancellation at stage and sampling boundaries …
mickqian Oct 31, 2026
028811f
[CI] chore: add CI permissions and diffusion CODEOWNERS for JackZeng0…
AgainstEntropy Oct 31, 2026
26452a6
[Doc] cookbook: Add Qwen3.8-Flash-Next B300 NVFP4 AgentX recipes (#47…
hshrivastava-droid Oct 31, 2026
d2cf237
[Fix] [NPU] DSA: Correct DSA Models Sparse offload index-cache budget…
JinyanYi Oct 31, 2026
9d33af8
[Feat] [NPU] GLM-4.1V: Add ModelSlim W8A8S support with multimodal se…
cx22757 Oct 31, 2026
fb33962
[Fix] [AMD] quark: Fix the EP>1 crash in the Quark MXFP4 W4A16 MoE pa…
yichiche Oct 31, 2026
9dbc462
[Fix] hicache: mlock pinned host buffers so page aging never scans th…
metamergebot Oct 31, 2026
5736498
[Perf] unified-memory: Bind the SWA tail's new pages without a host s…
metamergebot Oct 31, 2026
800de62
[Fix] model_loader: Neutral UE8M0 block scales for dummy weights (#47…
wwl2755 Oct 31, 2026
2ccbdab
[Perf] [NVIDIA] dsv41: Balance the SM90 paged decode indexer's tiles …
latrawy Oct 31, 2026
fbf9ba4
[CI] ci: Preload `sglang.test.test_utils` in CPU fork workers (#47592)
hnyls2002 Oct 31, 2026
b20fdd2
[Perf] DFLASH: route draft bf16 GEMMs through tuned dispatch on GB300…
wookjeHan Oct 31, 2026
566dbd2
[Perf] disaggregation: Stop device admission in a decode transfer pas…
metamergebot Oct 31, 2026
2f6e7f3
[Feat] [AMD] mamba: Store hicache checkpoints without a GPU slot (#44…
chien-an-chen Oct 31, 2026
b8d8635
[Refactor] scheduler: Track running requests in an owner list (#43899)
hnyls2002 Oct 31, 2026
d969264
[CI] test: Drop tests that never run in CI (#47603)
hnyls2002 Oct 31, 2026
6e4d53f
[Fix] megamoe: Allocate the symmetric buffers before KV cache sizing …
hnyls2002 Oct 31, 2026
1f834c1
[Fix] hicache: Resolve derived sidecars over every source transfer (#…
alphabetc1 Oct 31, 2026
7adb930
[Doc] cookbook: Update MiniMax-M3.1-Flash AgentX launches (#47623)
ispobock Oct 31, 2026
2e80af2
[Doc] [AMD] cookbook: Add GLM-5.2 MI355X AgentX recipe (#47615)
jiejingzhangamd Oct 31, 2026
ef4f3f1
[Perf] [AMD] moe: Resolve MoRI dispatch from the AITER activation con…
Duyi-Wang Oct 31, 2026
1c1166a
[Perf] [NVIDIA] muse-spark-open: Fuse MoE finalize with the 2shot_pus…
wookjeHan Oct 31, 2026
a4a6edd
[Fix] diffusion: Nomas KV budget counts freed cached memory, reject S…
OnePunchMonk Oct 31, 2026
419d31a
[CI] diffusion: Fix image preprocessing test dependency environment (…
alphabetc1 Oct 31, 2026
53a8a22
[CI] [NPU] tests: Add GSM8K PR gate for DSV4-Flash W8A8 8P GPQA (#45293)
pllimax Oct 31, 2026
e9c1825
[Fix] request_logger: Redact client watermark keys in request logs wh…
metamergebot Oct 31, 2026
ba78bce
[Fix] rust-server: End an /abort_request-ed request on the scheduler'…
metamergebot Oct 31, 2026
03a27e2
[Perf] cosmos3: Accelerate conditioning hashing with GPU BLAKE3 (#46028)
xupinjie Oct 31, 2026
f78d906
[Fix] disaggregation: Preserve sampler customized_info across PD (#40…
aurickq Oct 31, 2026
fb9846d
[Fix] hicache: Set host_only_checkpoint in the sidecar restore test (…
DarkSharpness Oct 31, 2026
07a8535
[Fix] kernels/jit: Correct the PDLTriggerSecondary comment on what gr…
DarkSharpness Oct 31, 2026
5c46d20
[Fix] diffusion: Server warmup reaches every data-parallel replica (#…
mickqian Oct 31, 2026
6e011a7
[Perf] [ROCM] diffusion: Skip dead math-SDPA work in the MiniMax-H3 V…
mickqian Oct 31, 2026
67bd749
[Perf] diffusion: Pipeline Ulysses attention over head groups down to…
mickqian Oct 31, 2026
37fc17b
[Feat] diffusion/model: Support Pika-Labs/Pika-H3-Flex (MiniMax-H3 + …
niehen6174 Oct 31, 2026
c0f710f
[Perf] [NVIDIA] sampling: Enable Cake softmax on SM100 (#44168)
bvolpato Oct 31, 2026
e93f817
[Perf] [ROCM] diffusion: Pipeline MiniMax-H3's Ulysses exchange over …
mickqian Oct 31, 2026
504f9cf
[Fix] diffusion: select Blackwell subtile attention by execution mode…
mickqian Oct 31, 2026
b44b02e
[Refactor] memory-pool: Build DSA indexer host pools from buffer info…
ispobock Oct 31, 2026
8368f86
[Perf] [NVIDIA] qwen3: Reuse fused QK-norm/RoPE kernel for decode (#4…
JiaFengYu Oct 31, 2026
dff2c4e
[Refactor] memory-pool: Build unified page-envelope host pools from b…
ispobock Oct 31, 2026
7585ccc
[Perf] [ROCM] diffusion: Bit-exact fast QK RMSNorm and fused RoPE for…
mickqian Oct 31, 2026
0f92dda
[Perf] [ROCM] diffusion: Encode the MP4 while the VAE decodes on ROCm…
mickqian Oct 31, 2026
dc79ae4
[Fix] diffusion: LTX-2.5 two-stage defaults to its 1920x1088 canvas (…
niehen6174 Oct 31, 2026
25c3fd8
[Fix] diffusion: MiniMax-H3 warms up at the short edge --warmup-resol…
niehen6174 Oct 31, 2026
140bb6d
[Fix] reasoning: Reject reasoning_effort outside the enum for M3 (#47…
ispobock Oct 31, 2026
9780c52
[Feat] hicache: Store MXFP8 KV pages in L3 as payload + scales object…
pengwu22 Oct 31, 2026
5eb7018
[Fix] disaggregation: Send aux_item_lens next to aux_ptrs in PD regis…
ShangmingCai Oct 31, 2026
9767ce3
[Perf] model_executor: Write small forward metadata with fill kernels…
metamergebot Oct 31, 2026
005b8e0
[Fix] tokenizer_manager: Do not dispatch a request whose abort was al…
metamergebot Oct 31, 2026
571ddad
[Feat] observability: Report a TTFT phase breakdown in response meta_…
metamergebot Oct 31, 2026
2410a09
[Refactor] memory-pool: Describe DeepSeek V4 device pools with buffer…
ispobock Oct 31, 2026
60b183d
[Feat] rust-server: Reach every attention-DP rank with aborts and pub…
metamergebot Oct 31, 2026
a65c9c5
[Feat] unified-memory: Evict Full and SWA KV from one shared LRU orde…
metamergebot Oct 31, 2026
e624174
[Fix] disaggregation: Ignore a late STAGING_REQ for an already-scatte…
ShangmingCai Oct 31, 2026
01fde5c
[CI] [AMD] pr-test: Run the MI35x 1-GPU stage as one job; move two e2…
michaelzhang-ai Oct 31, 2026
bd1f559
[CI] tests: Rendezvous the tile-sharding gloo test through a file sto…
leejnau Oct 31, 2026
1813806
[Fix] hicache: Withhold only protected bytes from page-cache credit (…
metamergebot Oct 31, 2026
3908210
[Perf] kimi-k3: Allow residual bank shards with NCCL (#47452)
Codoge Oct 31, 2026
fc32cbf
[Fix] disaggregation: Send decode transfer destinations to prefill on…
libertyeagle Oct 31, 2026
cda284e
[Fix] rust-server: Validate gathered listener ranks and address famil…
merrymercy Nov 1, 2026
8817f63
[Perf] diffusion: keep GQA and MOSS-Image attention on cuDNN SDPA on …
niehen6174 Nov 1, 2026
592754f
[Doc] cookbook: Add MiniMax-M3.1-Flash GB300 PD AgentX points (#47699)
zijiexia Nov 1, 2026
5032f98
[Perf] diffusion/minimax-h3: Skip attention for the packing's alignme…
mickqian Nov 1, 2026
4eba700
[CI] health: Retry transient GitHub API failures (#47563)
mickqian Nov 1, 2026
1457851
[Feat] model: support Unlimited-OCR v2 (#47196)
DonggeYang Nov 1, 2026
e3ec98b
[Refactor] diffusion: normalize diffusers output layout in one place …
OnePunchMonk Nov 1, 2026
b146a85
[Perf] [AMD] qwen3.5: Route 512-expert softmax MoE routing through ai…
omirosh Nov 1, 2026
8d12b65
[Fix] attention: Compute extend attention KV offsets in int64 (#47029)
xiaofei-zheng Nov 1, 2026
ee83f5a
[Refactor] memory-pool: Build QSA indexer host pools from buffer info…
ispobock Nov 1, 2026
cb7eaa2
[Feat] hicache: Add the fast_file local storage backend (#39880)
xiezhq-hermann Nov 1, 2026
d8c28b1
[Refactor] memory-pool: Rename index key format to layout (#47705)
ispobock Nov 1, 2026
5831112
[CI] [AMD] monokernels: Bump the sgl-amd-mono pin daily (#46685)
kevin-mii Nov 1, 2026
2d8462f
[Fix] mem_cache: Keep K/V in separate allocations under torch.compile…
DarkSharpness Nov 1, 2026
c108a65
[Fix] diffusion: keep tied vocab tables on device under layerwise off…
OnePunchMonk Nov 1, 2026
07efafb
[Fix] multimodal: Keep shared-memory feature transport for multi-node…
knlb-tml Nov 1, 2026
bebee56
[Fix] diffusion: Exit the worker after an unrecoverable device error …
Alex-Wengg Nov 1, 2026
fa5580f
[CI] ci: Re-tier CUDA tests across base, extra, nightly, and weekly b…
hnyls2002 Nov 1, 2026
b801787
[Doc] skills: Update the CI skills for the tier layout and current pi…
hnyls2002 Nov 1, 2026
4f45b15
[Perf] diffusion: Wan-Animate-2 1.7x faster denoising via exact FA3 i…
OnePunchMonk Nov 1, 2026
3edea93
[Fix] disaggregation: Match the whole bootstrap address when evicting…
ShangmingCai Nov 1, 2026
76066d6
[Fix] disaggregation: Initialize the bootstrap server loop and runner…
ShangmingCai Nov 1, 2026
928e285
[Feat] dcp: Support breakable prefill CUDA graphs under decode contex…
klshuster Nov 1, 2026
43881a7
[Fix] disaggregation: Call finalize_bootstrap outside the assert stat…
ShangmingCai Nov 1, 2026
1ad0149
[Perf] diffusion: Nomas camera attention reads each frame's value dir…
OnePunchMonk Nov 1, 2026
4aaa899
[CI] lora: Simplify LoRA tests and add a lora rerun group (#45981)
jybsuper Nov 1, 2026
c757c4b
[Fix] kv_events: Offset ipc endpoints per DP rank and finish shutdown…
ShangmingCai Nov 1, 2026
4ffc9e1
[Fix] diffusion: Fail fast on offline runtime failures (#47488)
Zeta-li Nov 1, 2026
cb0a7a9
[Feat] disagg: Write PD decode KV to the linker and skip pages the pr…
alphabetc1 Nov 1, 2026
e232ebf
[Perf] [AMD] moe/topk: Use the Triton softmax top-k router for Qwen3-…
chennyiiis Nov 1, 2026
f171859
[Perf] attention: Round the DeepSeek-V4.1 dense indexer logits tiles …
DarkSharpness Nov 1, 2026
e298298
[Fix] diffusion/bcg: Warm up LTX-2 two-stage at the output resolution…
mickqian Nov 1, 2026
0f93be7
[Refactor] memory-pool: Build draft sidecar host pools from buffer in…
ispobock Nov 1, 2026
b50d439
[Perf] [AMD] dsa: Replicate the DCP query projection on the owned rou…
EricKing626 Nov 1, 2026
a52ee80
[Perf] diffusion: opt-in Wan UMT5 padding trim (SGLANG_DIFFUSION_WAN_…
OnePunchMonk Nov 1, 2026
6dda2f0
[Fix] tests: Initialize DCP replay backend capability (#47736)
mickqian Nov 1, 2026
d3bf0b4
[Refactor] scheduler: Route every step launch through one hook and co…
hnyls2002 Nov 1, 2026
cdc2f7d
[CI] nightly: Disable the MiniMax-M3.1-Flash H200 test while it times…
em-dreyer Nov 1, 2026
1c0dc02
[Fix] scheduler: Decide early result processing before scheduling the…
hnyls2002 Nov 1, 2026
240f2a0
[Fix] disagg: Give the decode queue test scheduler a tree_cache (#47758)
DarkSharpness Nov 1, 2026
03f9054
[Feat] [AMD] minimax: gfx950 sparse attention, NVFP4 KV and DSpark ve…
kevin-mii Nov 1, 2026
9431d2c
[Fix] diffusion: LoRA alpha reload, auto-mode skip, critic coverage (…
OnePunchMonk Nov 1, 2026
fdf2266
[Feat] [AMD] distributed: JIT quick all-reduce for TP4 on gfx950 (#46…
kevin-mii Nov 1, 2026
e37c1fb
[Feat] [AMD] minimax: gfx950 MoE, router GEMV, block-fp8 GEMM and fp8…
kevin-mii Nov 1, 2026
c3654cd
[Chore] [AMD] docker: Build uTLX into the gfx950 rocm10 images (#46883)
RolaoDenthu Nov 1, 2026
1367b51
[Test] model_loader: Cover unindexed Mistral shard filtering (#47601)
pratik-naik003 Nov 1, 2026
e8bf40b
[Feat] diffusion: LRU eviction for LoRA merge cache (#47605)
OnePunchMonk Nov 1, 2026
38d9e4a
[Fix] diffusion: Keep each request's rollout trajectory and condition…
OnePunchMonk Nov 1, 2026
a202923
[Fix] kernels: Support OOT fused ops under torch.compile (#47229)
rumitdesai Nov 2, 2026
3a345c0
[Refactor] environ: Rename the SGLANG_Q35MF_* runtime switches to SGL…
YAMY1234 Nov 2, 2026
f96d5b1
[Refactor] [NPU] graph: Remove the NPU graph runner (#47173)
cen121212 Nov 2, 2026
009b167
[Fix] linear: Keep replicated outputs out of TP symmetric memory (#47…
Codoge Nov 2, 2026
a8c69d0
[Fix] attention: Zero padded KDA/GDN recurrent outputs (#46225)
Jzz1943 Nov 2, 2026
e1e0f4a
[Test] [Ascend] npu: Add GLM-5.2 2P2D gpqa cases and unify evalscope …
pllimax Nov 2, 2026
8088adb
[Fix] kernels: Hash tensors over 8 GiB in gpu_tensor_hash (#47066)
g0djan Nov 2, 2026
33c8d72
[Fix] diffusion: fix cosmos3 Bugs (#43194)
kediwu0331 Nov 2, 2026
9fcb7a7
[Perf] [NPU] attention: Support mlaprolog for kimik3 (#47373)
LinyuanLi0046 Nov 2, 2026
61d2005
[Fix] humming: Free replaced MoE weights after each sublayer transfor…
DarkraiHL Nov 2, 2026
22e6ff7
[Perf] [NVIDIA] fused_moe: Add H20 configs for GLM-5.3-Flash TP8/TP4 …
QNan0513 Nov 2, 2026
2b38f4a
[Doc] cookbook: Sync MiniMax-M3.1-Flash GB300 AgentX points (#47791)
zijiexia Nov 2, 2026
ca819c1
[Perf] [NVIDIA] moe: Rewrite moe_finalize_fuse_shared as a JIT kernel…
DarkSharpness Nov 2, 2026
572d3dc
[Refactor] [NPU] quant: Refactor shared quantization utilities and DS…
hanwlax Nov 2, 2026
cec4554
[Fix] kimi-k3: Honor initial expert placement in fused routing (#47742)
Codoge Nov 2, 2026
fbb847b
[Perf] rmsnorm: One SM100-tuned JIT template for plain, fused residua…
DarkSharpness Nov 2, 2026
2c362af
[Doc] [NPU] docs: clarify installation and quickstart docs for Ascend…
amote-i Nov 2, 2026
dfbbfc3
[Fix] disaggregation: Use the shared ZMQ context for staging prefetch…
ShangmingCai Nov 2, 2026
0c29b0b
[Perf] Hunyuan3D: Skip redundant OBJ exports when generating GLB (#46…
charliechenye Nov 2, 2026
297d2a1
[Test] [NPU] deepseek_v4_flash: Switch FP8 4P GPQA test from EAGLE to…
YZY00Raiser Nov 2, 2026
48db87c
[CI] disaggregation: Fail a PD test launch fast when an H20 runner is…
ShangmingCai Nov 2, 2026
a8e1539
[Perf] sampling: JIT softmax and argmax for the sampler and speculati…
DarkSharpness Nov 2, 2026
74ef460
[Perf] kda: Write convolution output directly into contiguous QKV pla…
BBuf Nov 2, 2026
46db5f7
[Fix] [NVIDIA] fused-a-gemm: Check the selected backend's K alignment…
lingxling Nov 2, 2026
8319feb
[Perf] kimi-k3: Seed a validated FlashInfer prefill MoE tactic (#47653)
Codoge Nov 2, 2026
a7c9c3c
[Refactor] memory-pool: Build MiniMax K-only index host pools from bu…
ispobock Nov 2, 2026
ea2fda7
[Fix] metrics: Handle CPU monitor clock resets after restore (#44494)
rst0git Nov 2, 2026
d644cfb
[Refactor] vlm/unlimited-ocr: Reduce scheduler coupling (#47735)
mickqian Nov 2, 2026
acd6f55
[Fix] scheduler: Publish KV events while the engine is paused (#46334)
sohom-cs Nov 2, 2026
a587350
[Fix] multimodal: Keep every image, video and audio of a request when…
sohom-cs Nov 2, 2026
e0c7992
[Fix] [AMD] quark: Load MXFP4 MoE checkpoints on gfx95 and gfx1250 wi…
ZhengGong-amd Nov 2, 2026
0abac08
[Fix] test: Stub _confidence_budget_prepare in the scheduler running-…
kevin-mii Nov 2, 2026
96b5d05
[Fix] mem_cache: Skip Mamba checkpoint donation when slots are exhaus…
yihong-w Nov 2, 2026
4ae54e9
[Perf] kda: Batch strided FlashInfer prefill input copies (#47651)
Codoge Nov 2, 2026
46e95bb
[Perf] [AMD] aiter: Bucket decode paged-attention partitions by seque…
xiaofei-zheng Nov 2, 2026
d82e124
[Fix] dp-controller: Keep one input_embeds request from killing a DP …
sohom-cs Nov 2, 2026
ae66493
[Fix] [AMD] aiter: Use the layer's KV scales in the legacy target-ver…
xiaofei-zheng Nov 2, 2026
f6127bf
[Feat] [AMD] moe: Add native GLM MXFP4 Triton Gluon backend (#41639)
jiejingzhangamd Nov 2, 2026
22e7897
[Fix] tokenizer_manager: Prevent scheduler crash and has_audio_unders…
enternal111 Nov 2, 2026
7605a94
[Perf] dsv4: Add native H16 CuTe sparse prefill backend (#36059)
xuqianmamba Nov 2, 2026
aa3e34a
[Feat] [NVIDIA] megakernels: Expose hybrid runtime and DSpark verific…
leejnau Nov 2, 2026
6aca419
[Feat] diffusion/model: support HiDream-O1-Image (#39682)
tthakkal Nov 2, 2026
2dc5bdd
[Perf] mla: Use masked K concatenation for padded head counts (#47743)
Codoge Nov 2, 2026
8fd3468
[Feat] attention: Add OOT linear-attention backend registration (#47234)
vvinniev34 Nov 2, 2026
b47e58e
[Fix] rust-server: Give each attention-DP listener its own frontend c…
phphlio1 Nov 2, 2026
768e562
[Refactor] scheduler: Move deferred responses into an object and rena…
hnyls2002 Nov 2, 2026
7cad887
[Perf] moe: Enable deferred finalize for NVFP4 on the routed TRT-LLM …
vincentzed Nov 2, 2026
806d863
[Fix] moe: Avoid stale prefix reads in ep_scatter (#44974)
Degeneracy-Evil Nov 2, 2026
3d33971
[Fix] cache: Roll back host slots when allocation raises (#46606)
YaoleWu Nov 2, 2026
180e7a1
[Feat] diffusion/model: add the BRIA Fibo 2.0 transformer and text co…
liquanfeng Nov 2, 2026
3e88483
[Doc] [AMD] cookbook: Verify MiniMax-M3.1-Flash High-Throughput on MI…
kevin-mii Nov 2, 2026
9dfa137
[Feat] [NPU] attention: support mixed-chunk on the ascend attention b…
iridiumine Nov 2, 2026
39a5d5b
[Feat] [XPU] graph: Add full prefill graph (#30980)
CaoE Nov 2, 2026
af5731c
[CI] [NPU] workflow: Gate pr-test-npu-finish on pr-gate and drop nigh…
shun8686 Nov 2, 2026
23c6442
[Fix] attention: Use ModelRunner query width for DSpark FlashAttentio…
charliechenye Nov 2, 2026
b0debe0
[Perf] dsv41: Fuse decode mHC post, combine and RMSNorm into one kern…
zcnrex Nov 2, 2026
669292b
[Test] diffusion: align warmup and LoRA fixtures with runtime contrac…
mickqian Nov 2, 2026
d952161
[Fix] PP: record received proxy tensors on the forward stream (#47614)
Phoenix3334 Nov 2, 2026
cc69762
[Fix] [Intel] gemma4: Bidirectional image attention on the intel_xpu …
SKRohit Nov 2, 2026
d2d3632
[Perf] Hunyuan3D: Speed up shape export with GPU-side grid generation…
charliechenye Nov 2, 2026
96f5a5d
[Test] linear: Expect replicated rows to skip TP symmetric memory (#4…
nvpohanh Nov 2, 2026
960e307
[Refactor] scheduler: List every request holder and check no request …
hnyls2002 Nov 2, 2026
26fde01
[Chore] owner: update (#47875)
HaiShaw Nov 2, 2026
d74749b
[Feat] [NPU] eplb: Enable dynamic expert load balancing on Ascend (#4…
AkiYangDev Nov 2, 2026
1599b16
[CI] [Intel] disaggregation: Run XPU P/D nightly on Mooncake and NIXL…
kpjeeja Nov 2, 2026
1ced9a9
[Perf] sgl-router: Encode tiktoken models with fastokens under --toke…
ShangmingCai Nov 2, 2026
b45a365
[Fix] kimi_k3: Use uuid tool_call_id instead of history-count id (#45…
yan-lgtm Nov 2, 2026
a133231
[Refactor] kernels: Centralize GPU architecture queries in sglang.ker…
DarkSharpness Nov 2, 2026
d0d50da
[Fix] kernels: Port rmsnorm and softmax bench off the removed get_jit…
DarkSharpness Nov 2, 2026
c0c7af4
[Fix] spec: Give the mocked draft decode backend a cache_loc_dtype in…
DarkSharpness Nov 2, 2026
427d8de
[Feat] trace: propagate per-request context to mooncake store RPC for…
staugust Nov 2, 2026
3f0051e
[Fix] graph: Restore XPU logger and Inkling backend mock dtype (#47858)
hanwlax Nov 2, 2026
a4e51e2
[Refactor] memory-pool: Build QSA indexer host pools as hybrid sideca…
ispobock Nov 2, 2026
c04bae2
[Test] [NPU] kimi_k3: add 128k prefill/decode perf acceptance cases o…
EdwardXuy Nov 2, 2026
58592ce
[Test] [NPU] DeepSeek_v4_pro: Add 16p mix in128k out1k performance an…
YZY00Raiser Nov 2, 2026
b55350b
[Fix] scheduler: Skip --prefill-decode-interval when nothing is decod…
akhilg-nv Nov 2, 2026
755783e
[Perf] allreduce: enable FlashInfer AllReduce Fusion auto-enable for …
brandonfzhang Nov 2, 2026
8c0544b
[Perf] http: Serialize /v1/embeddings responses with model_json_respo…
anmolgupt Nov 2, 2026
c35f89f
[Fix] moe: Preserve FP32 router precision across model gates (#38719)
seongyun1104 Nov 2, 2026
ad6ca42
[Perf] trtllm_mha: Split full-attention decode KV across CTAs at larg…
nbommi19 Nov 2, 2026
e33cd0c
[Perf] DeepSeek-V4.1: Support indexer TP shard (#42968)
yuan-luo Nov 2, 2026
114675a
[Perf] diffusion: Fuse MiniMax-H3 encoder SiLU and padding (#47852)
kevin-mii Nov 2, 2026
582168d
[CI] moe: Give the NVFP4 non-gated Marlin MoE test bf16-sized headroo…
leejnau Nov 2, 2026
16dd97d
[Perf] [AMD] deterministic: Use a fixed-order two-stage all-reduce fo…
aarnetalman Nov 2, 2026
e152f53
[Perf] [NVIDIA] dsv41: Split small-batch decode mHC post-combine-norm…
zcnrex Nov 2, 2026
251eb36
[Doc] [AMD] cookbook: Qwen3.8-Flash-Next MI355X, add the monokernel c…
ChangLiu0709 Nov 2, 2026
20f5f68
[Fix] [AMD] imports: Skip import-time device probes when no GPU is vi…
Arist12 Nov 2, 2026
33a1ba4
[Fix] [AMD] mamba: Import the causal conv1d kernels for Mamba2 layers…
aarnetalman Nov 2, 2026
3674c71
[Refactor] scheduler: Hold a launched prefill in one list until its n…
hnyls2002 Nov 2, 2026
bde43d6
[CI] [AMD] mi355x: Clean up interrupted nightly jobs and reduce runne…
Lzy17 Nov 2, 2026
efdcb42
[Perf] imports: Keep light modules off HF patches, model configs, and…
hnyls2002 Nov 2, 2026
9eec211
[Chore] [NVIDIA] deps: Bump sgl-deep-gemm to 0.2.5.post3 (#47894)
zcnrex Nov 2, 2026
c927e7c
[CI] test: Relax bs=1 perf limits for CI hosts with slower CPUs (#47989)
em-dreyer Nov 2, 2026
e7a6c35
[Fix] moe: Revert the JIT rewrite of `moe_finalize_fuse_shared` (#477…
hnyls2002 Nov 3, 2026
797af71
[Refactor] scheduler: Check request ownership once at step launch (#4…
hnyls2002 Nov 3, 2026
ba1ed00
[Fix] scheduler: Abort held decode requests, and read live requests f…
hnyls2002 Nov 3, 2026
6c9f5e6
[Doc] blogs: Sync LMSYS SGLang blog cards (#47997)
sglang-bot Nov 3, 2026
4387846
[Feat] deepseek-recipe: Expose convert and encode_conversation on Dee…
guapisolo Nov 3, 2026
8a0edcf
[Feat] paged-experts: size the resident experts automatically (#47476)
N3mezis Nov 3, 2026
d1dd0b7
[Test] [NPU] kimi_k3: enable L2 host HiCache for 32p gpqa accuracy te…
EdwardXuy Nov 3, 2026
b8100b7
[Feat] [NPU] kimi-k3: Support DCP with shared compact communication (…
qybnb Nov 3, 2026
133f712
[Doc] cookbook: Align navigation groups with model publishers (#47900)
mickqian Nov 3, 2026
602d045
[Fix] scheduler: Publish KV metrics when idle (#45785)
zhtshr Nov 3, 2026
1c13f95
[Refactor] rust-server: Make the advertised listener host an overrida…
metamergebot Nov 3, 2026
62f9f3f
[Feat] moe: Allow SiTU on the CuTe DSL NVFP4 MoE (Kimi-K3 W4A16) (#44…
ByronHsu Nov 3, 2026
a7d5354
[CI] [AMD] nightly-disagg: Run MI355X disagg nightly at 04:00 UTC (12…
bingxche Nov 3, 2026
dbeecc9
[Fix] scheduler: Tell a step's middle chunk from the batch's chunked …
hnyls2002 Nov 3, 2026
5bffd74
[Refactor] imports: Import names from where they are defined; remove …
hnyls2002 Nov 3, 2026
f6db18e
[Fix] attention: Keep fused RoPE KV writes off DCP layouts (#47261)
LLLL114 Nov 3, 2026
18b7144
[Doc] qwen: Use TRT-LLM MHA for the B300 Qwen3-VL-4B recipe (#48051)
leonardHONG Nov 3, 2026
cd41d53
[Doc] cookbook: Refresh SemiAnalysis AgentX launches to InferenceX ma…
nvpohanh Nov 3, 2026
7f878d7
Revert "[Feat] diffusion/model: add the BRIA Fibo 2.0 transformer and…
BBuf Nov 3, 2026
af98fcd
Revert "[Perf] kimi-k3: Seed a validated FlashInfer prefill MoE tacti…
BBuf Nov 3, 2026
c870481
[CI] lint: Reject importing a name through a module that only re-impo…
hnyls2002 Nov 3, 2026
4709a3a
[Perf] [NVIDIA] moe: Localize DeepGEMM Mega MoE weights across locali…
zcnrex Nov 3, 2026
c68083d
[Doc] cookbook: Add MiniMax-M3.1-Flash GB300 MonoKernel and DEP16 Age…
zijiexia Nov 3, 2026
825f8f6
[Perf] diffusion: Fuse SANA-Video depthwise convolution and GLU on SM…
BBuf Nov 3, 2026
cbfc093
[Perf] [AMD] deterministic: Allow AITER decode attention on gfx950 (#…
aarnetalman Nov 3, 2026
80121f2
[Perf] [NVIDIA] qwen3: Reuse the table QK-norm/RoPE kernel for MRoPE …
zcnrex Nov 3, 2026
ed8d232
[Refactor] scheduler: Share one interface across request lists and ab…
hnyls2002 Nov 3, 2026
11d6878
[CI] [AMD] diffusion/flux3: Make exact-quality VAE convolutions deter…
michaelzhang-ai Nov 3, 2026
63c6de2
[Doc] diffusion: Set measured cookbook defaults for MiniMax-H3, FastH…
niehen6174 Nov 3, 2026
2c89ac7
[CI] quant: Lower the Mixtral AWQ MMLU threshold to 0.62 to clear its…
leejnau Nov 3, 2026
57abc8a
[Fix] test: Import ResolvedView from model_override_base in test_mode…
aarnetalman Nov 3, 2026
f77064f
[CI] [AMD] diffusion/minimax-h3: Skip the group-attend contract test …
michaelzhang-ai Nov 3, 2026
a470d74
[Perf] [AMD] unquant: Pad gfx115x linear and lm_head weight rows to a…
twoask Nov 3, 2026
751fa99
[Perf] imports: Keep the detokenizer off TokenizerManager and the tok…
hnyls2002 Nov 3, 2026
96301ec
[Fix] Anthropic: accept thinking.display="updates" (#45129)
JustinTong0323 Nov 3, 2026
5845c0d
[CI] diffusion: Refresh the LTX-2 two-stage runtime VRAM baseline for…
leejnau Nov 3, 2026
8d4e1ce
[CI] [AMD] pr-test: Run the MI35x 1-GPU stage after MI300 stage-b; mo…
michaelzhang-ai Nov 3, 2026
021d17d
Merge branch 'main' into pr/rocm-qknorm-kernels
avjves Nov 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
25 changes: 25 additions & 0 deletions .agents/skills/ci-test-audit/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,31 @@ An audit is only useful if it is specific. "This suite could be trimmed" is not
finding; "`TestFooLargePage` is a strict subset of `TestFooRetractLargePage`, drop it"
is. Every finding needs the file, the pattern it matches, and what to do.

## Auditing at scale

An audit over hundreds of files (a whole tier, every CPU test) splits the reading from
the judgment:

- **Inventory first.** Collect each file's registrations, cost (live estimate x GPUs,
from the `sglang-ci-stats` model), and the exact items to check (CI-skipped classes,
overlapping configs) with a script, before any agent reads a file.
- **Fan out file reads to the cheapest model** with a fixed brief and a TSV answer, in
chunks of about 15 files. The brief lists the exact items to verify; "review every
case in the file" makes the pass slow and adds nothing.
- **Prosecutor stance.** Each file in the brief is deleted or demoted by default; keeping
it needs a reason: the change that would turn it red, and the grep that shows no other
test catches that change.
- **The main agent re-judges** every verdict, most expensive first, before reporting.
Treat cheap-model "covered by X" or "superseded by Y" verdicts as leads: in one audit
none of 20 proposed file deletions held on verification.
- **Re-tier before deleting.** For cost, moving a test down a tier (base, extra,
nightly, weekly) saves far more than deleting: the tier policy is in
[write-sglang-test, Choosing a Tier](../write-sglang-test/SKILL.md#choosing-a-tier).
Delete only what never runs in CI or is verified redundant.
- **Failure evidence comes from failed runs' job logs.** `sglang-ci-stats` records only
successful jobs, so whether a tier catches regressions the tier above misses is read
from the failed scheduled runs on main.

## Reporting

Group findings by confidence in the claim, not by pattern id. Lead with the ones where
Expand Down
15 changes: 10 additions & 5 deletions .agents/skills/ci-test-audit/action-items.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ un-skip them; skips older than the issue they were waiting on; registrations nam
retired `runner_config`; registered files whose TestCase classes never execute. Not
this pattern: a hardware-conditional skip (`skipUnless` on compute capability), which is
a gate; or a `disabled=` whose linked issue is still open, which is doing its job.
Examples: #32324, #39368, #38585, #34377, #33772, #34779, #33654, #34070.
Examples: #47603, #32324, #39368, #38585, #34377, #33772, #34779, #33654, #34070.

---

Expand All @@ -86,9 +86,12 @@ Examples: #34074, #33654, #33605, #34913, #35220.
Spot: a multi-GPU accuracy suite registered per-commit whose failures have never been
PR-specific; a nightly job that has been green for weeks, occupies several GPUs per run,
and whose coverage a weekly run would hold just as well; a retired platform still
running daily. Demoting is not
deleting -- the nightly and weekly grids exist to hold this coverage at the right cost.
Examples: #38011, #37990, #33809, #36814, #37532, #34204, #37586, #31749.
running daily; a model test on a per-commit or extra tier for a model that is old, little
used (check its downloads), or superseded by a newer model in its family on the same model
file. Demoting is not deleting -- the nightly and weekly grids exist to hold this coverage
at the right cost. The tier policy is in
[write-sglang-test, Choosing a Tier](../write-sglang-test/SKILL.md#choosing-a-tier).
Examples: #47619, #38011, #37990, #33809, #36814, #37532, #34204, #37586, #31749.

**B3. Declared cost matches measured cost; shard counts and timeouts derive from it.**
Spot: an `est_time` off from the measured run by an order of magnitude (the partitioner
Expand Down Expand Up @@ -221,7 +224,9 @@ copy of the package under test taking precedence over the working tree. Examples
**F3. Repeated work inside the test process is cut.**
Spot: a checker that re-parses the whole package on every scan; a fixture that reloads a
tokenizer or model per case; independent subprocesses run serially; lint environments
rebuilt on every run. Examples: #36240, #36235, #36241, #37203, #34309.
rebuilt on every run; a suite of short files where each file pays the same import
(measure the common test import with `python -X importtime` before reading test content).
Examples: #47592, #36240, #36235, #36241, #37203, #34309.

**F4. Work whose precondition already holds is skipped: installs, disk reclamation, cache refreshes.**
Spot: unconditional installs; a skip check whose matching is too strict to fire on real
Expand Down
Loading