perf(dcp): rotate destinations in the PCIe peer KV gather - #577

Open

myshytf wants to merge 72 commits into

local-inference-lab:dev/infernal-invocationfrom

myshytf:agent/kimi-k3-dcp-gather-rotate-20260902-pr

Commits on Aug 12, 2026

fix(structured-output): preserve grammar bitmask source widths
voipmonitor
committed
826bb40

Commits on Aug 17, 2026

[Bugfix] Stream Kimi K3 tool-call arguments incrementally

guptaishaan
authored and
voipmonitor
committed
b9eea38
tools: preserve partial Kimi XTML close markers

voipmonitor
and
codex
committed
efbaa95
fix(cache): preserve Mamba CoW after external hits

myshytf
authored and
voipmonitor
committed
9972d3c
test(cache): assert external-hit continuation precondition

myshytf
authored and
voipmonitor
committed
990a63d
Ignore Kimi tool calls after tools section close
voipmonitor
committed
3ebc896
fix(dspark): preserve draft graph capture contract
voipmonitor
committed
c805ebd
spec_decode: preserve DFlash-family checkpoint semantics

voipmonitor
and
codex
committed
3938209
attention: execute replicated KV groups without DCP partitioning

voipmonitor
and
codex
committed
8f7f6c6
models: document DFlash rotary layout detection

voipmonitor
and
codex
committed
f85d3d8

Commits on Aug 18, 2026

[II] Keep Kimi K3 protocol markers out of streamed content
voipmonitor
committed
04a6acf
Fix invalid block handling for hybrid KV cache groups

jungjiyu
authored and
voipmonitor
committed
c09bed0
test(scheduler): cover hybrid KV fail policy across 17 groups
voipmonitor
committed
6bafa63
test(scheduler): preserve hybrid KV recovery edge cases
voipmonitor
committed
3c296be
fix(structured-output): map compacted DSpark grammar rows
voipmonitor
committed
a18a86e

Commits on Aug 19, 2026

test(dspark): define graph-tail mode in capture fixture
voipmonitor
committed
e7ffc38

Commits on Aug 20, 2026

Commits on Aug 21, 2026

[Model][Spec Decode] Tap the pre-norm AttnRes mixture as the Kimi K3 DFlash aux state (#50487)

authored and
voipmonitor
committed
c27d117
test(kimi-k3): strengthen DFlash capture contracts
voipmonitor
committed
0a00dac
Bound Kimi vision RoPE allocation to input grids

voipmonitor
and
codex
committed
71964b3
Bound Kimi projector transients per image
voipmonitor
committed
18d9e27
[II] Let cache specs own DCP block-table geometry
voipmonitor
committed
4f34748
fix(mamba): resume prefix hits on checkpoint grid
voipmonitor
committed
edb1042
fix(multimodal): gather uneven vision shards without TP padding
voipmonitor
committed
832b1c9

Commits on Aug 22, 2026

fix(kimi-k3): preserve final AttnRes block
voipmonitor
committed
d19ef45
fix(attention): preserve aliased LSE during state merge
voipmonitor
committed
7aa4d61
fix(attention): align merge blocks to head groups
voipmonitor
committed
7671c66
fix(kimi-k3): bound chunked MLA output lifetime
voipmonitor
committed
62bead5
[Bugfix][Mamba] Fix overlapping state copy race (#50729)

authored and
voipmonitor
committed
f7180bc
Merge vLLM PR #414
voipmonitor
committed
4cfeb8a
Merge vLLM PR #295
voipmonitor
committed
bf5a9b2
Merge vLLM PR #294
voipmonitor
committed
e62cedb
Merge vLLM PR #320
voipmonitor
committed
6f78b5c
Merge vLLM PR #413
voipmonitor
committed
538ed2e
Merge vLLM PR #422
voipmonitor
committed
8b6e1de
Merge vLLM PR #310
voipmonitor
committed
799c2e7
Merge vLLM PR #415
voipmonitor
committed
bc3dd5a
Merge vLLM PR #418
voipmonitor
committed
afcbf4f
Merge vLLM PR #419
voipmonitor
committed
23e7093
Merge vLLM PR #459
voipmonitor
committed
f52fadc
Merge vLLM PR #460
voipmonitor
committed
94d3833
Merge vLLM PR #463
voipmonitor
committed
dc8f46d
Merge vLLM PR #464
voipmonitor
committed
3c9031e
Merge vLLM PR #467
voipmonitor
committed
7da8bd3
Merge vLLM PR #468
voipmonitor
committed
6ba940c
Merge vLLM PR #469
voipmonitor
committed
89b9db6
Merge upstream vLLM PR #50729 backport
voipmonitor
committed
1c93ca0

Commits on Aug 23, 2026

feat(spec_decode): run the Kimi-K3 DSpark draft on a dedicated remote GPU
myshytf
committed
a653e74
fix(spec_decode): allow scheduler-selected zero draft depth
myshytf
committed
23f9f27
fix(cache): avoid target EAGLE drop for disaggregated DSpark
myshytf
committed
f49f6c3
test(cache): cover DFlash target-cache policy
myshytf
committed
dbf86de
scheduler: skip speculative decoding when all scheduled requests need <=1 output token

malaiwah
authored and
myshytf
committed
1bed2da
fix: address CodeRabbit review — document lookahead reservation is harmless for single-token requests

malaiwah
authored and
myshytf
committed
27e71ca
fix(comm): prewarm FlashInfer PCIe graph shapes
myshytf
committed
74ec7af

Commits on Aug 29, 2026

feat(kquant): opt into W4A8 for coupled QSRT prefill
myshytf
committed
93917b3

Commits on Sep 1, 2026

perf(mla): fuse K3 DCP verification queries
myshytf
committed
c7fd9c8
[Attention][MLA] Fuse Kimi-K3 chunked-context K/V packing

authored and
myshytf
committed
9d50524
[Bugfix][MLA] Cast the gathered latent for a bf16 Kimi-K3 kv_b_proj

authored and
myshytf
committed
ee429fb
[Kimi-K3][DCP] Publish prefill KV directly in MLA layout

GirasoleY
authored and
myshytf
committed
865148a
perf(mla): publish K3 prefill KV over PCIe
myshytf
committed
96789b5
perf(mla): decouple context tiles and enable SM120 FA4
myshytf
committed
1ce3580
perf(kimi): retain large context projection workspace

myshytf
and
codex
committed
8a89a1d
perf(dcp): rotate destinations in the PCIe peer KV gather
myshytf
committed
6da7e41

Commits on Sep 2, 2026