Repository navigation

perf(pd): P/D first-token latency and API-server GC fixes (from Kimi-K3 AgentX 1M) - #2786

Open

wanzhenchn wants to merge 7 commits into

mainfrom

Open

perf(pd): P/D first-token latency and API-server GC fixes (from Kimi-K3 AgentX 1M)#2786
wanzhenchn wants to merge 7 commits into
mainfrom
zwan/k3-pd-api

Commits on Nov 5, 2026
- feat(kimi-k3): serve mono decode behind a Mooncake P/D decode
  
  wanzhenchn
  and
  cursoragent
  committed
  cb4f638
- perf(engine): poll KV completions again after the forward while requests wait for KV
  
  wanzhenchn
  and
  cursoragent
  committed
  af27f04
- perf(pd): pass a prefill's prompt ids to decode as base64 int32 (ATOM_PD_PROMPT_IDS_B64)
  
  wanzhenchn
  and
  cursoragent
  committed
  7fe018c
- perf(pd): stream the injected T0 when a decode's KV lands (ATOM_PD_EARLY_T0)
  
  wanzhenchn
  and
  cursoragent
  committed
  8534335
- perf(profiler): gzip torch traces at level 3
  
  wanzhenchn
  and
  cursoragent
  committed
  c6d0762
- docs(engine): name the steps that must drain early T0
  
  wanzhenchn
  and
  cursoragent
  committed
  2c87795
- docs(engine): say what the fresh KV poll waits on and why it is off
  
  wanzhenchn
  and
  cursoragent
  committed
  5329537