Repository navigation

feat(inference): router replay with KV cache offload - #3831

Merged

faresobeid merged 12 commits into

mainfrom

feat/router-replay-kv-offload

Oct 19, 2026

Merged

feat(inference): router replay with KV cache offload#3831
faresobeid merged 12 commits into
mainfrom
feat/router-replay-kv-offload

Commits on Oct 6, 2026
- feat(inference): router replay with disaggregated P/D on vLLM 0.31
  faresobeid
  committed
  a827863
- feat(inference): router replay with KV cache offload on vLLM 0.31
  faresobeid
  committed
  28c9c7a
Commits on Oct 12, 2026
Commits on Oct 13, 2026
- refactor(inference): take the end-anchored routing start from the packed /generate path
  faresobeid
  committed
  978146c
- Merge branch 'feat/router-replay-v2-pd' into feat/router-replay-kv-offload
  faresobeid
  committed
  f6cd446
Commits on Oct 14, 2026