Repository navigation

fix(cache-aware): fix load imbalance when decode is faster than prefill - #1714

Merged

slin1237 merged 4 commits into

smg-project:mainfrom

SYChen123:load-fix

Jun 16, 2026

Merged

fix(cache-aware): fix load imbalance when decode is faster than prefill#1714
slin1237 merged 4 commits into
smg-project:mainfrom
SYChen123:load-fix

Commits on Jun 16, 2026
- fix load imbalance problem for cache_aware policy when prefill is much slower than decode
  
  SYChen123
  authored and
  slin1237
  committed
  905bb24
- fix(cache-aware): complete least-load tiebreak on KV-aware fallback; drop dead decode param
  slin1237
  committed
  d47e4fd
- test lint fix
  
  SYChen123
  authored and
  slin1237
  committed
  6dba312
- test(cache-aware): exercise real token-tree affinity in empty-indexer test
  slin1237
  committed
  3228427