Repository navigation

perf(qwen-flash): 4096-token prefill chunks with an overlapped n-gram gather - #470

Merged

francescobozzo merged 7 commits into

gufo-org:mainfrom

Nickybuk:pr/a2-ngram-chunk

Oct 7, 2026

Merged

perf(qwen-flash): 4096-token prefill chunks with an overlapped n-gram gather#470
francescobozzo merged 7 commits into
gufo-org:mainfrom
Nickybuk:pr/a2-ngram-chunk

Commits on Oct 7, 2026
- fix(qwen-flash): prefill logits independent of chunk size
  
  authored and
  francescobozzo
  committed
  f540150
- perf(qwen-flash): 128 n-gram readers, woken per read
  
  authored and
  francescobozzo
  committed
  2d6bc73
- perf(qwen-flash): prefetch the next chunk's n-gram rows
  
  authored and
  francescobozzo
  committed
  0610c8f
- perf(qwen-flash): prefill in 4096-token chunks
  
  authored and
  francescobozzo
  committed
  f9a57de
- docs(qwen-flash): record dense split, prefetch, chunk results
  
  authored and
  francescobozzo
  committed
  90425ab
- perf(qwen-flash): reduce prefill scratch and short-request latency
  francescobozzo
  committed
  def2ed1
- docs(benchmarks): refresh Flash-Next Performance sweep
  francescobozzo
  committed
  5c6ebb4