[KV Offload] Grouped hybrid RAM caching, filesystem restoration and retention-sized Mamba pools (stacked on #617) - #598

Open

Leonccaa wants to merge 27 commits into

1CatAI:mainfrom

Leonccaa:fix/qsa-native-cpu-offload

Commits on Sep 13, 2026

fix: retain aligned Mamba prefixes across long prefills
Leonccaa
committed
1cbaa2c
docs: describe sparse retention for 1Cat
Leonccaa
committed
5cd53e1
Configure Mamba checkpoint retention in blocks
Leonccaa
committed
f0be84a
Backport upstream Mamba retention configuration and recovery boundaries

Leonccaa
and
codex
committed
820516b
Fix sparse cache events and non-EAGLE decode retention

Leonccaa
and
codex
committed
6c24974
Merge remote-tracking branch 'upstream/main' into fix/mamba-sparse-prefix-retention
Leonccaa
committed
5d4c0e4
Fix upstream README blank line for CI

Leonccaa
and
codex
committed
fdc0ffc
Move unrelated baseline CI fixes to a separate PR

Leonccaa
and
codex
committed
a9ab97a
fix: exclude scratch KV groups from native offload scheduling

Leonccaa
and
codex
committed
18dc5cb
fix: budget hybrid CPU offload pools by cache group

Leonccaa
and
codex
committed
ae3d484
test: verify grouped CPU eviction reuses the retired slot

Leonccaa
and
codex
committed
70ed46b
Validate grouped offload composition with existing filesystem tiers

Leonccaa
and
codex
committed
3a9d26e
Support group-local shared memory views in offload workers

Leonccaa
and
codex
committed
db8e258
Wire grouped shared regions through native tiering spec

Leonccaa
and
codex
committed
1c0c328
Let scheduler unlink shared offload files after worker shutdown

Leonccaa
and
codex
committed
e557cb6
Document stable prefix hash seed for filesystem restart reuse

Leonccaa
and
codex
committed
ac54716
Record V100 grouped filesystem restart and shutdown validation

Leonccaa
and
codex
committed
d8e7e2d
Fail early when shared offload memory cannot be pinned

Leonccaa
and
codex
committed
57c3fc4
Record complete-image V100 and single-group offload validation

Leonccaa
and
codex
committed
6847e06
Fix grouped offload retries when a pool defers storage

Leonccaa
and
codex
committed
5e12852
Test external LMCache group metadata compatibility

Leonccaa
and
codex
committed
70eb7cb
Document LMCache PR compatibility results and registration blocker

Leonccaa
and
codex
committed
52ce76c
Size grouped Mamba pools from the retention mask and propagate external junctions
Leonccaa
committed
eef147c
Document V100 offload validation on PR617 core

Leonccaa
and
codex
committed
0d497ca
fix: serialize shared offload mmap registration across workers
Leonccaa
committed
33b534e
docs: record concurrent pinning fix and V100 restart acceptance
Leonccaa
committed
fb808b7
Retry Mamba boundary hand-offs and reject sparse retention with a block-size factor
Leonccaa
committed
0b2d937

Navigation Menu

Uh oh!

[KV Offload] Grouped hybrid RAM caching, filesystem restoration and retention-sized Mamba pools (stacked on #617) - #598

[KV Offload] Grouped hybrid RAM caching, filesystem restoration and retention-sized Mamba pools (stacked on #617)#598
Leonccaa wants to merge 27 commits into
1CatAI:mainfrom
Leonccaa:fix/qsa-native-cpu-offload

Commits on Sep 13, 2026

fix: retain aligned Mamba prefixes across long prefills

docs: describe sparse retention for 1Cat

Configure Mamba checkpoint retention in blocks

Backport upstream Mamba retention configuration and recovery boundaries

Fix sparse cache events and non-EAGLE decode retention

Merge remote-tracking branch 'upstream/main' into fix/mamba-sparse-prefix-retention

Fix upstream README blank line for CI

Move unrelated baseline CI fixes to a separate PR

fix: exclude scratch KV groups from native offload scheduling

fix: budget hybrid CPU offload pools by cache group

test: verify grouped CPU eviction reuses the retired slot

Validate grouped offload composition with existing filesystem tiers

Support group-local shared memory views in offload workers

Wire grouped shared regions through native tiering spec

Let scheduler unlink shared offload files after worker shutdown

Document stable prefix hash seed for filesystem restart reuse

Record V100 grouped filesystem restart and shutdown validation

Fail early when shared offload memory cannot be pinned

Record complete-image V100 and single-group offload validation

Fix grouped offload retries when a pool defers storage

Test external LMCache group metadata compatibility

Document LMCache PR compatibility results and registration blocker

Size grouped Mamba pools from the retention mask and propagate external junctions

Document V100 offload validation on PR617 core

fix: serialize shared offload mmap registration across workers

docs: record concurrent pinning fix and V100 restart acceptance

Retry Mamba boundary hand-offs and reject sparse retention with a block-size factor