Repository navigation
Conversation
Ignore KV managers that explicitly opt out of prefix caching when deciding whether 64-token hash lookup is safe. KpoolTail remains transient and is rebuilt normally; reusable MLA and Mamba groups still constrain the cache hit. Keep the overlay additive for images that already contain its earlier DFlash2 hybrid-cache fix, update its patch test, and document the live append-only and semantic validation results.
plotarmordev
left a comment
plotarmordev
left a comment
There was a problem hiding this comment.
Hold update
Public head checked: 9f3eb7865b0ffe9b9675d0dda4ad89548b8ce2c1.
Why it is held: We need to choose how this rebased version of #125 fits alongside #125 and #171. This is a maintainer choice, not a failed author test.
What we tested: Earlier integrated cache candidates gained reuse on long and medium append requests, but reused fewer tokens on a short append. Those runs were not on this new head, so they do not qualify or reject it. A successful boot is also not proof of cache correctness.
Next action — me, @plotarmordev: Choose the implementation and supported scope, then review and qualify that exact integrated version. Preserve the replay-safety guard and the useful negative cases.
Author action now: None; no new rebase or run requested. If this is selected, the agreed comparison must include cold, repeated, short-append and long-append requests, completed-request cache counters, correct answers, and separate state/numerical checks.
Please keep the hold until the conditions above are met. When a new version is ready for review, include its commit ID, the checks actually run, their results, and anything not tested. Keep failed or incomplete runs in the report. Share only sanitized results—not private prompts, credentials, or machine configuration.
plotarmordev
added
merge-hold
|
Closing - keeping this local; thanks for the context on direction. |
Rebased version of #125 (by @rwl4) onto current main, resolving the conflicts against 9b9eb13 (per-KV-cache-group APC retention with DFlash SWA replay):
FINE_MARK,FINE_OLD/FINE_NEW(filterunsupported_partial_hit_managersbygroup.kv_cache_spec.participates_in_prefix_caching), three-block idempotentmain().DEFAULT_PAGE_TOKENS = 64+ HEAD's newer /reset_prefix_cache notes) and the README (keeps HEAD's table + reset-route text, adopts the 64-token page-math note).logger.warningwhen fine-grained lookup remains disabled by participating managers - on our TP=2 kit the merged overlay applies cleanly but warm hits still land on 4x3584 boundaries, so the log names exactly which manager still blocks 64-token hits on this coordinator build.Built and boot-tested on 2x GB10 TP=2 (image builds clean through the in-image test suite, incl. test_apc_per_group_retention composition).