Repository navigation
feat(compression): result memoization for deterministic engines (opt-in) — roadmap #21 - #5286
Merged
Merged
Conversation
Adds opt-in result memoization for deterministic compression modes (lite/standard/rtk + deterministic-only stacked pipelines). Off by default. Key includes principalId to prevent cross-principal leaks. Cache returns cloned bodies to prevent mutation of cached values. FIFO bound at 5000 entries, no new deps.
…(core review)
- invert engine classification to an opt-in whitelist {lite,caveman,rtk}: ccr and
session-dedup write to the cross-request CCR store (output depends on prior state),
so they are NOT cacheable; any new/unknown engine or mode is excluded by default
- skip the cache when principalId is absent (a missing principal would collapse
authenticated callers into the shared anonymous key space → cross-principal body leak)
- clone on store too (not just on read) so a later mutation of the caller's object
cannot corrupt the cached entry
- regression tests for all three
Contributor
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Adds opt-in result memoization for deterministic compression modes
(lite/standard/rtk + deterministic-only stacked pipelines). Off by
default. Key includes principalId to prevent cross-principal leaks.
Cache returns cloned bodies to prevent mutation of cached values.
FIFO bound at 5000 entries, no new deps.
isDeterministicMode uses an opt-IN whitelist {lite,caveman,rtk}: ccr,
session-dedup and ionizer write to the cross-request CCR store
(storeBlock), so memoizing their output would skip the side-effect on a
cache HIT and leave CCR markers pointing at never-stored blocks. They
are excluded; any unvetted/new engine defaults to not-cacheable.
diegosouzapw
force-pushed
the
feat/compression-memoization
branch
from
June 29, 2026 03:18
7a6d367 to
23dbc08
Compare
lite strips data:image URLs only when vision is unsupported, so the same (body, config, principal) yields a different result per target. Without these in the key a non-vision target's image-stripped body was served to a vision target. Adds a regression test. Plus CHANGELOG reconcile.
…ation branches)
Merged
tkgo11
pushed a commit
to tkgo11/OmniRoute
that referenced
this pull request
Sep 23, 2026
…in) — roadmap cheaper-inference#21 (cheaper-inference#5286) roadmap cheaper-inference#21: result memoization for deterministic engines (opt-in, default off). Fixed the cache-key under-specification (folded model+supportsVision so lite image-strip can't serve a wrong cached body across vision/non-vision targets) + regression test. Locally validated release-green. Integrated into release/v3.8.40.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Caches
(input, config) → resultfor provably pure, stateless compression so the hot path skips recompute. Opt-in, default off (memoizeCompressionResults). Perf, not token. Tier-3 roadmap item (#21).How
resultMemo.ts: FIFO-boundedMap(cap 5000, ported from the CCRboundedSetpattern — no new dep), sha256 key over{bodyHash, mode, config, principalId}. Memo seam insiderunCompression/runCompressionAsyncafter the guards.Core-review hardening (3 real issues caught + fixed)
{lite,caveman,rtk}.ccr/session-dedupwrite the cross-request CCR store (verified:ccr/index.tsccrStore,session-dedupimportsstoreBlock) → output depends on prior state → never cached;ultra/aggressive/llmlingua(model-backed) excluded; any new engine/mode defaults to not-cached.principalIdis absent (else authenticated callers collapse into the shared anonymous key space).Tests (TDD, node:test)
result-memo.test.ts(23: default-off, hit/miss, principal isolation, ultra-never-cached, FIFO evict, + 3 hardening regressions). Full compression suite green (1083/1083 — memo off changes nothing).Gates (local)
lint 0 errors · typecheck:core clean · check:complexity 1970 (baseline 1981) · cognitive 837/841.