feat(opencode): add double-buffer context management#15130
feat(opencode): add double-buffer context management#15130marklubin wants to merge 2 commits intoanomalyco:devfrom
Conversation
|
The following comment was made by an LLM, it may be inaccurate: Found related PRs! These are related to the compaction and context management work but address different aspects: Related PRs (Not Duplicates):
These are all related work in the compaction/context management space but none are duplicates of PR #15130. This PR is the first to implement the specific "double-buffer" two-phase compaction strategy with checkpoint phases. |
Implements a two-phase compaction strategy that checkpoints at ~50% capacity and swaps at ~75%, producing higher-quality summaries by summarizing while the model still has full attention rather than at the capacity cliff. Changes: - New `double-buffer.ts`: state machine tracking normal → checkpoint → concurrent → swap phases per session - `compaction.ts`: background checkpoint (fire-and-forget), swap check, and summary retrieval exports - `processor.ts`: skip overflow on compaction summaries (fixes anomalyco#13946) - `prompt.ts`: swap check before standard overflow, background checkpoint trigger after successful processing - `config.ts`: configurable `checkpointThreshold` and `swapThreshold` in compaction schema - Unit tests for state machine transitions The double-buffer is additive — existing single-pass compaction remains as the fallback when no checkpoint is available. Algorithm: https://marklubin.me/posts/hopping-context-windows/ Refs anomalyco#8140 anomalyco#11314 anomalyco#2945 anomalyco#4317 anomalyco#12479
Replace `require("./double-buffer")` with `await import("./double-buffer")`
in getCheckpointSummary and completeSwap to avoid issues with top-level
await in transitive dependencies. Update callers in prompt.ts accordingly.
marklubin
force-pushed
the
feat/double-buffer-context
branch
from
dda8503 to
d181d83
Compare
|
Is it better than the Copilot CLI implementation described here? |
It's the same idea. I didn't realize they were doing this already. Same functionality. |
I think the difference is that your implementation waits for swap, and Copilot CLI does not: the summary is injected as soon as it is ready. |
|
Right the reason it's
The reason it's implemented that way is mostly an artifact of designing the algorithm to support Claude Code. There, we have to client-drive the compaction on its own cadence for it to work correctly. There is probably a minor functional distinction here though — waiting to swap means more messages are processed with the original context before we swap in the summary. On the face of it, it seems like our approach might be slightly better performance-wise depending on the task, but we'd have to actually run the experiment to know for sure. |
2 participants
Issue for this PR
Refs #8140 #11314 #2945 #4317 #12479
Type of change
What does this PR do?
Implements double-buffer context management — a two-phase compaction strategy that produces higher-quality summaries by running summarization early (at ~50% capacity) while the model still has full attention, rather than at the capacity cliff.
How it works:
Changes:
double-buffer.ts: State machine trackingnormal → checkpoint_pending → concurrent → swapphases per sessioncompaction.ts: Four new exports —checkpointIfNeeded()(fire-and-forget background checkpoint),shouldSwapBuffer(),getCheckpointSummary(),completeSwap()processor.ts: Skip overflow on compaction summaries (fixes Bug:opencode runexits after compaction when compaction model's token usage exceeds overflow threshold #13946, also submitted separately as fix(opencode): skip overflow check on compaction summaries #15129)prompt.ts: Swap check before standard overflow + background checkpoint trigger after successful processingconfig.ts: ConfigurablecheckpointThresholdandswapThresholdin compaction schemaCommunity asks addressed:
Existing single-pass compaction remains as the fallback when no checkpoint is available.
Algorithm description: https://marklubin.me/posts/hopping-context-windows/
How did you verify your code works?
double-buffer.test.ts)Checklist