Skip to content

feat: use Codex-style compaction prompt for context compression - #915

Closed
teknium1 wants to merge 1 commit into
mainfrom
feat/codex-compaction-prompt
Closed

feat: use Codex-style compaction prompt for context compression#915
teknium1 wants to merge 1 commit into
mainfrom
feat/codex-compaction-prompt

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

Replaces the generic summarization prompt in ContextCompressor._generate_summary() with a task-oriented handoff prompt inspired by OpenAI's Codex CLI compaction flow (researched in #499).

What exactly changes

Before — the prompt sent to the summarization model was:

Summarize these conversation turns concisely. This summary will replace
these turns in the conversation history.

Write from a neutral perspective describing:
1. What actions were taken (tool calls, searches, file operations)
2. Key information or results obtained
3. Important decisions or findings
4. Relevant data, file names, or outputs

Keep factual and informative. Target ~2500 tokens.

After — the new prompt is:

You are performing a CONTEXT CHECKPOINT COMPACTION. Create a handoff
summary for the AI assistant that will resume this conversation.

Include:
- Current progress and key decisions made
- Important context, constraints, or user preferences discovered
- What remains to be done (clear next steps)
- Any critical data: file paths, variable names, URLs, error messages,
  or code snippets needed to continue
- Tool calls made and their key results

Be concise, structured, and focused on helping the assistant seamlessly
continue the work without re-doing what's already been done.

Target roughly 2500 tokens.

Why this is better

The old prompt treated compression as generic text summarization. The new prompt frames it as a handoff between LLMs — the summarization model understands the output will be consumed by another model to continue the work. This produces summaries that better preserve:

  • Task continuity — what was being done and what's left
  • Actionable context — file paths, error messages, specific data
  • User intent — preferences and constraints discovered during the conversation

What does NOT change

  • The compression algorithm itself (positional protection, role alternation, boundary alignment)
  • The [CONTEXT SUMMARY]: prefix on inserted summaries
  • The summarization model used (still auxiliary/Gemini Flash)
  • The compression trigger logic (85% threshold)
  • All existing tests pass unmodified

File changed

  • agent/context_compressor.py — prompt text in _generate_summary() only (18 insertions, 16 deletions)

Inspired by PR #776 by @kshitijk4poor and the research in #499.

Replace the generic summarization prompt ('Summarize these conversation
turns concisely') with a task-oriented handoff prompt inspired by
OpenAI's Codex CLI compaction flow (researched in #499).

The new prompt frames compression as a 'CONTEXT CHECKPOINT COMPACTION'
and instructs the summarization model to produce a structured handoff
summary that includes:
- Current progress and key decisions
- User preferences and constraints discovered
- Clear next steps remaining
- Critical data (file paths, URLs, error messages, code snippets)
- Tool calls made and their key results

This produces better summaries because the model understands the summary
will be used by another LLM to continue the work, rather than treating
it as a generic text compression task.

No behavioral change to the compression algorithm itself — same
positional protection, same role alternation, same [CONTEXT SUMMARY]:
prefix. Only the prompt sent to the summarization model changes.

Inspired by PR #776 by @kshitijk4poor.
@teknium1

Copy link
Copy Markdown
Contributor Author

Closing — this was implemented across several commits, most recently in PR #1273 (commit 5c479ee). The current _generate_summary() prompt in context_compressor.py already uses the Codex-style handoff framing described here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant