Skip to content

Add explicit OpenAI conversation compaction and fix Anthropic compaction - #61370

Merged
nathansobo merged 14 commits into
mainfrom
manual-openai-compaction
Jul 24, 2026
Merged

Add explicit OpenAI conversation compaction and fix Anthropic compaction#61370
nathansobo merged 14 commits into
mainfrom
manual-openai-compaction

Conversation

@nathansobo

@nathansobo nathansobo commented Jul 20, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Objective

Add explicit conversation compaction for OpenAI Responses API models, available through both direct OpenAI connections and Zed Cloud models.

Keep finalized replacement context provider-neutral and separate from the lifecycle events emitted while automatic compaction is in progress.

Solution

  • Add opt-in explicit compaction support to the LanguageModel trait.
  • Separate streamed CompactionUpdate lifecycle events from finalized CompactedContext.
  • Represent provider-native compacted context as an opaque, versioned state owned by the originating provider.
  • Construct compact requests through the same OpenAI request conversion used for completions, then narrow the request to fields supported by /responses/compact.
  • Preserve and validate the complete canonical replacement window returned by OpenAI.
  • Replay the replacement window before messages sent after compaction, without resending the superseded transcript.
  • Support explicit compaction through both the direct OpenAI provider and CloudLanguageModel.
  • Update automatic OpenAI and Anthropic compaction to use the same finalized context representation.
  • Reject empty, malformed, incorrectly versioned, and incorrectly owned provider state.

The corresponding Cloud change adds the /completions/compact proxy route: https://github.com/zed-industries/cloud/pull/3099.

The downstream Delta UI integration is https://github.com/zed-industries/delta/pull/1739.

Testing

  • cargo nextest run -p anthropic -p open_ai -p language_models_cloud
  • cargo check -p language_models --tests
  • cargo fmt --all --check
  • git diff --check

The focused tests cover request construction, canonical-window preservation and replay, direct and Cloud transports, automatic compaction lifecycle events, malformed provider state, provider ownership, format compatibility, and HTTP failure handling.

Self-Review Checklist:

  • I've reviewed my own diff for quality, security, and reliability
  • Unsafe blocks (if any) have justifying comments
  • The content adheres to Zed's UI standards (UX/UI and icon guidelines)
  • Tests cover the new/changed behavior
  • Performance impact has been considered and is acceptable

Release Notes:

  • Added: System messages are now sent through the instructions field in OpenAI Responses API requests.

@cla-bot cla-bot Bot added the cla-signed The user has signed the Contributor License Agreement label Jul 20, 2026
@zed-community-bot zed-community-bot Bot added the staff Pull requests authored by a current member of Zed staff label Jul 20, 2026
@nathansobo
nathansobo force-pushed the manual-openai-compaction branch from 1917db6 to bbd0f4f Compare July 22, 2026 14:07
nathansobo added 11 commits July 22, 2026 08:30
@nathansobo
Address review feedback
6da7b68 
- Document supersede semantics of provider-native compaction replay
- Add context to malformed compaction payload errors and assert on it
  instead of serde_json's internal error text
- Use the model's provider in the Cloud compaction request body instead
  of hardcoding OpenAI
@nathansobo
Scope provider-native compaction state to the producing backend
8a98df5 
Several backends share the OpenAI Responses request conversion and event
mapper (OpenAI itself, Zed Cloud's OpenAI models, OpenAI-compatible
endpoints, Codex, OpenCode, and Bedrock Mantle), but their encrypted
compaction items are not interchangeable: only the backend whose
infrastructure produced an item can decrypt it. Previously the shared
converter accepted any state stamped with the OpenAI provider id and the
shared mapper stamped every backend's compaction output with that id, so
compacting with OpenAI and then switching to another Responses-protocol
backend would discard the entire transcript in exchange for an opaque
blob that backend cannot read.

Thread the owning backend's provider id through into_open_ai_response,
OpenAiResponseEventMapper, and the compaction state helpers so replay is
limited to state the target backend produced; foreign state falls back
to replaying the full transcript.

Also surface malformed streams that end mid-compaction: Anthropic's
mapper now errors when the message stops with an unclosed compaction
block, and the OpenAI mapper errors when a response completes with a
compaction item that was added but never finished, instead of leaving
consumers with a Started event and no terminal signal.
@nathansobo
Round-trip Anthropic compaction encrypted_content and map failed comp…
b020145 
…actions

Anthropic's compaction protocol attaches opaque encrypted_content to
compaction blocks and requires it to be round-tripped verbatim. Deltas
carry it as a whole-value replacement (like thinking signatures), so the
mapper keeps the latest value rather than concatenating chunks. The
finalized summary now carries it as owner-scoped provider state, and
replay only includes it when the request targets the backend that
produced it, mirroring how OpenAI Responses compaction state is scoped.

A compaction block that closes without content is Anthropic's documented
representation of a failed compaction (the server treats it as a no-op),
so surface it as CompactionUpdate::Failed instead of erroring out the
rest of the response.
@nathansobo
Report token usage from explicit compaction
47fa7a9 
Compaction requests consume tokens like any other completion, but the
usage OpenAI reports on /responses/compact was parsed and then dropped.
LanguageModel::compact now returns a CompactionResult pairing the
compacted context with that usage so callers can display what the
compaction cost.
@nathansobo
Hoist system messages into instructions for the Responses API
3cdff5a 
The Responses API documents the top-level instructions field as a system
message inserted into the model's context on each request. Sending the
system prompt there instead of as system-role input items fixes a hole
in compaction replay: replaying provider compaction state replaces all
accumulated input items with the canonical window, which previously
discarded the current system prompt and left the model running on
whatever prompt was frozen into the window at compaction time.

This also unifies the Responses backends: the Codex backend rejects
system-role input items and always required the instructions form, so
its post-hoc extraction is no longer needed (it still always sends the
field, as it always has). Compact requests inherit the field, so
explicit compaction now runs with the current system prompt in context
as well.

NOTE: Zed Cloud's typed completion request currently has no
instructions field and will silently drop it; cloud must deploy that
field before this change ships to users.
@nathansobo
gpui: Allow accessibility activation in tests
c4f1752
@nathansobo
gpui: Report duplicate accessibility node provenance
57c096c
@nathansobo
gpui: Compile duplicate node diagnostics only in debug builds
78c9cdb
@nathansobo
Merge remote-tracking branch 'origin/main' into manual-openai-compaction
26a67e5
@nathansobo
Merge remote-tracking branch 'origin/main' into manual-openai-compaction
64088b7
@nathansobo
Remove unrelated accessibility changes
384aacb
@nathansobo nathansobo changed the title Add explicit OpenAI conversation compaction Add explicit OpenAI conversation compaction and fix Anthropic compaction Jul 24, 2026
@nathansobo
nathansobo enabled auto-merge July 24, 2026 16:53
@nathansobo
nathansobo added this pull request to the merge queue Jul 24, 2026
Merged via the queue into main with commit c28cf64 Jul 24, 2026
42 checks passed
@nathansobo
nathansobo deleted the manual-openai-compaction branch July 24, 2026 16:57
@zed-zippy zed-zippy Bot added the PR state:needs review Used to label PRs that are in need of a post-merge approval label Jul 24, 2026
@as-cii
as-cii self-requested a review July 24, 2026 17:04
as-cii
as-cii reviewed Jul 24, 2026
View reviewed changes

@as-cii as-cii left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved!

@as-cii

as-cii commented Jul 24, 2026

Copy link
Copy Markdown
Member

@zed-industries/approved

@zed-zippy zed-zippy Bot removed the PR state:needs review Used to label PRs that are in need of a post-merge approval label Jul 24, 2026
@JosephTLyons JosephTLyons mentioned this pull request Jul 24, 2026
Merged
mdz-axo added a commit to mdz-axo/zed-kask that referenced this pull request Jul 25, 2026
@mdz-axo
Merge upstream/main (27 commits) into zed-kask
c633460 
Upstream changes (zed-industries/zed main, 27 commits):
- agent: Add agent.compaction_model setting for context compaction (zed-industries#60012)
- agent: Show effort selector for anthropic compatible providers (zed-industries#61579)
- acp: Update agent-client-protocol SDK to 2.0.0 (zed-industries#61570)
- client: Extract proxy handshakes into new proxy_handshake crate (zed-industries#61427)
- collab: Fix multiworkspace location out of sync bugs (zed-industries#61598)
- editor: Fix sticky header drag cancels autoscroll (zed-industries#53592)
- editor: Fix crash when copying and pasting using multiple cursors (zed-industries#61545)
- editor: Skip untitled buffers when saving a multi-buffer (zed-industries#61380)
- gpui: Fix images not being drawn with rounded corners with ObjectFit::Cover (zed-industries#61383)
- gpui: Fix deadlock in performance profiler and reenable it (zed-industries#61584)
- git_ui: Prevent Git panel bindings in repository selector (zed-industries#61282)
- language_model: Add explicit OpenAI conversation compaction and fix Anthropic compaction (zed-industries#61370)
- markdown: Fix squashed Mermaid diagrams in markdown preview (zed-industries#61260)
- Opus 5 BYOK Support (zed-industries#61596)
- repl: Show add-cell controls in empty notebooks (zed-industries#61329)
- search: Escape seeded buffer search query in regex mode (zed-industries#57748)
- settings: Fix VS Code import appending duplicate file associations (zed-industries#61355)
- settings: Split VSCode and Zed keymap files (zed-industries#61532)
- Treat blank spawn_agent session IDs as absent (zed-industries#60893)
- worktree: Reload git state when a watcher rescan covers a repository (zed-industries#61541)
- Plus 7 more minor fixes.

Merge fixes:
- crates/agent/src/thread.rs: replay_tool_call used 'message_ix' (undefined)
  after auto-merge; renamed to 'owning_message_ix' (the parameter name).
- Cargo.toml: Removed stale workspace members hkask-wallet and hkask-git-cas
  (both directories deleted in prior commits but workspace entries remained).
- kask/crates/hkask-regulation/src/wallet_manager.rs: Stubbed consume() and
  settle_rjoules() on WalletBudgetPort — these were API-key encumbrance
  operations from the deleted hkask-wallet crate; regulation tracks per-agent
  gas balances, not per-key encumbrances.
- kask/crates/hkask-regulation/src/wallet_gas_calibrator.rs: Fixed test to
  use crate::agent_wallet_store::WalletStore instead of hkask_storage::WalletStore.
- kask/crates/hkask-regulation/Cargo.toml: Added tokio macros feature to
  dev-dependencies for #[tokio::test].
- kask/crates/kask_bridge/Cargo.toml: Added futures dependency (needed by
  context_injector.rs for futures::executor::block_on).
- kask/crates/kask_bridge/src/context_injector.rs: Fixed futures_util::executor
  to futures::executor (futures-util doesn't include executor module).

Release Notes:

- N/A
0arm pushed a commit to 0arm/zed that referenced this pull request Jul 26, 2026
@nathansobo @0arm
Add explicit OpenAI conversation compaction and fix Anthropic compact…
8eccd95 
…ion (zed-industries#61370)

# Objective

Add explicit conversation compaction for OpenAI Responses API models,
available through both direct OpenAI connections and Zed Cloud models.

Keep finalized replacement context provider-neutral and separate from
the lifecycle events emitted while automatic compaction is in progress.

## Solution

- Add opt-in explicit compaction support to the `LanguageModel` trait.
- Separate streamed `CompactionUpdate` lifecycle events from finalized
`CompactedContext`.
- Represent provider-native compacted context as an opaque, versioned
state owned by the originating provider.
- Construct compact requests through the same OpenAI request conversion
used for completions, then narrow the request to fields supported by
`/responses/compact`.
- Preserve and validate the complete canonical replacement window
returned by OpenAI.
- Replay the replacement window before messages sent after compaction,
without resending the superseded transcript.
- Support explicit compaction through both the direct OpenAI provider
and `CloudLanguageModel`.
- Update automatic OpenAI and Anthropic compaction to use the same
finalized context representation.
- Reject empty, malformed, incorrectly versioned, and incorrectly owned
provider state.

The corresponding Cloud change adds the `/completions/compact` proxy
route: https://github.com/zed-industries/cloud/pull/3099.

The downstream Delta UI integration is
https://github.com/zed-industries/delta/pull/1739.

## Testing

- `cargo nextest run -p anthropic -p open_ai -p language_models_cloud`
- `cargo check -p language_models --tests`
- `cargo fmt --all --check`
- `git diff --check`

The focused tests cover request construction, canonical-window
preservation and replay, direct and Cloud transports, automatic
compaction lifecycle events, malformed provider state, provider
ownership, format compatibility, and HTTP failure handling.

## Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

---

Release Notes:

- Added: System messages are now sent through the `instructions` field
in OpenAI Responses API requests.
@github-actions github-actions Bot mentioned this pull request Aug 6, 2026
Open
jolutz pushed a commit to jolutz/zed that referenced this pull request Aug 8, 2026
@nathansobo
Add explicit OpenAI conversation compaction and fix Anthropic compact…
f99ac43 
…ion (zed-industries#61370)

# Objective

Add explicit conversation compaction for OpenAI Responses API models,
available through both direct OpenAI connections and Zed Cloud models.

Keep finalized replacement context provider-neutral and separate from
the lifecycle events emitted while automatic compaction is in progress.

## Solution

- Add opt-in explicit compaction support to the `LanguageModel` trait.
- Separate streamed `CompactionUpdate` lifecycle events from finalized
`CompactedContext`.
- Represent provider-native compacted context as an opaque, versioned
state owned by the originating provider.
- Construct compact requests through the same OpenAI request conversion
used for completions, then narrow the request to fields supported by
`/responses/compact`.
- Preserve and validate the complete canonical replacement window
returned by OpenAI.
- Replay the replacement window before messages sent after compaction,
without resending the superseded transcript.
- Support explicit compaction through both the direct OpenAI provider
and `CloudLanguageModel`.
- Update automatic OpenAI and Anthropic compaction to use the same
finalized context representation.
- Reject empty, malformed, incorrectly versioned, and incorrectly owned
provider state.

The corresponding Cloud change adds the `/completions/compact` proxy
route: zed-industries/cloud#3099.

The downstream Delta UI integration is
zed-industries/delta#1739.

## Testing

- `cargo nextest run -p anthropic -p open_ai -p language_models_cloud`
- `cargo check -p language_models --tests`
- `cargo fmt --all --check`
- `git diff --check`

The focused tests cover request construction, canonical-window
preservation and replay, direct and Cloud transports, automatic
compaction lifecycle events, malformed provider state, provider
ownership, format compatibility, and HTTP failure handling.

## Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content adheres to Zed's UI standards
([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
and
[icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md)
guidelines)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

---

Release Notes:

- Added: System messages are now sent through the `instructions` field
in OpenAI Responses API requests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@as-cii as-cii as-cii left review comments

Assignees

No one assigned

Labels

cla-signed The user has signed the Contributor License Agreement staff Pull requests authored by a current member of Zed staff

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@nathansobo @as-cii