fix(ai-chat): fix regenerate stale messages, transport trigger, and identical-content reconciliation - #1014
Merged
Merged
Conversation
… transport Two bugs caused regenerate() to leave stale assistant messages in SQLite: 1. WebSocketChatTransport.sendMessages omitted the trigger field from the body payload, so the server never knew whether a request was a submit or a regenerate. Fixed by adding trigger: options.trigger to the serialized body. On the server, trigger is destructured out of the parsed body (like messages and clientTools) so it does not leak into options.body in onChatMessage. 2. persistMessages only performed upserts (INSERT ON CONFLICT UPDATE), never deletes. When regenerate() removed the last assistant message from the client array, the old row stayed in SQLite and reappeared on reload — causing Anthropic 400 errors and phantom messages. Fixed by adding an internal _deleteStaleRows option that the CF_AGENT_USE_CHAT_REQUEST handler passes. It deletes DB rows whose IDs are absent from the post-merge message set. The default behavior (upsert-only) is unchanged for all other callers. Closes #1012
🦋 Changeset detectedLatest commit: 3dc616c The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
commit: |
… messages Rewrites _reconcileAssistantIdsWithServerState with a two-pass approach to fix #1008. The old single-pass cursor advanced on both exact-ID and content matches. When an exact-ID match jumped the cursor forward (e.g. client had a server ID from the wrong position after state drift), it skipped server messages that a later incoming message needed for content matching. With identical assistant text like 'Sure' or 'I understand', the second message would match the wrong server message or fail to match entirely, creating duplicate rows. Pass 1: resolve all exact-ID matches, claiming server indices into a Set. Pass 2: content-based matching for remaining non-tool assistant messages, scanning only unclaimed server indices left-to-right. This ensures exact-ID matches cannot interfere with content matching, and each server message is claimed at most once. Closes #1008
threepointone
changed the title
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1012, fixes #1008
Three bugs in
@cloudflare/ai-chatcaused stale/duplicate assistant messages and broken regeneration:triggerfield — server never knew if a request was submit vs regeneratepersistMessagesnever deletes stale rows — regenerated messages persisted as ghostsBug 1: Transport drops
triggerfield (#1012)Root cause:
WebSocketChatTransport.sendMessagesserialized the body withouttrigger. The AI SDK passestrigger("submit-message"or"regenerate-message") but it was silently dropped.Fix: Added
trigger: options.triggerto the serialized body inws-chat-transport.ts.Cleanup: On the server,
triggeris destructured out of the parsed body alongsidemessagesandclientTools, so it does not leak intooptions.bodyinonChatMessage.Bug 2:
persistMessagesnever deletes stale rows (#1012)Root cause:
persistMessagesonly performed upserts. Whenregenerate()removed the last assistant message from the client array, the old DB row stayed and reappeared on reload — causing Anthropic 400 errors and phantom messages.Fix: Added an internal
_deleteStaleRowsoption topersistMessages. TheCF_AGENT_USE_CHAT_REQUESThandler passes{ _deleteStaleRows: true }, deleting DB rows whose IDs are absent from the post-merge message set.Why
_deleteStaleRowslives insidepersistMessagesThe delete logic must use post-merge message IDs.
_mergeIncomingWithServerStatecan remap client assistant IDs to server IDs. If reconciliation ran externally with raw client IDs, it would incorrectly delete remapped messages.Why the option is named
_deleteStaleRowsThe underscore prefix +
@internalJSDoc signal internal use only. Only the chat-request handler passes it. All other callers use the default (upsert-only).Why
CF_AGENT_USE_CHAT_REQUESTalways reconcilesThe transport always sends the full message array — it's the source of truth regardless of submit vs regenerate. Gating on trigger value would be fragile.
Bug 3: Content-based reconciliation mismatches identical text (#1008)
Root cause:
_reconcileAssistantIdsWithServerStateused a single-pass cursor for both exact-ID and content-based matching. When an exact-ID match jumped the cursor forward (e.g. client had a server ID from the wrong position after state drift), it skipped server messages that a later incoming message needed for content matching. With identical text like "Sure" or "I understand", the second message would match the wrong server message or fail to match entirely, creating duplicate rows.Fix: Rewrote with a two-pass approach:
Set<number>.This ensures exact-ID matches cannot interfere with content matching, and each server message is claimed at most once.
Files changed
packages/ai-chat/src/ws-chat-transport.tstriggerin body payloadpackages/ai-chat/src/index.tstriggerfromcustomBody; add_deleteStaleRowstopersistMessages; rewrite_reconcileAssistantIdsWithServerStatewith two-pass approachpackages/ai-chat/src/tests/chat-persistence.test.tspackages/ai-chat/src/tests/regenerate-message.test.tspackages/ai-chat/src/tests/ws-transport-trigger.test.tspackages/ai-chat/src/tests/reconcile-identical-content.test.ts.changeset/fix-regenerate-stale-messages.mdTest coverage
14 new tests across 3 new files + 1 updated file:
Regenerate / stale rows (7 tests):
CF_AGENT_CHAT_MESSAGESdoes NOT delete (backward compat)_deleteStaleRowsflag deletes stale rowstriggerstripped fromoptions.body(alone and with custom fields)Transport trigger (3 tests):
Identical-content reconciliation (4 tests):
Notes for reviewers
wait-mcp-connections.test.tstests now pass as well._deleteStaleRowsloop does N+1 queries (1 SELECT + N DELETEs). Fine for typical conversation sizes; batchable later if needed.persistMessagesis public and overridable. The new optional third parameter is backward-compatible.serverCursorentirely. Performance is still O(n·m) worst case (same as before), but the constant factor is slightly higher due to theSetlookups. Negligible for typical conversation sizes.