Skip to content

Virtualize long chat transcripts - #458

Merged
cfal merged 70 commits into
mainfrom
virtual-chat
Aug 6, 2026
Merged

Virtualize long chat transcripts#458
cfal merged 70 commits into
mainfrom
virtual-chat

Conversation

@cfal

@cfal cfal commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Long chat histories currently mount every loaded message, making deep conversations progressively slower and less stable. This change virtualizes the primary chat feed with TanStack Virtual while preserving pinned and detached scrolling, history paging, navigation, transient interaction state, accessibility announcements, and generation-aware transcript windows, with focused browser and state coverage for large variable-height conversations.

Stabilization rationale

  • TanStack Virtual core 3.17.7 lets a connected row with a stale data-index reach key lookup and measurement after the item count shrinks. The branch pins the exact upstream fix (#1246, commit d2cf98be) as a Bun package patch (patches/README.md documents the removal condition), deletes the local count-shrink measurement-reset protocol it replaced, and keeps text scale as the only global measurement reset.
  • Browser suites hard-fail on a stale web/build: the build-input hash covers client sources, the root bun.lock, and patches/, and the production build environment is normalized so NODE_ENV=test processes can validate production builds.

Chat open and switch behavior (user-visible)

  • The initial-bottom paint gate now holds through the staged initial reveal, viewport fill, and end restoration, and also hides the overlay scrollbar. Opening or switching chats shows a brief loading indicator and then the feed already settled at the bottom, instead of content flickering through intermediate scroll positions and thumb sizes.
  • Pinned publications (late history chunks, streamed growth, compaction) anchor to the physical end after the sizer commits but before the browser paints, so they no longer paint a frame at the pre-mutation offset.

Product decisions

  • Browser Find (Ctrl+F) cannot see unmounted virtualized history; deep-history search relies on in-app navigation. Accepted tradeoff.
  • Boundary prefetch uses one viewport height (minimum 100px) in both directions; 50px survives only as the live-end re-pin threshold.
  • The range extractor retains the pinned transcript tail in addition to the four retention reasons.

Release blockers (manual evidence pending)

  • iOS Safari: touch/momentum interrupt, prepend during momentum, detached streaming, touch re-pin.
  • VoiceOver/Safari and NVDA/Firefox: pinned streaming, detached one-shot response status, history paging silence, return-to-latest without replay.
  • Chrome performance profile at 2,000 and 20,000 loaded items during final-row streaming.

Migration notes

Client-only change. No server, API, WebSocket, or storage migrations.

@cfal
cfal force-pushed the virtual-chat branch 4 times, most recently from 7f38f29 to c23a13b Compare August 5, 2026 13:19
@cfal
cfal marked this pull request as ready for review August 5, 2026 14:10
cfal added 27 commits August 6, 2026 11:44
Revealing after the first settled batch let every remaining reveal batch
re-anchor the end in plain view, and the overlay scrollbar thumb was never
gated at all, so chat switches flickered through intermediate scroll
positions and thumb sizes before settling at the bottom.
Explicit end scrolls and destroy now cancel the hidden reading offset per
the cancellation ledger, destroyed controllers no longer arm pending end
scrolls, and hidden-restore offset writes revalidate surface identity so a
replacement during an in-flight restore cannot scroll the new surface to
the old offset. Adds direct coverage for user-intent cancellation, the
detached text-scale anchor branch, single subscription teardown, and
projection count-shrink emitting no global reset.
Samples every frame across chat switches to reject content painted away
from the physical end or visible through the initial paint gate, holds the
data revision across the stable-layout window, and adds pinned hide/show,
hidden text-scale, and post-shrink later-publication scenarios.
Gates the git-history Chromium suite on a current web build, fixes the
vacuous concurrent-change cache test, covers corrupt and wrong-version
build markers, tightens the build-cache type declarations, and renames the
stale Chromium workflow job label.
…ss probe

Pinned publications wrote their end correction one animation frame after
commit, painting a frame at the pre-mutation offset for every late history
chunk or streamed publication; the correction now lands after the sizer
commits but before paint, with the convergence loop verifying later frames.
The viewport-fill probe also treated itemSizeCache membership as proof of
measurement, so wrappers rendered exactly at their estimate probed forever;
rendered keys now resolve to their exact estimates.
The pinned show path's end convergence can be superseded by a concurrent
show-time publication, stranding the viewport short of the physical end
after late measurements; the scroll controller now rechecks the end once
after fill and layout settle. Anchor capture and root-offset observation
move to the runtime module, and the Lightpanda paging test awaits the
initial paint gate instead of asserting instant readiness.
Settle diagnostics now name the failing scenario, and the show-time end
recheck retries across up to three layout-settle windows so deferred scale
invalidation that lands after the first recheck still converges.
A root-offset change while pinned at the end moves the physical end outside
TanStack's anchoring with no repin owner; the margin observer now restores
the end when the viewport rested there. Stable-layout diagnostics carry the
failing scenario name inside the thrown error.
Remove the staged initial-reveal batching so activating a cached chat
publishes the bounded transcript window atomically; virtualization still
limits mounted row work.

Harden the TanStack virtual-core patch to ignore delayed in-range resize
entries whose stale index now resolves to a key owned by another connected
element, preventing misattributed row sizes during history prepends. Add
patch contract and geometry stability coverage.
@cfal
cfal merged commit 6fede5f into main Aug 6, 2026
9 checks passed
@cfal
cfal deleted the virtual-chat branch August 6, 2026 06:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant