perf: reduce WindowServer compositing cost and layout measurement overhead

[devin-ai-integration]

Summary

Four targeted performance improvements for the message list, addressing WindowServer compositing hangs (~10s) and layout measurement overhead (~1s) traced via spin dumps (LUM-713).

Why this is needed

Full-resolution images (e.g., 4000×3000 = 48MB RGBA bitmap) were uploaded as GPU textures even though display size is capped at ~580pt. With multiple images in a conversation, cumulative GPU texture memory caused WindowServer compositing stalls. A synchronous GeometryReader inside the LazyVStack created a layout dependency that blocked measurement. Interleaved messages (text + tool calls + images) were excluded from .compositingGroup(), leaving deep CALayer trees for the WindowServer to traverse.

Changes

1. Image downsampling via CGImageSourceCreateThumbnailAtIndex — Two downsampleForDisplay() overloads:

   • data: overload (preferred): Feeds raw compressed bytes directly to CGImageSourceCreateWithData, decoding only the pixels needed for the target size — no full-resolution bitmap allocation.
   • NSImage overload (fallback): For pre-decoded NSImage objects (tool-call cached images, thumbnailImage), extracts TIFF representation and feeds it to ImageIO.
   • Applied in InlineToolCallImageView and all AttachmentImageGrid decode paths. Task key includes displayScale so downsampling re-runs at the correct pixel density when moving between displays.

2. GeometryReader → .onGeometryChange() in InlineAudioAttachmentView.progressBar — Eliminates a synchronous layout dependency inside the LazyVStack. The track width is now measured asynchronously and stored in @State.

3. Extend .compositingGroup() to completed interleaved messages — Removes the cachedHasInterleavedContent exclusion so all non-streaming messages get layer flattening. Still skipped during streaming to avoid re-compositing on every token delta.

4. Lightbox full-resolution recovery — Since inline previews are now downsampled, the lightbox decodes full-resolution images asynchronously. decodeBase64LightboxImage uses thread-safe ImageIO APIs (CGImageSourceCreateWithData + CGImageSourceCreateImageAtIndex) on a Task.detached off the main thread, then sets fullResImage on completion. The lightbox initially shows the downsampled preview and swaps to full-res once decoded.

Why this is safe

• Image downsampling only affects inline display; the original full-resolution data is preserved for lightbox, copy, save, and drag operations.
• The GeometryReader replacement is a direct 1:1 swap — .onGeometryChange() provides the same width measurement without the synchronous layout dependency.
• The .compositingGroup() extension is gated by !message.isStreaming, so active token streaming is unaffected. Only completed messages get the flattened layer.
• The lightbox decode uses Task.detached with cancellation support (lightboxFetchTask) matching the existing lazy-fetch pattern.

Alternatives considered and rejected

• NSImage.preparingThumbnail(of:) — Only available on iOS/tvOS, not macOS. CGImageSourceCreateThumbnailAtIndex is the macOS equivalent recommended in WWDC18.
• Moving all image decoding to background actors — .task blocks inherit MainActor isolation for cancellation semantics tied to view lifecycle. Moving decode off-MainActor would require manual cancellation tracking. The direct data: → ImageIO path avoids the main-thread bitmap allocation that was the actual problem, making the actor migration unnecessary for this fix.
• Restoring the interleaved content exclusion — The original exclusion was added preemptively to avoid re-compositing during async completions. However, interleaved messages have the deepest CALayer trees, making them the most expensive without flattening. Since isStreaming already gates active token deltas and the remaining async work on completed messages is infrequent, the tradeoff favors flattening. If profiling with Core Animation Instruments shows regressions on interleaved-heavy conversations, the exclusion can be restored.
• Pre-computing cell heights to avoid LazyVStack re-measurement — This would address the deeper layout hang tracked in LUM-716, but is a larger architectural change outside the scope of this fix.

References

• WWDC18 — Images and Graphics Best Practices — CGImageSourceCreateThumbnailAtIndex for memory-efficient downsampling
• CGImageSourceCreateThumbnailAtIndex docs
• .onGeometryChange() docs — async geometry measurement replacing GeometryReader
• .compositingGroup() docs — flattening render tree into single compositing layer

Review & Testing Checklist for Human

• Build in Xcode — CI has no macOS build step; must verify locally that the project compiles without errors.
• Verify lightbox image quality — Open a lightbox for a non-lazy attachment (one with inline base64 data). Confirm it shows full-resolution after the async decode completes. Verify copy/save toolbar actions produce full-resolution output.
• Verify interleaved message rendering — Open a conversation with mixed content (images + tool calls + code blocks). Confirm no visual glitches or flickering on completed messages. If possible, profile with Core Animation Instruments before/after to confirm the compositing tradeoff is net-positive.
• Verify inline image quality — Check that downsampled images in chat don't appear blurry or pixelated on Retina displays. If multi-display setup available, drag the window between Retina and non-Retina screens and verify images re-render at correct resolution (expect a brief flash during re-decode).
• Verify audio progress bar — Play an audio attachment and scrub (tap-to-seek) the progress bar. Confirm the filled portion renders correctly during playback.

Notes

• The downsampleForDisplay(_:NSImage) fallback path calls image.tiffRepresentation on the main thread via .task. This is not a regression (pre-existing code called NSImage(data:) on the main thread), and this fallback is only hit for tool-call cached images and thumbnailImage — the common base64 paths use the direct data: overload.
• The fetchFullResLightboxImage path (for lazy-loaded attachments fetched from the server) still uses NSImage(data:), but runs on @MainActor where AppKit APIs are safe. Only the Task.detached base64 decode path uses CGImageSource for thread safety.
• Display scale changes (dragging between Retina/non-Retina) will cancel and re-run all visible image decode tasks. This is correct behavior but causes a brief flash to placeholder state — an acceptable tradeoff for rendering at the correct pixel density.
• The deeper 22–70s layout hangs from per-cell view hierarchy depth (50+ nested layout nodes causing recursive sizeThatFits / placeChildren stalls during scrolling and streaming) are tracked separately in LUM-716.

Link to Devin session: https://app.devin.ai/sessions/1f18acbafb0043a9afa2662c3fdb533c
Requested by: @ashleeradka

---

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[devin-ai-integration]

Devin Review found 1 new potential issue.

View 6 additional findings in Devin Review.

[devin-ai-integration]

🟡 Unnecessary loading spinner shown in lightbox when image is already full-resolution

The change to isLoadingFullRes: lazyAttachmentId != nil || base64Data != nil causes a spurious loading spinner to appear in the lightbox when callers pass an already-decoded full-resolution image alongside base64Data. Specifically, ComposerAttachments.openAttachmentPreview (ComposerAttachments.swift:102-121) decodes the full-res image synchronously via NSImage(data:) and passes both the decoded image and the raw base64Data to showImageLightbox. Before this PR, isLoadingFullRes was lazyAttachmentId != nil (false here), so no spinner appeared. Now isLoadingFullRes is true whenever base64Data is non-nil, triggering the spinner overlay (ImageLightboxOverlay.swift:31-35) on top of the already-visible full-res image while decodeBase64LightboxImage redundantly re-decodes the same data (~50-200ms). The user sees a confusing loading indicator over a fully loaded image.

Prompt for agents

The root cause is that showImageLightbox unconditionally sets isLoadingFullRes=true when base64Data is provided, but some callers (ComposerAttachments.openAttachmentPreview) have already decoded the full-res image and pass it as the image parameter. The fix should distinguish between callers that pass a downsampled thumbnail (need async full-res decode) and callers that pass the already-decoded full-res image (no decode needed).

Possible approaches:
1. Add a parameter like needsFullResDecode: Bool to showImageLightbox so callers can opt out of the async decode when they already have full-res.
2. Have showImageLightbox only set isLoadingFullRes and call decodeBase64LightboxImage when the passed image appears to be downsampled (e.g. check pixel dimensions vs some threshold).
3. Update ComposerAttachments.openAttachmentPreview to not pass base64Data (pass nil) since it already provides the full-res image, though this would prevent the lightbox toolbar from using base64Data for copy/save actions.

Option 1 is cleanest. The new parameter defaults to true for backward compatibility with the chat bubble path (which now passes downsampled images), and ComposerAttachments passes false.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

Devin is archived and cannot be woken up. Please unarchive Devin if you want to continue using it.