Skip to content

Fix SSH multi-image image drops - #3795

Closed
lawrencecchen wants to merge 2 commits into
mainfrom
fix-ssh-multi-image-drop
Closed

lawrencecchen wants to merge 2 commits into
mainfrom
fix-ssh-multi-image-drop

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented May 9, 2026 •

Copy link
Copy Markdown
Contributor

Split out the SSH-safe part from #3769 while the local Claude drag path is still under investigation.

Changes:

  • Add a regression for two image-data drops into a cmux-ssh terminal.
  • Teach the debug socket drop helper to simulate image-data pasteboard items.
  • Materialize every image item and upload the full list to the remote terminal.

Verification:

  • Red before fix: FinderFileDropRegressionTests/testImagePasteboardDropMaterializesEveryImageForRemoteUpload failed with 1 URL instead of 2.
  • Green after fix: focused XCTest passed.
  • Tagged socket regression passed: tests_v2/test_ssh_remote_image_drop_upload.py uploaded 2 remote files and hash-checked both.

Note

Medium Risk
Medium risk because it changes pasteboard image materialization and temporary-file lifecycle, which directly affects drag/drop behavior and remote SSH uploads. Failures could lead to missing uploads or leaked temp files across sessions.

Overview
Fixes remote (SSH) image drops to upload every image when the pasteboard contains multiple image items, instead of only materializing a single image.

Refactors GhosttyPasteboardHelper to extract image representations per NSPasteboardItem, normalize TIFF payloads to PNG, add APIs for multi-image materialization (materializeImageFileURLsIfNeeded/saveImageFileURLsIfNeeded), and ensure owned temporary clipboard images are cleaned up on app termination.

Extends the debug socket drop simulation to support an image_data payload (in addition to file_urls) and updates both XCTest and v2 Python regression tests to validate two-image remote upload behavior end-to-end.

Reviewed by Cursor Bugbot for commit 99aea35. Bugbot is set up for automated code reviews on this repo. Configure here.


Summary by cubic

Fixes SSH multi-image drops by materializing every image from the pasteboard and uploading the full set to the remote terminal. Adds a regression test and debug simulation for image-data drops.

  • Bug Fixes

    • Materialize all pasteboard images (direct types, RTFD attachments, TIFF→PNG) and upload the full list over SSH.
    • Enforce per-image size limit (10 MB) and avoid falling back to text/URLs when a real image is rejected.
    • Upload planner now returns all image file URLs, not just the first.
    • Clean up temporary image files after transfer and on app exit.
  • New Features

    • Debug drop simulation supports image-data payloads with a new payload param (image_data or file_urls) and returns route and payload.

Written for commit 99aea35. Summary will update on new commits.

Summary by CodeRabbit

  • New Features

    • Added support for handling multiple images in drag-and-drop and paste operations.
    • Images are now automatically normalized to PNG format for consistency.
  • Bug Fixes

    • Temporary clipboard image files are properly cleaned up on app shutdown.
    • Improved image drop and upload reliability for remote terminal connections.

Review Change Stack

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@vercel

vercel Bot commented May 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cmux Ready Ready Preview, Comment May 9, 2026 8:57am
cmux-staging Building Building Preview, Comment May 9, 2026 8:57am

@coderabbitai

coderabbitai Bot commented May 9, 2026 •

Copy link
Copy Markdown

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This PR extends clipboard image handling from single-image to multi-image materialization. It refactors GhosttyPasteboardHelper to extract image representations per NSPasteboardItem, normalizes TIFF to PNG, saves multiple images to temporary files, and adds cleanup on app shutdown. Debug file-drop simulation now supports an asImageData payload type, and tests validate multi-image drops on remote surfaces.

Changes

Multi-Image Clipboard Drop & Materialization

Layer / File(s) Summary
Data Shape & API Contract
Sources/GhosttyTerminalView.swift
Adds ImageFileListMaterializationResult enum with cases for saved multiple URLs, no decodable image, or rejected payload; adds materializeImageFileURLsIfNeeded and cleanupAllOwnedTemporaryImageFiles public methods.
Pasteboard Item Representation Pipeline
Sources/GhosttyTerminalView.swift
Refactors to extract image representations per NSPasteboardItem instead of pasteboard-wide; introduces TIFF detection, PNG normalization helpers (isTIFFType, normalizedPNGRepresentation), and per-item direct/RTFD/fallback extraction routes that aggregate into imageRepresentations(in:).
Multi-Image Materialization & Cleanup
Sources/GhosttyTerminalView.swift
materializeImageFileURLsIfNeeded iterates pasteboard items, builds representations, saves to temporary files, and tracks ownership; cleanupAllOwnedTemporaryImageFiles() removes all tracked temporary image files at once.
Debug Drop Simulation with Image Payload
Sources/GhosttyTerminalView.swift
debugSimulateFileDrop gains asImageData flag to simulate drops as image pasteboard items instead of file URLs; GhosttySurfaceScrollView wrapper updated to pass through the parameter.
Terminal Control & Image Transfer Integration
Sources/TerminalController.swift, Sources/TerminalImageTransfer.swift
v2DebugSimulateTerminalFileDrop adds TerminalFileDropSimulationPayload enum, parses payload parameter, validates constraints per route (rejects non-file_urls for text-destination), and forwards payload type to debug simulation; TerminalImageTransferPlanner switches to materializeImageFileURLsIfNeeded for multi-image support.
App Shutdown Cleanup
Sources/AppDelegate.swift
applicationWillTerminate calls GhosttyPasteboardHelper.cleanupAllOwnedTemporaryImageFiles() to remove all tracked temporary image files during app exit.
Tests & Test Helpers
cmuxTests/FinderFileDropRegressionTests.swift, tests_v2/cmux.py, tests_v2/test_ssh_remote_image_drop_upload.py
New regression test constructs multi-image pasteboard, validates remote-surface drop plan materializes multiple PNG files with clipboard- prefix; Python test client updated to support payload parameter and return RPC response; SSH remote-upload test refactored to drop, upload, and hash-validate two images end-to-end.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • manaflow-ai/cmux#1361: Modifies GhosttyPasteboardHelper's image-extraction paths to add RTFD-attachment-based fallback.
  • manaflow-ai/cmux#3755: Modifies the same debug terminal file-drop simulation flow and payload handling.
  • manaflow-ai/cmux#1907: Extends GhosttyPasteboardHelper's temporary-image ownership and cleanup behavior.

Poem

🐰 Hops through paste-boards, images galore,
TIFF to PNG, now we store!
Multi-drop magic, files multiply,
Cleanup on exit—goodbye, oh my!
Clipboard dreams come true, hooray! 🎉


Caution

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

  • Ignore

❌ Failed checks (3 errors, 1 warning)

Check name Status Explanation Resolution
Cmux Swift File And Package Boundaries ❌ Error GhosttyTerminalView.swift (13,801 lines) exceeds 800-line threshold. PR adds 319 lines (exceeds 250 limit). No extraction exception: file grew 261 lines, not decreased; no new package created. Extract image materialization logic into a new SwiftPM package owning ImageFileListMaterializationResult, materializeImageFileURLsIfNeeded, and cleanupAllOwnedTemporaryImageFiles, reducing GhosttyTerminalView by 200+ lines.
Cmux Swift Logging ❌ Error Unguarded print() and NSLog() statements in production code violate swift-logging.md: TerminalController.swift socket errors, GhosttyTerminalView initializeGhostty, AppDelegate error logging. Replace with os.log.Logger. Use #if DEBUG for temporary logs. Prefer: nonisolated private let logger = Logger(subsystem: Logging.subsystem, category: "Name")
Cmux Architecture Rethink ❌ Error Introduces NSLock-protected side-channel state for file lifecycle. Patches symptom without fixing invariant: silently drops undecodable items in multi-image pastes. Consolidate file lifecycle ownership in single component. Reject partial-payload pastes using ImageRepresentationScanResult pattern from review comment instead of silent drops.
Docstring Coverage ⚠️ Warning Docstring coverage is 2.17% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (11 passed)
Check name Status Explanation
Title check ✅ Passed The title directly and specifically addresses the main change: fixing SSH multi-image drops instead of single-image handling.
Description check ✅ Passed The description covers all required sections: a clear summary of what changed and why, detailed verification results from testing, and comprehensive checklists.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Cmux Swift Actor Isolation ✅ Passed No Swift 6 actor isolation mistakes introduced. Mutable static state properly synchronized with NSLock (Sendable). New enums/methods follow safe patterns. Tests correctly excluded.
Cmux Swift Blocking Runtime ✅ Passed Uses existing NSLock in established pattern for app shutdown cleanup. Lock is brief, non-blocking, consistent with prior code, not on latency-sensitive paths.
Cmux No Hacky Sleeps ✅ Passed No violations. All sleeps are in test-only code (tests_v2/) with polling timeouts, which is explicitly allowed. Swift code is covered by a different rule.
Cmux Swift Concurrency ✅ Passed Image functions are synchronous. No DispatchQueue.global(), completion handlers, or fire-and-forget Tasks in production code. NSLock appropriately used for sync thread-safety.
Cmux Swift @Concurrent ✅ Passed All Swift changes are synchronous. No async functions added, no incorrect @concurrent annotations. File I/O in sync helpers is acceptable.
Cmux Swiftui State Layout ✅ Passed This PR contains no new SwiftUI state patterns. All changes are to AppKit code and utility helpers (GhosttyPasteboardHelper enum). Existing @Published state in TerminalSurface was not modified.
Cmux Swift Auxiliary Window Close Shortcuts ✅ Passed PR modifies pasteboard image handling and debug drop simulation. No new or materially changed NSWindow, NSPanel, NSWindowController, or SwiftUI Window/WindowGroup. Test-only tests allowed.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix-ssh-multi-image-drop

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Closing this split because it still includes the multi-image SSH behavior. The intended first step, single-image SSH drag/upload, already landed in #3755.

@greptile-apps

greptile-apps Bot commented May 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a regression where dropping multiple images into an SSH-connected cmux terminal only uploaded the first image. The root cause was materializedFileURLs calling saveImageFileURLIfNeeded (which hard-stopped at the first match); the fix introduces a full per-item scan that collects every image representation from the pasteboard before materializing temp files.

  • GhosttyTerminalView.swift: Refactors GhosttyPasteboardHelper to extract image representations item-by-item across the whole pasteboard and adds saveImageFileURLsIfNeeded / materializeImageFileURLsIfNeeded as multi-image public APIs.
  • TerminalImageTransfer.swift: Wires the new multi-image path into materializedFileURLs with a one-line change.
  • Tests: Adds an XCTest regression for 2-item PNG drop producing 2 URLs and upgrades the tagged socket test to drop 2 images and SHA-verify both remote uploads.

Confidence Score: 3/5

The multi-image extraction logic is well-structured and the regression test directly covers the fixed path, but the new public APIs that create and manage named NSPasteboards are missing @mainactor, leaving a static soundness gap for future callers.

The core fix replaces a stop-at-first pattern with a full per-item scan, backed by both a unit regression and a tagged socket integration test. The actor isolation issue on new AppKit-stateful APIs is a real static guarantee gap. A secondary design concern: any single oversized image in a multi-image drop silently discards all valid images in the batch.

Sources/GhosttyTerminalView.swift — specifically the new public multi-image APIs and pasteboardFallbackImageRepresentations; the actor isolation annotations and size-rejection semantics both warrant a closer look before merging.

Important Files Changed

Filename Overview
Sources/GhosttyTerminalView.swift Core change: replaces single-image-stop-first extraction with per-item multi-image extraction, adds pasteboardFallbackImageRepresentations (creates/manages a named NSPasteboard lifecycle), and exposes saveImageFileURLsIfNeeded / materializeImageFileURLsIfNeeded without @mainactor annotation despite full AppKit pasteboard write operations.
Sources/TerminalImageTransfer.swift Minimal one-liner change in materializedFileURLs: replaces saveImageFileURLIfNeeded + wrap-in-array with the new saveImageFileURLsIfNeeded, correctly wiring in the multi-image list path for drops.
Sources/AppDelegate.swift Adds cleanupAllOwnedTemporaryImageFiles() call at app termination to sweep any temp image files that survived transfer; ordering intentional.
Sources/TerminalController.swift Extends the debug socket drop helper with a payload parameter (file_urls / image_data); guards image_data against the textDestination route.
cmuxTests/FinderFileDropRegressionTests.swift Adds regression test: writes two PNG items, asserts plan == .uploadFiles with 2 URLs, verifies both files exist on disk.
tests_v2/cmux.py Adds payload parameter to simulate_terminal_file_drop; backward-compatible default keeps existing callers unaffected.
tests_v2/test_ssh_remote_image_drop_upload.py Upgrades SSH image drop test to two images, polls for 2 remote paths, SHA-256 checks both with sorted comparison.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["NSPasteboard (drop)"] --> B["imageRepresentations(in pasteboard:)"]
    B --> C{"pasteboardItems?"}
    C -- "for each item" --> D["imageRepresentations(in item:)"]
    D --> E{"directImageRepresentation?"}
    E -- yes --> F["[(data, ext)]"]
    E -- no --> G{"rtfdAttachments?"}
    G -- yes --> F
    G -- no --> H{"fallbackImageRepresentation(item)?"}
    H -- yes --> F
    H -- no --> I["pasteboardFallbackImageRepresentations(for: item)"]
    I --> J["copiedPasteboardItem to new named NSPasteboard"]
    J --> K{"direct / rtfd / fallback?"}
    K -- found --> F
    K -- not found --> L["[]"]
    F --> M["flatMap all items to itemRepresentations"]
    M --> N{"empty?"}
    N -- no --> O["Return all item representations"]
    N -- yes --> P["Whole-pasteboard fallback (single-image)"]
    O --> Q["materializeImageFileURLs"]
    P --> Q
    Q --> R{"Each repr <= 10 MB?"}
    R -- yes --> S["Write temp file, register ownership"]
    R -- "any exceeds" --> T["cleanupTransferred, rejectedImagePayload"]
    S --> U[".saved([URL])"]
Loading

Reviews (1): Last reviewed commit: "Fix SSH multi-image image data drops" | Re-trigger Greptile

Comment on lines +708 to +732
private static func pasteboardFallbackImageRepresentations(
for item: NSPasteboardItem
) -> [(data: Data, fileExtension: String)] {
guard let copiedItem = copiedPasteboardItem(from: item) else { return [] }

let pasteboard = NSPasteboard(name: .init("cmux-single-image-item-\(UUID().uuidString)"))
pasteboard.clearContents()
defer {
pasteboard.clearContents()
pasteboard.releaseGlobally()
}
guard pasteboard.writeObjects([copiedItem]) else { return [] }

if let directImage = directImageRepresentation(in: pasteboard) {
imageData = directImage.data
fileExtension = directImage.fileExtension
} else if let rtfdAttachment = rtfdAttachmentImageRepresentation(in: pasteboard) {
imageData = rtfdAttachment.data
fileExtension = rtfdAttachment.fileExtension
} else {
guard hasImageData(in: pasteboard),
let image = NSImage(pasteboard: pasteboard),
let tiffData = image.tiffRepresentation,
let bitmap = NSBitmapImageRep(data: tiffData),
let pngData = bitmap.representation(using: .png, properties: [:]) else {
return .noDecodableImagePayload
return [directImage]
}
let rtfdAttachments = rtfdAttachmentImageRepresentations(in: pasteboard)
if !rtfdAttachments.isEmpty {
return rtfdAttachments
}
if let fallbackImage = fallbackImageRepresentation(in: pasteboard) {
return [fallbackImage]
}
return []
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Missing @MainActor on new AppKit-stateful public APIs

saveImageFileURLsIfNeeded, materializeImageFileURLsIfNeeded, and the private pasteboardFallbackImageRepresentations(for:) all reach into AppKit far beyond a simple read: pasteboardFallbackImageRepresentations creates a named NSPasteboard, calls clearContents(), writeObjects(), reads back data, and releases the pasteboard in a defer. copiedPasteboardItem also eagerly resolves lazy/promised pasteboard types via IPC. None of these carries @MainActor, so a future Task {} or DispatchQueue.global caller compiles cleanly but races against the main thread on AppKit internals. The existing single-read helpers already had this gap, but the new code adds full pasteboard lifecycle management (create → write → read → releaseGlobally) off-annotation, materially widening the exposure.

Rule Used: Flag new or materially worsened Swift 6 actor isol... (source)

Comment on lines +786 to +793
for representation in representations {
guard representation.data.count <= maxClipboardImageSize else {
#if DEBUG
cmuxDebugLog("terminal.paste.image.rejected reason=tooLarge bytes=\(imageData.count)")
cmuxDebugLog("terminal.paste.image.rejected reason=tooLarge bytes=\(representation.data.count)")
#endif
return .rejectedImagePayload
cleanupTransferredTemporaryImageFiles(fileURLs)
return .rejectedImagePayload
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 All-or-nothing size rejection drops valid images alongside oversized ones

When iterating a multi-image drop, hitting the 10 MB guard on any single representation calls cleanupTransferredTemporaryImageFiles(fileURLs) on all already-written files and returns .rejectedImagePayload for the entire batch. A user who drops two images — one 4 MB and one 12 MB — gets nothing uploaded. The old single-image code had the same early-return semantics but only ever considered one file, so this is a new all-or-nothing behaviour. The more intuitive outcome would be to skip (or reject individually) the oversized item while still returning the valid ones.

This branch was successfully deployed

1 active deployment
Preview – cmux — 99aea35d Deployed May 9, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant