Skip to content

Render local images in the Markdown Viewer - #2815

Closed
luoxi wants to merge 2 commits into
manaflow-ai:mainfrom
luoxi:feat/markdown-local-images
Closed

luoxi wants to merge 2 commits into
manaflow-ai:mainfrom
luoxi:feat/markdown-local-images

Conversation

@luoxi

@luoxi luoxi commented Apr 11, 2026 •

Copy link
Copy Markdown

Summary

  • add LocalFileImageProvider that loads local images via NSImage and delegates
    remote URLs to swift-markdown-ui's DefaultImageProvider so remote sizing stays
    on the upstream ResizeToFit path
  • add LocalFileImageLoader, a pure URL classifier that normalizes relative URLs
    via absoluteURL; for local file loads it ignores ?query / #fragment and
    handles percent-encoded paths cleanly
  • wire imageBaseURL (the .md file's parent directory) into Markdown(...) so
    relative image destinations resolve against it — regular markdown link
    resolution is untouched, baseURL is left at the default
  • cache decoded NSImages in an NSCache keyed by path + file mtime, bounded
    at 64 entries, so a remounted image view (panel closed and reopened after an
    edit) picks up the new bytes instead of reusing a stale entry
  • run the cache stat and image load off the main thread so the markdown panel
    re-render path stays free of blocking file I/O
  • render a localized "Image not found" placeholder for missing files and
    unsupported schemes

Addresses the local-image portion of #2069. Mermaid rendering is out of
scope and is being tracked separately in #2463.

Demo

cmux-markdown-local-images

Test plan

  • relative paths (./images/foo.png, images/foo.png, ../foo.png) render
  • absolute paths and file:// URLs render
  • percent-encoded filenames (spaces, unicode) round-trip cleanly
  • ?query / #fragment suffixes are stripped before load
  • missing files show the localized placeholder, no crash
  • https URLs keep rendering via DefaultImageProvider (no regression)
  • unit coverage in cmuxTests/LocalFileImageLoaderTests.swift for the
    URL classifier and the path+mtime cache key

Testing

  • No local unit/xcodebuild test run per repo policy; manual verification only
  • Built tagged Debug app: ./scripts/reload.sh --tag fix-md-local-images
  • Manual verification on the tagged Debug build with /tmp/md-test/edge.md
    fixture covering every case in the plan above

Summary by CodeRabbit

  • New Features

    • Markdown panels now display local file images; relative paths resolve from the markdown file location
    • Local and remote images supported with caching to improve load performance and reduce reloads
  • Bug Fixes

    • Failed or missing images show a user-friendly "Image not found" placeholder; stale image flashes prevented
  • Localization

    • Added English and Japanese strings for the missing-image message
  • Tests

    • Added tests covering URL classification, relative resolution, and cache key behavior

@vercel

vercel Bot commented Apr 11, 2026

Copy link
Copy Markdown

@luoxi is attempting to deploy a commit to the Manaflow Team on Vercel.

A member of the Team first needs to authorize it.

@coderabbitai

coderabbitai Bot commented Apr 11, 2026 •

Copy link
Copy Markdown
📝 Walkthrough

Walkthrough

Adds local-file image support for markdown: URL classification, a LocalFileImageProvider with mtime-keyed NSImage caching and cancellation-safe async loading, integration into MarkdownPanelView, a localized missing-image string, Xcode project entries, and unit tests for loader and cache.

Changes

Cohort / File(s) Summary
Project Configuration
GhosttyTabs.xcodeproj/project.pbxproj
Registered new source and test files with PBXFileReference/PBXBuildFile entries and added them to the app and test targets' sources build phases.
URL Classification
Sources/Panels/LocalFileImageLoader.swift
New LocalFileImageLoader enum and classify(_:) logic to normalize URLs and return .local(URL), .remote(URL), or .unsupported.
Image Provider & Cache
Sources/Panels/LocalFileImageProvider.swift
New LocalFileImageProvider: ImageProvider implementing remote delegation, async local NSImage loading, cancellation handling, and LocalFileImageCache (mtime-derived NSString keys, NSCache).
Markdown Integration
Sources/Panels/MarkdownPanelView.swift
Compute imageBaseURL from panel.filePath and attach LocalFileImageProvider() via .markdownImageProvider(...).
Localization
Resources/Localizable.xcstrings
Added markdown.image.missing.label with en ("Image not found") and ja ("画像が見つかりません") entries.
Tests
cmuxTests/LocalFileImageLoaderTests.swift
New tests for LocalFileImageLoader.classify(_:) (schemes, percent-decoding, relative resolution) and LocalFileImageCache.key(for:) behavior against filesystem mtime using temp files.

Sequence Diagram

sequenceDiagram
    participant MarkdownView
    participant LocalFileImageProvider
    participant LocalFileImageLoader
    participant LocalFileImageCache
    participant FileSystem

    MarkdownView->>LocalFileImageProvider: request image(url)
    LocalFileImageProvider->>LocalFileImageLoader: classify(url)
    LocalFileImageLoader-->>LocalFileImageProvider: Kind(.local | .remote | .unsupported)

    alt remote
        LocalFileImageProvider->>LocalFileImageProvider: delegate to DefaultImageProvider
        LocalFileImageProvider-->>MarkdownView: rendered remote image
    else local
        LocalFileImageProvider->>LocalFileImageCache: key = key(for: fileURL)?
        LocalFileImageCache-->>LocalFileImageProvider: cached image or nil
        alt cache miss
            LocalFileImageProvider->>FileSystem: load NSImage(contentsOf: fileURL)
            FileSystem-->>LocalFileImageProvider: NSImage or error
            LocalFileImageProvider->>LocalFileImageCache: store(image, key)
        end
        LocalFileImageProvider-->>MarkdownView: rendered local image or placeholder
    else unsupported/missing
        LocalFileImageProvider-->>MarkdownView: show placeholder + localized label
    end
Loading

Estimated Code Review Effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Poem

🐰
I hopped through paths both near and far,
Found images tucked inside each jar.
Cached by mtime, shown with care,
Missing ones get a gentle stare.
Hop on — markdown's brighter there! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: adding local image rendering support to the Markdown Viewer, which is the core objective of the PR.
Description check ✅ Passed The description covers all key aspects: detailed summary of what changed and why, comprehensive test plan with checkboxes showing verification, demo video, and checklist items completed except bot reviews.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 6 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="Sources/Panels/LocalFileImageProvider.swift">

<violation number="1" location="Sources/Panels/LocalFileImageProvider.swift:32">
P2: Local image state is reused across URL changes, allowing a stale previously loaded image to flash before the new load task resets state.</violation>

<violation number="2" location="Sources/Panels/LocalFileImageProvider.swift:64">
P2: Detached image-loading task is not cancellation-linked to the SwiftUI lifecycle task, so stale decode/I/O can continue after view task cancellation.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread Sources/Panels/LocalFileImageProvider.swift Outdated
Comment thread Sources/Panels/LocalFileImageProvider.swift Outdated
@greptile-apps

greptile-apps Bot commented Apr 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds local image rendering to the Markdown viewer by introducing LocalFileImageLoader (URL classifier), LocalFileImageProvider (SwiftUI ImageProvider), and an mtime-keyed NSCache, wired into MarkdownPanelView via imageBaseURL. All P2 findings — no blocking issues.

Confidence Score: 5/5

Safe to merge; all remaining findings are non-blocking style suggestions.

The core logic is correct: URL classification handles all cases (file/http/https/relative/schemeless/unsupported), percent-encoding and query/fragment stripping work correctly, the cache key uses path + mtime to avoid stale bitmap reuse, and state transitions in LocalFileImageView are sound. Both findings are P2: the Task.detached cancellation gap is a minor performance concern (not a correctness bug), and the pbxproj ID format is a stylistic deviation that doesn't affect build correctness.

GhosttyTabs.xcodeproj/project.pbxproj (non-standard object IDs) and Sources/Panels/LocalFileImageProvider.swift (Task.detached cancellation).

Important Files Changed

Filename Overview
Sources/Panels/LocalFileImageLoader.swift URL classifier that dispatches to local/remote/unsupported; correctly strips query/fragment via URL.path and handles percent-encoding and relative URLs via absoluteURL.
Sources/Panels/LocalFileImageProvider.swift ImageProvider that loads local files asynchronously with mtime-keyed NSCache; Task.detached doesn't propagate cancellation from the parent .task(id:) when fileURL changes rapidly.
Sources/Panels/MarkdownPanelView.swift Wires imageBaseURL (md file parent directory) and LocalFileImageProvider into Markdown; no changes to link resolution or baseURL.
cmuxTests/LocalFileImageLoaderTests.swift Good behavioural test coverage: URL classification (http/https/file/mailto/data/schemeless), percent-encoding, query/fragment stripping, relative URL resolution against a base, and mtime-keyed cache stability.
GhosttyTabs.xcodeproj/project.pbxproj Registers two new source files and one test file; uses non-standard 10-char alphanumeric object IDs (A500LFI*) instead of Xcode's conventional 24-char hex UUIDs.
Resources/Localizable.xcstrings Adds localized string for the "Image not found" placeholder label; consistent with the codebase's localization convention.

Sequence Diagram

sequenceDiagram
    participant MPV as MarkdownPanelView
    participant MUI as MarkdownUI
    participant LFIP as LocalFileImageProvider
    participant LFIL as LocalFileImageLoader
    participant LFIView as LocalFileImageView
    participant Cache as LocalFileImageCache
    participant FS as FileSystem

    MPV->>MUI: Markdown(content, imageBaseURL)
    MPV->>MUI: markdownImageProvider(LocalFileImageProvider)
    MUI->>LFIP: makeImage(url)
    LFIP->>LFIView: render LocalFileImageView
    LFIView->>LFIL: classify(url)

    alt local file
        LFIL-->>LFIView: .local(fileURL)
        LFIView->>Cache: key(for fileURL) - stat mtime
        Cache->>FS: attributesOfItem
        FS-->>Cache: mtime
        alt cache hit
            Cache-->>LFIView: NSImage
        else cache miss
            LFIView->>FS: NSImage contentsOf fileURL
            FS-->>LFIView: NSImage
            LFIView->>Cache: store image
        end
        LFIView-->>MPV: Image with ResizeToFit layout
    else remote URL
        LFIL-->>LFIView: .remote(url)
        LFIView->>MUI: DefaultImageProvider makeImage
        MUI-->>MPV: remote image
    else unsupported or nil
        LFIL-->>LFIView: .unsupported
        LFIView-->>MPV: Image not found placeholder
    end
Loading

Reviews (1): Last reviewed commit: "Render local images in the Markdown View..." | Re-trigger Greptile

Comment thread GhosttyTabs.xcodeproj/project.pbxproj Outdated
Comment on lines +39 to +40
A5001421 /* MarkdownPanelView.swift in Sources */ = {isa = PBXBuildFile; fileRef = A5001419 /* MarkdownPanelView.swift */; };
A500LFI011 /* LocalFileImageLoader.swift in Sources */ = {isa = PBXBuildFile; fileRef = A500LFI010 /* LocalFileImageLoader.swift */; };

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Non-standard pbxproj object IDs

The new object IDs (A500LFI010, A500LFI011, A500LFI020, A500LFI021, A500LFI030, A500LFI031) contain non-hex letters and are only 10 characters, whereas Xcode generates 24-character hex strings. While pbxproj accepts arbitrary string keys, manually crafted short IDs like these increase the chance of collisions as the project grows and may confuse some Xcode tooling (e.g. xcodebuild -list, merge conflict resolution). Consider regenerating these entries through Xcode's "Add Files…" flow to get canonical IDs.

Comment on lines +64 to +74
let loaded = await Task.detached(priority: .userInitiated) { () -> NSImage? in
let cacheKey = LocalFileImageCache.key(for: fileURL)
if let cacheKey, let cached = LocalFileImageCache.shared.object(forKey: cacheKey) {
return cached
}
guard let image = NSImage(contentsOf: fileURL) else { return nil }
if let cacheKey {
LocalFileImageCache.shared.setObject(image, forKey: cacheKey)
}
return image
}.value

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Task.detached doesn't inherit cancellation

When SwiftUI's .task(id: fileURL) cancels the outer task (because fileURL changes or the view is removed), the detached task is not a child and continues to completion. Because Task<NSImage?, Never>.value has a Never error type it never throws CancellationError, so the outer task silently waits for the full decode before honouring the cancellation signal. Back-to-back file navigations can therefore queue up multiple simultaneous background decodes.

Adding a Task.isCancelled guard inside the detached closure — both before the cache stat and before the NSImage decode — would let the hot path abort early when the result is no longer needed.

luoxi added 2 commits April 14, 2026 17:28
The Markdown Viewer panel used swift-markdown-ui's default image provider,
which only handles http(s) URLs, so every local image reference — relative
paths, absolute paths, and file:// URLs — rendered as a broken placeholder.

Introduce LocalFileImageProvider and a pure URL classifier that together
resolve every local form against the .md file's parent directory, load
NSImage off the main thread, and cache decoded bitmaps in a bounded
NSCache. The cache key combines path and file mtime so a remounted image
view (panel closed and reopened after an edit) picks up the new bytes
instead of reusing a stale entry. Local images follow swift-markdown-ui's
shrink-only sizing; remote URLs delegate straight to DefaultImageProvider;
missing files render a localized placeholder. Percent-encoded, fragmented,
and query-suffixed paths round-trip cleanly.

The viewer is wired through imageBaseURL only, so regular markdown link
resolution is unchanged.

Addresses the local-image portion of #2069. Mermaid rendering is out of
scope and is being tracked separately in #2463.
Gate LocalFileImageView's render on a URL match so a stale outcome from
a previous fileURL cannot flash through during a URL transition before
the new load task updates state.

Forward the parent .task(id:) cancellation into the detached image-loading
task via withTaskCancellationHandler so an in-flight stat + NSImage load
short-circuits when the view moves on to a new URL.

Renumber the six new pbxproj object IDs from the ad-hoc A500LFI0xx form
to 24-character hex so they line up with the rest of the project file.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
Sources/Panels/LocalFileImageProvider.swift (1)

113-127: Consider adding a memory-based limit alongside the count limit.

The cache is bounded by entry count (countLimit = 64), but image sizes can vary significantly. A single large image could consume substantial memory.

Optional enhancement: Add totalCostLimit
 static let shared: NSCache<NSString, NSImage> = {
     let cache = NSCache<NSString, NSImage>()
     cache.name = "cmux.markdown.localFileImage"
     cache.countLimit = 64
+    // ~50 MB rough memory bound
+    cache.totalCostLimit = 50 * 1024 * 1024
     return cache
 }()

Then when storing, pass the image size as cost:

 if let cacheKey {
-    LocalFileImageCache.shared.setObject(image, forKey: cacheKey)
+    let cost = Int(image.size.width * image.size.height * 4) // rough bytes estimate
+    LocalFileImageCache.shared.setObject(image, forKey: cacheKey, cost: cost)
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@Sources/Panels/LocalFileImageProvider.swift` around lines 113 - 127,
LocalFileImageCache currently only sets a countLimit (countLimit = 64) so a
single large NSImage can still blow memory; add a memory-based bound by setting
shared.totalCostLimit to a suitable byte limit (e.g., a few MBs to match app
requirements) and when inserting into the cache use
NSCache.setObject(_:forKey:cost:) with the image's approximate memory cost.
Locate the LocalFileImageCache.shared initializer to add totalCostLimit and
ensure callers that store images (places that call
LocalFileImageCache.shared.setObject or similar) compute a cost for the NSImage
(e.g., from tiffRepresentation?.count or using bitmapRepresentation bytesPerRow
* pixelsHigh) and pass that value to setObject(_:forKey:cost:).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@Sources/Panels/LocalFileImageProvider.swift`:
- Around line 113-127: LocalFileImageCache currently only sets a countLimit
(countLimit = 64) so a single large NSImage can still blow memory; add a
memory-based bound by setting shared.totalCostLimit to a suitable byte limit
(e.g., a few MBs to match app requirements) and when inserting into the cache
use NSCache.setObject(_:forKey:cost:) with the image's approximate memory cost.
Locate the LocalFileImageCache.shared initializer to add totalCostLimit and
ensure callers that store images (places that call
LocalFileImageCache.shared.setObject or similar) compute a cost for the NSImage
(e.g., from tiffRepresentation?.count or using bitmapRepresentation bytesPerRow
* pixelsHigh) and pass that value to setObject(_:forKey:cost:).

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 6e30c45d-b3ba-4900-a6a9-91b75d52c47a

📥 Commits

Reviewing files that changed from the base of the PR and between 67c9e30 and d46ab88.

📒 Files selected for processing (6)
  • GhosttyTabs.xcodeproj/project.pbxproj
  • Resources/Localizable.xcstrings
  • Sources/Panels/LocalFileImageLoader.swift
  • Sources/Panels/LocalFileImageProvider.swift
  • Sources/Panels/MarkdownPanelView.swift
  • cmuxTests/LocalFileImageLoaderTests.swift
✅ Files skipped from review due to trivial changes (2)
  • Sources/Panels/MarkdownPanelView.swift
  • Resources/Localizable.xcstrings
🚧 Files skipped from review as they are similar to previous changes (2)
  • GhosttyTabs.xcodeproj/project.pbxproj
  • Sources/Panels/LocalFileImageLoader.swift

@luoxi

luoxi commented Jun 5, 2026

Copy link
Copy Markdown
Author

Closing this PR — it's been superseded by #4288 ("Fix markdown viewer image rendering"), which already landed on main.

While this PR was open, the Markdown Viewer was rewritten from the swift-markdown-ui (Markdown(...)) rendering path to a WKWebView-based renderer (MarkdownWebRenderer). #4288 implements image rendering directly in that web renderer through custom URL scheme handlers:

  • Local images via the cmux-local-image scheme handler, resolved against the markdown file's directory — the local-image portion of Markdown viewer: render images and mermaid code blocks #2069 that this PR targeted.
  • data: images, which also rendered as broken placeholders before.
  • Remote images behind a hardened, HTTPS-only loader with a host allowlist, per-URL consent, and a size cap.

This PR's LocalFileImageProvider / LocalFileImageLoader were written against the swift-markdown-ui ImageProvider API, which the viewer no longer uses, so rebasing onto main would leave them as dead code targeting a rendering path that's been removed.

Closing as superseded. Leaving #2069 open for the remaining mermaid work.

@luoxi luoxi closed this Jun 5, 2026
@luoxi
luoxi deleted the feat/markdown-local-images branch June 5, 2026 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant