Skip to content

feat: adaptive progressive image loading for photo viewer#26636

Merged
alextran1502 merged 8 commits intomainfrom
push-zunuwtznrlpm
Mar 11, 2026
Merged

feat: adaptive progressive image loading for photo viewer#26636
alextran1502 merged 8 commits intomainfrom
push-zunuwtznrlpm

Conversation

@midzelis
Copy link
Collaborator

@midzelis midzelis commented Mar 2, 2026

In this screenshot, the load times are artificially slowed down so you can see all the different quality levels slowly stream into the browser. The artificial delays are as follows: 500ms delay before sending first byte, then throttle each photo as follows:
thumbnails=1s, preview=4s, original=10s

Initially, (b/c of 500ms delay) you'll set the thumbhash, then you'll see the thumb for 1s, then you'll see the preview stream in over 4s, and then when you zoom (which auto-loads the original quality) you'll see that stream in as well.

The demo video is repeated a 2nd time. The second time I start to zoom while preview is still loading, and you'll see the original start to load before the preview finishes.

Note, this is also using progressive-jpegs for preview. Non-progressive jpegs/webp files will stream/render from top to bottom.

Additionally, while all these images are loading, if you click next/prev, all of them are canceled, which is new feature too - this keeps the navigation responsive. If you are coming from the timeline, the preview image from the timeline is available immediately, so the delay between timeline and the full screen viewer should be almost imperceptible in most cases.

This video is an EXAGGERATION showing the worst case - very slow links/mobile devices. With preloading, and reusing thumbs from timeline, in practice, you won't see this at all. But, if you are on a slow link, you'll get faster feedback, better experience than staring at a black screen with a spinner like you currently do.

adaptive.mp4

The photo viewer currently loads images by swapping by showing a loading spinner until the 'preview' size image loads. And then on zoom, replacing that single <img> element's src from preview to original. This causes visible flicker between quality levels because the browser shows a blank state while the next resolution loads.

This PR replaces that with a layered progressive loading system. Three <img> elements (thumbnail, preview, original) are always present in the DOM, stacked by z-index. Each layer fades in on top of the previous one as it loads, so there's never a blank flash between quality transitions. A thumbhash canvas sits underneath everything as the initial placeholder.

The end result is - no spinner at all. The 'thumbnail' size image is VERY likely to already have been loaded (if your navigating from the timeline) - so you'll likely instantly see the thumbnail in most cases. However, if you do not (because this is a direct nav) you'll at least see the thumbhash, which is part of the asset response metdata and does not need a url fetch to get the image.

After the thumb is loaded, preview (or fullsize - or original - based on various conditions) image starts to load. If you start to zoom in, the fullsize (or original) image is loaded.

This change is fully transparent compatible. Transparent images (WEBP) have gray background set behind the image. Note - thumbhashes do support a transparent alpha channel, and the thumbhash will be drawn on top of the gray background.

The AdaptiveImageLoader is a state machine that manages cancelation and quality progression between: basic → loading-thumbnail → thumbnail → loading-preview → preview → loading-original → original

The asset viewer preloads and cancels the next/previous images using the same (headless) state-machine.

Because the "image" is now actually 3 layers - zoom-image was modified (upstream) and can now apply to an arbitary element instead of just tags - we now zoom the <AdaptiveImage> root div.

Added lots of e2e tests.

@midzelis midzelis force-pushed the push-zunuwtznrlpm branch from 394d962 to fb719d4 Compare March 2, 2026 02:36
@midzelis midzelis marked this pull request as ready for review March 2, 2026 02:55
@midzelis midzelis requested a review from danieldietzler as a code owner March 2, 2026 02:55
@mertalev
Copy link
Member

mertalev commented Mar 2, 2026

Is it possible to swap the image bitmaps as they're loaded rather than doing z-levels?

@midzelis
Copy link
Collaborator Author

midzelis commented Mar 2, 2026

Is it possible to swap the image bitmaps as they're loaded rather than doing z-levels?

The thumbhash (canvas) actually does get removed after the 'thumbnail' quality image is loaded successfully.

Something similar should be possible when the 'preview' image fully loads on top of the 'thumbnail' - once preview is loaded, thumbnail can be safely removed.

On zoom, same thing - once full/original is fully loaded, the thumbnail can be removed.

In general, we need to have 2 images while the higher quality is loading - so you see the lower quality 'under' the higher quality one. Once loaded, the lower quality one can be removed from DOM.

However, this does add overall complexity. Why are you asking? Is it for performance or memory reasons? For managing the z-index? Or something else?

@midzelis midzelis force-pushed the push-zunuwtznrlpm branch from fb719d4 to dffc150 Compare March 2, 2026 14:55
@mertalev
Copy link
Member

mertalev commented Mar 2, 2026

Swapping the bitmap is how it's handled in the mobile app, which is generally better for performance (less compositing, lower memory usage, etc.) and handles alpha. It basically needs an API that decouples decoding the image from displaying it. Not a huge deal if there's no good way to do that in web, but it's better if it can.

@midzelis midzelis force-pushed the push-rsywxvptwxuv branch from 6345dfd to 50ee10d Compare March 3, 2026 01:07
@midzelis midzelis force-pushed the push-zunuwtznrlpm branch from dffc150 to 1bf67a0 Compare March 3, 2026 01:07
@midzelis midzelis force-pushed the push-rsywxvptwxuv branch from 50ee10d to cb26611 Compare March 3, 2026 02:15
@midzelis midzelis force-pushed the push-zunuwtznrlpm branch from 1bf67a0 to 25a0c8f Compare March 3, 2026 02:16
@midzelis midzelis force-pushed the push-rsywxvptwxuv branch from cb26611 to 11a8a57 Compare March 3, 2026 02:24
@midzelis midzelis force-pushed the push-zunuwtznrlpm branch 3 times, most recently from 4d0ae32 to 9ecd6d8 Compare March 3, 2026 03:32
@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2026

Preview environment has been removed.

@midzelis midzelis force-pushed the push-rsywxvptwxuv branch from 11a8a57 to af8bf90 Compare March 3, 2026 04:05
@midzelis midzelis force-pushed the push-zunuwtznrlpm branch from 9ecd6d8 to c1308e5 Compare March 3, 2026 04:07
@midzelis midzelis force-pushed the push-zunuwtznrlpm branch from c1308e5 to 5c478cd Compare March 3, 2026 14:33
@immich-push-o-matic immich-push-o-matic bot added documentation Improvements or additions to documentation 📱mobile 🗄️server labels Mar 3, 2026
@midzelis midzelis changed the base branch from push-rsywxvptwxuv to main March 3, 2026 14:34
@midzelis midzelis force-pushed the push-zunuwtznrlpm branch from 3107b83 to 6fdf99c Compare March 9, 2026 13:43
midzelis added a commit that referenced this pull request Mar 9, 2026
Split out from #26636 (adaptive image loading). Leverages the
ContentMetrics extraction from #26310.

Moves face bounding boxes and OCR overlays from viewport-relative
to image-relative coordinates.

New features: zoom with face editor open, zoom with OCR boxes
visible, accurate face hover hit-testing at any zoom level.

Bug fixes: face tagging on stacked assets, faces loading when browsing
stacks, face editor auto-close on last face deletion, zoomTarget was
null, disablePointer option name mismatch.
@midzelis midzelis added changelog:enhancement and removed 📱mobile documentation Improvements or additions to documentation changelog:feature labels Mar 9, 2026
midzelis added a commit that referenced this pull request Mar 9, 2026
Split out from #26636 (adaptive image loading). Leverages the
ContentMetrics extraction from #26310.

Moves face bounding boxes and OCR overlays from viewport-relative
to image-relative coordinates.

New features: zoom with face editor open, zoom with OCR boxes
visible, accurate face hover hit-testing at any zoom level.

Bug fixes: face tagging on stacked assets, faces loading when browsing
stacks, face editor auto-close on last face deletion, zoomTarget was
null, disablePointer option name mismatch.
midzelis added 3 commits March 9, 2026 19:14
Replace ImageManager with a new AdaptiveImageLoader that progressively
loads images through quality tiers (thumbnail → preview → original).

New components and utilities:
- AdaptiveImage: layered image renderer with thumbhash, thumbnail,
  preview, and original layers with visibility managed by load state
- AdaptiveImageLoader: state machine driving the quality progression
  with per-quality callbacks and error handling
- ImageLayer/Image: low-level image elements with load/error lifecycle
- PreloadManager: preloads adjacent assets for instant navigation
- AlphaBackground/DelayedLoadingSpinner: loading state UI

Zoom is handled via a derived CSS transform applied to the content
wrapper in AdaptiveImage, with the zoom library (zoomTarget: null)
only tracking state without manipulating the DOM directly.

Also adds scaleToCover to container-utils and getAssetUrls to utils.
Copy link
Collaborator

@michelheusschen michelheusschen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I gave it another round of testing and no more issues, so all good from me. Thank you for the effort you put into this!

@alextran1502 alextran1502 enabled auto-merge (squash) March 11, 2026 01:11
alextran1502
alextran1502 previously approved these changes Mar 11, 2026
@alextran1502 alextran1502 disabled auto-merge March 11, 2026 01:33
@alextran1502 alextran1502 dismissed stale reviews from michelheusschen and themself March 11, 2026 01:41

I am thinking your approval somehow block the check job to run lol

@alextran1502 alextran1502 merged commit 8764a18 into main Mar 11, 2026
53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Notable blink while loading original image

7 participants