feat(web): OCR overlay interactivity during zoom by midzelis · Pull Request #27039 · immich-app/immich

midzelis · 2026-03-19T13:12:54Z

Problem

The zoom-image library intercepts all pointer, touch, and wheel events on the photo viewer container. This prevents users from interacting with overlay elements — OCR text can't be selected, and touch gestures on overlay boxes trigger zoom/pan instead of the intended interaction. On touch devices, OCR boxes require hover to reveal text, which isn't possible.

Features

OCR overlays scale and pan with the image during zoom — no positional drift
Touch: OCR text always visible (when OCR boxes are visible) with auto select-all on tap
Desktop: native text selection on OCR boxes (click-drag, double-click to select word)
Overlay-aware zoom: elements marked data-overlay-interactive opt out of zoom gesture handling

Minor Bug fixes / Annoyances

OCR text not selectable while zoomed — zoom library captured all pointer events
Pinching with one finger on an OCR box caused erratic zoom (split gesture)
Mouse wheel over an OCR box triggered image zoom instead of being ignored
Double-tapping an OCR box on Safari fired both tap-to-select and zoom-image's zoom handler
Touch devices couldn't reveal OCR text (required hover)

Implementation

Zoom the container div instead of individual images via zoomTarget, so overlays transform with the image automatically
Capture-phase event interception with pointer ID tracking for split gesture detection
Face editor marked as data-overlay-interactive to prevent zoom during face editing
Unified coordinate mapping with Point, Size, Rect, ContentMetrics types — mapNormalizedToContent, mapNormalizedRectToContent, mapContentRectToNatural
OCR and face bounding box functions refactored to use centralized coordinate mapping
Event listener cleanup via AbortController

Test plan

New E2E suite: ocr.e2e-spec.ts — OCR box toggle visibility, per-asset OCR data on stacked assets, zoom scaling, data-overlay-interactive attribute, focus on click, drag does not pan image, split touch gesture does not zoom
Unit tests for coordinate mapping (container-utils.spec.ts) — letterbox offsets, aspect ratio handling, round-trip mapping
Unit tests for OCR utilities (ocr-utils.spec.ts)
Unit tests for face bounding box (people-utils.spec.ts) — simplified after refactor

ocr_resize.mp4

github-actions · 2026-03-19T13:27:10Z

Preview environment has been removed.

jrasm91 · 2026-03-23T17:38:15Z

-      top: Math.round(coordinates.y1),
-      left: Math.round(coordinates.x1),
-      width: Math.round(coordinates.x2 - coordinates.x1),
-      height: Math.round(coordinates.y2 - coordinates.y1),


Why did you get rid of this rounding?

Browsers render fractional pixel values smoothly, so rounding only adds error.

This was done for consistency - this is used for faces. OCR boxes also do a similar calculation, but it never rounded. Technically, not rounding is a bit more accurate and more readable - but I don't care that much. Will be happy to re-add rounding, but if I do that, I'd like to be consistent with Face Editor, Bounding boxes, and OCR text boxes.

jrasm91

Code seems fine.

Change-Id: Id62e1a0264df2de0f3177a59b24bc5176a6a6964

….0) (#101) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [immich-app/immich](https://github.com/immich-app/immich) | minor | `v2.6.3` → `v2.7.0` | --- ### Release Notes <details> <summary>immich-app/immich (immich-app/immich)</summary> ### [`v2.7.0`](https://github.com/immich-app/immich/releases/tag/v2.7.0) [Compare Source](immich-app/immich@v2.6.3...v2.7.0) ### v2.7.0 Welcome to Immich `v2.7.0`! This release includes enhancements to the asset viewer, security improvements, changes to the duplicate APIs and viewer, and a bunch of bug fixes. Keep reading below for the complete highlights and a note on the upcoming `v3.0.0` release. > \[!NOTE]\ > We're working on a managed backup service for Immich with end-to-end encrypted backups of your library to a remote datacentre where only you hold the keys. > > We've put together a quick survey (\~5 mins) to get a better idea of how you're backing things up today and what you'd actually want from something like this. Your answers help us figure out what to prioritise, so we'd really appreciate it if you took a few minutes to fill it out. > > Leave your email at the end if you're interested in joining our free closed beta when it's ready. > > <https://futo-backups-survey.immich.app/> #### Known limitations - The machine learning service on `amd64` currently requires the `>= x86-64-v2` microarchitecture. This will be patched in an upcoming patch release for backward compatibility with very old processors (before \~2010), but it will become a minimum requirement in 3.0. `arm64` is not affected by this change. #### Highlights - Remove from album (asset viewer) - Move to locked folder (folder page) - Editor shortcuts - Create a new face on-the-fly in the face tag editor - Resolve duplicates - Helmet configuration - Version check infrastructure - Notable fix: live photo and video download in Safari - Notable fix: escape HTML in the Panorama Photo Viewer ##### Remove from album The web has a new action, "Remove from album," available in the asset viewer that makes it easier to remove an asset from an album. This action is available to both album and asset owners. <img width="360" height="202" alt="image" src="https://github.com/user-attachments/assets/e5facb24-ed10-4adc-957a-37147cca5634" /> ##### Move to locked folder in the Folder view Similarly, the folder view now includes the "Move to locked folder" action. <img width="1900" height="762" alt="image" src="https://github.com/user-attachments/assets/c39e792b-81da-4c31-a23f-03f96853fe8e" /> ##### Editor shortcuts Users on the web can now edit with keyboard shortcuts. Press `e` to open the editor. Once in the editor, press `[` or `]` to rotate the asset +/- 90 degrees. Finally, save any changes and close the editor with `ENTER`. <https://github.com/user-attachments/assets/969de104-b02d-41a6-830b-3e1a49541d14> ##### Create a new face on-the-fly in the face tag editor You can now create a new face/person on the fly from the face tagging editor interface <img width="350" alt="image" src="https://github.com/user-attachments/assets/c39db0e3-da47-4421-9040-5f51650deee9" /> <img width="350" alt="image" src="https://github.com/user-attachments/assets/0c81a1ee-c54e-4167-9dff-95719fe44595" /> ##### Deduplication improvements The duplicate screen has gone through a bunch of iterations since it was first introduced all the way back in May, 2024. The latest release moves a bunch of logic from the client to the server, which now automatically suggests which asset to keep based on image size and EXIF data. Additionally, the new server implementation will automatically synchronize metadata, including albums, favorite status, rating, description, visibility, location, and tags. For more information about this process, see the new [documentation](https://docs.immich.app/features/duplicates-utility). ##### Helmet configuration You can now opt in to using a [Content Security Policy (CSP)](https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/CSP) in Immich. The new environment variant `IMMICH_HELMET_FILE` accepts a boolean or a path to a [helmet](https://helmetjs.github.io/) configuration file. **Recommend action:** The team recommends setting `IMMICH_HELMET_FILE=true` to enable the default policy. Then, please let us know if you run into any issues with it. ##### Background and details Since Immich is deployed in so many different ways, it has been hard to figure out how to enable a CSP that would not conflict with or break existing installs that might use 3rd party map providers, custom CSS, embed Immich in an iframe, or other such features. In this release, we have added the ability to both opt in to a default policy and configure a custom one. To use the default policy, simply set the environment variable `IMMICH_HELMET_FILE=true`. To use a custom policy, set the environment variable to a path on disk (within the immich-server) that contains a valid helmet configuration (e.g. `IMMICH_HELMET_FILE=/opt/immich/helmet.json`). CSP can be used to control what scripts are allowed to run on the page, which domains to load images from, etc. Additionally, it can be used to configure headers for [Referrer-Policy](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/Referrer-Policy), [X-Powered-By](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Powered-By), [X-Frame-Options](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/X-Frame-Options), and others. ##### New version check infrastructure Prior to this release, instances that used the automatic version check feature would send HTTP requests to `github.com`. Now, we have set up a small service at `version.immich.cloud` to handle these types of requests. This avoids any privacy implications of connecting to `github.com` , as well as moves the request load to our own infrastructure. ##### Notable fix: live photo and video download in Safari When downloading files in Safari with the same name, it will simply overwrite the file instead of automatically renaming it. In this release, the still and motion parts of a live photo are now named differently to prevent this from happening. ##### Notable fix: escape HTML in panorama photo viewer In `v2.6.0`, we added the ability to show/view clip text in the panorama viewer, but introduced an XSS vulnerability, which has been fixed in this release. Interestingly, this was XSS using text in the image, which would then get read by OCR. ##### Notable fix: Immich User Agent for external requests Similar to the mobile app, the server now sends a custom [User Agent](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent) header. The format for the User Agent is `immich-server/{version}`. For example, `immich-server/2.7.0`. #### `v3.0.0` Just a heads up that this is the likely to be the last release before `v3.0.0`. Being a major release there will be a handful of breaking changes, *although it's worth noting that nothing is currently planned that requires user intervention*. It is mainly changes that impact 3rd party developers. More information and details should be available in the coming weeks. #### Support Immich <p align="center"> <img src="https://media.giphy.com/media/v1.Y2lkPTc5MGI3NjExbjY2eWc5Y2F0ZW56MmR4aWE0dDhzZXlidXRmYWZyajl1bWZidXZpcyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/87CKDqErVfMqY/giphy.gif" width="450" title="SUPPORT THE PROJECT!"> </p> If you find the project helpful, you can support Immich by purchasing a product key at <https://buy.immich.app> or our merchandise at <https://immich.store> ***  #### What's Changed ##### 🚀 Features - feat: add support for helmet configuration by [@jrasm91](https://github.com/jrasm91) in [#27058](immich-app/immich#27058) - feat: create new person in face editor by [@alextran1502](https://github.com/alextran1502) in [#27364](immich-app/immich#27364) ##### 🌟 Enhancements - feat(web): add a seperate tooltip for switching from dark to light mode by [@Vogeluff](https://github.com/Vogeluff) in [#27297](immich-app/immich#27297) - feat(web): focus on face-editor search input by [@cratoo](https://github.com/cratoo) in [#27136](immich-app/immich#27136) - feat(web): add RemoveFromAlbumAction to asset viewer nav bar by [@timonrieger](https://github.com/timonrieger) in [#27000](immich-app/immich#27000) - feat(web): add shortcuts to rotate images by [@squishykid](https://github.com/squishykid) in [#26927](immich-app/immich#26927) - feat(server): add checksum algorithm field by [@etnoy](https://github.com/etnoy) in [#26573](immich-app/immich#26573) - feat(server): resolve duplicates by [@Phlogi](https://github.com/Phlogi) in [#25316](immich-app/immich#25316) - chore(mobile): reduce spacing on video controls by [@uhthomas](https://github.com/uhthomas) in [#27313](immich-app/immich#27313) - perf(server): optimize people page query by [@ffchung](https://github.com/ffchung) in [#27346](immich-app/immich#27346) - feat(web): dim photo outside hovered face bounding box by [@midzelis](https://github.com/midzelis) in [#27402](immich-app/immich#27402) - feat(web): OCR overlay interactivity during zoom by [@midzelis](https://github.com/midzelis) in [#27039](immich-app/immich#27039) - feat: add move to lock folder in folder view by [@alextran1502](https://github.com/alextran1502) in [#27384](immich-app/immich#27384) - feat(web): highlight active person thumbnail in detail panel and edit faces panel by [@midzelis](https://github.com/midzelis) in [#27401](immich-app/immich#27401) - feat: move version checks to our own infrastructure by [@zackpollard](https://github.com/zackpollard) in [#27450](immich-app/immich#27450) - feat: add preview button when selecting images by [@johnmaguire](https://github.com/johnmaguire) in [#27305](immich-app/immich#27305) - fix: user-agent format by [@jrasm91](https://github.com/jrasm91) in [#27488](immich-app/immich#27488) - chore(mobile): reduce buffering timer duration by [@uhthomas](https://github.com/uhthomas) in [#27111](immich-app/immich#27111) - fix(mobile): use key on video controls by [@uhthomas](https://github.com/uhthomas) in [#27512](immich-app/immich#27512) - feat(server): Add support for .ts files by [@ray](https://github.com/ray) in [#27529](immich-app/immich#27529) ##### 🐛 Bug fixes - fix(server): refresh unedited asset dimensions on metadata extraction by [@michelheusschen](https://github.com/michelheusschen) in [#27220](immich-app/immich#27220) - fix(server): memory fragmentation by [@mertalev](https://github.com/mertalev) in [#27027](immich-app/immich#27027) - fix(database restores): don't assume onboarding has completed by [@insertish](https://github.com/insertish) in [#27052](immich-app/immich#27052) - fix(web): preserve timezone when changing timestamp (Closes [#25354](immich-app/immich#25354)) by [@updatemike](https://github.com/updatemike) in [#27095](immich-app/immich#27095) - fix: various command palette usages by [@danieldietzler](https://github.com/danieldietzler) in [#27304](immich-app/immich#27304) - fix(web): keep map view open after closing asset viewer by [@diiogofer](https://github.com/diiogofer) in [#26980](immich-app/immich#26980) - fix(web): prevent Safari from overwriting live photo image with video by [@saurav61091](https://github.com/saurav61091) in [#26898](immich-app/immich#26898) - fix(mobile): video icon not showing on memories by [@YarosMallorca](https://github.com/YarosMallorca) in [#27311](immich-app/immich#27311) - fix(mobile): mismatch between system and app color when using low-chroma system color scheme by [@putuprema](https://github.com/putuprema) in [#27282](immich-app/immich#27282) - fix(mobile): images loads sometimes cancel too early by [@LeLunZ](https://github.com/LeLunZ) in [#27067](immich-app/immich#27067) - fix(mobile): streamline error handling for live photo saving by [@LeLunZ](https://github.com/LeLunZ) in [#27337](immich-app/immich#27337) - fix(web): keep upload totals stable when dismissing items ([#27247](immich-app/immich#27247)) by [@Nicolas-micuda-becker](https://github.com/Nicolas-micuda-becker) in [#27354](immich-app/immich#27354) - fix(mobile): low upload timeout on android by [@mertalev](https://github.com/mertalev) in [#27399](immich-app/immich#27399) - fix(web): add drop shadow to asset viewer nav bar and prevent button shrinking by [@midzelis](https://github.com/midzelis) in [#27404](immich-app/immich#27404) - fix(mobile): favorite button not updating state by [@YarosMallorca](https://github.com/YarosMallorca) in [#27271](immich-app/immich#27271) - fix: detection of WebM container by [@chanb22](https://github.com/chanb22) in [#24182](immich-app/immich#24182) - fix(web): prevent AssetUpdate from adding unrelated timeline assets by [@michelheusschen](https://github.com/michelheusschen) in [#27369](immich-app/immich#27369) - fix: withFilePath select edited or unedited file by [@bwees](https://github.com/bwees) in [#27328](immich-app/immich#27328) - fix(web): Enable stack selector in shared album view by [@timonrieger](https://github.com/timonrieger) in [#24641](immich-app/immich#24641) - fix(server): use substring matching for person name search by [@okxint](https://github.com/okxint) in [#26903](immich-app/immich#26903) - fix: escape html by [@jrasm91](https://github.com/jrasm91) in [#27469](immich-app/immich#27469) - fix(mobile): ignore pointer events on toasts by [@uhthomas](https://github.com/uhthomas) in [#26990](immich-app/immich#26990) - fix(mobile): reset video controls hide timer when showing controls ch… by [@uhthomas](https://github.com/uhthomas) in [#26985](immich-app/immich#26985) - fix(mobile): don't update search filters in-place by [@uhthomas](https://github.com/uhthomas) in [#26965](immich-app/immich#26965) - fix(web): isFullScreen initial value check is incorrect by [@midzelis](https://github.com/midzelis) in [#27520](immich-app/immich#27520) - fix(mobile): transparent system navbar when trash bottom bar is visible by [@putuprema](https://github.com/putuprema) in [#27093](immich-app/immich#27093) - fix: timestamp handling for database backup in Web UI by [@AfonsoMendoncaRibeiro](https://github.com/AfonsoMendoncaRibeiro) in [#27359](immich-app/immich#27359) - fix: allow bots to access /s/ urls by [@domints](https://github.com/domints) in [#27579](immich-app/immich#27579) ##### 📚 Documentation - feat(docs): add keycloack example to oauth docs by [@robson90](https://github.com/robson90) in [#27425](immich-app/immich#27425) ##### 🌐 Translations - chore(web): update translations by [@weblate](https://github.com/weblate) in [#27029](immich-app/immich#27029) - chore(web): update translations by [@weblate](https://github.com/weblate) in [#27483](immich-app/immich#27483) #### New Contributors - [@Vogeluff](https://github.com/Vogeluff) made their first contribution in [#27297](immich-app/immich#27297) - [@updatemike](https://github.com/updatemike) made their first contribution in [#27095](immich-app/immich#27095) - [@diiogofer](https://github.com/diiogofer) made their first contribution in [#26980](immich-app/immich#26980) - [@squishykid](https://github.com/squishykid) made their first contribution in [#26927](immich-app/immich#26927) - [@Phlogi](https://github.com/Phlogi) made their first contribution in [#25316](immich-app/immich#25316) - [@saurav61091](https://github.com/saurav61091) made their first contribution in [#26898](immich-app/immich#26898) - [@putuprema](https://github.com/putuprema) made their first contribution in [#27282](immich-app/immich#27282) - [@ffchung](https://github.com/ffchung) made their first contribution in [#27346](immich-app/immich#27346) - [@chanb22](https://github.com/chanb22) made their first contribution in [#24182](immich-app/immich#24182) - [@robson90](https://github.com/robson90) made their first contribution in [#27425](immich-app/immich#27425) - [@okxint](https://github.com/okxint) made their first contribution in [#26903](immich-app/immich#26903) - [@johnmaguire](https://github.com/johnmaguire) made their first contribution in [#27305](immich-app/immich#27305) - [@ray](https://github.com/ray) made their first contribution in [#27529](immich-app/immich#27529) - [@AfonsoMendoncaRibeiro](https://github.com/AfonsoMendoncaRibeiro) made their first contribution in [#27359](immich-app/immich#27359) - [@domints](https://github.com/domints) made their first contribution in [#27579](immich-app/immich#27579) **Full Changelog**: <immich-app/immich@v2.6.3...v2.7.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.erwanleboucher.dev/eleboucher/homelab/pulls/101 Co-authored-by: bot-owl <bot@erwanleboucher.dev> Co-committed-by: bot-owl <bot@erwanleboucher.dev>

midzelis requested a review from danieldietzler as a code owner March 19, 2026 13:12

midzelis added the changelog:enhancement label Mar 19, 2026

immich-push-o-matic bot added the 🖥️web label Mar 19, 2026

midzelis mentioned this pull request Mar 19, 2026

fix(web): refresh bugs and image-relative overlays with zoom support for faces, OCR, and face editor #26777

Closed

midzelis added preview labels Mar 19, 2026

midzelis force-pushed the ocr-overlay-zoom branch 3 times, most recently from d103ab3 to f788aeb Compare March 22, 2026 19:56

jrasm91 reviewed Mar 23, 2026

View reviewed changes

midzelis force-pushed the ocr-overlay-zoom branch from f788aeb to ed04d87 Compare March 23, 2026 21:38

midzelis requested a review from jrasm91 March 23, 2026 22:30

jrasm91 approved these changes Mar 24, 2026

View reviewed changes

midzelis force-pushed the ocr-overlay-zoom branch from ed04d87 to 3156c03 Compare March 27, 2026 17:29

feat(web): OCR overlay interactivity during zoom

001782b

Change-Id: Id62e1a0264df2de0f3177a59b24bc5176a6a6964

midzelis force-pushed the ocr-overlay-zoom branch from 3156c03 to 001782b Compare March 30, 2026 23:36

alextran1502 merged commit 18201a2 into main Mar 31, 2026
54 checks passed

alextran1502 deleted the ocr-overlay-zoom branch March 31, 2026 00:19

immich-push-o-matic bot removed the preview label Mar 31, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(web): OCR overlay interactivity during zoom#27039

feat(web): OCR overlay interactivity during zoom#27039
alextran1502 merged 1 commit intomainfrom
ocr-overlay-zoom

midzelis commented Mar 19, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 19, 2026 •

edited

Loading

Uh oh!

jrasm91 Mar 23, 2026

Uh oh!

midzelis Mar 23, 2026

Uh oh!

jrasm91 left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

midzelis commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Features

Minor Bug fixes / Annoyances

Implementation

Test plan

Uh oh!

github-actions bot commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jrasm91 Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

midzelis Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

jrasm91 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

midzelis commented Mar 19, 2026 •

edited

Loading

github-actions bot commented Mar 19, 2026 •

edited

Loading