[BUG] Storage template migration ignores specific files

[PhasecoreX]

The bug

Modifying the storage template and then running storage template migration seems to not move the video portion of Live Photos nor archived photos. Re-running the migration will not fix it, and the repair section does not list any issues. If I look at the storage folder, I see most of the media in the correct location, but the Live Photo videos and archived media are in their old locations. The frontend still can show the Live Photo video, and still shows the archived photos, so I assume the database still points at the correct (old) location for the file.

The OS that Immich Server is running on

Ubuntu 22.04.4 LTS

Version of Immich Server

v1.108.0

Version of Immich Mobile App

v1.108.0

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

networks:
  proxy:
    name: proxy
    external: true

volumes:
  database-dump:
    name: database-dump
    external: true
  z-model-cache:
  z-redis-data:

services:
  immich-server:
    container_name: immich-server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    expose:
      - "3001"
    depends_on:
      - immich-redis
      - immich-postgres
    restart: always
    networks:
      - default
      - proxy

  immich-machine-learning:
    container_name: immich-machine-learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - z-model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-redis:
    container_name: immich-redis
    image: docker.io/redis:6.2-alpine@sha256:d6c2911ac51b289db208767581a5d154544f2b2fe4914ea5056443f62dc6e900
    volumes:
      - z-redis-data:/data
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  immich-postgres:
    container_name: immich-postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - ../volumes/immich/database:/var/lib/postgresql/data
      - database-dump:/database-dump
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/ext/immich

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=...

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich-postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich-redis

Reproduction steps

1. Upload a Live Photo
2. Upload another photo, and archive it
3. Change the storage template so that these photos will move
4. Run storage template migration
5. Notice that the photo part of the Live Photo moves to the correct location, but the video part is still in it's original spot
6. Notice also that the archived photo didn't move at all

Relevant log output

No response

Additional information

I see #6440 had an issue with Live Photo videos not being put in the right spot on initial upload, but that got fixed on its own. This issue is specifically when you try to move/migrate to a new layout.

[alextran1502]

@zackpollard When you have a chance, can you help taking a look at this issue?

[zackpollard]

Hey @PhasecoreX

I can reproduce and have fixed the issue of archived photos not being moved after changing the storage template structure, however I can't see the same problem with motion photos. I have tested with both android and iOS live photos and they both move correctly. If you could give me steps to reproduce the live photo problem I would be happy to look into it more 🙂

[PhasecoreX]

Hello @zackpollard! Thank you so much for the prompt response!

The listed steps is what I had done to get it to do that. Of course, the "Upload Live Photo" was actually my app uploading probably 5k of them, and then me manually uploading a bunch of older ones via CLI.

But I did notice something (I think). When I looked at the storage, I noticed that there was a jpeg file with associated mov that had successfully migrated over. However the HEIC ones seem to leave behind their mov files. I don't know if it has something to do with the issue? Actually that might just be coincidence, see below:

The only other thing I can think of is if you are uploading via CLI or via mobile app. Looking at the storage, it seems that the ones I have CLI uploaded do migrate both photo/video successfully, but iOS app uploaded ones only migrate the photo part. Maybe there's a difference between the two ways of uploading? That would be weird though. But it does explain how all the older ones in my library migrate correctly, and all the new app uploaded ones don't.

Sorry, I just keep finding more patterns and then keep writing them down before submitting. I looked at my wife's account/storage, and all of hers migrated correctly! The difference is that I have been using the app way longer than her, only just recently having her start using it. It looks like this upload difference between CLI and iOS app stopped being an issue sometime between January 7th 2024 and February 2024, as my library shows all the mov files being moved correctly for iOS uploads after February. So that explains how you couldn't reproduce it, as you were either using CLI, or using the newest iOS app. In an older iOS app, it wasn't doing something correctly, and those affected files have this weird bug.

Hopefully that helps! Let me know if you need anything else! And again, thank you!

[Shurov]

for me the job didn't move (update folder structure) for the deleted files, kept in recycle bin

[zackpollard]

for me the job didn't move (update folder structure) for the deleted files, kept in recycle bin

This is intentional, we don't move files that are already trashed as they are expected to be deleted in the future.

[zackpollard]

Hello @zackpollard! Thank you so much for the prompt response!

The listed steps is what I had done to get it to do that. Of course, the "Upload Live Photo" was actually my app uploading probably 5k of them, and then me manually uploading a bunch of older ones via CLI.

But I did notice something (I think). When I looked at the storage, I noticed that there was a jpeg file with associated mov that had successfully migrated over. However the HEIC ones seem to leave behind their mov files. I don't know if it has something to do with the issue? Actually that might just be coincidence, see below:

The only other thing I can think of is if you are uploading via CLI or via mobile app. Looking at the storage, it seems that the ones I have CLI uploaded do migrate both photo/video successfully, but iOS app uploaded ones only migrate the photo part. Maybe there's a difference between the two ways of uploading? That would be weird though. But it does explain how all the older ones in my library migrate correctly, and all the new app uploaded ones don't.

Sorry, I just keep finding more patterns and then keep writing them down before submitting. I looked at my wife's account/storage, and all of hers migrated correctly! The difference is that I have been using the app way longer than her, only just recently having her start using it. It looks like this upload difference between CLI and iOS app stopped being an issue sometime between January 7th 2024 and February 2024, as my library shows all the mov files being moved correctly for iOS uploads after February. So that explains how you couldn't reproduce it, as you were either using CLI, or using the newest iOS app. In an older iOS app, it wasn't doing something correctly, and those affected files have this weird bug.

Hopefully that helps! Let me know if you need anything else! And again, thank you!

Honestly I have no idea what could cause only newer files to move correctly, but it sounds like whatever bug did exist is now fixed as it doesn't occur in newer versions. I'm probably not going to invest time looking into this, if you want to do more digging and can come up with a concrete reproduction case, maybe I could take a look further. Cheers!

[PhasecoreX]

I did some research on this, it was caused by the exif table not having a record for the video asset. See #11324 (comment) for more details.