Lychee not importing S3 config from environment vars

[RickyRomero]

Lychee version

v5.5.0

Which PHP version are you using?

PHP 8.2

Detailed description of the problem

I'm using Lychee with Docker Compose, configuring it via environment variables fed in from a .env file adjacent to my docker-compose.yaml file using the env_file property.

For many of these variables, Lychee automatically copies them from my Docker environment into /conf/.env. I would expect Lychee to do the same when I specify an S3 config, however it doesn't do that. I had to manually move my S3 details into /conf/.env before Lychee recognized there was an S3 config at all.

(note: in the config below, I've specified AWS_IMAGE_VISIBILITY=private. I don't know what the correct value is for this yet, but I don't think it's related to this bug. I'm still figuring this out due to a lack of documentation. It sends the photos to S3 okay, but reading them back using pre-signed URLs isn't working yet.)

Steps to reproduce the issue

1. Configure your Lychee service like so, adding an appropriate database.

docker-compose.yaml

[...]

  lychee:
    image: 'lycheeorg/lychee:v5.5.0'
    restart: 'unless-stopped'
    volumes:
      - './config/conf:/conf'
      - './config/sym:/sym'
      - './config/uploads:/uploads'
      - './config/imported-items:/imported-items'
    networks:
      - 'proxy'
      - 'lychee'
    env_file: '.env'
    depends_on:
      - 'lychee-db'
    secrets:
      - 'POSTGRES_PASSWORD'

[...]

2. Add a .env file next to your docker-compose.yaml.

.env

[...]

PHOTO_PIPES=true
S3_ENABLED=true
AWS_ACCESS_KEY_ID=**redacted**
AWS_SECRET_ACCESS_KEY=**redacted**
AWS_DEFAULT_REGION=us-west-1
AWS_BUCKET=**redacted**
AWS_URL=https://s3.us-west-1.wasabisys.com/**redacted**
AWS_ENDPOINT=https://s3.us-west-1.wasabisys.com
AWS_IMAGE_VISIBILITY=private
AWS_USE_PATH_STYLE_ENDPOINT=true

[...]

3. Start the containers:

docker compose up

4. Set up your admin account.
5. Upload a photo. Notice that Lychee is storing the images in /conf/uploads.
6. Stop and remove the containers:

[^C]
docker compose down

7. Hand-modify config/conf/.env with the S3 config values in the Docker .env.
8. Start the containers:

docker compose up

9. Upload a photo. It should push to S3.

Diagnostics [REQUIRED]

    Diagnostics
    -----------
        Warning: Dropbox import not working. dropbox_key is empty.
        Warning: zend.assertions is disabled although Lychee is in debug mode. For easier debugging code generation for assertions should be enabled.
        Info: Latest version of PHP is 8.3
        Warning: git (software) is not available.
    
    System Information
    ------------------
        Lychee Version (tag):                    v5.5.0 (cc68474) -- Up to date (-4.8383424408015E-11 years ago).
        DB Version:                              5.5.0
        
        composer install:                        --no-dev
        APP_ENV:                                 production
        APP_DEBUG:                               true
        APP_URL:                                 set
        APP_DIR:                                 default
        LOG_VIEWER_ENABLED:                      true
        LIVEWIRE_ENABLED:                        true
        
        System:                                  Linux
        PHP Version:                             8.2.20
        PHP User agent:                          Lychee/4 (https://lycheeorg.github.io/)
        Timezone:                                America/Los_Angeles
        Max uploaded file size:                  100M
        Max post size:                           100M
        Livewire chunk size:                     12.00 MB
        Max execution time:                      3600
        PostgreSQL Version:                      PostgreSQL 16.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 13.2.1_git20240309) 13.2.1 20240309, 64-bit
        
        exec() Available:                        yes
        Imagick Available:                       1
        Imagick Enabled:                         1
        Imagick Version:                         1691
        GD Version:                              2.3.3
        Number of foreign key:                   12 found.
    
    Config Information
    ------------------
        version:                                 050500
        check_for_updates:                       0
        sorting_photos_col:                      taken_at
        sorting_photos_order:                    ASC
        sorting_albums_col:                      max_taken_at
        sorting_albums_order:                    ASC
        imagick:                                 1
        skip_duplicates:                         0
        small_max_width:                         0
        small_max_height:                        360
        medium_max_width:                        1920
        medium_max_height:                       1080
        lang:                                    en
        image_overlay_type:                      desc
        default_license:                         none
        compression_quality:                     90
        grants_full_photo_access:                1
        delete_imported:                         0
        mod_frame_enabled:                       1
        mod_frame_refresh:                       30
        thumb_2x:                                1
        small_2x:                                1
        medium_2x:                               1
        landing_page_enable:                     0
        force_migration_in_production:           0
        search_public:                           0
        hide_version_number:                     0
        SL_enable:                               0
        SL_for_admin:                            0
        recent_age:                              1
        grants_download:                         0
        photos_wraparound:                       1
        map_display:                             0
        zip64:                                   1
        map_display_public:                      0
        map_provider:                            Wikimedia
        force_32bit_ids:                         0
        map_include_subalbums:                   0
        update_check_every_days:                 3
        has_exiftool:                            1
        share_button_visible:                    0
        import_via_symlink:                      0
        has_ffmpeg:                              1
        location_decoding:                       0
        location_decoding_timeout:               30
        location_show:                           1
        location_show_public:                    0
        rss_enable:                              0
        rss_recent_days:                         7
        rss_max_items:                           100
        prefer_available_xmp_metadata:           0
        editor_enabled:                          1
        lossless_optimization:                   0
        swipe_tolerance_x:                       150
        swipe_tolerance_y:                       250
        local_takestamp_video_formats:           .avi|.mov
        log_max_num_line:                        1000
        unlock_password_photos_with_url_param:   0
        nsfw_visible:                            1
        nsfw_blur:                               0
        nsfw_warning:                            0
        nsfw_warning_admin:                      0
        nsfw_banner_override:                    
        map_display_direction:                   1
        album_subtitle_type:                     oldstyle
        upload_processing_limit:                 4
        new_photos_notification:                 0
        legacy_id_redirection:                   1
        zip_deflate_level:                       6
        default_album_protection:                1
        album_decoration:                        layers
        album_decoration_orientation:            row
        allow_username_change:                   1
        auto_fix_orientation:                    1
        use_job_queues:                          0
        random_album_id:                         starred
        use_last_modified_date_when_no_exif_date: 0
        layout:                                  justified
        date_format_photo_thumb:                 M j, Y, g:i:s A e
        date_format_photo_overlay:               M j, Y, g:i:s A e
        date_format_sidebar_uploaded:            M j, Y, g:i:s A e
        date_format_sidebar_taken_at:            M j, Y, g:i:s A e
        date_format_hero_min_max:                F Y
        date_format_hero_created_at:             M j, Y, g:i:s A T
        date_format_album_thumb:                 M Y
        upload_chunk_size:                       0
        nsfw_banner_blur_backdrop:               0
        search_pagination_limit:                 1000
        search_minimum_length_required:          4
        photo_layout_justified_row_height:       320
        photo_layout_masonry_column_width:       300
        photo_layout_grid_column_width:          250
        photo_layout_square_column_width:        200
        photo_layout_gap:                        12
        display_thumb_album_overlay:             always
        display_thumb_photo_overlay:             hover
        default_album_thumb_aspect_ratio:        1/1
        use_album_compact_header:                0
        login_button_position:                   left
        back_button_enabled:                     0
        back_button_text:                        Return to Home
        current_job_processing_visible:          1
        SA_random_thumbs:                        0
        job_processing_queue_visible:            1
        login_required:                          0
        enable_unsorted:                         1
        enable_starred:                          1
        enable_recent:                           1
        enable_on_this_day:                      1
        login_required_root_only:                1

Browser & System [REQUIRED]

Server: Debian 6.1.76-1 (2024-02-01) x86_64, Docker 27.0.2 build 912c1dd, Lychee v5.5.0 image behind Caddy v2.8.4
Client: macOS 14.5, Chrome 126.0.6478.127

Please confirm (incomplete submissions will not be addressed)

• I have provided easy and step-by-step instructions to reproduce the bug.
• I understand my bug report will be removed if I haven't met the criteria above.

[ildyria]

I'm using Lychee with Docker Compose, configuring it via environment variables fed in from a .env file adjacent to my docker-compose.yaml file using the env_file property.

That is because only part of the environment are passed to the conf/.env as you noticed.
The culprit is the injection: https://github.com/LycheeOrg/Lychee-Docker/blob/master/inject.sh

[RickyRomero]

If I were to open a PR to fix it, would that be helpful? 🙂

[ildyria]

For sure :)

[RickyRomero]

Just opened LycheeOrg/Lychee-Docker#200 – please take a look when you have a chance. Thanks!