Imagick install is failing randomly

[mvorisek]

Error description

Please see the Dockerfile below for a reproducible case.

Initially reported as Imagick issue - Imagick/imagick#582 - but I am not sure where the problem is. Maybe some race condition in the installer?

If you cannot reproduce it on the first run, please change the echo value in the Dockerfile and rerun the repro Dockerfile at least several times, as the issue is sometime present, sometime not. I can reproduce it using the Dockerfile localy in at least in 1 run of 10.

Minimal Dockerfile

FROM ghcr.io/mvorisek/image-php:8.3-zts-debian-base

# install basic system tools
RUN (seq 1 8 | xargs -I{} mkdir -p /usr/share/man/man{}) \
    && apt-get -y update \
    && apt-get -y upgrade \
    && apt-get -y install bash git make unzip gnupg ca-certificates apt-utils apt-transport-https netcat \
    && apt-get purge -y --auto-remove && apt-get clean && rm -rf /var/lib/apt/lists/* \
    && git config --system --add url."https://github.com/".insteadOf "[email protected]:" \
    && git config --system --add url."https://github.com/".insteadOf "ssh://[email protected]/" \
    # fix git repository directory is owned by someone else for Github Actions
    && { echo '#!/bin/sh'; echo 'if [ -n "$GITHUB_WORKSPACE" ] && [ "$(id -u)" -eq 0 ]; then'; echo '    (cd / && /usr/bin/git config --global --add safe.directory "$GITHUB_WORKSPACE")'; echo 'fi'; echo '/usr/bin/git "$@"'; } > /usr/local/bin/git && chmod +x /usr/local/bin/git

# install common PHP extensions
COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/local/bin/
RUN apt-get -y update && apt-get -y install fontconfig fontconfig-config fonts-dejavu-core imagemagick-6-common libaom0 libcairo2 libdatrie1 libdav1d4 libde265-0 libdeflate0 libdjvulibre-text libdjvulibre21 libfftw3-double3 libfontconfig1 libfreetype6 libfribidi0 libgraphite2-3 libharfbuzz0b libheif1 libilmbase25 libjbig0 libjpeg62-turbo liblcms2-2 liblqr-1-0 libltdl7 libmagickcore-6.q16-6 libmagickcore-6.q16-6-extra libmagickwand-6.q16-6 libnuma1 libopenexr25 libopenjp2-7 libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0 libpixman-1-0 libpng16-16 libthai-data libthai0 libtiff5 libwebp6 libwebpdemux2 libwebpmux3 libwmf0.2-7 libx265-192 libxcb-render0 libxcb-shm0 libxrender1 sensible-utils ucf automake autotools-dev libicu-dev libjpeg-dev libjpeg62-turbo-dev libmount-dev libpcre16-3 libpcre2-16-0 libpcre2-32-0 libpcre2-dev libpcre2-posix2 libpcre3-dev libpcre32-3 libpcrecpp0v5 media-types python3 python3-distutils python3-lib2to3 python3-minimal python3.9 python3.9-minimal shared-mime-info uuid-dev x11-common x11proto-dev x11proto-xext-dev xorg-sgml-doctools xtrans-dev zlib1g-dev
RUN apt-get -y install libdeflate-dev libdjvulibre-dev libelf1 libexif-dev libexif12 libexpat1-dev libffi-dev libfontconfig-dev libfontconfig1-dev libfreetype-dev libfreetype6-dev libgdk-pixbuf-2.0-0 libgdk-pixbuf-2.0-dev libgdk-pixbuf2.0-bin libgdk-pixbuf2.0-common libgirepository-1.0-1 libglib2.0-bin libglib2.0-data libglib2.0-dev libglib2.0-dev-bin libice-dev libice6 libilmbase-dev libjbig-dev liblcms2-dev liblqr-1-0-dev libltdl-dev liblzma-dev liblzo2-2 libopenexr-dev libopenjp2-7-dev libpixman-1-dev libpng-dev libpthread-stubs0-dev librsvg2-2 librsvg2-common librsvg2-dev libsm-dev libsm6 libtiff-dev libtiffxx5 libwmf-dev libx11-dev libxau-dev libxcb-render0-dev libxcb-shm0-dev libxcb1-dev libxdmcp-dev libxext-dev libxml2-dev libxrender-dev libxt-dev libxt6

# change the echo value to force Docker to rebuild the Dockerfile below in order it passed
RUN echo 5

RUN IPE_ICU_EN_ONLY=1 install-php-extensions \
    bcmath \
    imagick

# install Composer
RUN install-php-extensions @composer

RUN php --version

The repro Dockerfile is failing with:

In /tmp/pear/temp/imagick/Imagick.stub.php:
Unterminated preprocessor conditions
make: *** [Makefile:196: /tmp/pear/temp/imagick/Imagick_arginfo.h] Error 1
ERROR: `make -j6 INSTALL_ROOT="/tmp/pear/temp/pear-build-defaultuserDb1BBj/install-imagick-3.7.0" install' failed
The command '/bin/sh -c IPE_ICU_EN_ONLY=1 install-php-extensions     bcmath     imagick' returned a non-zero code: 1

here is a diff of a failing log vs. passing one: mvorisek/image-php@d357b3e (from exactly the same Dockerfile)

[mlocati]

It's rather hard to understand what's going wrong with such a big Dockerfile.

Here's what I've done to test your issue:

1. Create a file named Dockerfile.a with this contents:

   FROM php:8.2-cli-bullseye
   
   RUN apt-get update -qy \
      && apt-get upgrade -qy \
      && apt-get install libmagickwand-dev -qy \
      && apt-get purge --auto-remove -qy \
      && apt-get clean -qy \
      && rm -rf /var/lib/apt/lists/* \
      && curl -Lf -o /imagick-3.7.0.tgz https://pecl.php.net/get/imagick-3.7.0.tgz

2. Create a file named Dockerfile.b with this contents:

   FROM test-base:latest
   
   RUN printf '' | MAKE='make -j6' CPPFLAGS='' pecl install /imagick-3.7.0.tgz \
      && docker-php-ext-enable imagick \
      && php --ri imagick

3. Create a file named build-it with this contents:

   #!/bin/sh
   
   set -o nounset
   set -o errexit
   
   CDPATH= cd -- "$(dirname -- "$0")"
   
   if [ -z "$(docker images -q test-base:latest 2>/dev/null)" ]; then
      printf 'Building base image... '
      if ! docker build --no-cache --progress plain -t test-base:latest -f Dockerfile.a . >build.log 2>&1; then
         echo 'Failed!'
         cat build.log
         exit 1
      fi
      echo 'done.'
   fi
   
   cycles=100
   
   cycle=0
   while [ $cycle -lt $cycles ]; do
      cycle=$(( cycle + 1 ))
      printf "Build #$cycle/$cycles... "
      startTime=$(date +%s)
      if ! docker build --no-cache --progress plain -t test:latest -f Dockerfile.b . >build.log 2>&1; then
         echo 'Failed!'
         cat build.log
         exit 1
      fi
      endTime=$(date +%s)
      elapsedTime=$(( endTime - startTime ))
      echo "done in $elapsedTime seconds."
      docker rmi test:latest >/dev/null
   done
   
   echo 'All ok'

4. Make the build-it file executable:

   chmod +x build-it

5. Test the compilation of the imagick PHP extension 100 times:

   $ ./build-it
   Build #1/100... done in 4 seconds.
   Build #2/100... done in 6 seconds.
   Build #3/100... done in 5 seconds.
   [...omissis...]
   Build #98/100... done in 4 seconds.
   Build #99/100... done in 5 seconds.
   Build #100/100... done in 4 seconds.
   

So, I can't replicate the issue...

[mvorisek]

Thank you and I will try to reproduce it using the 1. and 2. step you have posted.

And one small feature request - can https://github.com/mvorisek/image-php/blob/d357b3e47e7a91500ad1656eb2d764854bc0a5bd/log.txt#L13 the full URL be displayed by the installer and possibly also the pecl command? Before I submit this issue, it was quite hard for me to understand what is called,

[mvorisek]

@mlocati now I looked into your examples more in depth. Please my FROM and my full 1st RUN. At least the base image is important to reproduce.

[mlocati]

You can write a very detailed log by using the IPE_DEBUG environment variable:

IPE_DEBUG=1 install-php-extensions imagick

[mvorisek]

It seems master branch of the Imagick package has this issue fixed.

Repro Dockerfile:

FROM ghcr.io/mvorisek/image-php:8.3-zts-debian-base

# install basic system tools
RUN (seq 1 8 | xargs -I{} mkdir -p /usr/share/man/man{}) \
    && apt-get -y update \
    && apt-get -y upgrade \
    && apt-get -y install bash git make unzip gnupg ca-certificates apt-utils apt-transport-https netcat \
    && apt-get purge -y --auto-remove && apt-get clean && rm -rf /var/lib/apt/lists/*

# install common PHP extensions
COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/local/bin/
RUN apt-get -y update && apt-get -y install fontconfig fontconfig-config fonts-dejavu-core imagemagick-6-common libaom0 libcairo2 libdatrie1 libdav1d4 libde265-0 libdeflate0 libdjvulibre-text libdjvulibre21 libfftw3-double3 libfontconfig1 libfreetype6 libfribidi0 libgraphite2-3 libharfbuzz0b libheif1 libilmbase25 libjbig0 libjpeg62-turbo liblcms2-2 liblqr-1-0 libltdl7 libmagickcore-6.q16-6 libmagickcore-6.q16-6-extra libmagickwand-6.q16-6 libnuma1 libopenexr25 libopenjp2-7 libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0 libpixman-1-0 libpng16-16 libthai-data libthai0 libtiff5 libwebp6 libwebpdemux2 libwebpmux3 libwmf0.2-7 libx265-192 libxcb-render0 libxcb-shm0 libxrender1 sensible-utils ucf automake autotools-dev libicu-dev libjpeg-dev libjpeg62-turbo-dev libmount-dev libpcre16-3 libpcre2-16-0 libpcre2-32-0 libpcre2-dev libpcre2-posix2 libpcre3-dev libpcre32-3 libpcrecpp0v5 media-types python3 python3-distutils python3-lib2to3 python3-minimal python3.9 python3.9-minimal shared-mime-info uuid-dev x11-common x11proto-dev x11proto-xext-dev xorg-sgml-doctools xtrans-dev zlib1g-dev
RUN apt-get -y install libdeflate-dev libdjvulibre-dev libelf1 libexif-dev libexif12 libexpat1-dev libffi-dev libfontconfig-dev libfontconfig1-dev libfreetype-dev libfreetype6-dev libgdk-pixbuf-2.0-0 libgdk-pixbuf-2.0-dev libgdk-pixbuf2.0-bin libgdk-pixbuf2.0-common libgirepository-1.0-1 libglib2.0-bin libglib2.0-data libglib2.0-dev libglib2.0-dev-bin libice-dev libice6 libilmbase-dev libjbig-dev liblcms2-dev liblqr-1-0-dev libltdl-dev liblzma-dev liblzo2-2 libopenexr-dev libopenjp2-7-dev libpixman-1-dev libpng-dev libpthread-stubs0-dev librsvg2-2 librsvg2-common librsvg2-dev libsm-dev libsm6 libtiff-dev libtiffxx5 libwmf-dev libx11-dev libxau-dev libxcb-render0-dev libxcb-shm0-dev libxcb1-dev libxdmcp-dev libxext-dev libxml2-dev libxrender-dev libxt-dev libxt6
RUN apt-get install -y libbz2-dev libmagickcore-6-arch-config libmagickcore-6-headers libmagickcore-6.q16-dev libmagickwand-6-headers libmagickwand-6.q16-dev libmagickwand-dev

RUN echo 0xxxxxxxxxxxxxxxxxxxxxxxxx

RUN curl -Lf -o imagick-pecl.tgz https://pecl.php.net/get/imagick-3.7.0.tgz
RUN git clone --depth 1 https://github.com/Imagick/imagick.git -b master imagick
RUN tar -czf imagick-git.tgz imagick # not sure if it is right, pecl install with it does not work

# native install
#RUN cd imagick && phpize && ./configure --with-php-config=/usr/local/bin/php-config --with-imagick && make -j6 &&  make -j6 install && docker-php-ext-enable imagick

# pecl install
#RUN printf '' | MAKE='make -j6' CPPFLAGS='' pecl install imagick-pecl.tgz && docker-php-ext-enable imagick

# CI install
RUN install-php-extensions Imagick/imagick@master

RUN php --ri imagick

Repro command: sed -E 's~RUN echo .+~\0x~' -i Dockerfile && docker build .

[b2un0]

can confirm, this works temporary for PHP 8.3

FROM php:8-apache

COPY --from=mlocati/php-extension-installer /usr/bin/install-php-extensions /usr/local/bin/

COPY --from=composer /usr/bin/composer /usr/local/bin/

ENV DEBIAN_FRONTEND=noninteractive

RUN apt update && \
    apt install --no-install-recommends -y ssl-cert unzip && \
    apt clean all && \
    a2enmod ssl && \
    install-php-extensions ast apcu xdebug intl mysqli opcache pdo_mysql zip Imagick/imagick@master

 install-php-extensions Imagick/imagick@master

[XFreitas]

It worked, thanks!