SSL support is not compiled (libpq)

[manifest]

I'm getting an error from the application with Diesel dependency that should have compiled statically with SSL

[2019-01-22T06:11:14Z ERROR r2d2] sslmode value "require" invalid when SSL support is not compiled in

I've compiled application with the following command

OPENSSL_STATIC=yes OPENSSL_LIB_DIR=/musl/lib/ OPENSSL_INCLUDE_DIR=/musl/include cargo build --release

Entire Dockerfile

## -----------------------------------------------------------------------------
## Build
## -----------------------------------------------------------------------------
FROM clux/muslrust as build-stage

WORKDIR "/build"
COPY . .
RUN OPENSSL_STATIC=yes OPENSSL_LIB_DIR=/musl/lib/ OPENSSL_INCLUDE_DIR=/musl/include cargo build --release

## -----------------------------------------------------------------------------
## Package
## -----------------------------------------------------------------------------
FROM alpine

COPY --from=build-stage "/build/target/x86_64-unknown-linux-musl/release/example" "/app/example"

WORKDIR "/app"
ENTRYPOINT ["/app/example"]

App's Cargo.toml

[dependencies]
diesel = { version = "1.3.3", features = ["postgres", "uuid", "serde_json", "r2d2"] }
openssl = "*"

[patch.crates-io]
diesel = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }
diesel_derives = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }

[manifest]

The same error with the command:

OPENSSL_STATIC=yes OPENSSL_LIB_DIR=/musl/lib/ OPENSSL_INCLUDE_DIR=/musl/include PQ_LIB_STATIC=yes PQ_LIB_DIR=/musl/lib/ cargo build --release

And Cargo.toml

[dependencies]
diesel = { version = "1.3.3", features = ["postgres", "uuid", "serde_json", "r2d2"] }
openssl = "*"
pq-sys = "*"

[patch.crates-io]
diesel = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }
diesel_derives = { git = "https://github.com/diesel-rs/diesel", rev = "59aa49b" }

[manifest]

I've played around libpg compilation adding --with-openssl to postgresql`s configure script and start getting an error on compilation

  = note: /build/target/x86_64-unknown-linux-musl/release/deps/libpq_sys-784bf4f1583faa91.rlib(fe-secure-openssl.o): In function `pgtls_init':
          fe-secure-openssl.c:(.text+0xf48): undefined reference to `OPENSSL_config'
          collect2: error: ld returned 1 exit status

     cd postgresql-$PQ_VER && \
     CC="musl-gcc -fPIE -pie" LDFLAGS="-L$PREFIX/lib" CFLAGS="-I$PREFIX/include" ./configure \
+    --with-openssl \

[clux]

Hey, thanks for raising this! This should indeed be supported. curl goes through the same dance.

It's probably a bit annoying to get it to work though. I'll try to have a look on the weekend, but can't promise anything.

In the mean time, any chance you can commit what you have as a test case in a PR?

[manifest]

@clux I've published a test case for the issue here.

Steps to reproduce:

1. Create a database

## Database conenction URL must contain `?sslmode=require`
## query string parameter to reproduce the issue
export DATABASE_URL='postgres://localhost:5432/diesel-muslrust-example?sslmode=require'
diesel setup --database-url ${DATABASE_URL}

2. Build and run a docker container

docker build -t diesel-muslrust-example -f docker/Dockerfile .
docker run --rm -ti --entrypoint=/bin/sh diesel-muslrust-example

3. Run the example app from within the container

## The app reads 'DATABASE_URL' from environment variable, so it should be exported
export DATABASE_URL=''
RUST_LOG=diesel_muslrust_example=info /app/diesel-muslrust-example

4. You should see an error after connection will fail with timeout

thread 'main' panicked at 'Error creating a database pool: Error(Some("sslmode value \"require\" invalid when SSL support is not compiled in\n"))', src/libcore/result.rs:999:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

[manifest]

@clux have you had time to take a look on this one?

[clux]

Ah, sorry not yet. Busy weeks 🙁

[clux]

Had a bit of a go today and added failing tests from your code in 1677445

Unfortunately, no matter what permutations i tried libpq will build, but cargo build will fail to link due to missing SSL_ symbols. Leaving a comment for the future with current status.

Testing method now is:

• tweak Dockerfile
• make clean
• make build
• make test-dieselpgssl

There's quite a lot of flags in libpq's ./configure though:

./configure -h

Usage: ./configure [OPTION]... [VAR=VALUE]...

To assign environment variables (e.g., CC, CFLAGS...), specify them as
VAR=VALUE.  See below for descriptions of some of the useful variables.

Defaults for the options are specified in brackets.

Configuration:
  -h, --help              display this help and exit
      --help=short        display options specific to this package
      --help=recursive    display the short help of all the included packages
  -V, --version           display version information and exit
  -q, --quiet, --silent   do not print `checking ...' messages
      --cache-file=FILE   cache test results in FILE [disabled]
  -C, --config-cache      alias for `--cache-file=config.cache'
  -n, --no-create         do not create output files
      --srcdir=DIR        find the sources in DIR [configure dir or `..']

Installation directories:
  --prefix=PREFIX         install architecture-independent files in PREFIX
                          [/usr/local/pgsql]
  --exec-prefix=EPREFIX   install architecture-dependent files in EPREFIX
                          [PREFIX]

By default, `make install' will install all the files in
`/usr/local/pgsql/bin', `/usr/local/pgsql/lib' etc.  You can specify
an installation prefix other than `/usr/local/pgsql' using `--prefix',
for instance `--prefix=$HOME'.

For better control, use the options below.

Fine tuning of the installation directories:
  --bindir=DIR            user executables [EPREFIX/bin]
  --sbindir=DIR           system admin executables [EPREFIX/sbin]
  --libexecdir=DIR        program executables [EPREFIX/libexec]
  --sysconfdir=DIR        read-only single-machine data [PREFIX/etc]
  --sharedstatedir=DIR    modifiable architecture-independent data [PREFIX/com]
  --localstatedir=DIR     modifiable single-machine data [PREFIX/var]
  --libdir=DIR            object code libraries [EPREFIX/lib]
  --includedir=DIR        C header files [PREFIX/include]
  --oldincludedir=DIR     C header files for non-gcc [/usr/include]
  --datarootdir=DIR       read-only arch.-independent data root [PREFIX/share]
  --datadir=DIR           read-only architecture-independent data [DATAROOTDIR]
  --infodir=DIR           info documentation [DATAROOTDIR/info]
  --localedir=DIR         locale-dependent data [DATAROOTDIR/locale]
  --mandir=DIR            man documentation [DATAROOTDIR/man]
  --docdir=DIR            documentation root [DATAROOTDIR/doc/postgresql]
  --htmldir=DIR           html documentation [DOCDIR]
  --dvidir=DIR            dvi documentation [DOCDIR]
  --pdfdir=DIR            pdf documentation [DOCDIR]
  --psdir=DIR             ps documentation [DOCDIR]

System types:
  --build=BUILD     configure for building on BUILD [guessed]
  --host=HOST       cross-compile to build programs to run on HOST [BUILD]

Optional Features:
  --disable-option-checking  ignore unrecognized --enable/--with options
  --disable-FEATURE       do not include FEATURE (same as --enable-FEATURE=no)
  --enable-FEATURE[=ARG]  include FEATURE [ARG=yes]
  --disable-integer-datetimes
                          obsolete option, no longer supported
  --enable-nls[=LANGUAGES]
                          enable Native Language Support
  --disable-rpath         do not embed shared library search path in
                          executables
  --disable-spinlocks     do not use spinlocks
  --disable-atomics       do not use atomic operations
  --disable-strong-random do not use a strong random number source
  --enable-debug          build with debugging symbols (-g)
  --enable-profiling      build with profiling enabled
  --enable-coverage       build with coverage testing instrumentation
  --enable-dtrace         build with DTrace support
  --enable-tap-tests      enable TAP tests (requires Perl and IPC::Run)
  --enable-depend         turn on automatic dependency tracking
  --enable-cassert        enable assertion checks (for debugging)
  --disable-thread-safety disable thread-safety in client libraries
  --disable-largefile     omit support for large files
  --disable-float4-byval  disable float4 passed by value
  --disable-float8-byval  disable float8 passed by value

Optional Packages:
  --with-PACKAGE[=ARG]    use PACKAGE [ARG=yes]
  --without-PACKAGE       do not use PACKAGE (same as --with-PACKAGE=no)
  --with-extra-version=STRING
                          append STRING to version
  --with-template=NAME    override operating system template
  --with-includes=DIRS    look for additional header files in DIRS
  --with-libraries=DIRS   look for additional libraries in DIRS
  --with-libs=DIRS        alternative spelling of --with-libraries
  --with-pgport=PORTNUM   set default port number [5432]
  --with-blocksize=BLOCKSIZE
                          set table block size in kB [8]
  --with-segsize=SEGSIZE  set table segment size in GB [1]
  --with-wal-blocksize=BLOCKSIZE
                          set WAL block size in kB [8]
  --with-wal-segsize=SEGSIZE
                          set WAL segment size in MB [16]
  --with-CC=CMD           set compiler (deprecated)
  --with-icu              build with ICU support
  --with-tcl              build Tcl modules (PL/Tcl)
  --with-tclconfig=DIR    tclConfig.sh is in DIR
  --with-perl             build Perl modules (PL/Perl)
  --with-python           build Python modules (PL/Python)
  --with-gssapi           build with GSSAPI support
  --with-krb-srvnam=NAME  default service principal name in Kerberos (GSSAPI)
                          [postgres]
  --with-pam              build with PAM support
  --with-bsd-auth         build with BSD Authentication support
  --with-ldap             build with LDAP support
  --with-bonjour          build with Bonjour support
  --with-openssl          build with OpenSSL support
  --with-selinux          build with SELinux support
  --with-systemd          build with systemd support
  --without-readline      do not use GNU Readline nor BSD Libedit for editing
  --with-libedit-preferred
                          prefer BSD Libedit over GNU Readline
  --with-uuid=LIB         build contrib/uuid-ossp using LIB (bsd,e2fs,ossp)
  --with-ossp-uuid        obsolete spelling of --with-uuid=ossp
  --with-libxml           build with XML support
  --with-libxslt          use XSLT support when building contrib/xml2
  --with-system-tzdata=DIR
                          use system time zone data in DIR
  --without-zlib          do not use Zlib
  --with-gnu-ld           assume the C compiler uses GNU ld [default=no]

Some influential environment variables:
  CC          C compiler command
  CFLAGS      C compiler flags
  LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries in a
              nonstandard directory <lib dir>
  LIBS        libraries to pass to the linker, e.g. -l<library>
  CPPFLAGS    (Objective) C/C++ preprocessor flags, e.g. -I<include dir> if
              you have headers in a nonstandard directory <include dir>
  CPP         C preprocessor
  PKG_CONFIG  path to pkg-config utility
  PKG_CONFIG_PATH
              directories to add to pkg-config's search path
  PKG_CONFIG_LIBDIR
              path overriding pkg-config's built-in search path
  ICU_CFLAGS  C compiler flags for ICU, overriding pkg-config
  ICU_LIBS    linker flags for ICU, overriding pkg-config
  LDFLAGS_EX  extra linker flags for linking executables only
  LDFLAGS_SL  extra linker flags for linking shared libraries only

Use these variables to override the choices made by `configure' or to help
it to find libraries and programs with nonstandard names/locations.

There might be one in there that'll fix it.
Otherwise, there might be some magic in https://github.com/sgrif/pq-sys/blob/master/build.rs that we can use.

[clux]

oh, and saw this once during compilation of libpq:

configure: WARNING: openssl/ssl.h: accepted by the compiler, rejected by the preprocessor!
configure: WARNING: openssl/ssl.h: proceeding with the compiler's result

got rid of this warning by passing:

    --with-libraries=$PREFIX/lib \
    --with-includes=$PREFIX/include \

to ./configure. didn't seem to help any further though.

[clux]

Another diff that may or may not help bring this closer:

diff --git a/Dockerfile b/Dockerfile
index 9899a4f..46448c1 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -45,7 +45,7 @@ RUN curl https://sh.rustup.rs -sSf | \
 ENV SSL_VER="1.0.2q" \
     CURL_VER="7.64.0" \
     ZLIB_VER="1.2.11" \
-    PQ_VER="10.6" \
+    PQ_VER="9.6.8" \
     SQLITE_VER="3270200" \
     CC=musl-gcc \
     PREFIX=/musl \
@@ -91,10 +91,14 @@ RUN curl -sSL https://curl.haxx.se/download/curl-$CURL_VER.tar.gz | tar xz && \
 
 # Build libpq
 # TODO: fix so that --with-openssl works with pqssl tests
-RUN curl -sSL https://ftp.postgresql.org/pub/source/v$PQ_VER/postgresql-$PQ_VER.tar.gz | tar xz && \
-    cd postgresql-$PQ_VER && \
+RUN curl -sSL https://ftp.postgresql.org/pub/source/v$PQ_VER/postgresql-$PQ_VER.tar.gz | tar xz
+RUN cd postgresql-$PQ_VER && \
     CC="musl-gcc -fPIE -pie" LDFLAGS="-L$PREFIX/lib" CFLAGS="-I$PREFIX/include" ./configure \
     --without-readline \
+    --with-openssl \
+    --with-libraries=$PREFIX/lib \
+    --with-includes=$PREFIX/include \
+    --oldincludedir=/usr/include \
     --prefix=$PREFIX --host=x86_64-unknown-linux-musl && \
     cd src/interfaces/libpq make -s -j$(nproc) all-static-lib && make -s install install-lib-static && \
     cd ../../bin/pg_config && make -j $(nproc) && make install && \
@@ -121,13 +125,19 @@ ENV PATH=$PREFIX/bin:$PATH \
     PKG_CONFIG_ALLOW_CROSS=true \
     PKG_CONFIG_ALL_STATIC=true \
     PQ_LIB_STATIC_X86_64_UNKNOWN_LINUX_MUSL=true \
+    PQ_LIB_STATIC=true \
+    PQ_LIB_DIR=$PREFIX/lib/ \
+    OPENSSL_LIB_DIR=$PREFIX/lib/ \
+    OPENSSL_INCLUDE_DIR=$PREFIX/include \
+    DEP_OPENSSL_INCLUDE=$PREFIX/include/ \
     PKG_CONFIG_PATH=$PREFIX/lib/pkgconfig \
     PG_CONFIG_X86_64_UNKNOWN_LINUX_GNU=/usr/bin/pg_config \
     OPENSSL_STATIC=true \
     OPENSSL_DIR=$PREFIX \
     SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt \
     SSL_CERT_DIR=/etc/ssl/certs \
-    LIBZ_SYS_STATIC=1
+    LIBZ_SYS_STATIC=1 \
+    TARGET=musl
 
 # Allow ditching the -w /volume flag to docker run
 WORKDIR /volume

[clux]

the issue might be libpq configuration based on this output from its detection:

checking for CRYPTO_new_ex_data in -lcrypto... yes
checking for SSL_new in -lssl... yes
checking for SSL_get_current_compression... yes
checking for OPENSSL_init_ssl... no
checking for BIO_get_data... no
checking for BIO_meth_new... no
checking for ASN1_STRING_get0_data... no
checking for RAND_OpenSSL... no
checking for CRYPTO_lock... yes

but maybe that's just openssl 1.1 stuff it's testing for 🤕

[emk]

Hello! I'm the maintainer of rust-musl-builder, and we ran into this problem lately. You might be interested in the following issues:

• diesel link order issues sometimes break build emk/rust-musl-builder#69
• Static linking needs to include openssl somehow. sgrif/pq-sys#25

TL;dr: It's possible to work around this by adding this to main.rs:

extern crate openssl;
#[macro_use]
extern crate diesel;

...but the workaround might break with certain projects or releases of Rust. I've filed a bug upstream to investigate further: sgrif/pq-sys#25

[clux]

Oh wow, thanks a lot @emk ! It does look like the order of the includes actually fixes this!

Had a quick check compiling pg with --with-openssl and make test-dieselpgssl now both links and runs in this repo without ssl errors.

[clux]

Have made openssl required now, since there's now way of having the support without pulling in the crate, but it's documented in the main README. Will keep an eye on pq-sys#25 for a potential future improvement on this.