Install Fails with macOS pyenv Python 3.8

[dbanty]

I seem to be unable to install this package under Python 3.8.0. Are there plans to support 3.8 and a rough timeline for it? Thanks!

In case it's helpful, here's the error output:

Collecting xmlsec
Using cached https://files.pythonhosted.org/packages/35/42/d7cd323c91d4706f3cc32ffe7d5f851ab8ef9898ccb350f6ba593dd8b89a/xmlsec-1.3.3.tar.gz
Requirement already satisfied: pkgconfig in ./.venv/lib/python3.8/site-packages (from xmlsec) (1.5.1)
Requirement already satisfied: lxml>=3.0 in ./.venv/lib/python3.8/site-packages (from xmlsec) (4.4.0)
Installing collected packages: xmlsec
Running setup.py install for xmlsec ... error
ERROR: Command errored out with exit status 1:
command: /Users/dylan/projects/cloud/simc-server/.venv/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec/setup.py'"'"'; file='"'"'/private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-record-151gkxi/install-record.txt --single-version-externally-managed --compile --install-headers /Users/dylan/projects/cloud/simc-server/.venv/include/site/python3.8/xmlsec
cwd: /private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec/
Complete output (23 lines):
running install
running build
running build_ext
building 'xmlsec' extension
creating build
creating build/temp.macosx-10.14-x86_64-3.8
creating build/temp.macosx-10.14-x86_64-3.8/private
creating build/temp.macosx-10.14-x86_64-3.8/private/var
creating build/temp.macosx-10.14-x86_64-3.8/private/var/folders
creating build/temp.macosx-10.14-x86_64-3.8/private/var/folders/xr
creating build/temp.macosx-10.14-x86_64-3.8/private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn
creating build/temp.macosx-10.14-x86_64-3.8/private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T
creating build/temp.macosx-10.14-x86_64-3.8/private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s
creating build/temp.macosx-10.14-x86_64-3.8/private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec
creating build/temp.macosx-10.14-x86_64-3.8/private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec/src
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include -DMODULE_NAME=xmlsec -DMODULE_VERSION=1.3.3 -D__XMLSEC_FUNCTION_=func -DXMLSEC_NO_SIZE_T -DXMLSEC_NO_GOST=1 -DXMLSEC_NO_GOST2012=1 -DXMLSEC_NO_CRYPTO_DYNAMIC_LOADING=1 -DXMLSEC_CRYPTO_OPENSSL=1 -I/usr/local/Cellar/libxmlsec1/1.2.28_1/include/xmlsec1 -I/usr/local/opt/[email protected]/include -I/usr/local/opt/[email protected]/include/openssl -I/usr/include/libxml2 -I/Users/dylan/projects/cloud/simc-server/.venv/lib/python3.8/site-packages/lxml/includes -I/Users/dylan/projects/cloud/simc-server/.venv/lib/python3.8/site-packages/lxml -I/Users/dylan/projects/cloud/simc-server/.venv/lib/python3.8/site-packages/lxml/includes/pycache -I/Users/dylan/projects/cloud/simc-server/.venv/include -I/Users/dylan/.pyenv/versions/3.8.0/include/python3.8 -c /private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec/src/constants.c -o build/temp.macosx-10.14-x86_64-3.8/private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec/src/constants.o -g -std=c99 -fno-strict-aliasing -Wno-error=declaration-after-statement -Werror=implicit-function-declaration -Os
In file included from /private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec/src/constants.c:11:
In file included from /private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec/src/constants.h:15:
/usr/local/Cellar/libxmlsec1/1.2.28_1/include/xmlsec1/xmlsec/xmlsec.h:14:10: fatal error: 'libxml/tree.h' file not found
#include <libxml/tree.h>
^~~~~~~~~~~~~~~
1 error generated.
error: command 'clang' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /Users/dylan/projects/cloud/simc-server/.venv/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec/setup.py'"'"'; file='"'"'/private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-install-ms21q76s/xmlsec/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /private/var/folders/xr/tq65qlbx2dn1qr8c74ft2rph0000gn/T/pip-record-_151gkxi/install-record.txt --single-version-externally-managed --compile --install-headers /Users/dylan/projects/cloud/simc-server/.venv/include/site/python3.8/xmlsec Check the logs for full command output.

[bgaifullin]

It seems like there is missing libxml-dev package

[dbanty]

I have libxml2 and libxmlsec1 installed from Homebrew and installing under Python 3.7.4 works fine.

Both my Python 3.7.4 and 3.8.0 are installed from pyenv, not the distributed binaries from python.org. Not sure if that's useful info or no.

[dbanty]

So this installs now. I have updated both lxml (pip) and libxml2 (hombrew) since opening this ticket. My only explanation is that one of the changes to those packages fixed this issue. lxml 4.4.2 mentions a fix to recursive includes, so maybe that? Doesn't matter much- it works so I'm closing this.

[dbanty]

My previous comment is incorrect, my virtual environment must have been setup wrong. This is still not working on macOS with Python 3.8 from pyenv, same error as the original.

This is, however, working in CI which uses the Python 3.8.0 Docker image (Debian), so my suspicion is this is a problem specific to either the Python 3.8 built from pyenv or one of the libraries from homebrew.

[dbanty]

After some googling and some guesswork, I ran
export PKG_CONFIG_PATH="/usr/local/opt/libxml2/lib/pkgconfig"
then tried to install xmlsec again (pip install xmlsec) and it worked! It seems (and again this is guesswork) that macOS comes with an older version of libxml2 (in /usr/lib) which isn't compatible but is used by default. By telling pkgconfig to use the version from Homebrew (/usr/local) everything works fine.