Heroku builds fail due to absence of libffi

[JonHannah]

I encountered this on upgrading to whitenoise 3.2 and adding brotlipy per the documented recommendation.

remote: -----> Using set buildpack heroku/python
remote: -----> Python app detected
remote: $ pip install -r requirements.txt
remote: Collecting brotlipy==0.3.0 (from -r requirements.txt (line 1))
remote: Downloading brotlipy-0.3.0.tar.gz (492kB)
remote: Complete output from command python setup.py egg_info:
remote: Package libffi was not found in the pkg-config search path.
remote: Perhaps you should add the directory containing `libffi.pc'
remote: to the PKG_CONFIG_PATH environment variable
remote: No package 'libffi' found

Adding cffi to requirements.txt was an easy fix, but would be good to add some acknowledgment to the whitenoise docs (until such time as Heroku 'notices' brotlipy!).

remote: -----> Using set buildpack heroku/python
remote: -----> Python app detected
remote: -----> _Noticed cffi. Bootstrapping libffi._
remote: $ pip install -r requirements.txt
remote: Collecting brotlipy==0.3.0 (from -r requirements.txt (line 1))
remote: Downloading brotlipy-0.3.0.tar.gz (492kB)
remote: Collecting cffi==1.2.1 (from -r requirements.txt (line 2))
remote: Downloading cffi-1.2.1.tar.gz (335kB) ...

[edmorley]

Thank you for filing. It's best practice to add all requirements to your requirements file for deployments, and Heroku's setup is pretty unique (and support can be added to it fairly easily, see below), so I wonder if it may be out of scope for the WhiteNoise docs to mention the need to add cffi to requirements.txt?

Adding automatic support to Heroku is a case of adding brotlipy to the list here:
https://github.com/heroku/heroku-buildpack-python/blob/master/bin/steps/cryptography#L22-L23

I was about to add it myself, but I wonder if it's best to wait for the official brotli PyPI package? (brotlipy is a stop-gap pending google/brotli#72). Guess we could add brotlipy now and tweak later?

[edmorley]

So another thing that would be good, is if the Heroku buildpack output a more helpful error message (like they already do for missing libmemcached for pylibmc for example).

I'm happy to open a PR against the buildpack to do that - however it would help to know how you got this output...?

remote: Complete output from command python setup.py egg_info:
remote: Package libffi was not found in the pkg-config search path.
remote: Perhaps you should add the directory containing `libffi.pc'
remote: to the PKG_CONFIG_PATH environment variable
remote: No package 'libffi' found

Since when I try to install brotlipy locally without libffi installed I instead get:

Complete output from command python setup.py egg_info:
c/_cffi_backend.c:15:17: fatal error: ffi.h: No such file or directory
compilation terminated.

Different version of pip/setuptools/Python perhaps? I just want to make sure the warning handling I add to the buildpack greps for the correct error strings in the pip output -- there may be several variations, but would be good to know why.

[evansd]

@JonHannah Yeah, I've realised that my docs aren't helpful here because I talk about adding brotlipy to requirements.txt when what I ought to say is: pip install brotlipy and then update your requirements.txt with the implication that the exact version of brotlipy and all its dependencies get added.

I guess that's what most users are doing anyway, but my docs are misleading in this regard.

@edmorley It's a shame the offical package still hasn't made it to PyPI. It all looks ready to go; I wonder what the hold up is?