psycopg2-binary 2.9.5 bundles wrong libpq version on Ubuntu 64 for Raspberry Pi 4B

[csotto]

NOTE: This ticket is very similar to "[psycopg2-binary 2.9.2 bundles wrong libpq version on aarch64 #1396" but differs on two details:
a) Linux Build / Hardware / Latest software version
b) Available temporary solution will become deprecated when pip bumps to version 23.1.x (Current pip version is 22.3.1)

** I am running on two Raspberry Py 4B: One for postgres server (docker) and other for client-postgres applications **

Client side raspberry Pi:

• OS: Ubuntu 22.04.1 LTS for Raspberry Pi 4B
• Psycopg version: psycopg2-binary 2.9.5
• Python version: Python 3.10.6
• PostgreSQL version: "PostgreSQL 14.6 (Debian 14.6-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit"
• pip version: 22.3.1

Client side Windows: Client exists but client connection to server do not present any problem.

Server side:

• OS: Ubuntu 22.04.1 LTS for Raspberry Pi 4B
• Psycopg version: psycopg2-binary 2.9.5
• Python version: Python 3.10.6
• PostgreSQL version: "PostgreSQL 14.6 (Debian 14.6-1.pgdg110+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit"
• pip version: 22.3.1

Describe the bug
Getting psycopg2.OperationalError: SCRAM authentication requires libpq version 10 or above (Client-side error msg) on client-side (raspberri Pi) after server was updated to use SCRAM authentication (which is the current and future default) instead of MD5.
NOTE: Windows client continued to able to connect to server, thus problem is restricted to Rasberry Pi clients.

Temporary solution was found at https://stackoverflow.com/questions/69573312/python-psycopg2-scram-authentication, which basically is:
$ pip uninstall psycopg2-binary
$ sudo apt update -y && sudo apt install -y build-essential libpq-dev
$ pip install psycopg2-binary --no-binary psycopg2-binary

The above fix resolved the problem at client side but deprecation warnings during build phase show that this fix will have to be revised when pip bumps to version 23.1.x. The warnings are:
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at pypa/pip#11453
DEPRECATION: psycopg2-binary is being installed using the legacy 'setup.py install' method, because the '--no-binary' option was enabled for it and this currently disables local wheel building for projects that don't have a 'pyproject.toml' file. pip 23.1 will enforce this behaviour change. A possible replacement is to enable the '--use-pep517' option. Discussion can be found at pypa/pip#11451

Please note missing project.toml file to be able to build with wheel

Thanks for your attention.

[dvarrazzo]

Hello,

which manylinux tag does Raspberry Pi architecture use? If you pip install numpy, which numpy package will it install?

[csotto]

I made a typo (numpi instead of numpy) but both where installed.
$ pip install numpi
Collecting numpi
Downloading NuMPI-0.3.1.tar.gz (48 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.7/48.7 kB 1.5 MB/s eta 0:00:00
Preparing metadata (setup.py) ... done
Collecting numpy>=1.15.0
Downloading numpy-1.23.5-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (13.9 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.9/13.9 MB 6.5 MB/s eta 0:00:00
Building wheels for collected packages: numpi
Building wheel for numpi (setup.py) ... done
Created wheel for numpi: filename=NuMPI-0.3.1-py3-none-any.whl size=55265 sha256=6c9da182c2fe192fc2eb1534e7ce3256fda072806a191b6df68537c48d44c3a8
Stored in directory: /home/csotto/.cache/pip/wheels/3b/82/0f/5ce1558c6fcb9be14ff320585e108f2f197cc82c895404a6a4
Successfully built numpi
Installing collected packages: numpy, numpi
Successfully installed numpi-0.3.1 numpy-1.23.5

[csotto]

Additional information:
Client Ubuntu is using Python 3.10.6 right from Ubuntu 22.04.1 LTS release. Since Python 3.10 is the original Ubuntu version of Python for Raspi. I did not install it from file. Python updates comes from original Ubuntu apt updates.
I run my application in a Python virtual environment in order not no mess up with Python used by system.

[dvarrazzo]

This seems exactly the same problem of #1396, no?

[csotto]

Daniele, I agree.
Please apologize my shallow knowledge of how psycopg2-binary is built. My prior understanding was that you compiled and distributed the binary code. Now I understand that the fix has to be worked via Ubuntu, who owns the official Ubuntu for Rasberry Pi release.
I could try to open a ticket at Ubuntu.
But if you agree I could ask you to kindly open a ticket at Ubuntu, complaining about the usage of usage of the old version of manylinux_2_17_aarch64.manylinux2014_aarch64.whl at latest Ubuntu LTS release (22.04.1) available today.
I understand you are more capable than me to clearly explain Ubuntu what is wrong and how it has to be fixed.

I am satisfied with the support for this ticket here. It should be closed, specially that we have a temporary fix that should work until pip moves to version 23.1.x

[dvarrazzo]

No, psycopg2-binary doesn't go through Ubuntu. It is compiled and uploaded on PyPI by us.

The currently available version bundles a libpq version too old to support scram. The solution is to redo the build script in order to compile the most recent libpq from source and ship that one.

This is something we know how to do: psycopg 3 binary packages for aarch64 architecture ships with such libpq. psycopg 2 packages predate that work.

It is just a matter of time for me to clean my backlog enough to reach #1396 and find half a day to do that job.

I am closing this as duplicate of the other task. Thank you very much for the detailed report and the collaboration!

[csotto]

Thanks for your support and for all the great work on psycopg for Postgres.
I do appreciate it.