python3: fix ssl support by removing libressl patches #20137

PolynomialDivision · 2022-12-20T08:27:55Z

Remove libressl specific patches. With commit
("tools/libressl: update to 3.7.0") they are no longer needed, rather they cause python3 to be compiled without working ssl-support.

Fixes: #20107
Suggested-by: Andre Heider a.heider@gmail.com
Signed-off-by: Nick Hainke vincent@systemli.org

Maintainer: @jefferyto
Compile tested: master
Run tested: tbd

dhewg · 2022-12-20T09:34:00Z

So my hunch was correct, the python package doesn't require any libressl patches anymore?

dangowrt

LGTM and fixes build issues related to pip https downloads.

PolynomialDivision · 2022-12-21T09:28:09Z

@jefferyto is now not active anymore since multiple months? I think we can merge this since it is fixing a substantial bug? @neheb @BKPepe

neheb · 2022-12-21T10:22:25Z

Maybe bump PKG_RELEASE

…

On Wed, Dec 21, 2022, 01:28 Nick Hainke ***@***.***> wrote: @jefferyto <https://github.com/jefferyto> is now not active anymore since multiple months? I think we can merge this since it is fixing a substantial bug? @neheb <https://github.com/neheb> @BKPepe <https://github.com/BKPepe> — Reply to this email directly, view it on GitHub <#20137 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAWITWLYXBAKGEAV5BXZTVTWOLETJANCNFSM6AAAAAATEHEEDE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

Remove libressl specific patches. With commit ("tools/libressl: update to 3.7.0") they are no longer needed, rather they cause python3 to be compiled without working ssl-support. Fixes: openwrt#20107 Suggested-by: Andre Heider <a.heider@gmail.com> Signed-off-by: Nick Hainke <vincent@systemli.org>

PolynomialDivision · 2022-12-21T13:26:33Z

Nice, thanks. Than I will merge now.

jefferyto · 2023-05-23T20:14:28Z

I've been looking at the original issue and this PR, and without this change this is the error message I see:

building '_ssl' extension
ccache gcc -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -O2 -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/host/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/hostpkg/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/host/include -O2 -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/host/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/hostpkg/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/host/include -fno-semantic-interposition -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/hostpkg/include -I./Include -I. -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/host/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/hostpkg/include -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/build_dir/hostpkg/Python-3.10.7/Include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/build_dir/hostpkg/Python-3.10.7 -c /media/jeff/Jekyll/Downloads/openwrt/testing/a/build_dir/hostpkg/Python-3.10.7/Modules/_ssl.c -o build/temp.linux-x86_64-3.10/media/jeff/Jekyll/Downloads/openwrt/testing/a/build_dir/hostpkg/Python-3.10.7/Modules/_ssl.o
Modules/_ssl.c:71:12: error: static declaration of 'SSL_CTX_get_security_level' follows non-static declaration
   71 | static int SSL_CTX_get_security_level(const SSL_CTX *ctx)
      |            ^~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from Modules/_ssl.c:60:
../../../staging_dir/host/include/openssl/ssl.h:1588:5: note: previous declaration of 'SSL_CTX_get_security_level' with type 'int(const SSL_CTX *)' {aka 'int(const struct ssl_ctx_st *)'}
 1588 | int SSL_CTX_get_security_level(const SSL_CTX *ctx);
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~

This appears to be caused by the definition of SSL_CTX_get_security_level() in 028-host-python-support-ssl-with-libressl.patch. This patch is no longer needed and removing it resolves the error.

This PR also removes 026-openssl-feature-flags.patch though, and this patch is still necessary; LibreSSL doesn't implement all of the hash functions supported by OpenSSL.

These are the errors I see after this PR:

building '_hashlib' extension
ccache gcc -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -O3 -Wall -O2 -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/host/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/hostpkg/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/host/include -O2 -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/host/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/hostpkg/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/target-arm_cortex-a15+neon-vfpv4_musl_eabi/host/include -fno-semantic-interposition -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -I./Include/internal -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/hostpkg/include -I./Include -I. -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/host/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/staging_dir/hostpkg/include -I/usr/include/x86_64-linux-gnu -I/usr/local/include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/build_dir/hostpkg/Python-3.10.7/Include -I/media/jeff/Jekyll/Downloads/openwrt/testing/a/build_dir/hostpkg/Python-3.10.7 -c /media/jeff/Jekyll/Downloads/openwrt/testing/a/build_dir/hostpkg/Python-3.10.7/Modules/_hashopenssl.c -o build/temp.linux-x86_64-3.10/media/jeff/Jekyll/Downloads/openwrt/testing/a/build_dir/hostpkg/Python-3.10.7/Modules/_hashopenssl.o
Modules/_hashopenssl.c:123:53: error: 'SN_sha512_224' undeclared here (not in a function); did you mean 'SN_sha224'?
  123 |     PY_HASH_ENTRY(Py_hash_sha512_224, "SHA512_224", SN_sha512_224, NID_sha512_224),
      |                                                     ^~~~~~~~~~~~~
Modules/_hashopenssl.c:110:25: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                         ^~~~~~~~~
Modules/_hashopenssl.c:123:68: error: 'NID_sha512_224' undeclared here (not in a function); did you mean 'NID_sha512'?
  123 |     PY_HASH_ENTRY(Py_hash_sha512_224, "SHA512_224", SN_sha512_224, NID_sha512_224),
      |                                                                    ^~~~~~~~~~~~~~
Modules/_hashopenssl.c:110:36: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                                    ^~~~~~~~
Modules/_hashopenssl.c:124:53: error: 'SN_sha512_256' undeclared here (not in a function); did you mean 'SN_sha256'?
  124 |     PY_HASH_ENTRY(Py_hash_sha512_256, "SHA512_256", SN_sha512_256, NID_sha512_256),
      |                                                     ^~~~~~~~~~~~~
Modules/_hashopenssl.c:110:25: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                         ^~~~~~~~~
Modules/_hashopenssl.c:124:68: error: 'NID_sha512_256' undeclared here (not in a function); did you mean 'NID_sha512'?
  124 |     PY_HASH_ENTRY(Py_hash_sha512_256, "SHA512_256", SN_sha512_256, NID_sha512_256),
      |                                                                    ^~~~~~~~~~~~~~
Modules/_hashopenssl.c:110:36: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                                    ^~~~~~~~
Modules/_hashopenssl.c:126:43: error: 'SN_sha3_224' undeclared here (not in a function); did you mean 'SN_sha224'?
  126 |     PY_HASH_ENTRY(Py_hash_sha3_224, NULL, SN_sha3_224, NID_sha3_224),
      |                                           ^~~~~~~~~~~
Modules/_hashopenssl.c:110:25: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                         ^~~~~~~~~
Modules/_hashopenssl.c:126:56: error: 'NID_sha3_224' undeclared here (not in a function); did you mean 'NID_sha224'?
  126 |     PY_HASH_ENTRY(Py_hash_sha3_224, NULL, SN_sha3_224, NID_sha3_224),
      |                                                        ^~~~~~~~~~~~
Modules/_hashopenssl.c:110:36: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                                    ^~~~~~~~
Modules/_hashopenssl.c:127:43: error: 'SN_sha3_256' undeclared here (not in a function); did you mean 'SN_sha256'?
  127 |     PY_HASH_ENTRY(Py_hash_sha3_256, NULL, SN_sha3_256, NID_sha3_256),
      |                                           ^~~~~~~~~~~
Modules/_hashopenssl.c:110:25: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                         ^~~~~~~~~
Modules/_hashopenssl.c:127:56: error: 'NID_sha3_256' undeclared here (not in a function); did you mean 'NID_sha256'?
  127 |     PY_HASH_ENTRY(Py_hash_sha3_256, NULL, SN_sha3_256, NID_sha3_256),
      |                                                        ^~~~~~~~~~~~
Modules/_hashopenssl.c:110:36: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                                    ^~~~~~~~
Modules/_hashopenssl.c:128:43: error: 'SN_sha3_384' undeclared here (not in a function); did you mean 'SN_sha384'?
  128 |     PY_HASH_ENTRY(Py_hash_sha3_384, NULL, SN_sha3_384, NID_sha3_384),
      |                                           ^~~~~~~~~~~
Modules/_hashopenssl.c:110:25: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                         ^~~~~~~~~
Modules/_hashopenssl.c:128:56: error: 'NID_sha3_384' undeclared here (not in a function); did you mean 'NID_sha384'?
  128 |     PY_HASH_ENTRY(Py_hash_sha3_384, NULL, SN_sha3_384, NID_sha3_384),
      |                                                        ^~~~~~~~~~~~
Modules/_hashopenssl.c:110:36: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                                    ^~~~~~~~
Modules/_hashopenssl.c:129:43: error: 'SN_sha3_512' undeclared here (not in a function); did you mean 'SN_sha512'?
  129 |     PY_HASH_ENTRY(Py_hash_sha3_512, NULL, SN_sha3_512, NID_sha3_512),
      |                                           ^~~~~~~~~~~
Modules/_hashopenssl.c:110:25: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                         ^~~~~~~~~
Modules/_hashopenssl.c:129:56: error: 'NID_sha3_512' undeclared here (not in a function); did you mean 'NID_sha512'?
  129 |     PY_HASH_ENTRY(Py_hash_sha3_512, NULL, SN_sha3_512, NID_sha3_512),
      |                                                        ^~~~~~~~~~~~
Modules/_hashopenssl.c:110:36: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                                    ^~~~~~~~
Modules/_hashopenssl.c:131:44: error: 'SN_shake128' undeclared here (not in a function); did you mean 'SN_sha512'?
  131 |     PY_HASH_ENTRY(Py_hash_shake_128, NULL, SN_shake128, NID_shake128),
      |                                            ^~~~~~~~~~~
Modules/_hashopenssl.c:110:25: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                         ^~~~~~~~~
Modules/_hashopenssl.c:131:57: error: 'NID_shake128' undeclared here (not in a function); did you mean 'NID_sha512'?
  131 |     PY_HASH_ENTRY(Py_hash_shake_128, NULL, SN_shake128, NID_shake128),
      |                                                         ^~~~~~~~~~~~
Modules/_hashopenssl.c:110:36: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                                    ^~~~~~~~
Modules/_hashopenssl.c:132:44: error: 'SN_shake256' undeclared here (not in a function); did you mean 'SN_sha256'?
  132 |     PY_HASH_ENTRY(Py_hash_shake_256, NULL, SN_shake256, NID_shake256),
      |                                            ^~~~~~~~~~~
Modules/_hashopenssl.c:110:25: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                         ^~~~~~~~~
Modules/_hashopenssl.c:132:57: error: 'NID_shake256' undeclared here (not in a function); did you mean 'NID_sha256'?
  132 |     PY_HASH_ENTRY(Py_hash_shake_256, NULL, SN_shake256, NID_shake256),
      |                                                         ^~~~~~~~~~~~
Modules/_hashopenssl.c:110:36: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                                    ^~~~~~~~
Modules/_hashopenssl.c:134:50: error: 'SN_blake2s256' undeclared here (not in a function)
  134 |     PY_HASH_ENTRY(Py_hash_blake2s, "blake2s256", SN_blake2s256, NID_blake2s256),
      |                                                  ^~~~~~~~~~~~~
Modules/_hashopenssl.c:110:25: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                         ^~~~~~~~~
Modules/_hashopenssl.c:134:65: error: 'NID_blake2s256' undeclared here (not in a function)
  134 |     PY_HASH_ENTRY(Py_hash_blake2s, "blake2s256", SN_blake2s256, NID_blake2s256),
      |                                                                 ^~~~~~~~~~~~~~
Modules/_hashopenssl.c:110:36: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                                    ^~~~~~~~
Modules/_hashopenssl.c:135:50: error: 'SN_blake2b512' undeclared here (not in a function)
  135 |     PY_HASH_ENTRY(Py_hash_blake2b, "blake2b512", SN_blake2b512, NID_blake2b512),
      |                                                  ^~~~~~~~~~~~~
Modules/_hashopenssl.c:110:25: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                         ^~~~~~~~~
Modules/_hashopenssl.c:135:65: error: 'NID_blake2b512' undeclared here (not in a function)
  135 |     PY_HASH_ENTRY(Py_hash_blake2b, "blake2b512", SN_blake2b512, NID_blake2b512),
      |                                                                 ^~~~~~~~~~~~~~
Modules/_hashopenssl.c:110:36: note: in definition of macro 'PY_HASH_ENTRY'
  110 |     {py_name, py_alias, ossl_name, ossl_nid, 0, NULL, NULL}
      |                                    ^~~~~~~~
Modules/_hashopenssl.c: In function 'EVPXOF_digest_impl':
Modules/_hashopenssl.c:737:10: error: implicit declaration of function 'EVP_DigestFinalXOF'; did you mean 'EVP_DigestFinal'? [-Werror=implicit-function-declaration]
  737 |     if (!EVP_DigestFinalXOF(temp_ctx,
      |          ^~~~~~~~~~~~~~~~~~
      |          EVP_DigestFinal
Modules/_hashopenssl.c: In function 'py_evp_fromname':
Modules/_hashopenssl.c:876:33: error: 'EVP_MD_FLAG_XOF' undeclared (first use in this function); did you mean 'EVP_MD_FLAG_FIPS'?
  876 |     if ((EVP_MD_flags(digest) & EVP_MD_FLAG_XOF) == EVP_MD_FLAG_XOF) {
      |                                 ^~~~~~~~~~~~~~~
      |                                 EVP_MD_FLAG_FIPS
Modules/_hashopenssl.c:876:33: note: each undeclared identifier is reported only once for each function it appears in
Modules/_hashopenssl.c: In function '_hashlib_scrypt_impl':
Modules/_hashopenssl.c:1389:14: error: implicit declaration of function 'EVP_PBE_scrypt'; did you mean 'EVP_PKEY_decrypt'? [-Werror=implicit-function-declaration]
 1389 |     retval = EVP_PBE_scrypt(NULL, 0, NULL, 0, n, r, p, maxmem, NULL, 0);
      |              ^~~~~~~~~~~~~~
      |              EVP_PKEY_decrypt
cc1: some warnings being treated as errors

I have restored an updated version of 026-openssl-feature-flags.patch in #21030.

PolynomialDivision marked this pull request as ready for review December 20, 2022 15:46

dangowrt approved these changes Dec 21, 2022

View reviewed changes

misaka36 mentioned this pull request Dec 21, 2022

WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available. openwrt/openwrt#11578

Closed

neheb approved these changes Dec 21, 2022

View reviewed changes

PolynomialDivision force-pushed the python3-fix-ssl branch from 3d6cf0d to ddbe3ca Compare December 21, 2022 11:33

PolynomialDivision merged commit 4ecd9d6 into openwrt:master Dec 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python3: fix ssl support by removing libressl patches #20137

python3: fix ssl support by removing libressl patches #20137

Uh oh!

PolynomialDivision commented Dec 20, 2022

Uh oh!

dhewg commented Dec 20, 2022

Uh oh!

dangowrt left a comment

Uh oh!

PolynomialDivision commented Dec 21, 2022

Uh oh!

neheb commented Dec 21, 2022 via email

Uh oh!

PolynomialDivision commented Dec 21, 2022

Uh oh!

jefferyto commented May 23, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

python3: fix ssl support by removing libressl patches #20137

python3: fix ssl support by removing libressl patches #20137

Uh oh!

Conversation

PolynomialDivision commented Dec 20, 2022

Uh oh!

dhewg commented Dec 20, 2022

Uh oh!

dangowrt left a comment

Choose a reason for hiding this comment

Uh oh!

PolynomialDivision commented Dec 21, 2022

Uh oh!

neheb commented Dec 21, 2022 via email

Uh oh!

PolynomialDivision commented Dec 21, 2022

Uh oh!

jefferyto commented May 23, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants