Change priority of PREFIX in CFLAGS and LDFLAGS #1031
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Move 'CFLAGS += -I/usr/local/include' below BoringSSL, so that if OS have OpenSSL header files installed in /usr/local/include, it won't mess with vendored BoringSSL. This should fix the issue with macOS CI job.
vixentael
reviewed
Oct 18, 2023
vixentael
reviewed
Oct 18, 2023
For some reason, previous commit broke usual OpenSSL builds on macOS. Since the main issue was with include dir, only change CFLAGS after boringssl part and leave LDFLAGS where it was before.
To trigger CI
I'm waiting green tests) We discussed this in slack and @iamnotacake mentioned that this change brought more failures and he will debug it |
Add VERBOSE option to Makefile. If set, print executed command (compiler, linker etc) even for successful runs. Remove WITH_EXPERIMENTAL_OPENSSL_3_SUPPORT flag from CI config since this option is no longer used.
iamnotacake
changed the title
Lagovas
approved these changes
Oct 20, 2023
Makefile
Outdated
| # Basic compiler flags (lower priority than selected engine) | ||
| # We got /usr/local as default PREFIX and not all platforms include that path in default search path. | ||
| # Make sure whatever PREFIX is used, includes and libs are searched there. | ||
| CFLAGS += -I$(PREFIX)/include |
There was a problem hiding this comment.
let's use
iamnotacake
merged commit 3a671e7
into
cossacklabs:master-ci-fixes
61 of 62 checks passed
iamnotacake
added a commit
that referenced
this pull request
Nov 9, 2023
Rix Rust CI builds (#1032) * Pin log dependency to 0.4.17 * Pin byteorder dependency to 1.4.3 Last versions that still work with current MSRV 1.58. Fix sanitizers CI job (#1033) * Switch to GCC 10 * Install `libgcc-10-dev` that provides file `libtsan_preinit.o` needed for thread sanitizer Change priority of PREFIX in CFLAGS and LDFLAGS (#1031) * Change include dir priority in CFLAGS Move 'CFLAGS += -I/usr/local/include' below engine selection macros, so that if OS have OpenSSL header files installed in /usr/local/include, it won't mess with vendored BoringSSL or any other selected engine. * Put /usr/local/lib in separate LDFLAGS Introduce ADDITIONAL_LDFLAGS macro and put it after both LDFLAGS and CRYPTO_ENGINE_LDFLAGS during linking. * Add optional verbose logging to Makefile Add VERBOSE option to Makefile. If set, print executed command (compiler, linker etc) even for successful runs, and run ldd on created shared libraries. * Enable verbose builds on macOS jobs
G1gg1L3s
added a commit
to G1gg1L3s/themis
that referenced
this pull request
Dec 4, 2023
commit 784033b Author: Anatolii Lishchynskyi <[email protected]> Date: Tue Nov 28 21:45:12 2023 +0200 New methods of building/installing PyThemis (cossacklabs#1023) Refactor Makefile: * new target pythemis_make_wheel to create a .whl Python package, current modern format to be installed into virtual environmants * new target pythemis_install_wheel to install it in currently active virtualenv * new target deb_python that builds a .deb package for system-wide installation of PyThemis * new target pythemis_install_deb, alias for pythemis_deb + apt install of the created pkg * new target rpm_python, similar to deb_python * new target pythemis_install_rpm, pythemis_install_deb Update GitHub Actions workflow, test .whl and .deb installation See cossacklabs/product-docs/pull/317 for related docs update commit 30578c8 Author: Anatolii Lishchynskyi <[email protected]> Date: Mon Nov 27 17:53:02 2023 +0200 Fix clippy and fmt issues, update MSRV (cossacklabs#1039) Fix clippy and fmt issues Update MSRV to 1.60 Freeze test deps so they compile on Rust 1.60, with no effect on themis itself (does not use those frozen crates) commit 6111766 Author: Anatolii Lishchynskyi <[email protected]> Date: Tue Nov 21 17:08:51 2023 +0200 Update emscripten requirements and WASM CI job (cossacklabs#1036) * Bump emsdk version to 3.1.47 Produces module importable in Node v18 With older (emsdk 3.0.0) version, generated `libthemis.js` that should load `libthemis.wasm` fails due to some internal autogenerated code working differently on v16 and v18 * Add link flag for WASM builds Needed to make new emscripten produce working module * Update Node testing versions Add v18 that should now work, also add v20 that is in active development as of now, but if tests are green then why not? * Switched integration tests to use v16 * Removed testing of quite old and deprecated v10 * Updated version of BoringSSL submodule to a newer one (not the latest though) commit 05cac26 Author: Anatolii Lishchynskyi <[email protected]> Date: Thu Nov 9 22:24:35 2023 +0200 CI fixes (cossacklabs#1034) Rix Rust CI builds (cossacklabs#1032) * Pin log dependency to 0.4.17 * Pin byteorder dependency to 1.4.3 Last versions that still work with current MSRV 1.58. Fix sanitizers CI job (cossacklabs#1033) * Switch to GCC 10 * Install `libgcc-10-dev` that provides file `libtsan_preinit.o` needed for thread sanitizer Change priority of PREFIX in CFLAGS and LDFLAGS (cossacklabs#1031) * Change include dir priority in CFLAGS Move 'CFLAGS += -I/usr/local/include' below engine selection macros, so that if OS have OpenSSL header files installed in /usr/local/include, it won't mess with vendored BoringSSL or any other selected engine. * Put /usr/local/lib in separate LDFLAGS Introduce ADDITIONAL_LDFLAGS macro and put it after both LDFLAGS and CRYPTO_ENGINE_LDFLAGS during linking. * Add optional verbose logging to Makefile Add VERBOSE option to Makefile. If set, print executed command (compiler, linker etc) even for successful runs, and run ldd on created shared libraries. * Enable verbose builds on macOS jobs commit 06d52f4 Author: Anatolii Lishchynskyi <[email protected]> Date: Tue Sep 26 13:15:37 2023 +0300 Fix make target rbthemis_uninstall (cossacklabs#1022) * Fix make target rbthemis_uninstall * Add RbThemis uninstall step to CI commit 8609650 Author: Nazar Serhiichuk <[email protected]> Date: Mon Jul 10 22:57:46 2023 +0300 Avoid specific versions in README (cossacklabs#1016) Just to avoid responsibility of updating it during release (•_•) ( •_•)>⌐■-■ (⌐■_■) commit 3219654 Author: Martin Arista <[email protected]> Date: Mon Jul 10 11:54:25 2023 -0400 Update README.md (cossacklabs#1015) update links for maven and java/kotlin links
Lagovas
pushed a commit
that referenced
this pull request
Dec 5, 2023
* Pre-release Themis 0.15.0 (#1011) * Fix rust issues (pin log, run bindgen) (#1005) * rust: Pin log version to =0.4.18 The 0.4.19 requires rustc 1.60, but currently we support 1.58. Pinning it is not a big deal since it's development dependecy for tests and examples. * rust: Regenerate and update lib.rs bindgen was updated again and changed something which resulted in new output (seems like some internal constants are removed). * Pythemis: introduce `pyproject.toml` (#1006) * pythemis: Add pyproject.toml Since setup.py is deprecated, let's try moving to the pyproject.toml and configuring it with the same data as in setup.py. Use setuptools as a backend for no particular reasons ¯\_(ツ)_/¯, just because the name is familiar and we have no reasons to not use it or use something else. Keep the old setup.py for backward compatibility so old systems can try to build the package. For now, keep 0.14.0, we will bump the version in another PR. * makefile: Use pyproject.toml for installing pythemis According to this [1] article, the correct command is pip install . in the project's root. Let's try that. Also, the other option is python -m build --wheel which builds the package but doesn't install it. We can provide something like `pythemis_build` for it for example. [1]: https://godatadriven.com/blog/a-practical-guide-to-setuptools-and-pyproject-toml/ * pythemis: Update classifiers to Python3.6+ With many hours and docker containers I tested that themis actually works up to python 3.4. The other versions require some changes in makefile so they are more like "grey area". However, python3.5 is deprecated and it produces warning like "DEPRECATION: Python 3.5 reached the end of its life on..." so many libraries don't support it. Instead they start with 3.6 which will do as well, I guess. Though, actually python3.6 is also deprecated [1]. The same will be true for python3.7 in a couple of days (Jun 27 2023), so the question is, should we declare support of these versions? [1]: https://devguide.python.org/versions/ * pythemis: Extend range of supported py versions * Update changelog * Run and pin bindgen (#1008) * rust-themis: Update bindgen It updated and broke something again 🤦 * rust-themis: Pin bindgen version It is pretty unstable with its frequent releases, so let's pin it. * Update changelog * Bump wrapper versions to 0.15.0 (#1007) * changelog: Add 0.15.0 summary * themis-core: Update version * pythemis: Update version * pythemis: Fix 8-year old typo in AUTHORS :) * rbthemis: Update version * jsthemis: Update versions * wasm-themis: Update versions * android-themis: Update version * rust-themis: Update versions * react-native-themis: Update versions * pythemis: https in AUTHORS Co-authored-by: vixentael <[email protected]> * rust-themis: Update bench versions Somehow missed that. * changelog: Forgot to mention rust 1.58 * changelog: Mention the new iteration count --------- Co-authored-by: vixentael <[email protected]> * Bump embedded BoringSSL (#1004) * Bump BoringSSL and fix makefile This is not the latest BoringSSL version yet, because there are a couple of fixes. So, treat it as the first. Here we also fix our makefile because the BoringSSL team fixed bug with the strange behaviour of absolute path to symbols.txt [1]. [1]: https://boringssl.googlesource.com/boringssl/+/8c75ed046f799f1d8b805036b1dea9c5ec0a0fb5%5E%21/#F0 * Bump BoringSSL and fix opaque EVP As OpenSSL, BoringSSL made many types opaque, so it will require updating some of the code to not use fields. * Bump BoringSSL again and fix RSA The same issue - RSA type became opaque, so we need to use accessors similar to what Openssl had. * Bump BoringSSL once more This is (hoperfully) the last bump. This time without issues but we will see what CI says. * Make bignum_to_bytes accept const bignum* It will prevent some of the warnings. This function doesn't mutate bignum anyway. * Update changelog * boringssl: Bump once again * msys2: Update hashes temporarily This are test values because we will move the tag. But for now, let's just test it. * phpthemis: Update version for the sake of testing They will fail probably, but just out of curiosity let's try to run the tests. * Update date of the release Solstice! --------- Co-authored-by: vixentael <[email protected]> * msys2: Update checksums (#1012) It's the egg-chicken problem: we can update those hashes only after the release. But then, the release tag will not point to the updated hashes. * Squashed commit of the following: commit 784033b Author: Anatolii Lishchynskyi <[email protected]> Date: Tue Nov 28 21:45:12 2023 +0200 New methods of building/installing PyThemis (#1023) Refactor Makefile: * new target pythemis_make_wheel to create a .whl Python package, current modern format to be installed into virtual environmants * new target pythemis_install_wheel to install it in currently active virtualenv * new target deb_python that builds a .deb package for system-wide installation of PyThemis * new target pythemis_install_deb, alias for pythemis_deb + apt install of the created pkg * new target rpm_python, similar to deb_python * new target pythemis_install_rpm, pythemis_install_deb Update GitHub Actions workflow, test .whl and .deb installation See cossacklabs/product-docs/pull/317 for related docs update commit 30578c8 Author: Anatolii Lishchynskyi <[email protected]> Date: Mon Nov 27 17:53:02 2023 +0200 Fix clippy and fmt issues, update MSRV (#1039) Fix clippy and fmt issues Update MSRV to 1.60 Freeze test deps so they compile on Rust 1.60, with no effect on themis itself (does not use those frozen crates) commit 6111766 Author: Anatolii Lishchynskyi <[email protected]> Date: Tue Nov 21 17:08:51 2023 +0200 Update emscripten requirements and WASM CI job (#1036) * Bump emsdk version to 3.1.47 Produces module importable in Node v18 With older (emsdk 3.0.0) version, generated `libthemis.js` that should load `libthemis.wasm` fails due to some internal autogenerated code working differently on v16 and v18 * Add link flag for WASM builds Needed to make new emscripten produce working module * Update Node testing versions Add v18 that should now work, also add v20 that is in active development as of now, but if tests are green then why not? * Switched integration tests to use v16 * Removed testing of quite old and deprecated v10 * Updated version of BoringSSL submodule to a newer one (not the latest though) commit 05cac26 Author: Anatolii Lishchynskyi <[email protected]> Date: Thu Nov 9 22:24:35 2023 +0200 CI fixes (#1034) Rix Rust CI builds (#1032) * Pin log dependency to 0.4.17 * Pin byteorder dependency to 1.4.3 Last versions that still work with current MSRV 1.58. Fix sanitizers CI job (#1033) * Switch to GCC 10 * Install `libgcc-10-dev` that provides file `libtsan_preinit.o` needed for thread sanitizer Change priority of PREFIX in CFLAGS and LDFLAGS (#1031) * Change include dir priority in CFLAGS Move 'CFLAGS += -I/usr/local/include' below engine selection macros, so that if OS have OpenSSL header files installed in /usr/local/include, it won't mess with vendored BoringSSL or any other selected engine. * Put /usr/local/lib in separate LDFLAGS Introduce ADDITIONAL_LDFLAGS macro and put it after both LDFLAGS and CRYPTO_ENGINE_LDFLAGS during linking. * Add optional verbose logging to Makefile Add VERBOSE option to Makefile. If set, print executed command (compiler, linker etc) even for successful runs, and run ldd on created shared libraries. * Enable verbose builds on macOS jobs commit 06d52f4 Author: Anatolii Lishchynskyi <[email protected]> Date: Tue Sep 26 13:15:37 2023 +0300 Fix make target rbthemis_uninstall (#1022) * Fix make target rbthemis_uninstall * Add RbThemis uninstall step to CI commit 8609650 Author: Nazar Serhiichuk <[email protected]> Date: Mon Jul 10 22:57:46 2023 +0300 Avoid specific versions in README (#1016) Just to avoid responsibility of updating it during release (•_•) ( •_•)>⌐■-■ (⌐■_■) commit 3219654 Author: Martin Arista <[email protected]> Date: Mon Jul 10 11:54:25 2023 -0400 Update README.md (#1015) update links for maven and java/kotlin links * Bump boringssl * Revert "Bump boringssl" This reverts commit dee8388. Okay, this time I checked, the previous commit of boiringssl was newer than one in the master. * msys2: Fix hashes for 0.15.0 release (#1040) Since we already have 0.15.0 assets on GitHub, we can specify those and expect successful builds on stable. --------- Co-authored-by: vixentael <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously we had broken BoringSSL build on macOS. Reason — include dir flags (
-I) and lib search dir flags (-L) were in wrong order.Then, discovered that
-L/usr/local/libis always before engine lib search path (stored inCRYPTO_ENGINE_LDFLAGS). That caused OpenSSL 1.1 and OpenSSL 3 macOS CI builds to actually use system OpenSSL instead (seems like macOS has OpenSSL 3.0 in/usr/local/lib).The final fix is:
-I$(PREFIX)/includetoCFLAGSafter engine CFLAGS are appended to CFLAGS-L$(PREFIX)/libto new macroADDITIONAL_LDFLAGSLDFLAGSis usedLDFLAGSandCRYPTO_ENGINE_LDFLAGS, thus make it lowest priorityIn process, extended Makefile with
VERBOSEvar. If set to anything, prints executed commands (compilation, linking) even for successfully completed commands. Because it's quite possible that a.cfile is successfully compiled with wrong flags, but then for some reason linking is failing. TL;DR is that it's useful. Also, after creation of Soter/Themis dynamic lib, print other dynamic libs it was linked with. So that we can clearly see in CI logs which OpenSSL lib is used in case of multiple ones exist in system like on macOS with Homebrew.This should fix the issue with macOS CI job.
During discussion we ended up with idea of additional macro, that it would be a more proper solution.
Alternative possible solution with linker flags:
CRYPTO_ENGINE_LDFLAGSbeforeLDFLAGS-L$(PREFIX)/libtoLDFLAGSADDITIONAL_LDFLAGSChecklist
Benchmark results are attached (if applicable)Public API has proper documentationExample projects and code samples are up-to-date (in case of API changes)Changelog is updated (in case of notable or breaking changes)