Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Musl support for htslib 1.10.2 (#193)
* Add alternative strategy (local dependencies install) to get musl support shipped into rust-htslib * Tweaking toolchain further for rust-htslib-musl and its dependencies * Circumvent Makefile openssl bug in openssl/openssl#11362 and many other compile time gotchas * Add newly docker baked image with the whole toolchain built statically with and for musl * Bypass unneeded CI steps for now to focus on musl * Tell openssl-sys crate where is our custom musl-openssl cross compiled directory... * It is possible that https://github.com/rust-bio/rust-htslib/pull/193/checks?check_run_id=518233498#step:5:253 is just a binutils bug according to https://bbs.archlinux.org/viewtopic.php?id=242682 and other sites... * Revert linting/formatting steps, those are done (and needed) anyway for the PR... * Nasty hack with ld, tell musl that its own includes are available via CPPFLAGS and hopefully bypass clang/llvm errors? * LDFLAGS and CPPFLAGS need to be passed to hts-sys, plus need to be surrounded by quotes since there are spaces in the variables * Be a bit more careful with env variables when clippy is running * A bit more classy to pass clippy? * Temporarily try to bypass formatting/linting again, not helping focus on musl * Passing clang_arg() to bindgen to compile the wrapper with -I/usr/local/musl/x86_64-linux-musl/include does not seem to help :/ * Hardcoded include path works for both musl and GNU bindgen building, removing .cargo/config, not necessary * Re-enabling formatting and linting Github action phases after green build on MUSL testing :) * hfile_s3.o is only included on libhts.a when ./configure is run previously according to James and Rob from the htslib team * Make sure we are in htslib directory, otherwise configure.ac and .in will not be found * Fix up redundant imports according to clippy, switch to alpine linux container, way more lightweight and less messy build process, also includes MUSL as base compiler * Explicitly cross-compile passing the right ./configure --host flag, to be generalized later * LDFLAGS was being pointed to CPPFLAGS, copy&paste hell. Also make cargo clippy happy * More clippy * Puzzled as of why this clippy error was not detected locally... also htslib compiles fine locally as well, no cross, just cargo * CPPFLAGS as an argument to make is causing trouble on Linux but not on OSX... * This include does not affect any build and it is crucial for MUSL ones * Back to non-alpine container, not sure why cargo is not being found * Disabling --no-default-features building for now, clang-sys not needed, suppress cppflags warnings * Following @pmarks advice on libz-sys, bump up both libc anb libz-sys while at it * Enable no-default-features back * Make clippy happy with libz * libz-sys should not be optional, thanks @pmarks * Cleaning up flags, trying to have libz as static to avoid //usr/lib64/libz.so.1: error adding symbols: DSO missing from command line, linking errors, thansk @wezm for the tips * Figured out next steps: need to incorporate static compilation features to both bzip2-sys and lzma-sys, otherwise MUSL will have DSO linker issues. Will implement alexcrichton/xz2-rs#62 and alexcrichton/bzip2-rs#56 next * Add missing static flag to lzma-sys, getting rid of the need to export the env variable, closing into the core of the issue: rust-htslib's build.rs needs a feature-flag revamp as well... in addition to env variables, it should be able to use cfg Cargo features as well... and not sed-modify htslib's makefile if it's a static build * Remove openssl-sys crate in favor of curl-sys, which already includes a vendored openssl-sys (hopefully more static-friendly) * Temporarily comment out sed modifications against htslib's makefile, focusing on static compilation * Semver + suffix on curl is not needed * make -B, that flag was the current culprit on static compilation, bizarre since it does not seem to be related at all to the libssl errors: https://www.gnu.org/software/make/manual/html_node/Options-Summary.html * Make extra sure that -fPIC flag is passed during all compile phases * Passing println(cargo:rustc-link-lib=static=hts), works, finally :) * Cleanup cflgags_patterns mess * Works locally, migrating to docker/cross builds, MUSL has regressions after build.rs changes, regular GNU toolchain works though * Cleanup build.rs * Add clux/muslrust Dockerfile example, as referred on https://users.rust-lang.org/t/musl-cross-compilation-how-to-reuse-curl-sys-libz-sys-bzip2-sys-lzma-sys-and-other-dependency-includes-under-target/42817/4 * Revert .github workflow minor changes because 'remote rejected] musl_support -> musl_support (refusing to allow an OAuth App to create or update workflow without scope)' * Adding rust workflow from master... * Remove GitHub actions workflows for now since they are blocking my progress * Thanks @wezm for feedback on MUSL dockerfiles and https://github.com/fornwall/rust-static-builder pointer * Finally compiles MUSL, passing CFLAGS=-I/usr/local/musl/include and using build.env cross to passthrough that var * Reintroducing the GitHub actions workflow... will I be able to push it now? * Nope, pushes are still blocked by '[remote rejected] musl_support -> musl_support (refusing to allow an OAuth App to create or update workflow...' * Cleanup and update README with the required cross commands. Also cleanup the different Dockerfiles tested but no longer needed. build.rs now runs make clean before reconfiguring htslib for better reproducible builds * Add back github workflow, now that I am a project collab/admin * Make sure MUSL builds find zlib.h et al * Sans quotes and let others concatenate CFLAGS * Remove clux rust container references and stray -j40 make flag Co-authored-by: Roman Valls Guimera <[email protected]>
- Loading branch information
1 parent
3a2ff4e
commit 8107160