[WIP] Qt 5.15 static build, help needed#136107
[WIP] Qt 5.15 static build, help needed#136107SCOTT-HAMILTON wants to merge 37 commits intoNixOS:masterfrom
Conversation
SCOTT-HAMILTON
requested review from
Ericson2314,
FRidh,
jonringer and
ttuegel
as code owners
github-actions
bot
added
6.topic: python
Sounds good. The next version will hopefully use Meson.
Why not do it in the main package?
Sounds like a build system bug.
Maybe we should bring back the use of
Afraid giscanner, and therefore also bindings generation, will not work without that. But do not think bindings would work with static libraries anyway (they are loaded at runtime).
After #136071, it might be possible to stop linking Qt against GTK so perhaps most of this will be unnecessary for static Qt. |
Alpine Linux handles it the same way we separate dev and bin outputs except that they don't call it outputs but subpackages. So they build the library with both shared and static libs and then the "fixup" phase automatically splits the static libraries to the static subpackage ($pkgname-static). cf https://wiki.alpinelinux.org/wiki/APKBUILD_Reference#static.28.29
« To interfer the least with the non static derivations ». |
Oh, I thought Alpine was building primarily static libs (must have confused it with some other distro). I wondered how they convince the projects to build statically but if they just use the static libs built by default like we do then that will not help us.
I assume non-static |
Also +1 for that. Switching |
pkgs/top-level/static.nix has been removed in staging, so you should modify the main derivations |
SCOTT-HAMILTON
requested review from
marsam and
thoughtpolice
as code owners
|
This PR is ready for review, I can build a Qt app with it, I fixed the qtsvg, qtdeclarative and qttools modules. |
| clangCxxFlags = "-nostdinc++ -isystem ${libcxxIncDir}"; | ||
| clangCxxLdflags = "-nostdlib++ -L ${libcxxIncDir} -Wl,-rpath,${libcxxLibDir} -lc++ -lc++abi"; | ||
|
|
||
| # Building with libc++ implies that all statically linked c++ deps are also |
There was a problem hiding this comment.
maybe you want to build pkgs.pkgsStatic.pkgsLLVM.qtbase then?
can you add a comment as to why you are using clang instead of the default compiler in stdenv? Doesn't that force all dependent apps to also compile with clang?
| }; | ||
|
|
||
| in { | ||
| in rec { |
There was a problem hiding this comment.
this file does was removed on staging next, please move this to the main derivations and rebase.
| at-spi2-core = super.at-spi2-core.overrideAttrs (old: { | ||
| NIX_MESON_DEPENDENCY_STATIC = true; | ||
| nativeBuildInputs = (old.nativeBuildInputs or []) ++ [ | ||
| super.mesonShlibsToStaticHook |
There was a problem hiding this comment.
maybe this hook should be propagated by meson when targetPlatform.isStatic ?
There was a problem hiding this comment.
Would that work ?
Details
diff --git a/pkgs/development/tools/build-managers/meson/default.nix b/pkgs/development/tools/build-managers/meson/default.nix
index 60ab7605944..219160c66a5 100644
--- a/pkgs/development/tools/build-managers/meson/default.nix
+++ b/pkgs/development/tools/build-managers/meson/default.nix
@@ -1,13 +1,14 @@
-{ lib
+{ stdenv
+, lib
, python3
-
+, mesonShlibsToStaticHook
, writeTextDir
, substituteAll
, fetchpatch
, installShellFiles
}:
-python3.pkgs.buildPythonApplication rec {
+python3.pkgs.buildPythonApplication (rec {
pname = "meson";
version = "0.57.1";
@@ -98,4 +99,6 @@ python3.pkgs.buildPythonApplication rec {
maintainers = with maintainers; [ jtojnar mbe ];
platforms = platforms.all;
};
-}
+} // lib.optionalAttrs stdenv.hostPlatform.isStatic {
+ propagatedBuildInputs = [ mesonShlibsToStaticHook ];
+})There was a problem hiding this comment.
actually putting this in propagatedBuildInputs is wrong because it also affects transitive dependencies. What about incorporating this to pkgs/development/tools/build-managers/meson/setup-hook.sh gated behind if [ -n $mesonForceReplaceSharedLibsByStaticLibs ] and $mesonForceReplaceSharedLibsByStaticLibs defaults to targetPlatform.isStatic (you can pass the value from nix to setup-hook.sh by using the fact that according to the doc of setupHook https://nixos.org/manual/nixpkgs/stable/#ssec-fixup-phase @envvar@ is replaced by the value of envvar during the build of meson)
| }; | ||
|
|
||
| shared-mime-info = super.shared-mime-info.overrideAttrs (old: { | ||
| NIX_MESON_DEPENDENCY_STATIC = true; |
There was a problem hiding this comment.
maybe this should be the default and one can disable this if this was wrong, like dontAddStaticConfigureFlag, what do you think ?
There was a problem hiding this comment.
I will add this logic in the makeStaticLibraries adapter. BTW I don't know why it's called like that because it's applied to both libraries and executables.
|
I think it would be preferable to focus on making pkgsStatic.pkgsLLVM.qtbase work and giveup on pkgsStatic.qtbase rather than hacking icu and postgresql to use llvm when compiling statically. |
|
I chose to use clang mainly because I don't know if this has something to do with https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/compilers/gcc/no-sys-dirs.patch |
gcc's c++ include is fine afaict, it just isn't in $out/include but in a different internal dir when cross-compiling. It still shows up in e.g. Not sure why icu has this linking issue, but setting postgresql's NIX_LDFLAGS to |
|
This contains quite a bunch of helpful fixes, any chance to get at least a subset into master before 21.11? :) |
|
Could you please split this change up into smaller PRs? It's too big to review all at once and make a decision on, especially because different people need to make a call about different bits. |
SuperSandro2000
added
the
2.status: merge conflict
|
Should I make 37 PRs ? One per commit ? |
qt can probably be combined and for the rest we can maybe also do batches of 3 or 5. |
| runHook preMesonShlibsToStatic | ||
| replaceMesonFunction() { |
There was a problem hiding this comment.
You are mixing tabs and spaces. I think nixpkgs is always formatted with spaces.
| # hence, libtasn1 is required in both native and build inputs. | ||
| nativeBuildInputs = [ autoreconfHook pkg-config which libtasn1 ]; | ||
| nativeBuildInputs = [ meson ninja pkg-config which libtasn1 ] ++ | ||
| lib.optional (stdenv.hostPlatform.isStatic) [ mesonShlibsToStaticHook ]; |
There was a problem hiding this comment.
| lib.optional (stdenv.hostPlatform.isStatic) [ mesonShlibsToStaticHook ]; | |
| lib.optional (stdenv.hostPlatform.isStatic) mesonShlibsToStaticHook; |
or use lib.optionals.
There was a problem hiding this comment.
@SuperSandro2000 as said above, this PR isn't meant to be merged any more, I'll make it a draft again and only use it as a reference for smaller PRs like this one #147600.
Once Qt5 statically builds on staging, I'll close it.
| nativeBuildInputs = [ meson ninja pkg-config which libtasn1 ] ++ | ||
| lib.optional (stdenv.hostPlatform.isStatic) [ mesonShlibsToStaticHook ]; | ||
| buildInputs = [ gettext libffi libiconv libtasn1 ]; | ||
| propagatedBuildInputs = [ bash-completion ]; |
There was a problem hiding this comment.
Do we need to propogate this when using the path in line 33?
| sha256 = "1cb2hwi859hds0fa2cbap014qaa7mah9p0rcxcm2cvj2ybl33qfc"; | ||
| }) | ||
| ]; | ||
| ] ++ lib.optional (stdenv.hostPlatform.isStatic) |
There was a problem hiding this comment.
FYI The () here are optional.
| sed -iE 's!^QMAKE_CXXFLAGS.*!QMAKE_CXXFLAGS += ${clangCxxFlags}!g'\ | ||
| mkspecs/linux-clang-libc++/qmake.conf | ||
| sed -iE 's!^QMAKE_LFLAGS.*!QMAKE_LFLAGS += ${clangCxxLdflags}!g'\ | ||
| mkspecs/linux-clang-libc++/qmake.conf |
There was a problem hiding this comment.
| sed -iE 's!^QMAKE_CXXFLAGS.*!QMAKE_CXXFLAGS += ${clangCxxFlags}!g'\ | |
| mkspecs/linux-clang-libc++/qmake.conf | |
| sed -iE 's!^QMAKE_LFLAGS.*!QMAKE_LFLAGS += ${clangCxxLdflags}!g'\ | |
| mkspecs/linux-clang-libc++/qmake.conf | |
| sed -iE -e 's!^QMAKE_CXXFLAGS.*!QMAKE_CXXFLAGS += ${clangCxxFlags}!g' \ | |
| -e 's!^QMAKE_LFLAGS.*!QMAKE_LFLAGS += ${clangCxxLdflags}!g' \ | |
| mkspecs/linux-clang-libc++/qmake.conf |
FYI this could be done like this but does not matter.
| outputs = [ "bin" "dev" "out" ]; | ||
| dontUnpack = true; | ||
|
|
||
| buildPhase = ""; |
There was a problem hiding this comment.
| buildPhase = ""; | |
| dontBuild = true; |
| # it's just easier to separate the build from the fixup | ||
| # and do/tweak it in a second/final stage. | ||
| in if (!stdenv.hostPlatform.isStatic) then stage1 else (stdenv.mkDerivation { | ||
| pname = "qtbase-final"; |
There was a problem hiding this comment.
| pname = "qtbase-final"; | |
| pname = "qtbase"; |
It would be nicer to name the stages differently as this might break other things.
| } // lib.optionalAttrs stdenv.hostPlatform.isStatic { | ||
| preConfigure = '' | ||
| NIX_LDFLAGS+=" -lXdmcp -lXau -lexpat -lbz2" | ||
| ''; |
There was a problem hiding this comment.
Since we are rebuilding everything anyway we can also inline this.
|
|
||
| LIBA=libselinux.a | ||
| -TARGET=libselinux.so | ||
| +# TARGET=libselinux.so |
There was a problem hiding this comment.
| +# TARGET=libselinux.so |
When rebasing this patch in the future it will be easier when the line is just deleted.
| nativeBuildInputs = (old.nativeBuildInputs or []) ++ [ | ||
| super.mesonShlibsToStaticHook | ||
| ]; | ||
| NIX_CFLAGS_COMPILE = "-DG_LOG_DOMAIN=\"\"\"\""; |
There was a problem hiding this comment.
I don't understand what is being done here.
|
Closing to fix confusion. |
|
@SCOTT-HAMILTON Are you available to create a PR for 292a04e? This should fix the static build of OpenGL. If not, may I create a PR with your changes and add your name as a package maintainer? |
|
I'm on it, will have the time this evening. |
9 participants
Motivation for this change
I think it's useful to have static builds for a few reasons :
Things done
I managed to get
pkgsStatic.libsForQt515.qtbasebuild successfully.But I couldn't getFixed !pkgsStatic.libsForQt515.qtsvgto build against it.The goal of this is to get at least one simple Qt Gui app to build statically.
The guidelines I tried to follow for this contribution are :
stdenv.hostPlatform.isStaticconditional extensively or only adding an override inpkgs/top-level/static.nixChanges made
static: truekwarg on all dependency() function calls (probably impossible to get write with a sed, it might be possible with some tools like beautifulsoup though)And whileFixed !pkgsStatic.libsForQt515.qtbasebuilds successfully, it doesn't seem to work,pkgsStatic.libsForQt515.qtsvgdoesn't build against it. This PR shouldn't be merge until bothpkgsStatic.postgresqlandpkgsStatic.libsForQt515.qtsvgbuild successfully.Status
I can build a simple Qt app of mine statically !
sandbox = trueset innix.conf? (See Nix manual)nix-shell -p nixpkgs-review --run "nixpkgs-review wip"./result/bin/)