Skip to content

pkgsMusl.firefox: fix build#191060

Closed
yu-re-ka wants to merge 250 commits intoNixOS:masterfrom
yu-re-ka:musl-firefox
Closed

pkgsMusl.firefox: fix build#191060
yu-re-ka wants to merge 250 commits intoNixOS:masterfrom
yu-re-ka:musl-firefox

Conversation

@yu-re-ka
Copy link
Contributor

@yu-re-ka yu-re-ka commented Sep 13, 2022 •
edited
Loading

Description of changes

Depends on #190796

I had some discussion in the musl irc channel about the malloc problem. There is two problems actually: lld crashes when built with the default stack size. But even when built with a larger stack size, it would crash when built with the musl mallocng allocator, because that allocator produces a very bad result for this allocation pattern and would exceed the default max_map_count value while linking firefox. Building musl with the oldalloc allocator works, but is very slow. Instead, I am LD_PRELOADing mimalloc here to work around this issue.

There is still an issue with dynamic linker path when running the resulting firefox. it is necessary to put /path-of-unwrapped-firefox/lib/firefox in the LD_LIBRARY_PATH manually, otherwise it will not start. i don't know why exactly. Fixed

Thunderbird builds and works as well.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.11 Release Notes (or backporting 22.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

Grumbel and others added 4 commits April 28, 2022 15:40
@Grumbel
guile: fix non-existing libunistring include path
2bd3cb6 
The include/ directory for libunistring is inside the libunistring.dev
package, not libunistring, thus the .pc file was pointing to a
non-existing path.
@SuperSandro2000
git: add git-jump to PATH
0ca0615
@r-ryantm
enchant: 2.3.2 -> 2.3.3
a5361b3
@r-ryantm
libaio: 0.3.112 -> 0.3.113
f2fffab
@github-actions github-actions bot added the 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. label Sep 13, 2022
@ofborg ofborg bot added 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. labels Sep 13, 2022
@Et7f3
cc-wrapper: remove duplicate C{C,XX}${role_post}
d937f6f 
This might be typo but I don't know which other variable was intended to be named. This can also come from a merge conflict.
@S-NA
Copy link
Contributor

S-NA commented Sep 14, 2022

There is still an issue with dynamic linker path when running the resulting firefox. it is necessary to put /path-of-unwrapped-firefox/lib/firefox in the LD_LIBRARY_PATH manually, otherwise it will not start. i don't know why exactly.

That seems oddly familiar to something I encountered awhile back when adding LTO support. It ended up that Mozilla had a Python script that tripped up on whitespace differences from GNU readelf and LLVM readelf which in turn caused /path-of-unwrapped-firefox/lib/firefox/dependentlibs.list to fail to be properly generated. I contributed a fix upstream awhile back for it so it is not the issue here but I suggest looking at the contents of dependentlibs.list and seeing if it has a list of shared objects. If it is missing items then something in the pkgsMusl environment is breaking an assumption of dependentlibs.py. Otherwise Firefox is failing to load dependentlibs.list.

@yu-re-ka yu-re-ka mentioned this pull request Sep 15, 2022
Merged
13 tasks
@yu-re-ka
Copy link
Contributor Author

yu-re-ka commented Sep 15, 2022

The dependentlibs.list was generated correctly. It is the same between the glibc and musl build. An strace reveals very different behavior:

musl

[yuka@yaya:~/musl-nixos/firefox-104.0.2]$ rg mozsandbox log
81:read(3, "libmozsandbox.so\nliblgpllibs.so\n"..., 1024) = 89
82:open("/home/yuka/musl-nixos/outputs/out/lib/firefox/libmozsandbox.so", O_RDONLY|O_LARGEFILE) = 4
87:open("/home/yuka/musl-nixos/outputs/out/lib/firefox/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = 4
1281:open("/nix/store/x5x7f006wbs09ww83pki17wli7qljaj2-icu4c-71.1/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1282:open("/nix/store/67i5mqcag7r35ydgh6g4yij0vg2z6xrd-libffi-3.4.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1283:open("/nix/store/0qyvydcbx9hfhmrzlfk7j7sfvyg2mwbv-nspr-4.34.1/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1284:open("/nix/store/vn1l1a1yxbwmp8nawc03p20vldla2r3f-zlib-1.2.12/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1285:open("/nix/store/7nirw85kh6zkjxlijjhwf2m9cd6iviiw-nss-3.82/lib64/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1286:open("/nix/store/sksgkzmxnj1q7ym2yzhhfr3f1rqyjji2-freetype-2.12.1/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1287:open("/nix/store/7ywq4vsqgl99p9vg06akay3yg1ri3hl0-fontconfig-2.14.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1288:open("/nix/store/kzqrav33fgal77hv1rryby0k2pj0mph7-glib-2.72.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1289:open("/nix/store/zzhyvzgmxhhhx13gcspwwb2ffir6gmyi-gtk+3-3.24.34/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1290:open("/nix/store/k5bfc8lvj8ks9bil5m1hcpd4pjqfd4rs-atk-2.38.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1291:open("/nix/store/m1l9cwkxvpabklf29x021k7rxc3jqrfx-cairo-1.16.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1292:open("/nix/store/mi47j9c9ixhjbzn1zq86y9l0lg4z39xh-gdk-pixbuf-2.42.8/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1293:open("/nix/store/b3c2ikzmzcj76ixds4i55xpzwc0wx5rw-pango-1.50.8/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1294:open("/nix/store/nbf5i7m3l8n56j0c5h6pciygz5xcxgin-harfbuzz-5.1.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1295:open("/nix/store/kcmiv0h368pgz3fic5p311hj97s1vy1c-libpng-apng-1.6.37/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1296:open("/nix/store/dgk6wjwsq004n8rmkmaydpyz4ly8azmp-libwebp-1.2.4/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1297:open("/nix/store/sd70as1lqg1nvysing79lcp89fi02bii-libevent-2.1.12/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1298:open("/nix/store/5nk3wcm7q3zdqp9z7wfk108i15yhzcaa-libvpx-1.12.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1299:open("/nix/store/qlvc1ympanf4qalp2pwqfjs8wpx5nczh-pixman-0.38.4/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1300:open("/nix/store/z055cqq1sxl6bmzxhkg19lj0dwvlmpj3-dbus-1.14.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1301:open("/nix/store/l8h95clnh22vvpjn9wb147jcq2w2q2yg-dbus-glib-0.112/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1302:open("/nix/store/2044lgh7qxpjkabb0i97y5clly5b3lpw-libXext-1.3.4/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1303:open("/nix/store/87k78lavc9ndkihjvvm8wrhqh9wl4jcs-libX11-1.7.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1304:open("/nix/store/yqx8yjbna5c9ndfq4bgkwd1w1qx77sp8-libxcb-1.14/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1305:open("/nix/store/8aspn3pkbj1a49hmqx81nkqxr2hq8sqr-libXcomposite-0.4.5/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1306:open("/nix/store/jxn57fm0jyf9qqnr2i7m1ckaw25ysgwh-libXfixes-6.0.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1307:open("/nix/store/bmmldybsss3d83w146lxib7sy6q1lyb0-libXcursor-1.2.0/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1308:open("/nix/store/k1x2rx52nbl9a67yrf7j9kk71gr5avbw-libXi-1.8/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1309:open("/nix/store/63wqbdk9zsb5gxpd05wgss8hcqrgrj87-libXrandr-1.5.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1310:open("/nix/store/lw0gsp8p4ws7mrhm4fxvybkbkxfjihns-libXdamage-1.1.5/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1311:open("/nix/store/via7achzpkbxh4pl4q0jmapp49r04bg4-libXtst-1.2.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1312:open("/nix/store/347fqzxbcm48ajcwhc80qyrppw2qldb4-gcc-11.3.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1313:open("/nix/store/lbyphf5vcnsjv9n1kx2h4rvws3hz0m6g-libXrender-0.9.10/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1314:open("/nix/store/fjmgvqwwj52bvk4nb64j3diyb5fwm9bl-alsa-lib-1.2.7.2/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1315:open("/nix/store/40fiaz98s2dzgi0pp87zkvx8hhv2i6c2-musl-1.2.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1316:open("/nix/store/347fqzxbcm48ajcwhc80qyrppw2qldb4-gcc-11.3.0-lib/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1317:open("/nix/store/40fiaz98s2dzgi0pp87zkvx8hhv2i6c2-musl-1.2.3/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1319:open("/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1320:open("/usr/local/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1321:open("/usr/lib/libmozsandbox.so", O_RDONLY|O_LARGEFILE|O_CLOEXEC) = -1 ENOENT (No such file or directory)
1326:Error loading shared library libmozsandbox.so: No such file or directory (needed by /home/yuka/musl-nixos/outputs/out/lib/firefox/libxul.so)) = 142

[yuka@yaya:~/musl-nixos/firefox-104.0.2]$

glibc

[yuka@mari:~]$ rg mozsandbox log
148:read(3, "libmozsandbox.so\nliblgpllibs.so\n"..., 4096) = 89
149:openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/libmozsandbox.so", O_RDONLY) = 4
154:openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = 4
27593:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/glibc-hwcaps/x86-64-v3/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27595:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/glibc-hwcaps/x86-64-v2/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27597:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/haswell/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27599:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/haswell/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27601:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27603:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/tls/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27605:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/haswell/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27607:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/haswell/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27609:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/x86_64/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
27611:[pid 17397] openat(AT_FDCWD, "/nix/store/8m4krn0lj5mi4hdajdrcwf6lflrs5258-firefox-unwrapped-104.0.1/lib/firefox/libmozsandbox.so", O_RDONLY|O_CLOEXEC) = 11
27932:[pid 17397] read(11, "libmozsandbox.so\nliblgpllibs.so\n"..., 4096) = 89

[yuka@mari:~]$

Note that glibc is only trying to open the libmozsandbox.so a second time in a child process

@github-actions github-actions bot added the 6.topic: stdenv Standard environment label Sep 21, 2022
@ofborg ofborg bot added the 8.has: package (new) This PR adds a new package label Sep 21, 2022
@github-actions github-actions bot removed 6.topic: stdenv Standard environment 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. labels Sep 23, 2022
@ofborg ofborg bot added 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. and removed 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. labels Sep 23, 2022
@github-actions github-actions bot added 6.topic: kernel The Linux kernel 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: printing Drivers, CUPS & Co. 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: qt/kde Object-oriented framework for GUI creation 6.topic: ruby A dynamic, open source programming language with a focus on simplicity and productivity. 6.topic: systemd Software suite that provides an array of system components for Linux operating systems. 6.topic: TeX Issues regarding texlive and TeX in general 8.has: module (update) This PR changes an existing module in `nixos/` labels Oct 31, 2022
@yu-re-ka
Copy link
Contributor Author

yu-re-ka commented Oct 31, 2022

I fucked up the rebase, opening a new PR to not create any more notifications

@yu-re-ka yu-re-ka closed this Oct 31, 2022
@ofborg ofborg bot added 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-linux-stdenv This PR causes stdenv to rebuild on Linux and must target a staging branch. 8.has: clean-up This PR removes packages or removes other cruft and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Oct 31, 2022
@ghost
Copy link

ghost commented Nov 1, 2022

I fucked up the rebase, opening a new PR to not create any more notifications

It's not your fault; github's UI for this is atrocious.

Hint: before changing the base, mark your PR as a draft. That way if this happens it won't ping half of humanity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

6.topic: kernel The Linux kernel 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: printing Drivers, CUPS & Co. 6.topic: python Python is a high-level, general-purpose programming language. 6.topic: qt/kde Object-oriented framework for GUI creation 6.topic: ruby A dynamic, open source programming language with a focus on simplicity and productivity. 6.topic: rust General-purpose programming language emphasizing performance, type safety, and concurrency. 6.topic: systemd Software suite that provides an array of system components for Linux operating systems. 6.topic: TeX Issues regarding texlive and TeX in general 8.has: clean-up This PR removes packages or removes other cruft 8.has: module (update) This PR changes an existing module in `nixos/` 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-darwin: 5001+ This PR causes many rebuilds on Darwin and must target the staging branches. 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. 10.rebuild-linux: 5001+ This PR causes many rebuilds on Linux and must target the staging branches. 10.rebuild-linux-stdenv This PR causes stdenv to rebuild on Linux and must target a staging branch.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.