nixos/binfmt: Add support for using statically-linked QEMU (continuation of #160802) by bltavares · Pull Request #300070 · NixOS/nixpkgs

bltavares · 2024-03-29T20:04:20Z

(I'm not familiar with the etiquette on updating the PR of others, and I did not find the right git commands to keep #160802 authorship during the merge. I'd like to thank @zhaofengli for the original PR and I'll investigate how to keep the authorship intact meanwhile)

Description of changes

Built on top of #160802, this commit
addresses necessary updates to bring it up to parity with nixpkgs-unstable:

Introduce pipewireSupport: false as a new override option
Remove 8.1.1 patch as qemu is 8.2.2 in nixpkgs-unstable
Introduce new patch to expose libaio static due to upstream meson.build changes

In order to compile, the perl dependency must also be fixed
(#299623) on the pkgsStatic environment.
With this additional changeset, nix-shell -p qemu-user-static compiles.

Tested on:

x86_64-linux
aarch64-linux

Depends on:

(pkgsStatic.perl): Patch Perl to avoid duplicate symbols with musl #299623

Things done

Add a 👍 reaction to pull requests you find important.

The plugin .so's do not build statically.

Not needed for a minimal qemu-user-static build.

This is synonymous with the "qemu-user-static" packages available in other distros.

The fixBinary flag will be enabled if a static emulator is in use.

Built on top of NixOS#160802, this commit addresses necessary updates to bring it up to parity with `nixpkgs-unstable`: - Introduce `pipewireSupport: false` as a new override option - Remove 8.1.1 patch as qemu is 8.2.2 in nixpkgs-unstable - Introduce new patch to expose libaio static due to upstream meson.build changes In order to compile, the `perl` dependency must also be fixed (NixOS#299623) on the `pkgsStatic` environment. With this additional changeset, `nix-shell -p qemu-user-static` compiles. **Tested on**: - `x86_64-linux` - `aarch64-linux` **Depends on:** - [ ] NixOS#299623

Ericson2314 · 2024-04-01T18:58:04Z

I don't like how lib needs to be adjusted for this. That doesn't feel right to me. Why can't we use the regular emulator function on pkgsStatic?

bltavares · 2024-04-01T19:57:56Z

@Ericson2314 I'm not familiar with the original intention on #160802 but a similar question was asked in the PR https://github.com/NixOS/nixpkgs/pull/160802/files#r1208073679

jcaesar · 2024-04-18T14:04:22Z

In the interest of this not being left lying around for 2 years again, would it be possible to get the changes to pkgs/ merged first and then get the options working? (If asked, I'll submit that as a PR, I'm just not sure having a third PR on the same topic would be nice.)

PS: I'm testing this right now, and I'm not actually sure its working right.

{lib, pkgs, ...}: {
  environment.etc."binfmt.d/nix-hack-qemu-user-statc.conf".text = let
    pr = pkgs.fetchFromGitHub {
      owner = "NixOS";
      repo = "nixpkgs";
      rev = "pull/300070/head"; #"d01bb6a1f7b820437406b4b341f77537c04bdc50";
      hash = "sha256-7uBcm17HVjPW5JBmEnyg+yVb1qDkiXHKfeLjR7wfyek=";
    };
    patched = import pr {system = "x86_64-linux";};
    # Workaround for https://github.com/NixOS/nixpkgs/issues/295608
    qus = patched.qemu-user-static.override {
      pkgsStatic =
        patched.pkgsStatic
        // {
          qemu = patched.pkgsStatic.qemu.override {
            texinfo = patched.pkgsStatic.texinfo.override {
              perl = pkgs.perl;
            };
            hostCpuTargets = ["aarch64-linux-user"];
          };
        };
    };
  in
    lib.concatStringsSep ":" [
      ""
      "aarch64-linux"
      "M"
      ""
      "\\x7fELF\\x02\\x01\\x01\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x02\\x00\\xb7\\x00"
      "\\xff\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xff\\xff\\xff\\xff\\xff\\x00\\xff\\xfe\\xff\\xff\\xff"
      "${qus}/bin/qemu-aarch64"
      "FOCP"
    ];
}

and running

docker run --rm -ti --platform linux/arm64 ubuntu:jammy bash -c 'apt-get update && apt-get -y install dnsutils'

yields me

…
Setting up dnsutils (1:9.18.18-0ubuntu0.22.04.2) ...
Processing triggers for libc-bin (2.35-0ubuntu3.6) ...
Segmentation fault (core dumped)
Segmentation fault (core dumped)
dpkg: error processing package libc-bin (--configure):
 installed libc-bin package post-installation script subprocess returned error exit status 139
Errors were encountered while processing:
 libc-bin
E: Sub-process /usr/bin/dpkg returned an error code (1)

Ericson2314 · 2024-04-18T14:28:16Z

The pkgs part can land first, but also needs to be reworked. We should not refer to pkgsStatic in pkgs like this.

bltavares · 2024-04-18T14:50:08Z

I'm not familiar enough with conventions to understand how staticPkgs should be used. Is there a reference where I can learn more about these conventions?

nixos-discourse · 2024-04-19T03:06:50Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/docker-ignoring-platform-when-run-in-nixos/21120/17

Atemu · 2024-04-19T03:16:29Z

nixos/modules/system/boot/binfmt.nix

@@ -30,6 +30,7 @@ let

  getEmulator = system: (lib.systems.elaborate { inherit system; }).emulator pkgs;


What @Ericson2314 probably means is to simply optionally do this:

Suggested change

getEmulator = system: (lib.systems.elaborate { inherit system; }).emulator pkgs;

getEmulator = system: (lib.systems.elaborate { inherit system; }).emulator pkgsStatic;

SuperSandro2000 · 2024-05-19T16:16:10Z

nixos/modules/system/boot/binfmt.nix

+      in [ ]
+        ++ lib.optional hasNonFixedRule "/run/binfmt"


Suggested change

in [ ]

++ lib.optional hasNonFixedRule "/run/binfmt"

in lib.optional hasNonFixedRule "/run/binfmt"

jcaesar · 2024-05-28T12:54:51Z

would it be possible to get the changes to pkgs/ merged first

I've gone ahead and opened the PR for that. If that does get merged, you can already use the static binfmt interpreter with

boot.binfmt.registrations = lib.genAttrs ["aarch64-linux" "armv7l-linux" "riscv64-linux"] (sys: {
  interpreter = "${pkgs.pkgsStatic.qemu-user}/bin/qemu-${(lib.systems.elaborate sys).qemuArch}";
  wrapInterpreterInShell = false;
  preserveArgvZero = true;
  matchCredentials = true;
  fixBinary = true;
});

so figuring out the module setup is less pressing. Though at least a test that ensures that pkgsStatic.qemu-user stays compiling/working would necessary, right?

jcaesar · 2024-05-28T12:58:02Z

pkgs/applications/virtualization/qemu/user-static.nix

+    guestAgentSupport = false;
+    numaSupport = false;
+    seccompSupport = false;
+    alsaSupport = false;
+    pulseSupport = false;
+    sdlSupport = false;
+    jackSupport = false;
+    pipewireSupport = false;
+    gtkSupport = false;
+    vncSupport = false;
+    smartcardSupport = false;
+    spiceSupport = false;
+    ncursesSupport = false;
+    libiscsiSupport = false;
+    smbdSupport = false;
+    tpmSupport = false;
+    uringSupport = false;
+    capstoneSupport = false;
+    enableDocs = false;
+    enableTools = false;
+    enableBlobs = false;
+    hostCpuTargets = qemuTargets;


This is somewhat redundant with https://github.com/NixOS/nixpkgs/pull/300070/files#diff-2165823a8d82c5dd1353601bd290df8bd431f9ee2096750d9ef655cf5d251998L256-L274, right? I think it might be nice if only one of the two could be kept.

JeffLabonte

With comments resolved, I approve. I need this

SuperSandro2000

This is already looking really good and I think we can almost merge this

SuperSandro2000 · 2024-06-20T14:54:25Z

pkgs/applications/virtualization/qemu/default.nix

@@ -254,7 +254,9 @@ stdenv.mkDerivation (finalAttrs: {

  # Add a ‘qemu-kvm’ wrapper for compatibility/convenience.
  postInstall = lib.optionalString (!toolsOnly) ''


Could we make this depend on an option we have? I think that would be nicer, to not accidentally drop it in some update.

SuperSandro2000 · 2024-06-20T14:56:43Z

pkgs/applications/virtualization/qemu/aio-find-static-library.patch

Can you rebase this commit and move the hunks into the initial commits to get a cleaner git history?

jcaesar · 2024-06-22T03:05:00Z

@SuperSandro2000 I have a significantly cleaned up version of the changes to qemu in #314998, maybe that's easier to merge? (Should have mentioned it here sooner.)

But if you do go for this PR, note:

the find-aio patch isn't necessary (aio is required for disk emulation only(?))
the indirect dependency on static perl can be avoided (texinfo is required for docs only)
using callPackage as is in all-packages isn't a good idea (see comment)
what's pkg-info (see comment)?

(Alternatively, there's also the option of having qemu-user as a completely separate package, like so.)

jcaesar · 2024-08-26T05:24:12Z

The changes to pkgs.qemu / pkgsStatic.qemu-user have been merged. I'm now trying to get the changes to the modules in with #334859.

I don't like how lib needs to be adjusted for this. That doesn't feel right to me. Why can't we use the regular emulator function on pkgsStatic?

Not sure it's meaningful, but I can now answer that question: Not all targets can have emulators from pkgsStatic. I don't think wine will ever work, e.g. So you need to decide somewhere, per target, whether to use the emulator from pkgsStatic or not. Adjusting lib for that is most natural, I think. (Though I did clean up that area a little.)

jcaesar · 2024-10-06T08:59:55Z

#334859 has been merged and this functionality is now usable. @bltavares Would you mind if I close this PR?

Atemu · 2024-10-06T11:59:02Z

#334859 explicitly supersedes this PR.

github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 8.has: module (update) This PR changes an existing module in `nixos/` 6.topic: lib The Nixpkgs function library labels Mar 29, 2024

This was referenced Mar 29, 2024

Emulated cross-architecture build leaves a broken rust-std derivation on NixOS #297340

Open

nixos/binfmt: Add support for using statically-linked QEMU #160802

Closed

zhaofengli added 9 commits April 1, 2024 15:31

qemu: Skip doc-related dependencies when docs are disabled

00e2f8e

qemu: Support disabling TCG plugins

1368e90

The plugin .so's do not build statically.

qemu: Support disabling tools

1922e9d

Not needed for a minimal qemu-user-static build.

qemu: Support disabling blobs

21b7d48

Not needed for a minimal qemu-user-static build.

qemu: Only link qemu-kvm if it exists

3aad7c8

qemu: Fix static build

96054ca

qemu-user-static: init

eb53281

This is synonymous with the "qemu-user-static" packages available in other distros.

nixos/binfmt: Add option to use static emulators when available

2a913d6

The fixBinary flag will be enabled if a static emulator is in use.

nixos/tests/systemd-binfmt: Add chroot test

af44ef1

bltavares force-pushed the pkgs.qemu-user-static branch from 353b714 to 65da6e6 Compare April 1, 2024 18:36

bltavares force-pushed the pkgs.qemu-user-static branch from 65da6e6 to d01bb6a Compare April 1, 2024 18:36

bltavares marked this pull request as ready for review April 1, 2024 18:37

bltavares requested review from Ericson2314, alyssais and dasJ as code owners April 1, 2024 18:37

Atemu reviewed Apr 19, 2024

View reviewed changes

wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label May 3, 2024

SuperSandro2000 reviewed May 19, 2024

View reviewed changes

jcaesar mentioned this pull request May 27, 2024

qemu: add user space emulator only variant #314998

Merged

13 tasks

jcaesar reviewed May 28, 2024

View reviewed changes

JeffLabonte reviewed Jun 20, 2024

View reviewed changes

SuperSandro2000 changed the title ~~(pkgs.qemu-user-static) Update #160802 into nixpkgs-master~~ nixos/binfmt: Add support for using statically-linked QEMU (continuation of #160802) Jun 20, 2024

SuperSandro2000 reviewed Jun 20, 2024

View reviewed changes

jcaesar mentioned this pull request Aug 15, 2024

nixos/binfmt: Add option to use static emulators when available #334859

Merged

13 tasks

Atemu closed this Oct 6, 2024

		@@ -30,6 +30,7 @@ let

		getEmulator = system: (lib.systems.elaborate { inherit system; }).emulator pkgs;

	in [ ]
	++ lib.optional hasNonFixedRule "/run/binfmt"
	in lib.optional hasNonFixedRule "/run/binfmt"

		@@ -254,7 +254,9 @@ stdenv.mkDerivation (finalAttrs: {

		# Add a ‘qemu-kvm’ wrapper for compatibility/convenience.
		postInstall = lib.optionalString (!toolsOnly) ''

Uh oh!

Conversation

bltavares commented Mar 29, 2024

Description of changes

Things done

Uh oh!

Ericson2314 commented Apr 1, 2024

Uh oh!

bltavares commented Apr 1, 2024

Uh oh!

jcaesar commented Apr 18, 2024

Uh oh!

Ericson2314 commented Apr 18, 2024

Uh oh!

bltavares commented Apr 18, 2024

Uh oh!

nixos-discourse commented Apr 19, 2024

Uh oh!

Atemu Apr 19, 2024

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 May 19, 2024

Choose a reason for hiding this comment

Uh oh!

jcaesar commented May 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jcaesar May 28, 2024

Choose a reason for hiding this comment

Uh oh!

JeffLabonte left a comment

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 left a comment

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Jun 20, 2024

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Jun 20, 2024

Choose a reason for hiding this comment

Uh oh!

jcaesar commented Jun 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jcaesar commented Aug 26, 2024

Uh oh!

jcaesar commented Oct 6, 2024

Uh oh!

Atemu commented Oct 6, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

jcaesar commented May 28, 2024 •

edited

Loading

jcaesar commented Jun 22, 2024 •

edited

Loading