Skip to content

de-vendorify kernel config (build full NixOS kernel)#341

Merged
yuyuyureka merged 3 commits into
nix-community:mainfrom
yuyuyureka:minimize-patches
Sep 29, 2025
Merged

de-vendorify kernel config (build full NixOS kernel)#341
yuyuyureka merged 3 commits into
nix-community:mainfrom
yuyuyureka:minimize-patches

Conversation

@yuyuyureka
Copy link
Copy Markdown
Collaborator

@yuyuyureka yuyuyureka commented Aug 23, 2025

I've been running this (NixOS-derived) kernel config for many months.

The downside is it takes a few times longer to build, since it compiles all possible modules. But this should no longer be a problem now that we have a CI and binary cache.

@dramforever
Copy link
Copy Markdown
Contributor

dramforever commented Aug 23, 2025

If we drop that all the way down to these (which are necessary to get DRM_ASAHI=m it seems?)

    # For DRM_ASAHI
    DRM = yes;
    ARM64_16K_PAGES = yes;

the difference with yours is (simple -> yours)

CONFIG_HID_APPLE                y -> m
CONFIG_APPLE_WATCHDOG           m -> y
CONFIG_DRM_VGEM                 m -> n
CONFIG_APPLE_M1_CPU_PMU         n -> y

My take on these, from the perspective of whether the "simple" config needs changing:

  • HID_APPLE=y is fine.
  • I don't know which of y or m for APPLE_WATCHDOG is better. I think what happens if we get to Linux and it doesn't initialize apple_wdt in time, the machine forcibly reboot, and after three strikes it goes to recovery. So if we do m here, I think we should also add it to boot.initrd.availableKernelModules so you have like more than a minute or something to do initrd debugging.
  • DRM_VGEM=m should be fine. It helps with software renderers that need it like llvmpipe, and doesn't hurt otherwise.
  • APPLE_M1_CPU_PMU=n is annoying. it can't be m because it uses non-EXPORT_SYMBOL symbols. I personally would have it at y as long as we build our own kernel. In the far future where upstream kernel works on M1 I think we wouldn't need n-a-s by that point.

@yuyuyureka
Copy link
Copy Markdown
Collaborator Author

@dramforever what is this "simple" config you are talking about, where can I find it?

Can you perhaps make a specific suggestion using the diff feature in github?

@yuyuyureka
Copy link
Copy Markdown
Collaborator Author

yuyuyureka commented Aug 23, 2025

Ah, you dropped all options from the extraStructuredConfig except those two and then diffed the generated config, meaning all other statements are no-ops?

I would think we also need ARCH_APPLE = yes;?

I also don't understand how HID_APPLE can be "y" when not specified, since it can be built as a module the generate-config.pl would always opt for that.

@dramforever
Copy link
Copy Markdown
Contributor

HID_APPLE is default !EXPERT || SPI_HID_APPLE, which is default y, which the config script thing honors instead of making that m

@dramforever
Copy link
Copy Markdown
Contributor

Oh, missed this: I actually don't know why ARCH_APPLE defaults to y, but it does, and in fact does for all ARCH_*

@yuyuyureka yuyuyureka force-pushed the minimize-patches branch 2 times, most recently from fffc941 to 33935ed Compare August 23, 2025 17:43
@yuyuyureka
Copy link
Copy Markdown
Collaborator Author

Removed redundant config (and some more; I think DRM_SCHED-related things are also differing, but it seems to work fine with the simple version on my machine).

Also attempted integrating it into the CI.

@yuyuyureka yuyuyureka mentioned this pull request Aug 23, 2025
@tpwrules
Copy link
Copy Markdown
Collaborator

Looks like the CI build is working, thanks for the experiment. I will test it more thoroughly later. We should definitely not be building for x86-64 though, although I guess that's neat that it worked.

@yuyuyureka yuyuyureka force-pushed the minimize-patches branch 3 times, most recently from 1d34925 to d157365 Compare August 23, 2025 21:03
@yuyuyureka
Copy link
Copy Markdown
Collaborator Author

From what I can tell we're not building for x86_64, but just on x86_64, which is good since it will cache the derivation for cross-compilation users.

@waltmck
Copy link
Copy Markdown
Contributor

waltmck commented Aug 25, 2025

I just built this with the latest nixpkgs and have a few issues:

  1. There is no GPU acceleration (Vulkan nor OpenGL). I saw that there was a rust-related issue and the commit fixing it is not in this branch, so maybe this is to be expected.
  2. options hid_apple swap_fn_leftctrl=1 no longer works to swap the control and function keys.

@dramforever
Copy link
Copy Markdown
Contributor

I just built this with the latest nixpkgs and have a few issues:

  1. There is no GPU acceleration (Vulkan nor OpenGL). I saw that there was a rust-related issue and the commit fixing it is not in this branch, so maybe this is to be expected.

Yes, this is a known issue worked around in #343 and should be fixed for properly in NixOS/nixpkgs#436245

  1. options hid_apple swap_fn_leftctrl=1 no longer works to swap the control and function keys.

This is expected for HID_APPLE=y. You'd need to move/add that to boot.kernelParams like:

boot.kernelParams = [ "hid_apple.swap_fn_leftctrl=1" ];

@yuyuyureka
Copy link
Copy Markdown
Collaborator Author

Ohh I guess that's a reason to keep HID_APPLE=m...

I just rebased, so the Rust workaround should be included now

@yuyuyureka yuyuyureka force-pushed the minimize-patches branch 2 times, most recently from 94b36d1 to 2f642f7 Compare August 25, 2025 08:28
@waltmck
Copy link
Copy Markdown
Contributor

waltmck commented Aug 25, 2025

We talked about this in the matrix chat, but I will summarize my results testing the latest version of this PR here for transparency:

  • The built-in speakers are very quiet compared to the existing kernel. This same problem happened with @zzywysm's custom kernel, and the fix was to tweak some config options. I do not have insight into why those changes were necessary, and I have not yet tested whether they fix the issue on 6.16 as well.

  • IWD 802.1x support is broken---when I try to connect to such a network, I get Error loading client private key. The fix is to add

    boot.kernelModules = ["pkcs8_key_parser"];

    This may be an issue with the IWD module, but I do not have any non-ARM wifi-capable hardware to test this on. I will file an upstream bug when I have the time.

  • The kernel no longer builds with -march=armv8.6a, which is a shame although it probably isn't supported. The error message is

    /build/cc2qPCGH.s: Assembler messages:
    /build/cc2qPCGH.s:23760: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:24801: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:25196: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:25487: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:26704: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:27997: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:28784: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:29808: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:29923: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:30010: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:30034: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:30232: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:30423: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:30510: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:30721: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:31037: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:32079: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:32271: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:32528: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:32768: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:33037: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:33298: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:33508: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:33710: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:33958: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:34176: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:34630: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:34872: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:35147: Error: selected processor does not support `retaa'
    /build/cc2qPCGH.s:35415: Error: selected processor does not support `retaa'
    make[8]: *** [../scripts/Makefile.build:287: drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.o] Error 1
    

@waltmck
Copy link
Copy Markdown
Contributor

waltmck commented Aug 31, 2025

i fixed the audio issue by building some things as modules, see my branch.

@yuyuyureka
Copy link
Copy Markdown
Collaborator Author

I think what's still missing here is a somewhat trustworthy binary cache & CI. At the moment, I am not convinced I would want to build even the installer images in the nix-community CI which builds derivations from untrusted PRs.

@yuyuyureka
Copy link
Copy Markdown
Collaborator Author

i fixed the audio issue by building some things as modules, see my branch.

Thanks Walter! I squashed your commits together and added them to my branch.

@yuyuyureka yuyuyureka force-pushed the minimize-patches branch 2 times, most recently from 6d67867 to a5ee443 Compare September 7, 2025 09:10
@ghost
Copy link
Copy Markdown

ghost commented Sep 19, 2025

Any update on this? I would really like to see this merged.

@waltmck
Copy link
Copy Markdown
Contributor

waltmck commented Sep 19, 2025

This kernel takes 2-3 hours to compile on my M2 air, compared to around 20 minutes for the minimal one, so I think that a community cache is necessary before it can be merged. Otherwise, the kernel works fine (I have been using it).

@ghost
Copy link
Copy Markdown

ghost commented Sep 19, 2025

For me it was less than 2 hours (1 hour 49 minutes) on my MBA m1 2020. There's probably a lot of factors that affect how long the build process takes.

A step towards a binary cache would be to consolidate n-a-s to what we only need: See #349 and #333 and #329

@waltmck
Copy link
Copy Markdown
Contributor

waltmck commented Sep 19, 2025

For me it was less than 2 hours (1 hour 49 minutes) on my MBA m1 2020. There's probably a lot of factors that affect how long the build process takes.

A step towards a binary cache would be to consolidate n-a-s to what we only need: See #349 and #333 and #329

Upstreaming m1n1 and other components to nixpkgs would be nice and is a worthy effort, but they barely add any compile time so I don't think that they should be considered a blocker for moving the vender-specific kernel build to a binary cache. Maybe someone involved in the nix-community infra could tell us if there is anything blocking that.

@yuyuyureka
Copy link
Copy Markdown
Collaborator Author

The last status here was that we (the nixos-apple-silicon maintainers, but mostly me) were not ok with the idea of encouraging people to use the kernels and install images built on the same CI which also builds arbitrary untrusted derivations (PRs on other nix-community projects).
So either the security concept of the nix-community CI & binary cache needs to be clarified and possibly adjusted to separate untrusted and trusted builds on different builders & signing keys, or we build the kernels and installer images using GitHub Actions and push them to our own cache (hosting for that would need to be figured out).

@astratagem
Copy link
Copy Markdown

I've just checked this out in my personal fork and I'm happy to report that it completely resolves whatever was causing #294! So I would hope that once this is merged, there might be fewer reports of issues with hardware-accelerated Wayland compositors like Niri and Hyprland... what exactly the "fix" was, I have no idea.

@ghost
Copy link
Copy Markdown

ghost commented Sep 22, 2025

Some nits, remove unused arguments and also this change means we aren't really tracking the asahi-wip tree or basing off of fedora releases.

diff --git a/apple-silicon-support/packages/linux-asahi/default.nix b/apple-silicon-support/packages/linux-asahi/default.nix
index 397edf8..a0a5552 100644
--- a/apple-silicon-support/packages/linux-asahi/default.nix
+++ b/apple-silicon-support/packages/linux-asahi/default.nix
@@ -11,10 +11,9 @@ let
       stdenv,
       lib,
       fetchFromGitHub,
-      fetchpatch,
       buildLinux,
       ...
-    }@args:
+    }:
     buildLinux rec {
       inherit stdenv lib;
 
@@ -23,7 +22,6 @@ let
       extraMeta.branch = "6.16";
 
       src = fetchFromGitHub {
-        # tracking: https://github.com/AsahiLinux/linux/tree/asahi-wip (w/ fedora verification)
         owner = "AsahiLinux";
         repo = "linux";
         tag = "asahi-6.16.5-2";
@@ -65,11 +63,6 @@ let
       ++ _kernelPatches;
     };
 
-  linux-asahi = (callPackage linux-asahi-pkg { }).overrideAttrs (_: {
-    # FIXME: Remove when https://github.com/NixOS/nixpkgs/pull/436245 lands
-    preConfigure = ''
-      export RUST_LIB_SRC KRUSTFLAGS
-    '';
-  });
+  linux-asahi = (callPackage linux-asahi-pkg { });
 in
 lib.recurseIntoAttrs (linuxPackagesFor linux-asahi)

The last change requires a flake update since it was merged on Aug 23rd and the nixpkgs in n-a-s is from Aug 19th.

@yuyuyureka yuyuyureka force-pushed the minimize-patches branch 3 times, most recently from 02faa63 to 358f408 Compare September 26, 2025 14:31
@yuyuyureka yuyuyureka marked this pull request as ready for review September 26, 2025 14:31
Comment thread apple-silicon-support/packages/linux-asahi/default.nix
yuyuyureka and others added 3 commits September 28, 2025 11:25
Credit to zzywysm for this solution. According to James Calligeros
(chadmed), this is most likely necessary because of some race condition.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants