Skip to content

tests: run more of them in ci and hydra #428706

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
jopejoe1 wants to merge 3 commits into
base: master
Choose a base branch
from
Draft

tests: run more of them in ci and hydra #428706

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c0071be
tests.cross: eval more tests
jopejoe1 Jul 26, 2025
81be307
tests.cross.sanity: turn into attrset
jopejoe1 Jul 26, 2025
4a79d7e
tests.cross.sanity: disable eval failing tests
jopejoe1 Aug 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
76 changes: 38 additions & 38 deletions pkgs/test/cross/default.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ let

mapMultiPlatformTest =
crossSystemFun: test:
lib.dontRecurseIntoAttrs (
lib.recurseIntoAttrs (
lib.mapAttrs (
name: system:
lib.recurseIntoAttrs (test rec {
Expand Down Expand Up @@ -176,43 +176,43 @@ let
# so we can ask @ofborg to check it, yet should have good examples
# of things that often break. So, no buckshot `mapTestOnCross`
# calls here.
sanity = [
mbuffer
#pkgs.pkgsCross.gnu64.bash # https://github.com/NixOS/nixpkgs/issues/243164
pkgs.gcc_multi.cc
pkgs.pkgsMusl.stdenv
pkgs.pkgsLLVM.stdenv
pkgs.pkgsStatic.bash
#pkgs.pkgsCross.gnu64_simplekernel.bash # https://github.com/NixOS/nixpkgs/issues/264989
pkgs.pkgsCross.arm-embedded.stdenv
pkgs.pkgsCross.sheevaplug.stdenv # for armv5tel
pkgs.pkgsCross.raspberryPi.stdenv # for armv6l
pkgs.pkgsCross.armv7l-hf-multiplatform.stdenv
pkgs.pkgsCross.m68k.stdenv
pkgs.pkgsCross.aarch64-multiplatform.pkgsBuildTarget.gcc
pkgs.pkgsCross.powernv.pkgsBuildTarget.gcc
pkgs.pkgsCross.s390.stdenv
pkgs.pkgsCross.mips64el-linux-gnuabi64.stdenv
pkgs.pkgsCross.mips64el-linux-gnuabin32.stdenv
pkgs.pkgsCross.mingwW64.stdenv
# Uses the expression that is used by the most cross-compil_ed_ GHCs
pkgs.pkgsCross.riscv64.haskell.compiler.native-bignum.ghc948

]
++ lib.optionals (with pkgs.stdenv.buildPlatform; isx86_64 && isLinux) [
# Musl-to-glibc cross on the same architecture tends to turn up
# lots of interesting corner cases. Only expected to work for
# x86_64-linux buildPlatform.
pkgs.pkgsMusl.pkgsCross.gnu64.hello

# Two web browsers -- exercises almost the entire packageset
pkgs.pkgsCross.aarch64-multiplatform.qutebrowser-qt5
pkgs.pkgsCross.aarch64-multiplatform.firefox

# Uses pkgsCross.riscv64-embedded; see https://github.com/NixOS/nixpkgs/issues/267859
pkgs.spike
];

sanity = lib.recurseIntoAttrs (
Copy link
Contributor

@wolfgangwalther wolfgangwalther Aug 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can actually do this recursion, because:

  • This takes too many resources to Eval, and
  • some of these use "variants" (pkgsLLVM, etc.), which are disabled for CI. The long-term goal is to not allow any variants used within nixpkgs.

Copy link
Member Author

@jopejoe1 jopejoe1 Aug 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some of these use "variants" (pkgsLLVM, etc.), which are disabled for CI. The long-term goal is to not allow any variants used within nixpkgs.

If that is the case, I'm not sure that we should have those tests in their current state at all.

Copy link
Contributor

@wolfgangwalther wolfgangwalther Aug 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was introduced in #243321.

Using them via ofborg still potentially makes sense, right? I'm not sure whether that's possible without recurseIntoAttrs, though. Now I wonder how recurseForDerivations behaves exactly - what happens when I do nix-build -A tests.cross.sanity and the sanity attrset does not have recurseForDerivations set? Would it still go through that attrset, because I specified it explicitly or would this already require the recurse attribute?

{
inherit mbuffer;
# gnu64-bash = pkgs.pkgsCross.gnu64.bash; # https://github.com/NixOS/nixpkgs/issues/243164
gcc_multi-cc = pkgs.gcc_multi.cc;
# musl-stdenv = pkgs.pkgsMusl.stdenv;
# llvm-stdenv = pkgs.pkgsLLVM.stdenv;
static-bash = pkgs.pkgsStatic.bash;
# gnu64_simplekernel-bash = pkgs.pkgsCross.gnu64_simplekernel.bash; # https://github.com/NixOS/nixpkgs/issues/264989
arm-embedded-stdenv = pkgs.pkgsCross.arm-embedded.stdenv;
sheevaplug-stdenv = pkgs.pkgsCross.sheevaplug.stdenv; # for armv5tel
raspberrypi-stdenv = pkgs.pkgsCross.raspberryPi.stdenv; # for armv6l
armv7l-hf-multiplatform-stdenv = pkgs.pkgsCross.armv7l-hf-multiplatform.stdenv;
m68k-stdenv = pkgs.pkgsCross.m68k.stdenv;
aarch64-multiplatform-build-target-gcc = pkgs.pkgsCross.aarch64-multiplatform.pkgsBuildTarget.gcc;
powernv-build-target-gcc = pkgs.pkgsCross.powernv.pkgsBuildTarget.gcc;
s390-stdenv = pkgs.pkgsCross.s390.stdenv;
mips64el-linux-gnuabi64-stdenv = pkgs.pkgsCross.mips64el-linux-gnuabi64.stdenv;
mips64el-linux-gnuabin32-stdenv = pkgs.pkgsCross.mips64el-linux-gnuabin32.stdenv;
mingwW64-stdenv = pkgs.pkgsCross.mingwW64.stdenv;
# Uses the expression that is used by the most cross-compil_ed_ GHCs
riscv64-ghc = pkgs.pkgsCross.riscv64.haskell.compiler.native-bignum.ghc948;
}
// lib.optionalAttrs (with pkgs.stdenv.buildPlatform; isx86_64 && isLinux) {
# Musl-to-glibc cross on the same architecture tends to turn up
# lots of interesting corner cases. Only expected to work for
# x86_64-linux buildPlatform.
# musl-gnu64-hello = pkgs.pkgsMusl.pkgsCross.gnu64.hello;

# Two web browsers -- exercises almost the entire packageset
aarch64-multiplatform-qutebrowser-qt5 = pkgs.pkgsCross.aarch64-multiplatform.qutebrowser-qt5;
aarch64-multiplatform-firefox = pkgs.pkgsCross.aarch64-multiplatform.firefox;

# Uses pkgsCross.riscv64-embedded; see https://github.com/NixOS/nixpkgs/issues/267859
inherit (pkgs) spike;
}
);
in
{
gcc = lib.recurseIntoAttrs (
Expand Down
Loading