Skip to content

lib.mkPackageVariants: init; openssl: migrate to mkPackageVariants#496617

Draft
qweered wants to merge 5 commits intoNixOS:masterfrom
qweered:lib/mkPackageVariants
Draft

lib.mkPackageVariants: init; openssl: migrate to mkPackageVariants#496617
qweered wants to merge 5 commits intoNixOS:masterfrom
qweered:lib/mkPackageVariants

Conversation

@qweered
Copy link
Copy Markdown
Contributor

@qweered qweered commented Mar 4, 2026

Adds lib.mkPackageVariants to standardize multi-version package management in nixpkgs. Currently, packages like openssl, perl, etc. each reinvent their own variant handling — this provides a shared abstraction for it. Extensive tests added to lib/test/customisation.nix

Inspired by corepkgs' mkManyVariants, but adapted for nixpkgs:

corepkgs mkManyVariants nixpkgs mkPackageVariants
callPackage Uses internal callFromScope Caller-provided callPackage, pre-bound via splice.nix (same pattern as callPackage itself)
allowAliases Reads from config directly Explicit parameter, defaults to config.allowAliases in the splice binding
Return value defaultPackage.override defaultPackage directly (override is already available via callPackage/makeOverridable)
Feature variants Not supported Per-variant overrideArgs allows feature variants (e.g., oqs, legacy) that share version metadata but change build arguments
File detection Explicit paths required Auto-detects variants.nix and generic.nix in caller's directory via unsafeGetAttrPos

The generic builder follows a curried pattern — first receives variant-specific args (version, hash, patches, plus helpers like packageOlder/packageAtLeast), then package args resolved by callPackage. All variants are accessible via passthru (e.g., openssl.v1_1, openssl.v3_6, openssl.oqs), and pkg.override { ... } flows through to the generic builder without per-package boilerplate.

I migrated openssl as a proof-of-concept to verify everything works end-to-end, including by-name compatibility. The old monolithic default.nix is split into generic.nix (curried builder), variants.nix (version metadata), and a thin package.nix wrapper. openssl_oqs and openssl_legacy are also migrated from inline .override calls to proper variants. All backward-compat aliases are preserved.

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

@qweered qweered changed the title lib.mkPackageVariants: init, migrate openssl lib.mkPackageVariants: init, migrate openssl to mkPackageVariant Mar 4, 2026
@nixpkgs-ci nixpkgs-ci bot requested review from a team, Ericson2314, alyssais, hsjobeki, infinisil and thillux March 4, 2026 17:26
@nixpkgs-ci nixpkgs-ci bot added 2.status: merge conflict This PR has merge conflicts with the target branch 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 6.topic: lib The Nixpkgs function library labels Mar 4, 2026
@qweered qweered force-pushed the lib/mkPackageVariants branch 2 times, most recently from 96ec04d to 6653d83 Compare March 4, 2026 17:30
Copy link
Copy Markdown
Member

@RossComputerGuy RossComputerGuy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach so far seems reasonable, only have this nit so far.

@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 4, 2026
@qweered
Copy link
Copy Markdown
Contributor Author

qweered commented Mar 4, 2026

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 496617
Commit: 6653d83ae8968aaa3a4dd937d6944d7d73ec47cf


x86_64-linux

⏩ 2 packages blacklisted:
  • nixos-install-tools
  • tests.nixos-functions.nixos-test
✅ 4 packages built:
  • extra-container
  • nixos-container
  • nixpkgs-manual
  • tests.lib-tests

@qweered qweered changed the title lib.mkPackageVariants: init, migrate openssl to mkPackageVariant lib.mkPackageVariants: init, openssl: migrate to mkPackageVariants Mar 4, 2026
@qweered qweered force-pushed the lib/mkPackageVariants branch 2 times, most recently from b3c1085 to 912f142 Compare March 4, 2026 18:40
Comment on lines +1282 to +1289
pkg.overrideAttrs (o: {
passthru =
o.passthru or { }
// mkVariantPassthru variantArgs
// {
inherit variantArgs;
};
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we tell the nixpkgs search which attributes should be shown?
How do we tell hydra where all the package variants are located that need to be evaluated?

As far as I know this is currently done through recurseForDerivations = true;, so all attribute sets with that attribute are also searched for derivations.

This does not work for attributes in passthru. We can't search every package for derivations.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, derivations inside derivations are a non‐starter without several changes outside of Nixpkgs. nix search and nix-env can’t handle them either; it’s baked into the codebase.

Copy link
Copy Markdown
Contributor

@quantenzitrone quantenzitrone Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While writing down the new version of my RFC 197 I thought we could maybe make all versioned packages package sets like we currently do for nix with nixVersions.
If we do that it would be consistent to also have versioned package sets be in a package-set-set like currently for ocamlPackages with ocaml-ng and lixPackages with lixPackageSets.

@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 18, 2026
@qweered qweered marked this pull request as draft March 23, 2026 16:14
@qweered qweered changed the title lib.mkPackageVariants: init, openssl: migrate to mkPackageVariants lib.mkPackageVariants: init; openssl: migrate to mkPackageVariants Mar 23, 2026
qweered added 2 commits March 23, 2026 18:16
Add `lib.mkPackageVariants`, a utility for managing multiple variants
(versions or feature variants) of a single package with a shared
generic builder.

The function is curried in three stages:
1. Scope args (callPackage, allowAliases) — pre-bound in splice.nix
2. Override args — captured via @Args, forwarded to the generic builder
3. Package config — variant selection, paths, aliases

The generic builder follows a curried pattern:
  variantArgs -> packageArgs -> derivation-attrs

Each variant's passthru includes references to all other variants,
enabling access patterns like `pkg.v1_1`, `pkg.v3`, etc.
Preparatory rename for the mkPackageVariants migration to preserve git
history
@qweered qweered force-pushed the lib/mkPackageVariants branch from 912f142 to cbd10b6 Compare March 23, 2026 18:54
@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Mar 23, 2026
@qweered qweered force-pushed the lib/mkPackageVariants branch from cbd10b6 to bf731f2 Compare March 23, 2026 20:01
qweered added 3 commits March 24, 2026 18:29
Refactor openssl to use the new `lib.mkPackageVariants` utility:

- generic.nix: Convert from monolithic callPackages pattern to curried
  builder (variantArgs -> packageArgs -> derivation-attrs). Replace
  lib.versionOlder/AtLeast with injected packageOlder/packageAtLeast.
  Move withDocs and extraMeta logic into the builder using version
  checks instead of per-variant metadata.

- variants.nix: Extract version metadata (version, hash, patches) into
  a dedicated file resolved via callPackage.

- default.nix: Thin mkPackageVariants entry point that only specifies
  the default variant selector.

- all-packages.nix: Simplify openssl wiring. Old names (openssl_1_1,
  openssl_3, openssl_3_6) become aliases to openssl.v1_1 etc.
Move openssl from pkgs/development/libraries/openssl to
pkgs/by-name/op/openssl, renaming default.nix to package.nix
per by-name convention.
@qweered qweered force-pushed the lib/mkPackageVariants branch from bf731f2 to db6f0b0 Compare March 24, 2026 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: lib The Nixpkgs function library 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux.

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

4 participants