Skip to content

jetbrains: switch builder to use extendMkDerivation pattern#478575

Merged
MattSturgeon merged 2 commits intoNixOS:masterfrom
theCapypara:jetbrains/extendMkDerivation
Feb 4, 2026
Merged

jetbrains: switch builder to use extendMkDerivation pattern#478575
MattSturgeon merged 2 commits intoNixOS:masterfrom
theCapypara:jetbrains/extendMkDerivation

Conversation

@theCapypara
Copy link
Member

@theCapypara theCapypara commented Jan 9, 2026

Follow-up to #475183

I recommend disabling whitespaces for reviewing.

⚠️ The Darwin and aarch64 builds need attention. I can't test them and I can't explain why only these specific 5 IDEs are rebuilt on Darwin

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

Add a 👍 reaction to pull requests you find important.

@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes. labels Jan 10, 2026
@theCapypara

This comment was marked as outdated.

Copy link
Contributor

@MattSturgeon MattSturgeon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a much cleaner implementation overall!

I've scrolled through the diff a few times and left a mixture of design questions/comments and impl nitpicks. I'll stop scrolling now, to avoid overwhelming you with too much review spam. 🙈

meta = meta // {
teams = [ lib.teams.jetbrains ];
};
meta = (lib.optionalAttrs (args ? meta) args.meta) // {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same feedback(s) as the other meta

Comment on lines +17 to +28
# Args to not pass to mkDerivation in the base builders. Since both get the same args
# passed in, both have the same list of args to ignore, even if they don't both use
# all of them.
excludeDrvArgNames = [
"product"
"productShort"
"wmClass"
"libdbm"
"fsnotifier"
"extraLdPath"
"extraWrapperArgs"
];
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. This intuitively feels a bit ugly, but I get why you've taken this approach... I currently don't have a better suggestion.

If the goal is being DRY, you either do this or have both implementations import ./exclude-drv-arg-names.nix. If the goal is to be explicit about which args each implementation expects, then calling code would need to apply args conditionally.


Maybe related:

This also makes me question: does the current structure (common builder -> calls platform base builder) make more sense than the alternative (platform builder -> calls common base builder)... What do you think?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regading your quote: I think which order we do this in kinda makes no difference, since we'll end up with the Darwin builder getting most of these attrs, even though it doesn't use them, regardless, unless we would specify separate drvs for Linux and Darwin for all IDEs, which doesn't make any sense really.

I'll think about this a bit, because I also kinda don't like that there isn't one place that documents the arguments. I guess having the excludeDrvArgNames here in this file at least allows seeing all of them in one file and we could even add documentation comments for them here as well.

I couldn't really find any example of how this is solved elsewhere, the pattern of needing to have a completely different base drv for Darwin and Linux seems to be somewhat unique.

We could maybe put all the Linux-specific attributes into a sub-attrset linux but this seems very confusing and may cause issues in the future, if we need some of them for the Darwin builder as well.

mkPyCharmProduct,
mkJetBrainsSource,
pyCharmCommonOverrides,
...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are callPackage args, right? Why the ...?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't use mkJetBrainsProduct anymore and I didn't want to introduce an extra function just for PyCharm in the default.nix, so it also gets mkPyCharmProduct passed. Happy for any suggestions to make this cleaner!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll probably need to clone this locally to fully understand, but my instinct is that unnecessary ... is a code smell. How come mkJetBrainsProduct and mkPyCharmProduct get supplied unconditionally?

fsnotifier,
pyCharmCommonOverrides,
musl,
...
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same

@theCapypara theCapypara force-pushed the jetbrains/extendMkDerivation branch 2 times, most recently from dfb133f to 8347f2b Compare January 11, 2026 10:19
@theCapypara

This comment was marked as outdated.

@theCapypara
Copy link
Member Author

theCapypara commented Jan 11, 2026

The fact that there are now more rebuilds than before really confuses me. I can't really figure out why? I guess maybe because of the default attrs and how they are merged? Or some interactions with the overrides in the IDEs? But then why is mps rebuild on Darwin?

@theCapypara
Copy link
Member Author

@ofborg build jetbrains.idea-oss.tests.plugins.stored-correctly

@MattSturgeon
Copy link
Contributor

Testing locally, I was able to avoid rebuilds for jetbrains.idea with the following patch:

diff --git a/pkgs/applications/editors/jetbrains/builder/default.nix b/pkgs/applications/editors/jetbrains/builder/default.nix
index c0bf69d462c2..b710bb9cc0d9 100644
--- a/pkgs/applications/editors/jetbrains/builder/default.nix
+++ b/pkgs/applications/editors/jetbrains/builder/default.nix
@@ -23,6 +23,7 @@ lib.extendMkDerivation {
     excludeDrvArgNames = [
       "product"
       "productShort"
+      "buildNumber"
       "wmClass"
       "libdbm"
       "fsnotifier"
diff --git a/pkgs/applications/editors/jetbrains/builder/linux.nix b/pkgs/applications/editors/jetbrains/builder/linux.nix
index 76a08a4220a9..5194df00de23 100644
--- a/pkgs/applications/editors/jetbrains/builder/linux.nix
+++ b/pkgs/applications/editors/jetbrains/builder/linux.nix
@@ -81,21 +81,19 @@ lib.extendMkDerivation {
     {
       inherit desktopItem vmoptsIDE vmoptsFile;
 
-      buildInputs = [
+      buildInputs = buildInputs ++ [
         stdenv.cc.cc
         fontconfig
         libGL
         libX11
-      ]
-      ++ buildInputs;
+      ];
 
-      nativeBuildInputs = [
+      nativeBuildInputs = nativeBuildInputs ++ [
         makeWrapper
         patchelf
         unzip
         autoPatchelfHook
-      ]
-      ++ nativeBuildInputs;
+      ];
 
       postPatch = ''
         rm -rf jbr
  1. buildInputs order was accidentally changed (not important, but does technically change the drv)
  2. buildNumber was accidentally added as a derivation attr

Maybe this is enough for all IDEs?

I'm not suggesting that avoiding all rebuilds should be the end goal, but it does make the initial switch easier to review 🙂

To fully take advantage of fixpoint-args args, making them more accessible from finalAttrs and overridable via overrideAttrs, we'll want to allow more args to propagate into derivationAttrs (rather than being excluded). But it probably makes sense to do this after verifying that the initial change is rebuild-free.

@theCapypara
Copy link
Member Author

Thanks! I will test later, this may get rid of all rebuilds.

If it does, I'll add additional commits to reduce the list of excluded attrs again, we'll have to see what makes sense there. This will cause rebuilds again, but we'll know why.

Everything else would then go in follow-up PRs.

@theCapypara theCapypara force-pushed the jetbrains/extendMkDerivation branch from 8347f2b to 25cff2b Compare January 17, 2026 21:42
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. and removed 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Jan 17, 2026
@theCapypara
Copy link
Member Author

theCapypara commented Jan 17, 2026

Alright, got rid of all rebuilds now except for idea-oss and pycharm-oss. I diffed their drvAttrs and the only difference is that preferLocalBuild is now false instead of being undefined. I think that's fine?

So we can now talk about changing excludeDrvArgNames. I'm not entirely sure which of these attributes it would actually make sense to be override-able...

Edit: On Darwin there are more rebuilds: plugins is no longer part of drvAttrs, it was always empty before and was unused.

@theCapypara theCapypara force-pushed the jetbrains/extendMkDerivation branch from 25cff2b to b4f6a47 Compare January 17, 2026 22:07
@theCapypara theCapypara force-pushed the jetbrains/extendMkDerivation branch from b4f6a47 to 02c853a Compare January 17, 2026 22:25
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. and removed 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Jan 17, 2026
@theCapypara theCapypara force-pushed the jetbrains/extendMkDerivation branch 2 times, most recently from 69f3bca to 4264290 Compare January 17, 2026 22:39
@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. and removed 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-darwin: 1 This PR causes 1 package to rebuild on Darwin. labels Jan 17, 2026
@theCapypara
Copy link
Member Author

theCapypara commented Jan 17, 2026

I now made it so that the first commit does not cause any rebuilds* (see the fact that the labels changed to "rebuild-*: 0". I'll now push a second commit that removes the unused plugins again on Darwin and refactors preferLocalBuild on Linux.

I also had to disable the buildInputs for dataspell on Darwin. I'm not sure anybody uses DataSpell on Darwin, I can't imagine those missing is actually correct, but the old builder for Darwin just ignored them, so I'll keep it like this for now and added a note.

*This first commit allegedly removes 6 packages, see the diff generated by CI:

{"added":[],"changed":["release-checks"],"rebuilds":["release-checks"],"removed":["jetbrains.idea.aarch64-darwin","jetbrains.phpstorm.aarch64-darwin","jetbrains.pycharm.aarch64-darwin","jetbrains.rider.aarch64-darwin","jetbrains.ruby-mine.aarch64-darwin","jetbrains.webstorm.aarch64-darwin"]}

It doesn't actually remove these.

@nixpkgs-ci nixpkgs-ci bot added 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. and removed 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels Jan 17, 2026
@MattSturgeon
Copy link
Contributor

*This first commit technically removes 6 packages, see the diff generated by CI:

Odd. Testing locally, all of these are still present. Not sure why the eval comparison would find them missing.

@nixpkgs-ci nixpkgs-ci bot added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jan 30, 2026
@theCapypara theCapypara force-pushed the jetbrains/extendMkDerivation branch from 69f3bca to a6276de Compare February 3, 2026 18:11
@nixpkgs-ci nixpkgs-ci bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Feb 3, 2026
Copy link
Contributor

@MattSturgeon MattSturgeon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's get this merged to unblock further refactoring. LGTM!

Comment on lines +51 to +56
inherit
buildNumber
product
libdbm
fsnotifier
;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At this layer, these arguments are only used to populate passthru. I understand the motivation to avoid duplicating the passthru wiring in both linux.nix and darwin.nix, but it does mean this wrapper needs to be aware of arguments it otherwise doesn't consume. That's a bit of an abstraction leak: ideally each layer would fully own the arguments it accepts.

That said, this isn't something introduced by this change. I'm happy to defer that to a follow-up.

Stepping back, builder/default.nix itself is fairly thin, and it's worth reconsidering whether this level of indirection is buying us much at all.

@MattSturgeon
Copy link
Contributor

MattSturgeon commented Feb 4, 2026

For the record, I'm still not sure why the eval comparison thinks these packages have been removed; they are still there:

Added packages (0)
Removed packages (6)
Changed packages (9)

cc @NixOS/nixpkgs-ci

@MattSturgeon MattSturgeon added this pull request to the merge queue Feb 4, 2026
Merged via the queue into NixOS:master with commit d96826b Feb 4, 2026
29 of 31 checks passed
@theCapypara
Copy link
Member Author

theCapypara commented Feb 8, 2026

@MattSturgeon Seems that the packages got removed on Darwin because (somehow) musl got added as a dependency. So CI was right.

#488169

Edit: Yeah I somehow managed to miss the buildInputs of these packages, they need to be marked as Linux-only, since before they also effectively were (since the Darwin builder ignored them. Will fix later today.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

10.rebuild-darwin: 1-10 This PR causes between 1 and 10 packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 11.by: package-maintainer This PR was created by a maintainer of all the package it changes.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants