modules/nixos/buildbot: add stylix

[danth]

See nix-community/stylix#1961.

In short, there isn't enough disk space on a standard GitHub Actions runner to build all of our checks, especially when starting from an empty cache. This is because each testbed depends on the application it's theming, some of which are quite large. We also can't use a separate runner for each check because there are more checks than the maximum number of parallel jobs.

How much local disk space does BuildBot have? Cache space shouldn't be an issue because most of our dependencies are already on https://cache.nixos.org, so wouldn't need to be uploaded to Cachix.

[trueNAHO]

How much local disk space does BuildBot have? Cache space shouldn't be an issue because most of our dependencies are already on https://cache.nixos.org, so wouldn't need to be uploaded to Cachix.

Stylix overlays are generally not cached by https://cache.nixos.org and may consume arbitrary disk space:

some of our overlays invalidate https://cache.nixos.org caches, resulting in local re-compilations. Most notably, the GNOME module has custom patches, triggering re-compilations everytime the color scheme changes.

-- nix-community/stylix#1862 (comment)

For improved CI time, overlays should ideally be cached. Might be good to know how much invalid cache Stylix currently requires in total.

See nix-community/stylix#1961.

In short, there isn't enough disk space on a standard GitHub Actions runner to build all of our checks

Cross-posting the potential CI time concern:

Considering Stylix consumes considerably more CI time than most other Nix projects 1 2 3 (maybe second to Nixpkgs), would we overload CI usage?

This is definitely something to consider. Assuming packages can be cached locally on the build machine, our main consumption would be evaluation time. We could reduce this later by simplifying the testbed machines, for example using a more lightweight desktop environment, disabling documentation packages, and so on.

-- nix-community/stylix#1961 (comment)

[zowoq]

Disk space shouldn't be an issue, the darwin runner has 1TB and linux runners are >= 2TB. We only has one darwin runner to handle both darwin arches so may need to only run tests on aarch64-darwin like home-manger and nixvim.

I've triggered a CI run manually, once it is finished I'll trigger another so we can see a cached CI run.

https://buildbot.nix-community.org/#/builders/6570/builds/1

[trueNAHO]

We only has one darwin runner to handle both darwin arches so may need to only run tests on aarch64-darwin like home-manger and nixvim.

I've triggered a CI run manually, once it is finished I'll trigger another so we can see a cached CI run.

For reference, nix-community/stylix#1968 attempts to fix the legitimately broken CI. In nix-community/stylix#1969, I am experimenting with increasing nothing-but-nix's aggressiveness, otherwise removing the aarch64-linux runner successfully (nix-community/stylix@5602d1a) avoids running out of space and should make our CI pass again.

I will continue investigating nix-community/stylix#1969 tomorrow.

[zowoq]

I think the first CI run crashed. Buildbot doesn't really do well handling repos with hundreds of CI checks. home-manager and nixvim group the checks to reduce the overall number.

Here is the cached run: https://buildbot.nix-community.org/#/builders/6570/builds/6

[danth]

Buildbot doesn't really do well handling repos with hundreds of CI checks. home-manager and nixvim group the checks to reduce the overall number.

Grouping the checks for each application would reduce the total number by about 5×. Would that be enough?

[zowoq]

Grouping the checks for each application would reduce the total number by about 5×. Would that be enough?

Yes, that should work well.

[trueNAHO]

With nix-community/stylix#1985 merged, I suppose this here can be merged as well.

[0xda157]

@zowoq is there anything we (stylix committers) need to do so buildbot status will be displayed by github?

[zowoq]

No. If you rebase a PR or add a commit you'll see it.