Skip to content

[Backport release-24.11] ci: allow running jobs locally#406136

Merged
wolfgangwalther merged 15 commits intorelease-24.11from
backport-404466-to-release-24.11
May 11, 2025
Merged

[Backport release-24.11] ci: allow running jobs locally#406136
wolfgangwalther merged 15 commits intorelease-24.11from
backport-404466-to-release-24.11

Conversation

@nixpkgs-ci
Copy link
Copy Markdown
Contributor

@nixpkgs-ci nixpkgs-ci bot commented May 11, 2025

Bot-based backport to release-24.11, triggered by a label in #404466.

  • Before merging, ensure that this backport is acceptable for the release.
    • Even as a non-committer, if you find that it is not acceptable, leave a comment.

They are not really related to eval.yml.

(cherry picked from commit e908941)
This allows running the lib-tests locally in exactly the same way that
they are run in CI:

  nix-build ci -A lib-tests

(cherry picked from commit 7097614)
The NixOS manual can now be built locally the same way as in CI with:

  nix-build ci -A manual-nixos

(cherry picked from commit 89520b9)
The Nixpkgs manual can now be built locally the same way as in CI with:

  nix-build ci -A manual-nixpkgs -A manual-nixpkgs-tests

(cherry picked from commit d253ad1)
The dev shell can now be built locally the same way as in CI with:

  nix-build ci -A shell

(cherry picked from commit 7c16d15)
The nix-parse workflow can now be run locally the same way as in CI.

To do this, the CI's workflow was slightly adjusted. Instead of testing
only the changed files, we're now testing all files in the repository.

This is possible in two ways:

1. By calling nix-instantiate once with all files as arguments. This
will be rather fast, but only the first error is shown before it errors
out.
2. By calling nix-instantiate once for each file. This will be much
slower, but has the advantage that we see all errors at once.

To avoid running the long variant every time, we first do a quick check
with the fast version. If that fails, we run the slower one to report
the errors. This gives us the best of both.

(cherry picked from commit a553ef2)
Path interpolation syntax is not supported in the minimum nix version.

(cherry picked from commit 8980c1f)
This adds the minimum nix version and the latest lix version to the
matrix of parse checks. Especially the minimum nix version is relevant,
because parsing routinely breaks because of introduction of newer
syntax.

Adding lix just completes the picture.

(cherry picked from commit 101a271)
@nixpkgs-ci nixpkgs-ci bot mentioned this pull request May 11, 2025
6 tasks
@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: continuous integration Affects continuous integration (CI) in Nixpkgs, including Ofborg and GitHub Actions 4.workflow: backport This targets a stable branch 6.topic: policy discussion Discuss policies to work in and around Nixpkgs labels May 11, 2025
@Mic92
Copy link
Copy Markdown
Member

Mic92 commented May 11, 2025

Merge is blocked on #404466 (comment)

@Mic92 Mic92 marked this pull request as draft May 11, 2025 10:35
@github-actions github-actions bot added 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux. labels May 11, 2025
@Mic92 Mic92 marked this pull request as ready for review May 11, 2025 10:44
@wolfgangwalther
Copy link
Copy Markdown
Contributor

Cherry-picked the fixes from #406138 in here, too.

Will still wait a bit more to see that CI stays green.

ARM runners are supposed to be more energy efficient than x86. Also,
from limited testing, they appear to be faster for the eval jobs as
well. Average run time for the "Outpaths (x86_64-linux)" job was 4m 27s,
so far. In the first run, this job came in at 3m 9s. This effect did not
show for other jobs, yet.

The following two exceptions are made right now:
- nixpkgs-lib-tests currently fails on the ARM runner building Nix 2.3
- nixpkgs-vet is currently pinned to a x86_64-linux only binary release

(cherry picked from commit d3e4865)
A bit easier to read.

(cherry picked from commit c148dc7)
GitHub has all the different runners, so we can just as well check that
shell.nix works.

(cherry picked from commit a56fa7a)
I added a lint-action.sh script in .github/workflows a while ago while
fixing some warnings. But I haven't run it myself ever since. This needs
to be part of CI to make any use of it.

(cherry picked from commit 8b65ba5)
@wolfgangwalther
Copy link
Copy Markdown
Contributor

Added the two failed backports of #406114 and #405943 in here as well.

@wolfgangwalther
Copy link
Copy Markdown
Contributor

I'm running all the recently merged PRs together (aka current master) once more in my fork, triggering all those jobs.

@wolfgangwalther
Copy link
Copy Markdown
Contributor

All looks good, except the job for the nixpkgs-manual. It fails weirdly: https://github.com/wolfgangwalther/nixpkgs/actions/runs/14955192741/job/42009953079?pr=639

It works just fine for me locally.

Seems to be some networking error:

  File "/nix/store/b7g7y0dkgbza3wh3nypyz5l6crd6pw7p-python3-3.12.10-env/lib/python3.12/site-packages/aiohttp/client.py", line 707, in _request
    raise ConnectionTimeoutError(
aiohttp.client_exceptions.ConnectionTimeoutError: Connection timeout to host https://www.gnu.org/software/coreutils/manual/html_node/cat-invocation.html

I don't see this related to the recent changes.

@wolfgangwalther
Copy link
Copy Markdown
Contributor

wolfgangwalther commented May 11, 2025

This also reproduces against master in nixpkgs: #406152

@wolfgangwalther
Copy link
Copy Markdown
Contributor

That was a temporary issue, re-running a few hours later and all is good.

@wolfgangwalther wolfgangwalther merged commit 53b8217 into release-24.11 May 11, 2025
18 checks passed
@wolfgangwalther wolfgangwalther deleted the backport-404466-to-release-24.11 branch May 11, 2025 15:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.workflow: backport This targets a stable branch 6.topic: continuous integration Affects continuous integration (CI) in Nixpkgs, including Ofborg and GitHub Actions 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: policy discussion Discuss policies to work in and around Nixpkgs 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants