Skip to content

testers.nixosTest: Deprecate#293891

Open
roberth wants to merge 5 commits intoNixOS:masterfrom
hercules-ci:deprecate-nixosTest
Open

testers.nixosTest: Deprecate#293891
roberth wants to merge 5 commits intoNixOS:masterfrom
hercules-ci:deprecate-nixosTest

Conversation

@roberth
Copy link
Member

@roberth roberth commented Mar 6, 2024 •
edited
Loading

Description of changes

It's time to migrate.
runNixOSTest is a better alternative, that supports modules properly, as well as running on macOS.
At this point, the existence of nixosTest leads to uncertainty, and its users are missing out on the new features.

🚀

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 8.has: documentation This PR adds or changes documentation label Mar 6, 2024
@roberth roberth mentioned this pull request Mar 6, 2024
Merged
@nixos-discourse
Copy link

nixos-discourse commented Mar 6, 2024

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixostest-vs-runtest/26419/6

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 7, 2024

Why is this not working?

DanielSidhion
DanielSidhion reviewed Mar 7, 2024
View reviewed changes
@roberth
Copy link
Member Author

roberth commented Mar 7, 2024

Why is this not working?

nixosTest is not completely broken, but has issues, as described.

Maybe I don't understand the question, but I think I've explained that in the diff, and I've added extra context in the PR description.

@AndersonTorres
Copy link
Member

AndersonTorres commented Mar 7, 2024 •
edited
Loading

I was talking about the ofborg red status.
Apologies for not being clear.

DanielSidhion
DanielSidhion approved these changes Mar 7, 2024
View reviewed changes
Copy link
Member

@DanielSidhion DanielSidhion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on the docs changes. Idk what's the reasoning behind the ofborg red status though.

@delroth delroth added the 12.approvals: 1 This PR was reviewed and approved by one person. label Mar 7, 2024
infinisil
infinisil reviewed Mar 7, 2024
View reviewed changes
pkgs/build-support/testers/default.nix Outdated
Copy link
Member

@infinisil infinisil Mar 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ofborg failure could be fixed by moving this warning into the function body under test:. Non ideal that this is necessary though..

Copy link
Member

@infinisil infinisil Apr 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind, looks like that doesn't work either..

Thinking about this again, I believe we need to not expose the attribute at all if config.allowAliases = false (which then also prevents uses in Nixpkgs).

@delroth delroth removed the 12.approvals: 1 This PR was reviewed and approved by one person. label Mar 10, 2024
@ShamrockLee ShamrockLee mentioned this pull request Mar 10, 2024
Merged
13 tasks
roberth and others added 3 commits April 22, 2024 21:38
@roberth
testers.nixosTest: Deprecate
43bb838
@roberth @DanielSidhion
doc/testers: Edit deprecation warning for nixosTest
4e69ce4 
Co-authored-by: Daniel Sidhion <DanielSidhion@users.noreply.github.com>
@roberth
testers.nixosTest: Move warning and add link
4e54377 
This way ofborg doesn't see it and fail.
Unfortunately, it will cause more warnings than it otherwise
would, if multiple tests are evaluated in one go.
@roberth roberth force-pushed the deprecate-nixosTest branch 2 times, most recently from c1dad81 to 73cdcdd Compare April 22, 2024 23:53
@roberth roberth force-pushed the deprecate-nixosTest branch from 73cdcdd to c44c917 Compare April 23, 2024 11:01
@github-actions github-actions bot added 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: testing Tooling for automated testing of packages and modules labels Apr 23, 2024
@ofborg ofborg bot added 8.has: clean-up This PR removes packages or removes other cruft 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. labels Apr 23, 2024
@roberth roberth requested a review from infinisil April 23, 2024 12:36
@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 4, 2024
@thiagokokada thiagokokada mentioned this pull request Jul 26, 2024
Merged
yfyf added a commit to yfyf/playos that referenced this pull request Sep 20, 2024
@yfyf
Switch to testers.runNixOSTest
bb68b4b 
`nixosTest` is to-be-depercated, see here:
        NixOS/nixpkgs#293891
yfyf added a commit to yfyf/playos that referenced this pull request Sep 24, 2024
@yfyf
Switch to testers.runNixOSTest
5dda177 
`nixosTest` is to-be-depercated, see here:
        NixOS/nixpkgs#293891
@wegank wegank added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jan 2, 2025
@nixos-discourse
Copy link

nixos-discourse commented Mar 25, 2025

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/nixostest-with-flake-configurations/11542/8

@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Mar 25, 2025
@nixpkgs-ci nixpkgs-ci bot added 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 12.approvals: 1 This PR was reviewed and approved by one person. and removed 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md labels Jun 25, 2025
@nixpkgs-ci nixpkgs-ci bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Sep 22, 2025
@bitbloxhub bitbloxhub mentioned this pull request Sep 29, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@infinisil infinisil Awaiting requested review from infinisil

1 more reviewer

@DanielSidhion DanielSidhion DanielSidhion approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md 6.topic: nixos Issues or PRs affecting NixOS modules, or package usability issues specific to NixOS 6.topic: testing Tooling for automated testing of packages and modules 8.has: clean-up This PR removes packages or removes other cruft 8.has: documentation This PR adds or changes documentation 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild on Darwin. 10.rebuild-linux: 1-10 This PR causes between 1 and 10 packages to rebuild on Linux. 12.approvals: 1 This PR was reviewed and approved by one person.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@roberth @nixos-discourse @AndersonTorres @DanielSidhion @infinisil @delroth @wegank

Comments