Skip to content

feat(nix): opt-in NixOS VM integration test for the module - #69496

Draft
alt-glitch wants to merge 1 commit into
mainfrom
feat/nixos-vm-test
Draft

feat(nix): opt-in NixOS VM integration test for the module#69496
alt-glitch wants to merge 1 commit into
mainfrom
feat/nixos-vm-test

Conversation

@alt-glitch

Copy link
Copy Markdown
Collaborator

Summary

The NixOS module (1008 lines — systemd unit, tmpfiles, activation script, container mode) had zero VM-level coverage; all existing checks are runCommand-based. This adds the repo's first testers.runNixOSTest:

  • boots a VM importing nixosModules.default with services.hermes-agent.enable = true and a dummy provider key
  • asserts: state-dir tree is 2770 hermes:hermes (tmpfiles), hermes version/--help work as the hermes user from the sealed venv, the activation .managed marker exists, and the credential-less gateway starts and stays active — asserted via its No messaging platforms enabled. startup log instead of a bare wait_for_unit that could pass during a restart loop
  • exposed as packages.x86_64-linux.nixos-vm-test (opt-in), not a flake check, so nix flake check cost is unchanged; runs on manual trigger locally or in the ci(nix): restore nix CI as trigger-only workflow with local entrypoint #69463 CI via nix build .#nixos-vm-test -L (needs kvm + nixos-test system features)

Test plan

Full local run: VM boots, all assertions pass, gateway active, test script finished in 38.5s (log excerpt in the build output).

First VM-level coverage for nixosModules.default: boots a VM with
services.hermes-agent enabled, then asserts tmpfiles state-dir modes
(2770 hermes:hermes across the .hermes tree), the sealed CLI working as
the hermes user, the activation .managed marker, and a credential-less
gateway that starts and STAYS active (asserted via the 'No messaging
platforms enabled.' startup log rather than a bare wait_for_unit, which
could pass mid-restart-loop).

Exposed as packages.x86_64-linux.nixos-vm-test, deliberately NOT a flake
check: booting a VM over the ~700MB closure is too costly for every
'nix flake check'. Run explicitly: nix build .#nixos-vm-test -L
(requires kvm/nixos-test system features).
@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

૮ >ﻌ< ა ci review

running on da20df2

looks good to me!

@alt-glitch

Copy link
Copy Markdown
Collaborator Author

Review of the opt-in NixOS VM integration test (Nix is out of calldiff's scope, so this is a plain diff read). Clean, well-targeted addition — packaged as packages.nixos-vm-test rather than a flake check (correct, given the VM boot cost) and gated to x86_64-linux.

🔴 Issues

None blocking.

🟡 Duplication / abstraction

  • nix/vmTest.nix:47-49 — the HOME / HERMES_HOME layout is re-hardcoded here (/var/lib/hermes, /var/lib/hermes/.hermes) alongside the tmpfiles assertions at :33-43. These paths are defined by the NixOS module under test, so the test string-duplicates the module's own contract. If the module ever moves the state dir, this test silently keeps asserting the old path. Consider deriving the expected paths from the module option (or at least a let stateDir = "/var/lib/hermes"; binding at the top of the test) so there's one source of truth.
  • The two stat -c '%a %U %G' … = '2770 hermes hermes' assertions (:36 and :39-43) repeat the same mode/owner tuple; a small shell helper in testScript (assert_mode <path>) would DRY the loop and the single check.

✅ Looks good

  • Not relying on a bare wait_for_unit() — asserting the live ActiveState=active and the "No messaging platforms enabled." startup log is exactly right; it defends against a passing-during-restart-loop false green. Good comment explaining why.
  • The inline dummy OPENROUTER_API_KEY via writeText is a fine credential-less startup fixture (placeholder value, no real secret).
  • lib.optionalAttrs (… == "x86_64-linux") keeps it from breaking eval on darwin/aarch64 builders.
  • memorySize/diskSize are reasonable for the closure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant