feat(config): add env.d/ drop-in directory for additional env files

[yzx9]

What does this PR do?

Add $HERMES_HOME/env.d/ drop-in directory. Any *.env files placed there are loaded alphabetically after the main .env with override=True. Useful for separating secrets from config, providing managed defaults that users shouldn't edit, or integrating with secret managers (sops-nix, agenix) via symlinks.

• Add $HERMES_HOME/env.d/ drop-in directory for loading additional .env files
• NixOS module now symlinks environmentFiles into env.d/ instead of copying secrets into .env. Non-secret environment vars are written to env.d/nix-environment.env.

cc @alt-glitch as you create the NixOS modules

Related Issue

Previously, the NixOS activation script concatenated all environmentFiles (e.g., sops-nix/agenix secrets) into $HERMES_HOME/.env via cat >>. This defeated the purpose of secret managers — secrets were copied into persistent state instead of staying as symlinks to secrets.

Fixes #

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

hermes_cli/env_loader.py: After loading the main .env, load_hermes_dotenv() now scans $HERMES_HOME/env.d/*.env (sorted alphabetically) and loads each file with override=True. This works for all users, not just NixOS — drop any .env file into ~/.hermes/env.d/ and it gets picked up.

nix/nixosModules.nix: The activation script now:

• Writes cfg.environment to env.d/nix-environment.env (instead of .env)
• Symlinks each cfg.environmentFiles entry to env.d/nix-<i>.env (instead of cat merging)
• Adds env.d/ to tmpfiles rules
• Documents that sops-nix/agenix secrets need owner/group set to the hermes user

Docs: Updated environment-variables.md (env loading order) and nix-setup.md (secrets management, directory layout, options reference, troubleshooting).

How to Test

1. pytest tests/hermes_cli/test_env_loader.py -v
2. Create ~/.hermes/env.d/test.env with TEST_VAR=from-env-d, run hermes config to verify it's loaded
3. NixOS: nix build .#nixosConfigurations.<host> and verify env.d/ is created with correct symlinks

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: linux (nixos with latest nixos-unstable)

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

• This skill is broadly useful to most users (if bundled) — see Contributing Guide
• SKILL.md follows the standard format (frontmatter, trigger conditions, steps, pitfalls)
• No external dependencies that aren't already available (prefer stdlib, curl, existing Hermes tools)
• I've tested the skill end-to-end: hermes --toolsets skills -q "Use the X skill to do Y"

Screenshots / Logs

[alt-glitch]

hi @yzx9
thank you so much for this PR.

will triage and get this into upstream soon 🫡

[alt-glitch]

@yzx9 pushed 4 commits on top of your branch — container mode was broken and needed fixes.

The problem

Symlinks to /run/secrets/... don't resolve inside the container (those paths aren't mounted). Stale nix-N.env entries also stuck around when environmentFiles shrinks.

What changed

The key commit is 5ddbf89 — the activation script now branches on cfg.container.enable:

• Native: ln -sfn (your original approach, works perfectly)
• Container: install -m 0640 (copies content since symlink targets aren't accessible)

Also added find -delete before creating entries to handle the stale cleanup, and a few test commits (pytest symlink edge cases, runCommand check with 4 scenarios, nixosTest VM integration).

Container mode security

The copy means secrets hit persistent disk in container mode, which is the same as the old cat >> .env behavior — not a regression, but native mode gets the security win while container mode doesn't.

Ideally we wouldn't need to diverge the native/container paths in the NixOS module. One approach would be a tmpfs-backed staging dir (/run/hermes-env/) bind-mounted into the container so secrets stay in RAM. Would you be up for looking into that? We'd rather not keep the copy path long-term.

Tested on NixOS unstable, container mode, sops-nix, Docker — service runs, all env vars load inside the container.

[yzx9]

@alt-glitch How about mounting the secrets directly into the container? This way, we can avoid copying them and instead let the secret manager handle the entire lifecycle.

${lib.concatStringsSep " " (lib.imap0 (i: f:
  "--volume ${f}:${containerDataDir}/.hermes/env.d/nix-${toString i}.env:ro"
) cfg.environmentFiles)}

and skip the secret symlink in the activation script of container mode:

${lib.concatStringsSep "\n" (lib.imap0 (i: f: ''
  ${lib.optionalString (!cfg.container.enable) ''
    ln -sfn "${f}" "${cfg.stateDir}/.hermes/env.d/nix-${toString i}.env"
  ''}
'') cfg.environmentFiles)}

[alt-glitch]

@yzx9
this runs into a container-lifecycle issue.

Docker bakes --volume args at create time, not start time. Since environmentFiles would now affect the docker create command, it'd need to be part of containerIdentity (nix/nixosModules.nix:169-176) — otherwise the container keeps its original mount set forever and secret rotation silently doesn't take effect.

But adding it to the identity hash means every change to environmentFiles (rotation, new key, path change) triggers docker rm -f + recreate. That wipes the writable layer: apt packages, pip/uv installs, ~/.venv, npm globals — all gone, and first-boot provisioning re-runs. See the recreation table in website/docs/getting-started/nix-setup.md:597-600.

[yzx9]

@alt-glitch Ah yeah, I completely forgot about this constraint on the Docker side.

--secret would be nice, but it only works with Swarm services, which doesn’t fit our current standalone container model.

docker pass also isn’t a good fit here. It’s experimental and depends on a local keychain/DBus setup, which doesn’t work well for NixOS services.

I’m good with the current approach. I will add a comment explaining the divergence in container mode and its potential security implications.

[alt-glitch]

Would you like to take a look at the docker fixes in another PR? Would prefer standardisation between nix and nix container mode!

[yzx9]

@alt-glitch sure, happy to take a look

[alt-glitch]

@yzx9 if you can add a PR / update this PR for the docker updates as well, would love to get both of them merged together preferably.

[yzx9]

Hi @alt-glitch,
I’m willing to proceed, but I think this topic likely requires more detailed consideration. At the moment, I don’t have a clear approach for handling it.

As discussed above, we currently have two possible ideas:

1. Read the secrets directly and pass them into the container as environment variables. The downside of this approach is that all processes inside the container would automatically inherit these environment variables, which introduces potential security risks.

2. Introduce Docker Compose or Swarm for improved secret management. However, this would require relatively large changes to our current setup.

[yzx9]

@alt-glitch Hi, any updates? I think copying the secret with a note is enough for now. In practice, agenix also provides this workaround via age.secrets.<name>.symlink

[teknium1]

Thanks for the focused Nix secret-lifecycle work. Current origin/main still concatenates environmentFiles into .env (nix/nixosModules.nix:832-846) and has no env.d loader (hermes_cli/env_loader.py:245-265), so the premise remains valid.

Problems

• nix/nixosModules.nix:847 wraps the cleanup in cfg.environment != {} || cfg.environmentFiles != []. If both values are removed after a prior deployment, the find ... -delete block is omitted and old Nix-managed env files remain loaded. The added test covers three files becoming one, not becoming zero (nix/checks.nix:647-664).
• website/docs/getting-started/nix-setup.md:360 and :861 say secret files are symlinked with no copying, but container mode uses install to copy them (nix/nixosModules.nix:864-869). This should retain the container security caveat from the PR discussion.

Suggested changes

• Make targeted cleanup unconditional and add a transition-to-empty test.
• Document native symlinks versus container copies in both affected docs locations.

Automated hermes-sweeper review.

[teknium1]

This guard also suppresses find ... -delete when a user removes the last environmentFiles entry and clears environment. Existing nix-*.env / nix-environment.env files then survive and continue to load. Please run the targeted cleanup outside this conditional and add a transition-to-empty test.

[teknium1]

Container mode does copy each secret via install in nix/nixosModules.nix:864-869; this statement's universal “symlinked (no copying)” claim is inaccurate. Please distinguish native symlinks from container copies and retain the persistent-disk security caveat here and in the option table.