fix(env_loader): prune previously-seeded keys when removed from .env - #18784
Open
rayzhux wants to merge 1 commit into
Open
fix(env_loader): prune previously-seeded keys when removed from .env#18784rayzhux wants to merge 1 commit into
rayzhux wants to merge 1 commit into
Conversation
Problem
-------
load_hermes_dotenv() calls python-dotenv's load_dotenv(override=True),
which writes keys present in the file but never *removes* keys you
deleted from .env. Long-running Hermes processes (gateway, dashboard,
CLI sessions) keep stale values in os.environ for the lifetime of the
process. The same key can ghost across SIGUSR1-triggered config reloads
and across explicit re-invocations of load_hermes_dotenv() inside the
same process. Restarting the process is the only mitigation.
Real-world failure: a key deleted from ~/.hermes/.env continued routing
browser_navigate calls to its old endpoint for 8 days across three
long-lived CLI processes. macOS `ps eEww -p <pid>` returns nothing for
env on non-self processes, so detecting the ghost requires lldb or
process restart.
Fix
---
Track keys that load_hermes_dotenv itself put into os.environ (per-
process _HERMES_SEEDED_KEYS set, populated only when a key was absent
from os.environ before our load). Before each subsequent load:
1. Parse the .env file(s) with dotenv_values to compute declared keys.
2. Pop any seeded key not in that declared set, and drop it from
the registry.
3. Then run load_dotenv as before.
Genuine shell exports (keys we never seeded) are never popped — only
keys whose presence in os.environ originated from one of our own
load_dotenv calls. If any .env file fails to parse, pruning is skipped
to avoid clobbering keys whose declaration we couldn't read.
A bare `FOO` line (no `=`) is treated as undeclared. dotenv_values
returns it as `FOO -> None` but load_dotenv does not actually seed it,
so counting it as declared would let a previously-seeded value linger.
`FOO=` (empty value) parses as `FOO -> ""` and IS seeded, so it stays
declared.
Tests
-----
6 new test cases plus an autouse fixture that snapshots and restores
the seeded-keys registry between tests:
- test_seeded_key_removed_from_file_is_pruned_on_reload
- test_shell_exported_key_never_pruned
- test_seeded_key_value_change_propagates_on_reload
- test_bare_key_without_value_is_pruned_on_reload
- test_empty_value_keeps_key_declared
- test_seeded_key_pruned_when_only_in_one_of_two_files
All 11 tests in tests/hermes_cli/test_env_loader.py pass.
Relationship to PR NousResearch#18734
-------------------------
PR NousResearch#18734 addresses a related but distinct bug: 12-factor precedence
between shell-exported env vars and .env file values. That PR flips the
default to override=False with a HERMES_DOTENV_OVERRIDE=1 opt-in.
This PR is complementary. It fixes the multi-load-in-same-process ghost
under either precedence default — when override=True is in effect (or
when override=False but .env did originally seed a value), a deletion
in .env now actually takes effect on reload instead of pinning the
old value forever.
teknium1
reviewed
Jul 12, 2026
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for identifying a real long-lived-process gap. Current main still reloads through load_hermes_dotenv() on gateway turns (gateway/run.py:1336-1340), while the helper loads user .env with override=True (hermes_cli/env_loader.py:245-247) and has no deletion tracking.
Problems
- The added
newly_seededexpression records only names absent before loading (hermes_cli/env_loader.py:239, PR right side). If the shell starts withFOO=shelland.envhasFOO=dotenv, the loader overwrites the shell value but does not track it. RemovingFOOfrom.envthen leavesFOO=dotenv, so the reported ghost persists for that important case. - Current main now also loads
.op.envand applies external/managed secret sources after the user/project files (hermes_cli/env_loader.py:249-268,:273-302); the proposed declared-key pass covers only user/project paths.
Suggested changes
- Track each overwritten key’s prior state and restore it when its dotenv declaration disappears; add the shell-overwrite/delete regression test.
- Reconcile the implementation with current secret-source and managed-environment ordering before salvage.
This is an automated hermes-sweeper review.
| # already seeded) are not added a second time, and shell exports | ||
| # remain ineligible for pruning. | ||
| newly_seeded = (set(os.environ.keys()) - pre_load_keys) & declared_keys | ||
| _HERMES_SEEDED_KEYS.update(newly_seeded) |
Contributor
There was a problem hiding this comment.
This tracks only names that were absent before loading. With shell FOO=shell and .env FOO=dotenv, override=True replaces the shell value but FOO is not registered; deleting FOO later leaves dotenv ghosted. Preserve the prior value/missing state for overwritten keys and restore it on removal.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
load_hermes_dotenv()callspython-dotenv'sload_dotenv(override=True), which writes keys present in the file but never removes keys you deleted from.env. Long-running Hermes processes (gateway, dashboard, CLI sessions) keep stale values inos.environfor the lifetime of the process. The same key can ghost across SIGUSR1-triggered config reloads and explicit re-invocations ofload_hermes_dotenv()inside the same process. Restarting the process is the only mitigation today.Real-world failure mode
A key deleted from
~/.hermes/.envcontinued routingbrowser_navigatecalls to its old endpoint for 8 days across three long-lived CLI processes. macOSps eEww -p <pid>returns nothing for env on non-self processes, so detecting the ghost requireslldbor a process restart — there is no surfaced signal.Fix
Track keys that
load_hermes_dotenvitself put intoos.environin a per-process_HERMES_SEEDED_KEYSset, populated only when a key was absent fromos.environbefore our load. Before each subsequent load:.envfile(s) withdotenv_valuesto compute declared keys.load_dotenvas before.Genuine shell exports (keys we never seeded) are never popped — only keys whose presence in
os.environoriginated from one of our ownload_dotenvcalls. If any.envfile fails to parse, pruning is skipped to avoid clobbering keys whose declaration we couldn't read.A bare
FOOline (no=) is treated as undeclared.dotenv_valuesreturns it asFOO -> Nonebutload_dotenvdoes not actually seed it, so counting it as declared would let a previously-seeded value linger.FOO=(empty value) parses asFOO -> \"\"and IS seeded, so it stays declared.Relationship to #18734
#18734 addresses a related but distinct bug: 12-factor precedence between shell-exported env vars and
.envfile values. That PR flips the default tooverride=Falsewith aHERMES_DOTENV_OVERRIDE=1opt-in.This PR is complementary. It fixes the multi-load-in-same-process ghost under either precedence default — when
override=Trueis in effect (or whenoverride=Falsebut.envdid originally seed a value), a deletion in.envnow actually takes effect on reload instead of pinning the old value forever.Test plan
Six new test cases plus an autouse fixture that snapshots and restores the seeded-keys registry between tests:
test_seeded_key_removed_from_file_is_pruned_on_reloadtest_shell_exported_key_never_prunedtest_seeded_key_value_change_propagates_on_reloadtest_bare_key_without_value_is_pruned_on_reloadtest_empty_value_keeps_key_declaredtest_seeded_key_pruned_when_only_in_one_of_two_filesAll 11 tests in
tests/hermes_cli/test_env_loader.pypass:```
$ pytest tests/hermes_cli/test_env_loader.py
11 passed in 1.51s
```
🤖 Generated with Claude Code