fix(tests): defer live module env load to test execution, not collection - #80350
Draft
wali-reheman wants to merge 1 commit into
Draft
wali-reheman wants to merge 1 commit into
wali-reheman wants to merge 1 commit into
Conversation
Move _load_user_env() out of module scope and behind an autouse fixture that runs after _hermetic_environment. The old module-level call ran at pytest collection time, before the credential scrubber, leaking developer-local env vars (e.g. SEARXNG_URL) into os.environ and causing non-deterministic test failures that depended on what was in ~/.hermes/.env. The autouse fixture is the right mechanism here: it fires for every test in the module (skipped or not), but only after the hermetic scrub has already run, so developer credentials can still flow in for live test execution without contaminating the collection phase. Also add tests/test_env_leak_guard.py: a regression test that walks every test module's AST and fails if any module writes a file-read result into os.environ at module scope — catching the exact pattern of NousResearch#80343.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixes #80343
What
Move
_load_user_env()out of module scope and behind an autouse fixture thatruns after
_hermetic_environment. The old module-level call ran at pytestcollection time, before the credential scrubber, leaking developer-local
env vars (e.g.
SEARXNG_URL) intoos.environand causing non-deterministictest failures.
Why
When
test_sequential_chats_live.pywas collected,_load_user_env()read~/.hermes/.envand wrote intoos.environimmediately — before any pytestautouse fixtures had a chance to scrub credential-shaped variables. With
SEARXNG_URLleaked,_get_backend()intest_web_tools_config.pyreturnedsearxnginstead offirecrawl, causing those two tests to fail on machinesthat had
SEARXNG_URLconfigured locally.How
The fix defers the env load to fixture time, not collection time. An autouse
fixture
_live_test_envcalls_load_user_env()after_hermetic_environmenthas already run — so live tests still get
OPENROUTER_API_KEYfor execution,but collection is clean.
Also adds
tests/test_env_leak_guard.py: a regression test that walks everytest module's AST and fails if any module writes a file-read result into
os.environat module scope.Verification