fix(tests): live module must not import the developer's .env at collection

[rodrigogs]

Symptom

Two tests passed in isolation and failed in a wide run:

tests/tools/test_web_tools_config.py::TestBackendSelection::test_fallback_no_keys_defaults_to_firecrawl
    AssertionError: assert 'searxng' == 'firecrawl'
      - firecrawl
      + searxng

tests/tools/test_web_tools_config.py::TestCheckWebApiKey::test_null_backend_value_does_not_crash
    AssertionError: assert True is False

Root cause

tests/run_agent/test_sequential_chats_live.py is skipped unless HERMES_LIVE_TESTS=1, but the skip governs execution, not import. Its _load_user_env() call sat at module scope, so pytest ran it during collection — before any fixture, including the hermetic _hermetic_environment autouse fixture that scrubs credential-shaped variables. It copied every key from ~/.hermes/.env into os.environ via setdefault.

On the machine where this reproduced, that file holds 23 keys. Roughly 12 of them — SEARXNG_URL, TELEGRAM_*, WHATSAPP_*, HERMES_SPOTIFY_* — match neither _CREDENTIAL_NAMES nor the _CREDENTIAL_SUFFIXES heuristic in tests/conftest.py, so the scrubber never removed them. With SEARXNG_URL leaked, _get_backend() reached its searxng candidate and returned searxng instead of firecrawl.

The suite's outcome therefore depended on what the developer happened to have configured locally.

Why this was hard to bisect

Two properties, both worth recording for the next person:

It is collection-time, not execution-order. Selecting only the two victims out of the full tree still fails, with zero other tests executing:

$ pytest tests/ -k '<the two victim names>'
2 failed, 4 skipped, 26400 deselected

Order is irrelevant. Listing the victim file first still fails, because pytest imports every collected module before running anything:

$ pytest tests/tools/test_web_tools_config.py tests/run_agent/test_sequential_chats_live.py -k ...
2 failed, 37 deselected in 0.29s

This is why a pairwise bisect over the 39 candidate files found nothing, and why feeding pytest the full 180-item ordered prefix of node ids passes cleanly. Delta debugging over the collected file set converged on a single file in 22 runs.

Fix

Import only the key the module actually needs (OPENROUTER_API_KEY), and only when live runs are enabled. A skipped module should not touch shared process state merely by being imported.

The added regression test walks every test module's AST and fails on any that copies file contents into os.environ at import time. It deliberately targets that combination rather than module-scope env writes in general: a fixed os.environ["TERMINAL_ENV"] = "local" is deterministic and reviewable, and tests/tools/test_code_execution.py and test_code_execution_modes.py both use it on purpose. Reading the developer's filesystem is what makes the result differ per machine.

Verification

Shortest reproducer, 0.31s, before the fix:

$ pytest tests/run_agent/test_sequential_chats_live.py \
    'tests/tools/test_web_tools_config.py::TestBackendSelection::test_fallback_no_keys_defaults_to_firecrawl' \
    'tests/tools/test_web_tools_config.py::TestCheckWebApiKey::test_null_backend_value_does_not_crash' -q
sFF  ->  2 failed, 1 skipped in 0.31s

After: 2 passed, 1 skipped in 0.31s.

• Wide run -k "parallel or web_tools or lazy_deps": both victims pass. The only remaining failures in that selection are TestParallelClientConfig, fixed independently by fix(web): a benched lazy-install must not disable an importable SDK #80022 — with both branches applied the selection is 213 passed, 5 skipped, 0 failed.
• The new regression test fails when the production change is reverted (git stash the module, test goes red), so it is not a tautology.
• tests/tools/ in full passes; ruff clean on both touched files.

Found alongside #79839, #79840 and #80022 while investigating a single incident — independent subsystems, separate PRs.

[rodrigogs]

Hi — heads-up that CI has never actually run on this PR: every workflow run (CI + Docker Build) since it was opened ends in action_required with zero jobs, and the run page says 'This workflow is awaiting approval from a maintainer'. The branch is mergeable with no conflicts and doesn't touch any workflow files. Could a maintainer approve the workflow runs (or enable CI for outside collaborators)? Happy to address anything the checks find.

[rodrigogs]

Rebased onto current upstream/main (0a171fffe); new head 1163d9f42997.

Conflict-free, and 492 commits of drift changed nothing this PR depends on:

• tests/conftest.py still gates its hermetic scrubber on the same _looks_like_credential() heuristic
  (_CREDENTIAL_NAMES + _CREDENTIAL_SUFFIXES), and it still matches none of SEARXNG_URL, TELEGRAM_*,
  WHATSAPP_* or HERMES_SPOTIFY_*;
• the _hermetic_environment autouse fixture is still test-setup-time, so it still cannot undo a
  collection-time write;
• both victim tests still exist under their original names in tests/tools/test_web_tools_config.py
  (lines 294 and 533);
• the only upstream commit to touch tests/run_agent/test_sequential_chats_live.py since this branch is
  5797728ca, which created it — the module-scope _load_user_env() call is still there verbatim.

The single commit replayed with no conflicts and the resulting diff is byte-identical to the pre-rebase one.
Verification on the new head: tests/test_env_isolation_at_collection.py → 1 passed.

This repository does not run CI on pull requests from forks, so the checks tab stays empty and protect-main's required All required checks pass context never reports — which is why this PR shows mergeable: true with mergeStateStatus: BLOCKED. Approving the workflow run (or landing it on the strength of the local evidence) is all that is left from my side.

[rodrigogs]

Force-pushed a metadata-only fix so the contributor attribution check can pass.

.github/workflows/contributor-check.yml exits 1 on any commit-author email that has no file under
contributors/emails/, and ci.yaml calls it, so it feeds the required All required checks pass context.
This branch carried a placeholder author identity from a misconfigured local git config, which the check
would have rejected the moment a maintainer approved the workflow runs.

Every commit's author is now Rodrigo Gomes <2362425+rodrigogs@users.noreply.github.com> — the account's
GitHub noreply address, which the check auto-resolves via its +…@users.noreply.github.com rule, so no
mapping file is needed. Author dates are preserved, and the tree is byte-identical: git diff <old-head> <new-head> is empty, so nothing about the change under review moved and the verification I posted earlier
still stands.