Conversation
65a5c15 to
1e7e9fc
Compare
Duplicate of #57182 — both patch the same 5 config.set model tests in |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for isolating the five model-switch tests.
Problems
- The targeted mocks are correct for the current path:
tui_gateway/server.py:2795-2812callscli.save_config_value, whose fallback writes repo-rootcli-config.yamlwhencli._hermes_home/config.yamlis absent (cli.py:3644-3646). The five affected tests remain unmocked on current main, includingtests/test_tui_gateway_server.py:3497-3562. - This PR does not include a suite-wide regression guard. Earlier open PR #57182 includes the same five mocks plus an autouse
tests/conftest.pytripwire for repo-rootcli-config.yamlwrites; current main has no equivalent guard.
Suggested changes
- Salvage this patch together with #57182's tripwire, or add equivalent protection so future real
save_config_value()calls fail at the offending test.
Automated hermes-sweeper review.
| @@ -3526,6 +3526,7 @@ def switch_model(self, **_kwargs): | |||
| "hermes_cli.model_switch.switch_model", lambda **_kwargs: result | |||
| ) | |||
| monkeypatch.setattr(server, "_restart_slash_worker", lambda sid, session: None) | |||
| monkeypatch.setattr("cli.save_config_value", lambda *_a, **_k: True) | |||
There was a problem hiding this comment.
This mock fixes this test's real persistence path. Please pair the five targeted mocks with a suite-wide repo-root cli-config.yaml write guard (as in earlier open PR #57182), so future unmocked callers fail at the source.
test_config_set_model_{requires_confirmation_for_expensive_model,
explicit_provider_skips_broken_default_init,does_not_leak_inference_provider_env,
records_per_session_override_not_env,switches_agent_without_touching_env}
exercised the full config.set flow with switch_model mocked but let
_persist_model_switch call the real cli.save_config_value — which resolves
the config path from cli._hermes_home captured at import. Run without
HERMES_HOME (any plain 'pytest tests/'), they clobber the live user config:
last writer left model.default=anthropic/claude-sonnet-4.6 +
base_url=https://api.anthropic.com, silently repointing every fleet worker
at an Anthropic endpoint (observed live 2026-07-08 16:38; ~2000 worker
deaths on non-retryable 400s until restored).
Mock cli.save_config_value in all five, same as
test_config_set_model_global_persists already does.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…test guard) Per review: the five targeted mocks stand; add the missing suite-wide regression guard so no future test can write the developer's live config.yaml or the repo-root cli-config.yaml fallback. Two layers in tests/conftest.py: 1. Call-time interception — _arm_save_config_tripwire() wraps cli.save_config_value (marker-tagged, armed process-wide, re-armed by the autouse fixture so it survives importlib.reload(cli)) and fails the offending test at its call site BEFORE the write lands, mirroring the function's own target resolution (cli._hermes_home/config.yaml if present, else repo-root cli-config.yaml). Writes into per-test tmp homes pass through untouched; leaked background threads calling save_config_value between tests are blocked too (the failure dies inside that thread). cli is deliberately not force-imported (its module-level load_hermes_dotenv would mutate os.environ mid-test). 2. Before/after watch (autouse fixture) — snapshot both forbidden files around every test; restore the pre-test bytes and fail the test in whose window the write landed if anything slips past the wrapper (cli imported mid-test, direct YAML writers, subprocesses), so one polluting write cannot cascade into later runs. Forbidden paths resolve at conftest-import time from the PRE-isolation environment (hermes_constants.get_hermes_home()), i.e. the home pytest was invoked with — per-test tmp homes are never affected. tests/test_config_write_guard.py pins both layers: repo-root fallback blocked, pre-isolation user config blocked (both shapes), tmp-home writes allowed through to the real implementation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1e7e9fc to
2c422a7
Compare
|
Added the suite-wide guard as an autouse tripwire in tests/conftest.py — equivalent to #57182's but extended to also cover the live user config.yaml, not just the repo-root cli-config.yaml fallback. Two layers: |
test_config_set_model_{requires_confirmation_for_expensive_model,
explicit_provider_skips_broken_default_init,does_not_leak_inference_provider_env,
records_per_session_override_not_env,switches_agent_without_touching_env}
exercised the full config.set flow with switch_model mocked but let
_persist_model_switch call the real cli.save_config_value — which resolves
the config path from cli._hermes_home captured at import. Run without
HERMES_HOME (any plain 'pytest tests/'), they clobber the live user config:
last writer left model.default=anthropic/claude-sonnet-4.6 +
base_url=https://api.anthropic.com, silently repointing every fleet worker
at an Anthropic endpoint (observed live 2026-07-08 16:38; ~2000 worker
deaths on non-retryable 400s until restored).
Mock cli.save_config_value in all five, same as
test_config_set_model_global_persists already does.
🤖 Generated with Claude Code