fix(tests): stop config.set model tests writing cli-config.yaml into the repo root - #57182
fix(tests): stop config.set model tests writing cli-config.yaml into the repo root#57182lEWFkRAD wants to merge 1 commit into
Conversation
…the repo root
Five tests in tests/test_tui_gateway_server.py drive a real /model
switch through server.handle_request without patching
cli.save_config_value. A plain /model persists globally by default
(model.persist_switch_by_default → True), so _persist_model_switch ran
the real save — and because the hermetic per-test HERMES_HOME tempdir
never contains a config.yaml, save_config_value fell through to its
project-config fallback and wrote cli-config.yaml into the developer's
repo root:
model:
default: anthropic/claude-sonnet-4.6
provider: anthropic
base_url: https://api.anthropic.com
The file is gitignored, so the pollution is invisible in git status,
and later runs read it back as the project-config fallback — silently
breaking tests/hermes_cli/test_ignore_user_config_flags.py and
tests/test_hermes_state.py.
- Patch cli.save_config_value in the five offending tests
(test_config_set_model_requires_confirmation_for_expensive_model,
test_config_set_model_explicit_provider_skips_broken_default_init,
test_config_set_model_does_not_leak_inference_provider_env,
test_config_set_model_records_per_session_override_not_env,
test_config_set_model_switches_agent_without_touching_env), matching
the existing pattern in test_config_set_model_global_persists.
- Add an autouse tripwire fixture in tests/conftest.py that fails any
test that creates or mutates the repo-root cli-config.yaml, restoring
the pre-test state so one offender can't cascade into unrelated
failures runs later.
Found during the PR NousResearch#57066 / issue NousResearch#57068 triage.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Thanks for isolating the test pollution. The premise remains valid on current main: The five targeted tests still exercise successful plain GitHub reports the PR cleanly mergeable against current main. This is an automated hermes-sweeper review. |
|
Thanks @lEWFkRAD — closing as resolved on main: cli.save_config_value now resolves get_hermes_home()/config.yaml live with no repo-root cli-config.yaml fallback. Verified empirically: /model persistence tests create no repo-root file and leave the real config byte-identical. Your issue #57147 documented the class correctly. |
What does this PR do?
Stops the test suite from silently writing a
cli-config.yamlinto the repo root of the developer's checkout, and adds a tripwire so any future offender fails loudly by name.cli.save_config_value()writes the user config atcli._hermes_home / 'config.yaml'only if it already exists, otherwise it falls back to the project config —cli-config.yamlnext tocli.py, i.e. the repo root. The suite's hermetic fixture points HERMES_HOME at a fresh empty per-test tempdir, so during tests the user config never exists and any realsave_config_value()call lands in the repo root. The file is gitignored (invisible ingit status) and is read back as the project-config fallback by every later run, silently breakingtests/hermes_cli/test_ignore_user_config_flags.pyandtests/test_hermes_state.py.Five tests in
tests/test_tui_gateway_server.pyhit this: they drive a real/modelswitch throughserver.handle_requestwithout patchingcli.save_config_value— and a plain/model(no flag) persists globally by default (resolve_persist_behavior→model.persist_switch_by_defaultdefaultsTrue), so_persist_model_switchran the real save withmodel.default: anthropic/claude-sonnet-4.6,provider: anthropic,base_url: https://api.anthropic.com.Related Issue
Fixes #57147
Type of Change
Changes Made
tests/test_tui_gateway_server.py— patchcli.save_config_valuein the five offending tests (test_config_set_model_requires_confirmation_for_expensive_model,test_config_set_model_explicit_provider_skips_broken_default_init,test_config_set_model_does_not_leak_inference_provider_env,test_config_set_model_records_per_session_override_not_env,test_config_set_model_switches_agent_without_touching_env), matching the existing pattern intest_config_set_model_global_persists.tests/conftest.py— new autouse_guard_repo_root_cli_configfixture: fails any test that creates or mutates the repo-rootcli-config.yaml, and restores the pre-test state so one offender can't cascade into unrelated failures. (This tripwire is how the five tests above were identified.)How to Test
python -m pytest tests/test_tui_gateway_server.py -q→ an untrackedcli-config.yamlappears at the repo root afterwards (it's gitignored — check withls, notgit status).cli-config.yaml.Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests pass —tests/test_tui_gateway_server.pyis fully green with this change (303 passed, 0 errors, repo root clean afterwards), and a full-suite run viascripts/run_tests_parallel.pyon native Windows (together with the Windows: parallel test runner children share the console process group — one os.kill(pid, 0) probe broadcasts KeyboardInterrupt across the whole run #57145 fix) ends with no repo-rootcli-config.yaml. Remaining full-suite failures are pre-existing native-Windows failures that reproduce under plainpython -m pyteston unmodifiedmain— none involve the new guard.Documentation & Housekeeping
docs/, docstrings) — the fixture docstring documents the fallback trapcli-config.yaml.exampleif I added/changed config keys — N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — N/A