Skip to content

test_config.py::TestSanitizeEnvLines::test_migrate_reports_normalized_line_formatting fails only in full tests/hermes_cli/ suite (order-dependent state leak) #73144

Description

@iap

Summary

tests/hermes_cli/test_config.py::TestSanitizeEnvLines::test_migrate_reports_normalized_line_formatting fails only when run as part of the full tests/hermes_cli/ suite (~15% in). It passes in isolation and passes when its own file is run alone. This is a pre-existing test-isolation / global-state-leakage bug on main, not a regression from any contributor PR.

Verification (clean main-equivalent)

  • Single test alone: pytest "tests/hermes_cli/test_config.py::TestSanitizeEnvLines::test_migrate_reports_normalized_line_formatting"1 passed
  • Whole file alone: pytest tests/hermes_cli/test_config.py186 passed
  • Full suite: pytest tests/hermes_cli/1 failed at that test (the only failure in an otherwise-green run of ~1500+ tests before -x stop). Repros on a branch whose only differences from nousresearch/main are unrelated plugin-display/memory commits that do not touch hermes_cli/config logic or test_config.py.

Repro

pytest tests/hermes_cli/
# -> FAILED tests/hermes_cli/test_config.py::TestSanitizeEnvLines::test_migrate_reports_normalized_line_formatting

Root cause (hypothesis, not confirmed)

The test asserts env-line normalization formatting. Because it passes standalone + in-file but fails after earlier suite tests, some earlier test in tests/hermes_cli/ is leaking shared global state (an env var, or a config-file/global mutated without restoration) that this test depends on. A bisect of tests/hermes_cli/ ordering would pinpoint the polluting test.

Impact

This makes the hermes_cli CI suite red on any PR that runs it, independent of the PR's changes — likely causing maintainers to misread CI failure as a contributor regression. Flagging so it can be fixed/quarantined separately from PR review.

Note

I have not bisected which earlier test pollutes state — posting the verified facts + minimal repro so it can be picked up without re-deriving.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium — degraded but workaround existscomp/cliCLI entry point, hermes_cli/, setup wizardtype/testTest coverage or test infrastructure

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions