Skip to content

fix(tests): isolate suite fixtures from host auth - #82500

Merged
teknium1 merged 1 commit into
NousResearch:mainfrom
DanDo385:fix/test-suite-isolation-and-pool-routing
Aug 13, 2026
Merged

teknium1 merged 1 commit into
NousResearch:mainfrom
DanDo385:fix/test-suite-isolation-and-pool-routing

Conversation

@DanDo385

@DanDo385 DanDo385 commented Aug 9, 2026 •

Copy link
Copy Markdown

What does this PR do?

Makes the affected test fixtures independent of credentials and source trees present on the host machine.

  • Removes a test-local sys.path insertion that could import a stale ~/.hermes/hermes-agent checkout instead of the checkout under test.
  • Prevents the Anthropic credential-pool fixture from auto-seeding host Claude Code OAuth or Anthropic environment credentials, and asserts that the pool contains exactly the configured fixture entries.

This fixes machine-dependent failures without changing runtime credential behavior.

Related Issue

No existing issue found. This updates the previously opened fixture-isolation PR on a current main base.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✅ Tests (adding or improving test coverage)

Changes Made

  • tests/agent/test_anthropic_output_field_leak.py: remove the host-specific import-path override.
  • tests/agent/test_credential_pool_routing.py: isolate the fixture from host auth discovery and assert the exact credential-pool contents.

How to Test

  1. Run the focused regression tests:
    scripts/run_tests.sh tests/agent/test_anthropic_output_field_leak.py tests/agent/test_credential_pool_routing.py -q
    Result: 21 passed.
  2. Run the cross-platform guard:
    python3 scripts/check-windows-footguns.py --diff origin/main
    Result: No Windows footguns found (2 file(s) scanned).
  3. Run the full suite:
    scripts/run_tests.sh
    The changed modules pass, but the full suite is currently blocked by unrelated failures outside this diff. A targeted rerun of all reported failure files on untouched origin/main reproduced 20 failing tests across 10 files and a 300-second timeout in tests/cli/test_worktree.py. Reproduced upstream failures include Hindsight provider, readiness, systemd socket, worktree, approval, computer-use, file-tools, execution-flag, and voice-mode tests.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass. Blocked by pre-existing upstream failures documented above.
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS 13.7.8

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

Focused test result:

21 passed, 0 failed

The full-suite blocker is documented above with a matching origin/main reproduction.

@alt-glitch alt-glitch added type/test Test coverage or test infrastructure comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have labels Aug 9, 2026
@DanDo385
DanDo385 force-pushed the fix/test-suite-isolation-and-pool-routing branch from 129b464 to f0fe219 Compare August 10, 2026 15:18
@DanDo385 DanDo385 changed the title test(agent): isolate suite fixtures from host Claude/Hermes installs fix(tests): isolate suite fixtures from host auth Aug 10, 2026
@DanDo385

Copy link
Copy Markdown
Author

Unblock request: first-time fork workflow approval

mergeStateStatus=BLOCKED is not a test failure on this diff.

Required check from protect-main:

  • All required checks pass

GitHub Actions never started jobs for this head. Current runs are stuck at action_required with zero jobs:

This is the first-time external contributor workflow approval gate. A maintainer needs to approve the pending workflow runs for this fork PR so CI can execute.

Local verification already completed on current main base:

  • focused: scripts/run_tests.sh tests/agent/test_anthropic_output_field_leak.py tests/agent/test_credential_pool_routing.py -q → 21 passed
  • Windows footguns: clean on the 2-file diff
  • independent Codex review: no findings

Please approve workflows for this PR when convenient. I cannot self-approve them with fork permissions.

@DanDo385

Copy link
Copy Markdown
Author

Maintainer note (companion to #83278)

This remains the focused test-isolation PR only.

Already labeled by triage:

  • type/test
  • comp/agent
  • P3

Local verification:

  • scripts/run_tests.sh tests/agent/test_anthropic_output_field_leak.py tests/agent/test_credential_pool_routing.py -q → 21 passed
  • Windows footguns clean on the 2-file diff
  • Codex review: no findings

Still blocked only by first-time fork workflow approval (no jobs started):

Please approve workflows when convenient. Related auth fix is #83278 (carries forward @alt-glitch triage from superseded #80581).

@teknium1
teknium1 merged commit c28114a into NousResearch:main Aug 13, 2026
44 checks passed
@DanDo385
DanDo385 deleted the fix/test-suite-isolation-and-pool-routing branch August 25, 2026 23:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have type/test Test coverage or test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants