Skip to content

test(agent): update stub agents for per-turn fallback-policy plumbing - #291

Merged
OmarB97 merged 1 commit into
mainfrom
fix/stale-stub-agents-fallback-policy
Aug 2, 2026
Merged

test(agent): update stub agents for per-turn fallback-policy plumbing#291
OmarB97 merged 1 commit into
mainfrom
fix/stale-stub-agents-fallback-policy

Conversation

@OmarB97

@OmarB97 OmarB97 commented Aug 2, 2026

Copy link
Copy Markdown
Owner

What does this PR do?

Fixes 11 tests that fail on main at collection/run time with AttributeError.

PRs #269 and #272 added fallback-policy plumbing to the shared turn prologue
(agent/turn_context.py now calls _refresh_fallback_policy() and
_emit_pending_fallback_notice()) and structured status emission to the
fallback activation path (_emit_fallback_status, which reads
self.status_callback directly).

Three test files build agents that deliberately bypass __init__ — either
AIAgent.__new__(AIAgent) or a hand-rolled _FakeAgent — and were never
updated with the new attributes, so they raise AttributeError as soon as the
prologue or the fallback path runs.

This is test-side only. agent/agent_init.py sets status_callback
unconditionally during real construction (in a plain sequential
callback-assignment block, with nothing emitting before it), so no shipped code
path is affected. Verified by reading the init ordering, not just by the tests
going green.

One detail worth flagging for review: in test_compressor_fallback_update.py
the breakage was masked. The missing status_callback raised inside
try_activate_fallback, whose blanket except Exception swallowed it, logged
Failed to activate fallback gpt-4o, and then recursed through the remainder of
the fallback chain before finally failing on the terminal status emit. A status
emission failure being able to walk the whole fallback chain is arguably its own
robustness issue — I have not changed that behavior here, since it is out of
scope for a test fix, but it may deserve a follow-up.

Related Issue

No filed issue — these surfaced once #286 unblocked uv sync --locked and the
Python test slices ran for the first time since 2026-07-10.

Type of Change

  • ✅ Tests (adding or improving test coverage)

Changes Made

  • tests/run_agent/test_compressor_fallback_update.py — set agent.status_callback = None on the __new__-built stub, with a comment explaining the masking behavior above.
  • tests/agent/test_gateway_turn_sidecar.py — add _refresh_fallback_policy() / _emit_pending_fallback_notice() to _FakeAgent.
  • tests/agent/test_api_content_sidecar.py — same two stubs on its _FakeAgent.

Stub shapes match the convention already used by the maintained sibling
tests/agent/test_turn_context.py (which stubs exactly these three methods).

How to Test

Per-file runs — single-process multi-file pytest cross-contaminates in this repo:

pytest tests/run_agent/test_compressor_fallback_update.py tests/agent/test_gateway_turn_sidecar.py tests/agent/test_api_content_sidecar.py -p no:randomly -q

Before: 2 failed / 4 failed, 4 passed / 5 failed, 28 passed.
After: 2 passed / 8 passed / 33 passed.

⚠️ Run with an isolated HERMES_HOME. The suite reads the developer's real
~/.hermes/config.yaml; a local fallback_policy: 'off' makes 5 unrelated
tests in tests/test_tui_gateway_server.py::TestResolveRuntimeWithFallback fail
locally that pass on CI (CI has no user config, so the policy defaults to
any). That is a pre-existing test-isolation gap, not something this PR
changes.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix
  • I've run the affected test files and all tests pass
  • I've added tests for my changes — N/A, this PR repairs existing tests
  • I've tested on my platform: macOS 15 (Darwin 25.6.0), Python 3.11.15

Documentation & Housekeeping

  • I've updated relevant documentation — N/A
  • I've updated cli-config.yaml.example — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md — N/A
  • I've considered cross-platform impact — N/A, test-only stub attributes
  • I've updated tool descriptions/schemas — N/A

PRs #269 and #272 added fallback-policy plumbing to the shared turn
prologue (`_refresh_fallback_policy`, `_emit_pending_fallback_notice`)
and structured status emission to the fallback activation path
(`_emit_fallback_status`, which reads `self.status_callback` directly).

Three test files build agents that deliberately bypass `__init__` —
either `AIAgent.__new__(AIAgent)` or a hand-rolled `_FakeAgent` — and
were never updated with the new attributes, so they raise
AttributeError as soon as the prologue or fallback path runs. This is
test-side only: `agent_init.py` sets `status_callback` unconditionally
during real construction, so no shipped code path is affected.

In `test_compressor_fallback_update.py` the breakage was masked: the
missing `status_callback` raised inside `try_activate_fallback`, whose
`except` swallowed it, logged "Failed to activate fallback", and
recursed through the rest of the chain before failing on the terminal
status emit.

Stubs match the convention already used by tests/agent/test_turn_context.py.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@OmarB97
OmarB97 merged commit 07e3159 into main Aug 2, 2026
22 of 35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant