Skip to content

fix(run-agent): rotate credential pool on billing-classified 400s - #6940

Closed
helix4u wants to merge 1 commit into
NousResearch:mainfrom
helix4u:fix/credential-pool-billing-400-rotation
Closed

fix(run-agent): rotate credential pool on billing-classified 400s#6940
helix4u wants to merge 1 commit into
NousResearch:mainfrom
helix4u:fix/credential-pool-billing-400-rotation

Conversation

@helix4u

@helix4u helix4u commented Apr 10, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes a credential-pool recovery gap where billing-style HTTP 400 errors, such as Anthropic out of extra usage, were correctly classified as rotation-worthy but still aborted instead of rotating to the next credential.

The core issue was that classify_api_error could mark billing-style 400s as should_rotate_credential true, while pool recovery in run_agent.py still only keyed off raw status codes 402, 429, and 401. This PR lets credential-pool recovery honor the classified failover reason, so billing, rate-limit, and auth recovery still work when a provider reports the condition under a different raw status code.

Related Issue

No linked upstream issue yet.

Type of Change

  • Bug fix
  • Tests

Changes Made

  • Updated run_agent.py so _recover_with_credential_pool accepts the classified failover reason and uses that to choose billing, rate-limit, or auth recovery behavior.
  • Preserved existing pool semantics: billing rotates immediately, rate limits retry once then rotate, auth failures try refresh then rotate.
  • Added a regression test in tests/run_agent/test_run_agent.py covering a billing-classified HTTP 400 that should rotate the credential pool.

How to Test

  1. Configure a same-provider credential pool with at least two Anthropic-style credentials.
  2. Force one credential to return a billing-style HTTP 400 such as out of extra usage.
  3. Confirm Hermes rotates to the next credential in the pool instead of aborting immediately.

Focused automated checks run for this PR:

  1. python -m pytest tests/run_agent/test_run_agent.py -k recover_with_pool -q
  2. python -m pytest tests/agent/test_credential_pool_routing.py -q

Observed focused results:

  • recover_with_pool slice: 7 passed
  • credential_pool_routing file: 14 passed

Full suite status from a clean single-process run:

  • python -m pytest tests/ -q -n 0
  • Result: 25 failed, 9799 passed, 34 skipped, 50 deselected, 1 xpassed

The full-suite failures appear unrelated to this credential-pool fix and are spread across other areas such as CLI defaults, cron scheduler delivery, gateway approvals, Discord bot filtering, browser provider handling, and tirith security tests.

Checklist

  • Read CONTRIBUTING.md
  • Used a conventional commit message
  • PR contains only changes related to this fix
  • Full pytest suite is green
  • Added tests for the bug fix
  • Considered cross-platform impact

@helix4u
helix4u marked this pull request as ready for review April 10, 2026 04:09
@teknium1

Copy link
Copy Markdown
Contributor

Merged via PR #7112. Your commit was cherry-picked onto current main with your authorship preserved in git log. Thanks for the fix @helix4u!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants