Skip to content

fix(credential-pool): back off repeated exhaustion - #74282

Closed
konsisumer wants to merge 1 commit into
NousResearch:mainfrom
konsisumer:fix/credential-pool-exponential-backoff-15296
Closed

konsisumer wants to merge 1 commit into
NousResearch:mainfrom
konsisumer:fix/credential-pool-exponential-backoff-15296

Conversation

@konsisumer

Copy link
Copy Markdown
Contributor

Fixes #15296

What changed and why

Per the follow-up's lifecycle clarification, credential exhaustion now persists a per-credential failure streak and uses capped exponential cooldowns (for a 429: 1h → 2h → 4h → 8h). Automatic cooldown expiry makes an entry eligible to try again without clearing its streak, so a repeated provider outage no longer falls back to a one-hour retry loop. Genuine credential recovery paths and reset_statuses() clear the streak; malformed persisted values are safely coerced.

Addressing maintainer feedback

No maintainer instruction was posted, but this incorporates the related-work analysis: #15455 (closed) supplied the complete lifecycle behavior, while #43856 (open) was identified as resetting the streak during normal expired-entry selection. This branch preserves the streak across that selection path and resets it only on recovery or an operator reset.

How to test

  • pytest tests/agent/test_credential_pool.py -q -x --timeout=60 — 111 passed.
  • ruff check agent/credential_pool.py tests/agent/test_credential_pool.py — passed.
  • The required full pytest tests/ -q -x --timeout=60 suite reaches collection but cannot run in this sandbox because its default Python lacks the optional fastapi and uvicorn dashboard dependencies.

What platforms tested on

  • macOS (Darwin arm64)

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint area/auth Authentication, OAuth, credential pools P2 Medium — degraded but workaround exists needs-decision Awaiting maintainer decision before any implementation labels Jul 29, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

This was generated by AI during triage.

Related: #43856. Both implement exponential credential-pool exhaustion backoff for #15296, but this patch preserves the failure streak when a cooldown merely expires while #43856 clears it. These are competing lifecycle semantics, not duplicate patches; maintainer selection is needed.

@konsisumer

Copy link
Copy Markdown
Contributor Author

Closing in favour of #43856, same as the earlier #73750 — that PR was opened on 2026-06-11 and changes the same agent/credential_pool.py for the same purpose.

This PR was re-created automatically after I closed the previous one; that was an oversight on my side, not a disagreement with the duplicate call. Suppressing it properly now so it does not come back a third time.

@konsisumer konsisumer closed this Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Credential pool: no exponential backoff on repeated exhaustion — flat TTL causes 429 retry loops

2 participants