Repository navigation
feat(resilience): skip CLIProxy-backed combo targets from management health - #14301
Conversation
…health Combo dispatch currently learns CLIProxyAPI account cooldown only after a real 429. Query GET /v0/management/auth-files through a fail-open TTL cache and skip a target before dispatch when every relevant credential is unavailable or rejected for that model family. Keep the skip family-scoped so Claude cooldown never takes Gemini, Codex, or xAI on the same instance out of rotation. Per-model quota rejections are only honored while fresh: an account-level recovery clears stale rejected flags so a healthy credential is not skipped (live 2026-09-21: 68x 200s served while model_quotas carried leftover rejected=true). Management errors, unknown models, and generic OpenAI-compatible connections fail open. Closes cheaper-inference#14206
…ip semantics - Probe exclusively the operator-configured CLIProxy endpoint (CLIPROXYAPI_HOST/PORT or the embedded loopback instance). Connection baseUrl is never a probe target; a cliproxy-shaped baseUrl pointing at a foreign instance fails open to avoid exfiltrating CLIPROXYAPI_MANAGEMENT_KEY or inheriting a foreign snapshot. - Normalize cache keys by host and port (canonicalizing localhost aliases onto 127.0.0.1) so equivalent URL variants share one snapshot without retaining the management secret in the key. - Align protected-priority skip semantics with quota_cutoff so pure management cooldown/quota skips fall through to backup targets while non-quota infra failures halt with 503.
|
Security & correctness hardening (896e406) Reviewing the gate against hostile operator data surfaced three fixes, now on top of the branch:
Deep-mode connections ( New coverage: no-probe-on-foreign-baseUrl, operator-endpoint probe verification, cross-instance fail-open, marker-only and deep-mode trusted reuse, cache-key normalization and localhost canonicalization, and protected-priority fall-through parity. Evidence on the worktree: 55 focused tests green (preflight, execute-target-gates, account-health), |
|
Really well-built feature — the SSRF-safety design (never probing a connection-controlled baseUrl, failing open on any mismatch) and the fail-open/family-isolation logic are exactly right, and the test coverage here is excellent (1300+ test lines). Your CI run is currently red across typecheck/eslint/docs-gates/all 4 unit shards though, wider than the #13866 base-red you flagged. Could you rebase onto the current release/v3.8.51 tip and re-run so we can confirm the reds are only the known #13866 set before merging? |
|
Rebased onto the current |
…y-management-health-sync
|
Re-synced onto the current |
|
Re-homed to |
# Conflicts: # open-sse/services/combo/decisionTrace.ts # src/lib/services/cliproxyAccountHealth.ts # tests/unit/combo/execute-target-gates.test.ts
The singleflight/TTL tests used a 50ms TTL, which expired between the coalesced call and the "served within TTL" call on a loaded runner (fetchCount 2, expected 1). Use a 1s TTL and wait 1.1s for expiry; every assertion is unchanged. Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
f276b62
into
cheaper-inference:release/v3.8.52
Summary
rejectedsignals as authoritative only while their own retry window or the producing account-level cooldown remains active. A recovered account with a stale rejection is routable again. This covers the 2026-09-21 production observation where 68 requests returned HTTP 200 whilemodel_quotasstill carried a stale rejected signal.Related Issues
Validation
Choose the change type and focused loop from the
Contribution Golden Path. The full unit suite,
Vitest, the 60% coverage gate, and the production build all run in CI on this PR (#8329):
npm run lint(full global gate left to CI; scoped ESLint on every changed TypeScript file passed with no findings)Focused validation:
tests/unit/services/cliproxy-management-preflight.test.ts: 25/25 passedtests/unit/combo/execute-target-gates.test.ts: 17/17 passedtests/unit/services/cliproxy-account-health.test.ts: 5/5 passedcheck:mutation-test-coverage -- --strictreports the same 12 pre-existing missing registrations on the unmodified base and this branch; this PR adds no mutation-coverage drift.Tests Added Or Updated
tests/unit/services/cliproxy-management-preflight.test.tstests/unit/combo/execute-target-gates.test.tstests/unit/services/cliproxy-account-health.test.tsCoverage Notes
evaluateExecuteTargetGateswith injected management health and verify skip/proceed behavior before dispatch, including the stale-rejected regression from feat(resilience): consume CLIProxyAPI management account health as a pre-dispatch skip gate #14206.Reviewer Notes
cliproxy_management_health.8317alone is not sufficient without an explicit CLIProxy marker.rejectedflag is not a durable scheduler decision. It blocks only whileobserved_at + Retry-Afteris in the future or the account-level cooldown is still active.release/v3.8.51checkout and this branch (12 entries on both); the PR-specific delta is zero.