Skip to content

feat(resilience): skip CLIProxy-backed combo targets from management health - #14301

Merged
diegosouzapw merged 7 commits into
cheaper-inference:release/v3.8.52from
sprintberlin:feat/cliproxy-management-health-sync
Oct 8, 2026
Merged

diegosouzapw merged 7 commits into
cheaper-inference:release/v3.8.52from
sprintberlin:feat/cliproxy-management-health-sync

Conversation

@sprintberlin

Copy link
Copy Markdown
Contributor

Summary

  • Add a pre-dispatch management-health gate for demonstrably CLIProxyAPI-backed combo targets.
  • Keep decisions model- and backend-family-scoped so Claude cooldowns do not suppress healthy Gemini/Antigravity, Codex, or xAI targets on the same proxy.
  • Use a bounded fail-open health read with a 45-second TTL and singleflight coalescing.
  • Treat per-model rejected signals as authoritative only while their own retry window or the producing account-level cooldown remains active. A recovered account with a stale rejection is routable again. This covers the 2026-09-21 production observation where 68 requests returned HTTP 200 while model_quotas still carried a stale rejected signal.
  • Project and cache only sanitized health fields; management credentials, paths, account identifiers, emails, and raw payloads never reach diagnostics.

Related Issues

Validation

Choose the change type and focused loop from the
Contribution Golden Path. The full unit suite,
Vitest, the 60% coverage gate, and the production build all run in CI on this PR (#8329):

  • Change type: routing / resilience
  • Focused tests and category gates from the golden path
  • npm run lint (full global gate left to CI; scoped ESLint on every changed TypeScript file passed with no findings)
  • Reconciled with the current active release base; focused checks rerun afterward
  • Production-code changes include new or updated automated tests in this PR
  • SonarQube is temporarily opt-in while the private project has no quota; it is not a PR gate.

Focused validation:

  • tests/unit/services/cliproxy-management-preflight.test.ts: 25/25 passed
  • tests/unit/combo/execute-target-gates.test.ts: 17/17 passed
  • tests/unit/services/cliproxy-account-health.test.ts: 5/5 passed
  • Prettier passed on all changed files.
  • Scoped ESLint passed on all changed TypeScript files.
  • Commit hooks passed, including documentation sync, tracked-artifact checks, and explicit-any budgets.
  • check:mutation-test-coverage -- --strict reports the same 12 pre-existing missing registrations on the unmodified base and this branch; this PR adds no mutation-coverage drift.

Tests Added Or Updated

  • tests/unit/services/cliproxy-management-preflight.test.ts
  • tests/unit/combo/execute-target-gates.test.ts
  • tests/unit/services/cliproxy-account-health.test.ts

Coverage Notes

  • Pure decision tests cover family isolation, unanimous blocking, model-level rejection, stale-rejection recovery, elapsed retry windows, fresh sibling rejections, unknown families, empty families, TTL, singleflight, and fail-open management states.
  • Gate-level tests drive evaluateExecuteTargetGates with injected management health and verify skip/proceed behavior before dispatch, including the stale-rejected regression from feat(resilience): consume CLIProxyAPI management account health as a pre-dispatch skip gate #14206.
  • Account-health tests cover the allowlisted projection and prove sensitive fields and values are discarded.
  • No touched-file coverage was intentionally reduced.

Reviewer Notes

  • The gate is intentionally placed immediately before the credential gate so a management-health skip follows the existing combo fallback path and is recorded as cliproxy_management_health.
  • Strict opt-in prevents generic OpenAI-compatible connections from making management requests. Port 8317 alone is not sufficient without an explicit CLIProxy marker.
  • Management failures, malformed responses, unknown model families, and absent family accounts all fail open.
  • A bare stale rejected flag is not a durable scheduler decision. It blocks only while observed_at + Retry-After is in the future or the account-level cooldown is still active.
  • ⚠️ base-red inherited: 🔴 Release branch not green: release/v3.8.51 #13866. The mutation-coverage drift is identical on the clean release/v3.8.51 checkout and this branch (12 entries on both); the PR-specific delta is zero.
  • No production deployment or live patch is included.

…health

Combo dispatch currently learns CLIProxyAPI account cooldown only after a
real 429. Query GET /v0/management/auth-files through a fail-open TTL cache
and skip a target before dispatch when every relevant credential is
unavailable or rejected for that model family.

Keep the skip family-scoped so Claude cooldown never takes Gemini, Codex, or
xAI on the same instance out of rotation. Per-model quota rejections are only
honored while fresh: an account-level recovery clears stale rejected flags so
a healthy credential is not skipped (live 2026-09-21: 68x 200s served while
model_quotas carried leftover rejected=true).

Management errors, unknown models, and generic OpenAI-compatible
connections fail open.

Closes cheaper-inference#14206
…ip semantics

- Probe exclusively the operator-configured CLIProxy endpoint (CLIPROXYAPI_HOST/PORT
  or the embedded loopback instance). Connection baseUrl is never a probe target;
  a cliproxy-shaped baseUrl pointing at a foreign instance fails open to avoid
  exfiltrating CLIPROXYAPI_MANAGEMENT_KEY or inheriting a foreign snapshot.
- Normalize cache keys by host and port (canonicalizing localhost aliases onto
  127.0.0.1) so equivalent URL variants share one snapshot without retaining
  the management secret in the key.
- Align protected-priority skip semantics with quota_cutoff so pure management
  cooldown/quota skips fall through to backup targets while non-quota infra
  failures halt with 503.
@sprintberlin

Copy link
Copy Markdown
Contributor Author

Security & correctness hardening (896e406)

Reviewing the gate against hostile operator data surfaced three fixes, now on top of the branch:

  • SSRF / secret-exfiltration guard. Management probes now target only the operator-configured CLIProxy endpoint (CLIPROXYAPI_HOST/CLIPROXYAPI_PORT, or the embedded loopback instance). A connection baseUrl is never used as a probe target. A cliproxy-shaped baseUrl that resolves to a different host:port fails open, so CLIPROXYAPI_MANAGEMENT_KEY cannot travel to a connection-controlled host and a foreign instance's snapshot cannot decide this instance's targets. A connection whose whole family is blocked on the trusted instance still skips normally.
  • Cache-key normalization. Keys are now host:port::credentialed|default with localhost/::1 canonicalized onto 127.0.0.1, so path/case/trailing-slash/alias URL variants share one snapshot and singleflight actually coalesces them. The management secret itself is never part of a key.
  • Protected-priority parity with quota_cutoff. The skip now goes through observeFailure(true, ...) and the same trust check the quota-cutoff gate uses: pure management cooldown/quota evidence lets a protected-priority target fall through to its backup chain instead of terminating the combo with 503.

Deep-mode connections (cliproxyapiMode: "claude-native") and marker-only connections deliberately keep reusing the trusted endpoint: the executor routes them through CLIPROXYAPI_HOST/cliproxyapi_url regardless of their native provider baseUrl, so the snapshot matches where their traffic actually goes.

New coverage: no-probe-on-foreign-baseUrl, operator-endpoint probe verification, cross-instance fail-open, marker-only and deep-mode trusted reuse, cache-key normalization and localhost canonicalization, and protected-priority fall-through parity.

Evidence on the worktree: 55 focused tests green (preflight, execute-target-gates, account-health), typecheck:core exit 0, complexity/cognitive ratchets green, prettier clean.

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Really well-built feature — the SSRF-safety design (never probing a connection-controlled baseUrl, failing open on any mismatch) and the fail-open/family-isolation logic are exactly right, and the test coverage here is excellent (1300+ test lines). Your CI run is currently red across typecheck/eslint/docs-gates/all 4 unit shards though, wider than the #13866 base-red you flagged. Could you rebase onto the current release/v3.8.51 tip and re-run so we can confirm the reds are only the known #13866 set before merging?

@sprintberlin

Copy link
Copy Markdown
Contributor Author

Rebased onto the current release/v3.8.51 tip (893fef9). All 55 preflight/gate tests pass locally alongside typecheck:core and the complexity ratchets (2904 / baseline 3218). Verified that the remaining CI failures match the known #13866 base-red set (rerankProviderNodes.ts/antigravity.ts TS2677, missing .env.example keys, and skill drift).

@sprintberlin

Copy link
Copy Markdown
Contributor Author

Re-synced onto the current release/v3.8.51 tip after #14290 landed (34113170). Local preflight/gate tests still pass (55/55), typecheck:core is clean, and ESLint on the PR-owned files is clean. Remaining CI failures continue to match the known #13866 base-red set (rerankProviderNodes.ts/antigravity.ts TS2677, omni-version-manager skill drift, missing .env.example keys, and ESLint in untouched files).

@diegosouzapw diegosouzapw changed the title feat(resilience): skip CLIProxy-backed combo targets from management health [defer] feat(resilience): skip CLIProxy-backed combo targets from management health Sep 25, 2026
@diegosouzapw diegosouzapw added the deferred-v3.8.52 Grande demais / suspeito para o lote atual; precisa de sessão dedicada no ciclo v3.8.52 label Sep 25, 2026
@diegosouzapw
diegosouzapw changed the base branch from release/v3.8.51 to release/v3.8.52 September 29, 2026 11:22
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Re-homed to release/v3.8.52: v3.8.51 entered its release freeze, so the branch now belongs to the release captain and development continues on the next cycle. Nothing is wrong with this PR — it just needed a live base. No action needed from you; CI will re-run against the new base.

@diegosouzapw diegosouzapw changed the title [defer] feat(resilience): skip CLIProxy-backed combo targets from management health feat(resilience): skip CLIProxy-backed combo targets from management health Oct 1, 2026
@diegosouzapw diegosouzapw removed the deferred-v3.8.52 Grande demais / suspeito para o lote atual; precisa de sessão dedicada no ciclo v3.8.52 label Oct 1, 2026
diegosouzapw and others added 2 commits October 7, 2026 23:38
# Conflicts:
#	open-sse/services/combo/decisionTrace.ts
#	src/lib/services/cliproxyAccountHealth.ts
#	tests/unit/combo/execute-target-gates.test.ts
The singleflight/TTL tests used a 50ms TTL, which expired between the
coalesced call and the "served within TTL" call on a loaded runner
(fetchCount 2, expected 1). Use a 1s TTL and wait 1.1s for expiry; every
assertion is unchanged.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw
diegosouzapw merged commit f276b62 into cheaper-inference:release/v3.8.52 Oct 8, 2026
21 of 41 checks passed
@diegosouzapw diegosouzapw mentioned this pull request Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(resilience): consume CLIProxyAPI management account health as a pre-dispatch skip gate

2 participants