Skip to content

\ feat: operator-configurable account rotation\ #6763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
diegosouzapw merged 8 commits into from
Jul 11, 2026
Merged

\ feat: operator-configurable account rotation\ #6763

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
5f24bb4
feat(resilience): operator-configurable account rotation
artickc Jul 10, 2026
9f351bc
docs(env): document configurable account-rotation env vars in ENVIRON…
artickc Jul 10, 2026
17b2098
refactor(rotation): extract rotation gate/context helpers to keep acc…
artickc Jul 10, 2026
30c93cd
chore(changelog): re-sync CHANGELOG.md to release tip (restore lost b…
artickc Jul 10, 2026
9bb218b
test(stryker): register rotation-config test in tap.testFiles for mut…
artickc Jul 10, 2026
d2c0820
Merge remote-tracking branch 'origin/release/v3.8.47' into HEAD
diegosouzapw Jul 10, 2026
f4cea63
test(stryker): register ollama-quota covering tests (drift from #6731…
artickc Jul 10, 2026
6ae6080
Merge branch 'release/v3.8.47' into feat/configurable-account-rotation
diegosouzapw Jul 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2103,3 +2103,24 @@ QUOTA_STORE_DRIVER=sqlite # sqlite | redis
# BIFROST_API_KEY=
# BIFROST_STREAMING_ENABLED=true
# BIFROST_TIMEOUT_MS=30000

# ─────────────────────────────────────────────────────────────────────────────
# Account rotation config (operator-managed; consumed by open-sse/services/rotationConfig.ts)
# Lets a supervising front-end mirror its rotation rules onto the backend's account-fallback
# engine. All optional; defaults preserve the historical behavior.
# ─────────────────────────────────────────────────────────────────────────────
# OMNIROUTE_ROTATION_ENABLED=true
# OMNIROUTE_ROTATION_RATE_LIMIT_RESET_SECONDS=0
# OMNIROUTE_ROTATION_DISABLE_TAG_WITHOUT_RESET=true
# OMNIROUTE_ROTATE_ON_429=true
# OMNIROUTE_ROTATE_429_THRESHOLD=1
# OMNIROUTE_ROTATE_429_WINDOW_SECONDS=120
# OMNIROUTE_ROTATE_ON_500=true
# OMNIROUTE_ROTATE_500_THRESHOLD=1
# OMNIROUTE_ROTATE_500_WINDOW_SECONDS=120
# OMNIROUTE_ROTATE_ON_502=true
# OMNIROUTE_ROTATE_502_THRESHOLD=1
# OMNIROUTE_ROTATE_502_WINDOW_SECONDS=120
# OMNIROUTE_ROTATE_ON_400=false
# OMNIROUTE_ROTATE_400_THRESHOLD=1
# OMNIROUTE_ROTATE_400_WINDOW_SECONDS=120
1 change: 1 addition & 0 deletions changelog.d/features/6763-operator-configurable-account-rotation.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **feat(resilience):** operator-configurable account rotation policy — a new `rotationConfig` layer lets operators tune how connections rotate on failure, wired into `accountFallback` (#6763 — thanks @artickc).
15 changes: 15 additions & 0 deletions docs/reference/ENVIRONMENT.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1094,6 +1094,21 @@ Provider quota endpoints, network tunnels (Tailscale, Ngrok, MITM debug proxy),
| `QDRANT_EMBEDDING_MODEL` | `text-embedding-3-small` | _(opt-in cluster profile)_ | Default embedding model name recorded in the Qdrant collection metadata. Actual embeddings are generated by whatever provider the `embeddingModel` field in OmniRoute's settings points to. |
| `QDRANT_VECTOR_SIZE` | `1536` | _(opt-in cluster profile)_ | Embedding vector dimension. Must match the model you embed with (text-embedding-3-small → 1536; ada-002 → 1536; nomic-embed-text → 768). |
| `QDRANT_HNSW_EF_CONSTRUCT` | `128` | _(opt-in cluster profile)_ | HNSW index construction-time accuracy. Higher = slower build, faster search. |
| `OMNIROUTE_ROTATION_ENABLED` | `true` | `open-sse/services/rotationConfig.ts` | Master switch for operator-configurable account rotation. When `false`, none of the `OMNIROUTE_ROTATE_*` classes below trigger account fallback (the master-off state also blocks the default-enabled 429/500/502 classes). Lets a supervising front-end (e.g. the VibeProxy desktop app) mirror its own rotation rules onto the backend's account-fallback engine. |
| `OMNIROUTE_ROTATION_RATE_LIMIT_RESET_SECONDS` | `0` | `open-sse/services/rotationConfig.ts` | Cooldown (seconds) applied to a rate-limited account when the upstream gives no explicit reset hint. `0` = use the engine default cooldown instead of a fixed override. |
| `OMNIROUTE_ROTATION_DISABLE_TAG_WITHOUT_RESET` | `true` | `open-sse/services/rotationConfig.ts` | Mirror of the front-end "don't tag as rate-limited without a reset time" preference. |
| `OMNIROUTE_ROTATE_ON_429` | `true` | `open-sse/services/rotationConfig.ts` | Per-status fallback enable for `429` errors. When `false` (and `OMNIROUTE_ROTATION_ENABLED=true`), a `429` no longer triggers account rotation and is returned to the client instead. |
| `OMNIROUTE_ROTATE_429_THRESHOLD` | `1` | `open-sse/services/rotationConfig.ts` | Number of `429` errors within `OMNIROUTE_ROTATE_429_WINDOW_SECONDS` required before the account is rotated. `1` (default) rotates immediately, preserving historical behavior. |
| `OMNIROUTE_ROTATE_429_WINDOW_SECONDS` | `120` | `open-sse/services/rotationConfig.ts` | Sliding window (seconds) over which `429` errors are counted toward `OMNIROUTE_ROTATE_429_THRESHOLD`. |
| `OMNIROUTE_ROTATE_ON_500` | `true` | `open-sse/services/rotationConfig.ts` | Per-status fallback enable for `5xx` server errors (excluding `502`, which has its own class). When `false`, these errors no longer trigger account rotation. |
| `OMNIROUTE_ROTATE_500_THRESHOLD` | `1` | `open-sse/services/rotationConfig.ts` | Number of `5xx` errors within `OMNIROUTE_ROTATE_500_WINDOW_SECONDS` required before the account is rotated. `1` (default) rotates immediately. |
| `OMNIROUTE_ROTATE_500_WINDOW_SECONDS` | `120` | `open-sse/services/rotationConfig.ts` | Sliding window (seconds) over which `5xx` errors are counted toward `OMNIROUTE_ROTATE_500_THRESHOLD`. |
| `OMNIROUTE_ROTATE_ON_502` | `true` | `open-sse/services/rotationConfig.ts` | Per-status fallback enable for `502` (bad gateway) errors. When `false`, `502`s no longer trigger account rotation. |
| `OMNIROUTE_ROTATE_502_THRESHOLD` | `1` | `open-sse/services/rotationConfig.ts` | Number of `502` errors within `OMNIROUTE_ROTATE_502_WINDOW_SECONDS` required before the account is rotated. `1` (default) rotates immediately. |
| `OMNIROUTE_ROTATE_502_WINDOW_SECONDS` | `120` | `open-sse/services/rotationConfig.ts` | Sliding window (seconds) over which `502` errors are counted toward `OMNIROUTE_ROTATE_502_THRESHOLD`. |
| `OMNIROUTE_ROTATE_ON_400` | `false` | `open-sse/services/rotationConfig.ts` | Opt-in (default OFF): when `true`, a plain `400` (bad request) also triggers account rotation. This is additive only — it never blocks the engine's existing behavior where a `400` carrying rate-limit/quota text still falls over regardless of this flag. |
| `OMNIROUTE_ROTATE_400_THRESHOLD` | `1` | `open-sse/services/rotationConfig.ts` | Number of `400` errors within `OMNIROUTE_ROTATE_400_WINDOW_SECONDS` required before the account is rotated (only consulted when `OMNIROUTE_ROTATE_ON_400=true`). |
| `OMNIROUTE_ROTATE_400_WINDOW_SECONDS` | `120` | `open-sse/services/rotationConfig.ts` | Sliding window (seconds) over which `400` errors are counted toward `OMNIROUTE_ROTATE_400_THRESHOLD`. |

---

Expand Down
38 changes: 38 additions & 0 deletions open-sse/config/errorConfig.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,3 +199,41 @@ export function matchErrorRuleByStatus(statusCode: number): ErrorRule | null {
export function findMatchingErrorRule(statusCode: number, message: unknown): ErrorRule | null {
return matchErrorRuleByText(message) || matchErrorRuleByStatus(statusCode);
}

export interface ServiceSupervisorCooldown {
shouldFallback: true;
cooldownMs: number;
baseCooldownMs: number;
newBackoffLevel: 0;
reason: string;
skipProviderBreaker: true;
}

/**
* G-02: detect embedded service supervisor failures (X-Omni-Fallback-Hint: connection_cooldown).
* These are NOT upstream AI provider failures — they are local supervisor state changes. Returns
* a short 5s connection-cooldown decision (no provider circuit-breaker trip), or null when the
* status/header don't match.
*/
export function serviceSupervisorCooldown(
status: number,
headers: Headers | Record<string, string> | null
): ServiceSupervisorCooldown | null {
if (status !== 503 || !headers) return null;
const hintValue =
typeof (headers as Headers).get === "function"
? (headers as Headers).get("x-omni-fallback-hint")
: (headers as Record<string, string>)["x-omni-fallback-hint"] ||
(headers as Record<string, string>)["X-Omni-Fallback-Hint"];
if (typeof hintValue !== "string" || hintValue.toLowerCase() !== "connection_cooldown") {
return null;
}
return {
shouldFallback: true,
cooldownMs: 5_000,
baseCooldownMs: 5_000,
newBackoffLevel: 0,
reason: "service_not_running",
skipProviderBreaker: true,
};
}
46 changes: 18 additions & 28 deletions open-sse/services/accountFallback.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,10 @@ import {
findMatchingErrorRule,
matchErrorRuleByText,
matchErrorRuleByStatus,
serviceSupervisorCooldown,
} from "../config/errorConfig.ts";
import { getProviderErrorRuleMatch } from "../config/providerErrorRules.ts";
import * as rot from "./rotationConfig.ts";
import { getPassthroughProviders, getProviderCategory } from "../config/providerRegistry.ts";
import {
DEFAULT_RESILIENCE_SETTINGS,
Expand Down Expand Up @@ -1267,7 +1269,8 @@ export function checkFallbackError(
provider: string | null = null,
headers: Headers | Record<string, string> | null = null,
profileOverride: ProviderProfile | null = null,
structuredError?: { code?: string | null; type?: string | null } | null
structuredError?: { code?: string | null; type?: string | null } | null,
rotation?: { account?: unknown } | null
): {
shouldFallback: boolean;
cooldownMs: number;
Expand All @@ -1287,27 +1290,10 @@ export function checkFallbackError(
* caller can persist an explicit reset window instead of the engine's scaled cooldown. */
configuredCooldownMs?: number;
} {
// G-02: detect embedded service supervisor failures (X-Omni-Fallback-Hint: connection_cooldown).
// These are NOT upstream AI provider failures — they are local supervisor state changes.
// Apply a short 5s connection cooldown without tripping the provider circuit breaker.
if (status === 503 && headers) {
const hintValue =
typeof (headers as Headers).get === "function"
? (headers as Headers).get("x-omni-fallback-hint")
: (headers as Record<string, string>)["x-omni-fallback-hint"] ||
(headers as Record<string, string>)["X-Omni-Fallback-Hint"];
if (typeof hintValue === "string" && hintValue.toLowerCase() === "connection_cooldown") {
return {
shouldFallback: true,
cooldownMs: 5_000,
baseCooldownMs: 5_000,
newBackoffLevel: 0,
reason: "service_not_running",
skipProviderBreaker: true,
};
}
}

const svc = serviceSupervisorCooldown(status, headers);
if (svc) return svc;
const rg = rot.gateFor(status, rotation?.account);
if (rg) return rg;
const errorStr = (errorText || "").toString();
const profile = profileOverride ?? (provider ? getProviderProfile(provider) : null);
const maxBackoffSteps = profile?.maxBackoffSteps ?? BACKOFF_CONFIG.maxLevel;
Expand Down Expand Up @@ -1396,6 +1382,8 @@ export function checkFallbackError(
};
}

const ro = rot.overrideFor(reason, rotation?.account);
if (ro) return ro;
const scaled = getScaledBaseCooldown(reason, backoffLevel);
return {
shouldFallback: true,
Expand Down Expand Up @@ -1761,13 +1749,15 @@ export function resetAccountState<T extends AccountState | null | undefined>(
export function applyErrorState<T extends AccountState | null | undefined>(
account: T,
status: number,
errorText: string | null,
provider: string | null = null
errText: string | null,
prov: string | null = null
): T | AccountState {
if (!account) return account;

const backoffLevel = account.backoffLevel || 0;
const fallbackDecision = checkFallbackError(status, errorText, backoffLevel, null, provider);
const lvl = account.backoffLevel || 0;
const fallbackDecision = checkFallbackError(status, errText, lvl, null, prov, null, null, null, {
account,
});
const { cooldownMs, reason } = fallbackDecision;
const newBackoffLevel =
"newBackoffLevel" in fallbackDecision ? fallbackDecision.newBackoffLevel : undefined;
Expand All @@ -1789,8 +1779,8 @@ export function applyErrorState<T extends AccountState | null | undefined>(
const nextState: T | AccountState = {
...account,
rateLimitedUntil: effectiveCooldownMs > 0 ? getUnavailableUntil(effectiveCooldownMs) : null,
backoffLevel: newBackoffLevel ?? backoffLevel,
lastError: { status, message: errorText, timestamp: new Date().toISOString(), reason },
backoffLevel: newBackoffLevel ?? lvl,
lastError: { status, message: errText, timestamp: new Date().toISOString(), reason },
status: "error",
};

Expand Down
Loading
Loading