Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
## Project

Unified AI proxy/router — route any LLM through one endpoint. Multi-provider support
with **227 provider entries** (OpenAI, Anthropic, Gemini, DeepSeek, Groq, xAI, Mistral, Fireworks,
with **231 provider entries** (OpenAI, Anthropic, Gemini, DeepSeek, Groq, xAI, Mistral, Fireworks,
Cohere, NVIDIA, Cerebras, Pollinations, Puter, Cloudflare AI, HuggingFace, DeepInfra,
SambaNova, Meta Llama API, Moonshot AI, AI21 Labs, Databricks, Snowflake, and many more)
with **MCP Server** (87 tools), **A2A v0.3 Protocol**, and **Electron desktop app**.

> **Live counts (v3.8.24)**: providers 227 · MCP tools 87 · MCP scopes 30 · A2A skills 6 ·
> **Live counts (v3.8.31)**: providers 231 · MCP tools 87 · MCP scopes 30 · A2A skills 6 ·
> open-sse services 115 · routing strategies 15 · auto-combo scoring factors 9 ·
> DB modules 83 · DB migrations 97 · base tables 17 · search providers 11 ·
> i18n locales 42. **Refresh with `npm run check:docs-all`.**
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ _In development — bullets added per PR; finalized at release._

- **fix(mitm): exact host membership in the MITM hosts test (CodeQL false positive)** — `tests/unit/mitm-tool-hosts.test.ts` checked host membership with `Array.includes(host)`, which CodeQL's `js/incomplete-url-substring-sanitization` heuristic misreads as a `String.includes()` URL-substring sanitization test (HIGH false positive). Switched to `.some((h) => h === host)` — identical semantics, no flagged pattern. ([#4386](https://github.com/diegosouzapw/OmniRoute/pull/4386))

### 📝 Maintenance

- **docs: one-time feature-documentation catch-up (v3.8.20 → v3.8.30)** — reconciled the docs with every user-facing feature shipped since v3.8.20: a new README **✨ What's New** section; new guides for [CLI integrations](docs/guides/CLI-INTEGRATIONS.md), [MITM TPROXY transparent decrypt](docs/security/MITM-TPROXY-DECRYPT.md) and [delegated Anthropic Context Editing](docs/compression/CONTEXT_EDITING.md); refreshed AUTO-COMBO (`auto/<category>:<tier>` + Arena-ELO), API_REFERENCE (`x-omniroute-no-memory`), MEMORY (int8 quantization, off-by-default), RESILIENCE (model-lockout success-decay), RTK, AGENTBRIDGE, TRAFFIC_INSPECTOR, GUARDRAILS, CLOUD_AGENT, ENVIRONMENT; regenerated PROVIDER_REFERENCE (231 providers) and synced the provider count in README/CLAUDE/AGENTS. Going forward this runs every release (generate-release step 6b). ([#4391](https://github.com/diegosouzapw/OmniRoute/pull/4391))

---

## [3.8.30] — 2026-06-20
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ For full test matrix, see `CONTRIBUTING.md` → "Running Tests". For deep archit

## Project at a Glance

**OmniRoute** — unified AI proxy/router. One endpoint, 227 LLM providers, auto-fallback.
**OmniRoute** — unified AI proxy/router. One endpoint, 231 LLM providers, auto-fallback.

| Layer | Location | Purpose |
| ------------- | ----------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
Expand Down
40 changes: 30 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

# 🚀 OmniRoute — The Free AI Gateway

### Never stop coding. Connect every AI tool to **227 providers** — **50+ free** — through one endpoint.
### Never stop coding. Connect every AI tool to **231 providers** — **50+ free** — through one endpoint.

**Plug Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini. Auto-fallback.**
<br/>
Expand All @@ -19,8 +19,8 @@

<br/>

[![227 AI Providers](https://img.shields.io/badge/227-AI_Providers-6C5CE7?style=for-the-badge)](#-227-ai-providers--50-free)
[![50+ Free](https://img.shields.io/badge/50%2B-Free_Tiers-00B894?style=for-the-badge)](#-227-ai-providers--50-free)
[![231 AI Providers](https://img.shields.io/badge/231-AI_Providers-6C5CE7?style=for-the-badge)](#-231-ai-providers--50-free)
[![50+ Free](https://img.shields.io/badge/50%2B-Free_Tiers-00B894?style=for-the-badge)](#-231-ai-providers--50-free)
[![1.6B Free Tokens/mo](https://img.shields.io/badge/1.6B-Free_Tokens%2Fmo-00B894?style=for-the-badge)](docs/reference/FREE_TIERS.md)
[![Token Savings](https://img.shields.io/badge/up_to_95%25-Token_Savings-E17055?style=for-the-badge)](#%EF%B8%8F-save-1595-tokens--automatically)
[![15 Strategies](https://img.shields.io/badge/15-Routing_Strategies-0984E3?style=for-the-badge)](#-combos--the-flagship)
Expand Down Expand Up @@ -59,7 +59,7 @@

<br/>

[**🚀 Quick Start**](#-quick-start) • [**🎯 Combos**](#-combos--the-flagship) • [**🌐 Providers**](#-227-ai-providers--50-free) • [**🔌 CLI & MCP**](#-full-cli--a2a--mcp) • [**🗜️ Compression**](#%EF%B8%8F-save-1595-tokens--automatically) • [**🌍 Website**](https://omniroute.online)
[**🚀 Quick Start**](#-quick-start) • [**🎯 Combos**](#-combos--the-flagship) • [**🌐 Providers**](#-231-ai-providers--50-free) • [**🔌 CLI & MCP**](#-full-cli--a2a--mcp) • [**🗜️ Compression**](#%EF%B8%8F-save-1595-tokens--automatically) • [**🌍 Website**](https://omniroute.online)

[💥 The Promise](#-the-promise) • [🤔 Why](#-why-omniroute) • [🏆 What Sets Apart](#-what-sets-omniroute-apart) • [🤖 Compatible CLIs](#-compatible-clis--coding-agents) • [🖥️ Where It Runs](#%EF%B8%8F-where-omniroute-runs--anywhere) • [🔒 Private](#-private--local-first) • [🎬 In Action](#-omniroute-in-action) • [📚 Explore More](#-explore-more) • [📧 Support](#-support--community)

Expand Down Expand Up @@ -137,11 +137,11 @@

</div>

> One endpoint. **227 providers.** Never stop building — and let OmniRoute pick the cheapest one that works.
> One endpoint. **231 providers.** Never stop building — and let OmniRoute pick the cheapest one that works.

<table>
<tr>
<td width="33%" valign="top"><b>🚫 Never hit limits</b><br/><sub>Auto-fallback across 227 providers in milliseconds. Quota out? Next provider takes over — zero downtime.</sub></td>
<td width="33%" valign="top"><b>🚫 Never hit limits</b><br/><sub>Auto-fallback across 231 providers in milliseconds. Quota out? Next provider takes over — zero downtime.</sub></td>
<td width="33%" valign="top"><b>💸 Save up to 95% tokens</b><br/><sub>RTK + Caveman stacked compression cuts 15–95% of eligible tokens (~89% avg on tool-heavy sessions).</sub></td>
<td width="33%" valign="top"><b>🆓 $0 to start</b><br/><sub>50+ providers with a free tier, 11 free <i>forever</i> (Kiro, Qoder, Pollinations, LongCat…). No card needed.</sub></td>
</tr>
Expand Down Expand Up @@ -264,7 +264,7 @@ Result: 4 layers of fallback = zero downtime

| Feature | OmniRoute | Other routers |
| -------------------------------------- | ----------------------------------------------------------- | ------------- |
| 🌐 Providers | **227** | 20–100 |
| 🌐 Providers | **231** | 20–100 |
| 🆓 Free providers | **50+ (11 free forever)** | 1–5 |
| 🔀 Routing strategies | **15** (priority, weighted, cost-optimized, context-relay…) | 1–3 |
| 🗜️ Token compression | **RTK + Caveman stacked (15–95%)** | None / 20–40% |
Expand All @@ -283,6 +283,26 @@ Result: 4 layers of fallback = zero downtime

<div align="center">

# ✨ What's New

</div>

> Recent highlights from **v3.8.20 → v3.8.30**. Full history in [`CHANGELOG.md`](CHANGELOG.md).

- **🤖 One-command CLI/agent setup** — a dedicated `setup-*` command configures each coding tool to route through OmniRoute (Claude Code, Codex, Cline, Continue, Cursor, Roo Code, Kilo Code, Crush, Goose, Qwen Code, Aider, OpenCode, Gemini CLI); `omniroute launch` / `omniroute launch-codex` are zero-config launchers. → [CLI Integrations](docs/guides/CLI-INTEGRATIONS.md)
- **🛰️ Remote mode** — drive a remote OmniRoute from any machine with scoped access tokens (`omniroute connect` / `omniroute contexts` / `omniroute tokens`). → [Remote Mode](docs/guides/REMOTE-MODE.md)
- **🧭 Smarter auto-routing** — OpenRouter-style `auto/<category>:<tier>` combos (e.g. `auto/coding:fast`, `auto/reasoning:pro`), live Arena-ELO + models.dev model intelligence, and per-step account allowlists. → [Auto-Combo](docs/routing/AUTO-COMBO.md)
- **🗜️ Pluggable compression** — an async compression pipeline with Compression Studios, a stable LLMLingua-2 ONNX engine, RTK, and delegated Anthropic Context Editing. → [Compression](docs/compression/COMPRESSION_ENGINES.md)
- **🕵️ Transparent MITM decrypt (TPROXY)** — capture & translate traffic from CLIs that ignore proxy env vars, with a per-SNI certificate authority and a trust-store installer. → [MITM/TPROXY](docs/security/MITM-TPROXY-DECRYPT.md)
- **💸 Cost telemetry everywhere** — `X-OmniRoute-*` cost/usage headers on every endpoint (including media), a non-token cost engine, a cache-HIT `X-OmniRoute-Cost-Saved` header, and per-key USD spend quotas. → [API Reference](docs/reference/API_REFERENCE.md)
- **🧠 Memory you control** — opt-in int8 vector quantization (Qdrant + sqlite-vec), memory off by default, and a per-request `x-omniroute-no-memory` header. → [Memory](docs/frameworks/MEMORY.md)
- **🛡️ Security** — a prompt-injection guard across every LLM route (backed by a red-team suite), plus a free DuckDuckGo last-resort web search. → [Guardrails](docs/security/GUARDRAILS.md)
- **🤝 More providers & agents** — Cursor Cloud Agent (a 4th cloud agent), a refreshed 231-provider catalog (OrcaRouter, Wafer AI, OpenAdapter, dit.ai, TokenRouter, …), and Vertex AI media generation (speech / transcription / music / video). → [Providers](docs/reference/PROVIDER_REFERENCE.md)

<br/>

<div align="center">

# 🤖 Compatible CLIs & Coding Agents

> One config — `http://localhost:20128/v1` — and **every** AI IDE or CLI runs on free & low-cost models.
Expand Down Expand Up @@ -320,11 +340,11 @@ Result: 4 layers of fallback = zero downtime

<div align="center">

# 🌐 227 AI Providers — 50+ Free
# 🌐 231 AI Providers — 50+ Free

</div>

> The most complete catalog of any open-source router: **227 providers**, **50+ with a free tier**, **11 free forever**.
> The most complete catalog of any open-source router: **231 providers**, **50+ with a free tier**, **11 free forever**.

<div align="center">

Expand Down Expand Up @@ -767,7 +787,7 @@ Compression: aggressive (~50%) → double your free quota · Cost: $0/mo
**Will I be charged by OmniRoute?** No — it's free, open-source software on your machine. You only pay paid providers directly. OmniRoute has no billing system.
**Are FREE providers really unlimited?** Yes — Kiro, Qoder, Pollinations, LongCat, Cloudflare. No catch.
**Will compression hurt quality?** No — it only compresses the **input**; code, URLs, JSON are always protected.
**Does it work where AI is blocked?** Yes — 3-level proxy + 1proxy marketplace reach all 227 providers.
**Does it work where AI is blocked?** Yes — 3-level proxy + 1proxy marketplace reach all 231 providers.

📖 [User Guide](docs/guides/USER_GUIDE.md) · [API Reference](docs/reference/API_REFERENCE.md) · [Environment Config](docs/reference/ENVIRONMENT.md)

Expand Down
45 changes: 43 additions & 2 deletions docs/architecture/RESILIENCE_GUIDE.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Resilience Guide"
version: 3.8.18
lastUpdated: 2026-06-09
version: 3.8.31
lastUpdated: 2026-06-20
---

# Resilience Guide
Expand Down Expand Up @@ -116,6 +116,47 @@ Lists active lockouts with: provider, connection, model, reason, expiresAt. Oper
- `GET /api/resilience/model-cooldowns` — list active lockouts
- `DELETE /api/resilience/model-cooldowns` — manual re-enable. Body: `{provider, connection, model}`. Auth: management.

### Lockout settings UI + success-decay recovery (v3.8.23)

Model lockout went from always-on hardcoded behavior to a fully configurable,
opt-in feature with its own settings card and a self-healing recovery path.

**Settings card:** Settings → Model Lockout
(`src/app/(dashboard)/dashboard/settings/components/ModelLockoutCard.tsx`).
This is **distinct** from the read-only `ModelCooldownsCard` above (which only
*lists* active lockouts) — the new card *configures the parameters*. Defaults
live in `DEFAULT_MODEL_LOCKOUT_SETTINGS`
(`src/lib/resilience/modelLockoutSettings.ts`):

| Setting | Default | Meaning |
| ----------------------- | -------------------------------- | --------------------------------------------------------------- |
| `enabled` | `false` | Master toggle — model lockout is **off by default**. |
| `errorCodes` | `[403, 404, 429, 502, 503, 504]` | Upstream statuses that count as a model-scoped failure. |
| `baseCooldownMs` | `120_000` (120 s) | Initial lockout duration for the first failure. |
| `maxCooldownMs` | `1_800_000` (30 min) | Cap on the escalated cooldown. |
| `maxBackoffSteps` | `10` | Max exponential-backoff escalation steps. |
| `useExponentialBackoff` | `true` | Whether repeated failures escalate the cooldown exponentially. |

Settings persist through the normal settings store and validate via the
resilience settings schema; the card clamps `baseCooldownMs`/`maxCooldownMs`
(with `maxCooldownMs ≥ baseCooldownMs`) and `maxBackoffSteps`.

**Success-decay recovery:** recovery is **not** purely timer expiry. A healthy
response walks the model's failure count back down so a model that recovered
mid-window stops escalating (and clears) before its timer would. On a successful
combo target, `open-sse/services/combo.ts` calls `decayModelFailureCount()`
(`open-sse/services/accountFallback.ts`), which **halves** the stored
`failureCount` (`Math.floor(failureCount / 2)`); when it reaches `0` the lockout
entry is deleted entirely. The counterpart `recordModelLockoutFailure()`
increments the count (and escalates the cooldown) on failures within the
escalation window. This success-decay is in addition to plain timer expiry —
either path can re-enable a model.

**State:** lockouts are held **in-memory** (per-process `Map`s of
`ModelLockoutEntry` keyed by `provider:connectionId:model`), not persisted to
the DB — they are lost on restart. The *settings* are persisted; the active
lockout *state* is ephemeral.

---

## Other Resilience Features
Expand Down
Loading
Loading