Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,12 @@
## Project

Unified AI proxy/router — route any LLM through one endpoint. Multi-provider support
with **236 provider entries** (OpenAI, Anthropic, Gemini, DeepSeek, Groq, xAI, Mistral, Fireworks,
with **237 provider entries** (OpenAI, Anthropic, Gemini, DeepSeek, Groq, xAI, Mistral, Fireworks,
Cohere, NVIDIA, Cerebras, Pollinations, Puter, Cloudflare AI, HuggingFace, DeepInfra,
SambaNova, Meta Llama API, Moonshot AI, AI21 Labs, Databricks, Snowflake, and many more)
with **MCP Server** (94 tools), **A2A v0.3 Protocol**, and **Electron desktop app**.

> **Live counts (v3.8.40)**: providers 236 · MCP tools 94 · MCP scopes 30 · A2A skills 6 ·
> **Live counts (v3.8.40)**: providers 237 · MCP tools 94 · MCP scopes 30 · A2A skills 6 ·
> open-sse services 298 · routing strategies 17 · auto-combo scoring factors 12 ·
> DB modules 94 · DB migrations 106 · base tables 17 · search providers 11 ·
> i18n locales 42. **Refresh with `npm run check:docs-all`.**
Expand Down
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ For full test matrix, see `CONTRIBUTING.md` → "Running Tests". For deep archit

## Project at a Glance

**OmniRoute** — unified AI proxy/router. One endpoint, 236 LLM providers, auto-fallback.
**OmniRoute** — unified AI proxy/router. One endpoint, 237 LLM providers, auto-fallback.

| Layer | Location | Purpose |
| ------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
Expand Down
22 changes: 11 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

# 🚀 OmniRoute — The Free AI Gateway

### Never stop coding. Connect every AI tool to **236 providers** — **90+ free** — through one endpoint.
### Never stop coding. Connect every AI tool to **237 providers** — **90+ free** — through one endpoint.

**Plug Claude Code, Codex, Cursor, Cline, Copilot & Antigravity into FREE Claude / GPT / Gemini. Auto-fallback.**
<br/>
Expand All @@ -26,8 +26,8 @@

<a href="https://trendshift.io/repositories/23589" target="_blank"><img src="https://trendshift.io/api/badge/repositories/23589" alt="diegosouzapw%2FOmniRoute | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>

[![236 AI Providers](https://img.shields.io/badge/236-AI_Providers-6C5CE7?style=for-the-badge)](#-236-ai-providers--90-free)
[![90+ Free](https://img.shields.io/badge/90%2B-Free_Tiers-00B894?style=for-the-badge)](#-236-ai-providers--90-free)
[![237 AI Providers](https://img.shields.io/badge/237-AI_Providers-6C5CE7?style=for-the-badge)](#-237-ai-providers--90-free)
[![90+ Free](https://img.shields.io/badge/90%2B-Free_Tiers-00B894?style=for-the-badge)](#-237-ai-providers--90-free)
[![1.6B Free Tokens/mo](https://img.shields.io/badge/1.6B-Free_Tokens%2Fmo-00B894?style=for-the-badge)](docs/reference/FREE_TIERS.md)
[![Token Savings](https://img.shields.io/badge/up_to_95%25-Token_Savings-E17055?style=for-the-badge)](#%EF%B8%8F-save-1595-tokens--automatically)
[![17 Strategies](https://img.shields.io/badge/17-Routing_Strategies-0984E3?style=for-the-badge)](#-combos--the-flagship)
Expand Down Expand Up @@ -56,7 +56,7 @@
![Docker Pulls](https://img.shields.io/docker/pulls/diegosouzapw/omniroute?label=docker%20pulls&logo=docker&color=2496ED)
![Electron Downloads](https://img.shields.io/github/downloads/diegosouzapw/omniroute/total?style=flat&label=electron%20downloads&logo=electron&color=47848F)

[**🚀 Quick Start**](#-quick-start) • [**🎯 Combos**](#-combos--the-flagship) • [**🌐 Providers**](#-236-ai-providers--90-free) • [**🔌 CLI & MCP**](#-full-cli--a2a--mcp) • [**🗜️ Compression**](#%EF%B8%8F-save-1595-tokens--automatically) • [**🌍 Website**](https://omniroute.online)
[**🚀 Quick Start**](#-quick-start) • [**🎯 Combos**](#-combos--the-flagship) • [**🌐 Providers**](#-237-ai-providers--90-free) • [**🔌 CLI & MCP**](#-full-cli--a2a--mcp) • [**🗜️ Compression**](#%EF%B8%8F-save-1595-tokens--automatically) • [**🌍 Website**](https://omniroute.online)

[💥 The Promise](#-the-promise) • [🤔 Why](#-why-omniroute) • [🏆 What Sets Apart](#-what-sets-omniroute-apart) • [🤖 Compatible CLIs](#-compatible-clis--coding-agents) • [🖥️ Where It Runs](#%EF%B8%8F-where-omniroute-runs--anywhere) • [🔒 Private](#-private--local-first) • [🎬 In Action](#-omniroute-in-action) • [📚 Explore More](#-explore-more) • [📧 Support](#-support--community)

Expand Down Expand Up @@ -144,11 +144,11 @@

</div>

> One endpoint. **236 providers.** Never stop building — and let OmniRoute pick the cheapest one that works.
> One endpoint. **237 providers.** Never stop building — and let OmniRoute pick the cheapest one that works.

<table>
<tr>
<td width="33%" valign="top"><b>🚫 Never hit limits</b><br/><sub>Auto-fallback across 236 providers in milliseconds. Quota out? Next provider takes over — zero downtime.</sub></td>
<td width="33%" valign="top"><b>🚫 Never hit limits</b><br/><sub>Auto-fallback across 237 providers in milliseconds. Quota out? Next provider takes over — zero downtime.</sub></td>
<td width="33%" valign="top"><b>💸 Save up to 95% tokens</b><br/><sub>RTK + Caveman stacked compression cuts 15–95% of eligible tokens (~89% avg on tool-heavy sessions).</sub></td>
<td width="33%" valign="top"><b>🆓 $0 to start</b><br/><sub>90+ providers with a free tier, 11 free <i>forever</i> (Kiro, Qoder, Pollinations, LongCat…). No card needed.</sub></td>
</tr>
Expand Down Expand Up @@ -308,7 +308,7 @@ Result: 4 layers of fallback = zero downtime

| Feature | OmniRoute | Other routers |
| -------------------------------------- | ------------------------------------------------------------------- | ------------- |
| 🌐 Providers | **236** | 20–100 |
| 🌐 Providers | **237** | 20–100 |
| 🆓 Free providers | **90+ (11 free forever)** | 1–5 |
| 🔀 Routing strategies | **17** (priority, weighted, cost-optimized, context-relay, fusion…) | 1–3 |
| 🗜️ Token compression | **RTK + Caveman stacked (15–95%)** | None / 20–40% |
Expand Down Expand Up @@ -344,7 +344,7 @@ Result: 4 layers of fallback = zero downtime
- **💸 Cost telemetry everywhere** — `X-OmniRoute-*` cost/usage headers on every endpoint (including media), a non-token cost engine, a cache-HIT `X-OmniRoute-Cost-Saved` header, and per-key USD spend quotas. → [API Reference](docs/reference/API_REFERENCE.md)
- **🧠 Memory you control** — opt-in int8 vector quantization (Qdrant + sqlite-vec), memory off by default, and a per-request `x-omniroute-no-memory` header. → [Memory](docs/frameworks/MEMORY.md)
- **🛡️ Security** — a prompt-injection guard across every LLM route (backed by a red-team suite), plus a free DuckDuckGo last-resort web search. → [Guardrails](docs/security/GUARDRAILS.md)
- **🤝 More providers & agents** — Cursor Cloud Agent (a 4th cloud agent), CodeBuddy CN (`copilot.tencent.com`), a Google Flow video-generation provider, new gateways **DGrid** and **Pioneer AI** (Fastino Labs), inbound **xAI Grok** translators plus **Grok Build (xAI)** with an OAuth import-token flow, GPT-4 / GPT-4o-mini on the GitHub Copilot provider, multi-model **Factory Droid**, **ZenMux Free** (session-cookie free tier), **Alibaba DashScope** text-to-video (`wan2.7-t2v`), a refreshed 236-provider catalog (OrcaRouter, Wafer AI, OpenAdapter, dit.ai, TokenRouter, …), Vertex AI media generation (speech / transcription / music / video), and one-click account import from CLIProxyAPI (`~/.cli-proxy-api/`). → [Providers](docs/reference/PROVIDER_REFERENCE.md)
- **🤝 More providers & agents** — Cursor Cloud Agent (a 4th cloud agent), CodeBuddy CN (`copilot.tencent.com`), a Google Flow video-generation provider, new gateways **DGrid** and **Pioneer AI** (Fastino Labs), inbound **xAI Grok** translators plus **Grok Build (xAI)** with an OAuth import-token flow, GPT-4 / GPT-4o-mini on the GitHub Copilot provider, multi-model **Factory Droid**, **ZenMux Free** (session-cookie free tier), **Alibaba DashScope** text-to-video (`wan2.7-t2v`), a refreshed 237-provider catalog (OrcaRouter, Wafer AI, OpenAdapter, dit.ai, TokenRouter, …), Vertex AI media generation (speech / transcription / music / video), and one-click account import from CLIProxyAPI (`~/.cli-proxy-api/`). → [Providers](docs/reference/PROVIDER_REFERENCE.md)
- **⚡ Local performance & infra** — a one-click local Redis launcher (`omniroute redis up`, plus a dashboard Redis panel), one-click **Cloudflare Workers** and **Deno Deploy** relay deployers wired into the proxy pool, and an optional Bifrost Go sidecar that offloads the hottest relay path (`BIFROST_BASE_URL`, with automatic fallback to the TypeScript path on timeout) — now with a relay-backend selector (`OMNIROUTE_RELAY_BACKEND=ts|bifrost|auto`) so the `/v1/relay` endpoint stays the stable surface while choosing the fastest backend internally. → [Environment](docs/reference/ENVIRONMENT.md)

<br/>
Expand Down Expand Up @@ -387,11 +387,11 @@ Result: 4 layers of fallback = zero downtime

<div align="center">

# 🌐 236 AI Providers — 90+ Free
# 🌐 237 AI Providers — 90+ Free

</div>

> The most complete catalog of any open-source router: **236 providers**, **90+ with a free tier**, **11 free forever**.
> The most complete catalog of any open-source router: **237 providers**, **90+ with a free tier**, **11 free forever**.

<div align="center">

Expand Down Expand Up @@ -899,7 +899,7 @@ Compression: aggressive (~50%) → double your free quota · Cost: $0/mo
**Will I be charged by OmniRoute?** No — it's free, open-source software on your machine. You only pay paid providers directly. OmniRoute has no billing system.
**Are FREE providers really unlimited?** Mostly — Qoder, Pollinations, LongCat, and Cloudflare are free with no per-account credit cap. Kiro is free too but capped at ~50 credits/month per account. Stack multiple free providers in a combo and auto-fallback keeps you serving for $0.
**Will compression hurt quality?** No — it only compresses the **input**; code, URLs, JSON are always protected.
**Does it work where AI is blocked?** Yes — 3-level proxy + 1proxy marketplace reach all 236 providers.
**Does it work where AI is blocked?** Yes — 3-level proxy + 1proxy marketplace reach all 237 providers.

📖 [User Guide](docs/guides/USER_GUIDE.md) · [API Reference](docs/reference/API_REFERENCE.md) · [Environment Config](docs/reference/ENVIRONMENT.md)

Expand Down
5 changes: 3 additions & 2 deletions docs/reference/PROVIDER_REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ lastUpdated: 2026-06-30
> Regenerate with: `npm run gen:provider-reference`
> **Last generated:** 2026-06-30

Total providers: **236**. See category breakdown below.
Total providers: **237**. See category breakdown below.

## Categories

Expand Down Expand Up @@ -247,7 +247,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
| `zai` | `zai` | Z.AI | API key | [link](https://open.bigmodel.cn) | — |
| `zenmux` | `zm` | ZenMux | API key | [link](https://zenmux.ai) | Use your ZenMux API key in Authorization: Bearer <key>. ZenMux is fully OpenAI-compatible. Base URL: https://zenmux.ai/api/v1. |

## Local Providers (11)
## Local Providers (12)

| ID | Alias | Name | Tags | Website | Notes |
|----|-------|------|------|---------|-------|
Expand All @@ -257,6 +257,7 @@ Use the dashboard at `/dashboard/providers` to enable, configure, and test each
| `llama-cpp` | `llamacpp` | llama.cpp | Local, self-hosted | [link](https://github.com/ggml-org/llama.cpp) | API key optional (use any value, e.g. sk-no-key-required). Configure the llama-server OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). Note: if Llamafile is also installed, both default to port 8080 — run only one at a time or override the port. |
| `llamafile` | `llamafile` | Llamafile | Local, self-hosted | [link](https://github.com/Mozilla-Ocho/llamafile) | API key optional. Configure the local Llamafile OpenAI-compatible base URL (default: http://127.0.0.1:8080/v1). |
| `lm-studio` | `lmstudio` | LM Studio | Local, self-hosted | [link](https://lmstudio.ai) | API key optional. Configure the local LM Studio OpenAI-compatible base URL (default: http://localhost:1234/v1). |
| `ollama-local` | `ollama` | Ollama | Local, self-hosted | [link](https://ollama.com) | No API key required. Ollama runs locally — configure its OpenAI-compatible base URL (default: http://localhost:11434/v1) and make sure Ollama is running before connecting. |
| `oobabooga` | `ooba` | oobabooga | Local, self-hosted | [link](https://github.com/oobabooga/text-generation-webui) | API key optional. Configure the local oobabooga OpenAI-compatible base URL (default: http://localhost:5000/v1). |
| `sdwebui` | `sdwebui` | SD WebUI | Local | [link](https://github.com/AUTOMATIC1111/stable-diffusion-webui) | No API key required. Configure the local WebUI base URL (default: http://localhost:7860). |
| `triton` | `triton` | NVIDIA Triton | Local, self-hosted | [link](https://developer.nvidia.com/triton-inference-server) | API key optional. Configure the Triton OpenAI-compatible base URL (default: http://localhost:8000/v1). |
Expand Down
1 change: 1 addition & 0 deletions open-sse/executors/default.ts
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,7 @@ export class DefaultExecutor extends BaseExecutor {
const baseUrl = credentials?.providerSpecificData?.baseUrl || this.config.baseUrl;
return normalizeOpenAIChatUrl(baseUrl);
}
case "ollama-local":
case "llama-cpp":
case "lm-studio":
case "modal":
Expand Down
1 change: 1 addition & 0 deletions src/shared/constants/providers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,7 @@ export function isLocalProvider(providerId: unknown): boolean {
}

export const SELF_HOSTED_CHAT_PROVIDER_IDS = new Set([
"ollama-local",
"lm-studio",
"vllm",
"lemonade",
Expand Down
13 changes: 13 additions & 0 deletions src/shared/constants/providers/local.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,19 @@
* Pure data literal; re-exported by the providers.ts barrel. No behavior change.
*/
export const LOCAL_PROVIDERS = {
"ollama-local": {
id: "ollama-local",
alias: "ollama",
name: "Ollama",
icon: "pets",
color: "#58A6FF",
textIcon: "OL",
website: "https://ollama.com",
authHint:
"No API key required. Ollama runs locally — configure its OpenAI-compatible base URL (default: http://localhost:11434/v1) and make sure Ollama is running before connecting.",
localDefault: "http://localhost:11434/v1",
passthroughModels: true,
},
"lm-studio": {
id: "lm-studio",
alias: "lmstudio",
Expand Down
50 changes: 50 additions & 0 deletions tests/unit/ollama-local-provider.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
import { test } from "node:test";
import assert from "node:assert/strict";

import {
LOCAL_PROVIDERS,
isLocalProvider,
isSelfHostedChatProvider,
} from "@/shared/constants/providers";
import { DefaultExecutor } from "@omniroute/open-sse/executors/default.ts";

// #5578: Ollama is the most popular local runtime, yet OmniRoute only shipped
// `ollama-cloud` (api-key, cloud) and `ollama-search` (web search). There was no
// first-class card for the local Ollama runtime (localhost:11434). This adds
// `ollama-local` to the local catalog so users get a dedicated card instead of
// falling back to the generic `openai-compatible-*` provider.

test("ollama-local is a first-class entry in the local catalog", () => {
const entry = LOCAL_PROVIDERS["ollama-local"];
assert.ok(entry, "ollama-local must be defined in LOCAL_PROVIDERS");
assert.equal(entry.id, "ollama-local");
assert.equal(entry.name, "Ollama");
// Ollama exposes an OpenAI-compatible surface at /v1 on its default port.
assert.equal(entry.localDefault, "http://localhost:11434/v1");
// Models are listed via the OpenAI-compatible /v1/models passthrough.
assert.equal(entry.passthroughModels, true);
});

test("ollama-local is classified as a local, self-hosted chat provider", () => {
assert.equal(isLocalProvider("ollama-local"), true);
assert.equal(isSelfHostedChatProvider("ollama-local"), true);
});

test("ollama-local buildUrl routes to the configured local baseUrl, not OpenAI", () => {
const executor = new DefaultExecutor("ollama-local");
const url = executor.buildUrl("llama3.2", true, 0, {
providerSpecificData: { baseUrl: "http://127.0.0.1:11434/v1" },
});

assert.equal(url, "http://127.0.0.1:11434/v1/chat/completions");
assert.equal(new URL(url).hostname, "127.0.0.1", `expected local host, got ${url}`);
});

test("ollama-local buildUrl falls back to localhost:11434, never OpenAI, when no baseUrl is set", () => {
const executor = new DefaultExecutor("ollama-local");
const url = executor.buildUrl("llama3.2", true, 0, {});

assert.equal(url, "http://localhost:11434/v1/chat/completions");
assert.equal(new URL(url).hostname, "localhost", `expected local default host, got ${url}`);
assert.notEqual(new URL(url).hostname, "api.openai.com");
});
Loading