Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/maintenance/12316-mode-packs-doc-gate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- **docs(auto-combo):** The mode pack table in `docs/routing/AUTO-COMBO.md` now lists all six shipped packs with every weight each one sets, replacing a four-pack table whose numbers had also drifted from the source. It states plainly that no pack sets `quality`, so selecting any pack silences the observed-quality signal. Six more documents that quote the scoring factor count joined the `check:docs-counts` gate, which caught five stale claims — including one naming nine factors that do not exist — and two stale mode pack counts. The dashboard routing panel, which offered four of the six packs and labelled the default strategy "6-Factor Scoring", is now covered by a test; the two packs it was missing are `reliability-first` and `chaos-mode`, the latter labelled as the fault-injection profile it is rather than as one more routing preference ([#12316](https://github.com/diegosouzapw/OmniRoute/pull/12316))
12 changes: 8 additions & 4 deletions docs/architecture/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -370,14 +370,18 @@ Key capabilities:
**auto**, lkgp, context-optimized, context-relay, **fusion**, plus a fallback path) —
auto is the headline addition in v3.8.0; `fusion` (panel fan-out + judge synthesis,
`open-sse/services/fusion.ts`) is new in v3.8.36.
- **9-factor scoring**: cost, latency p95, success rate, quota headroom, lockout
proximity, breaker state, recent failures, model availability, and tag affinity.
- **15-factor scoring**: quota, health, inverse cost, inverse latency, task fit and
ten more. The canonical table of factors and their default weights lives in
[`docs/routing/AUTO-COMBO.md`](../routing/AUTO-COMBO.md) — restating it here would
give it a second place to go stale.
- **Virtual factory** materializes ephemeral combos when no matching named combo
exists, sourcing candidates from healthy active provider connections.
- **Auto prefixes**: `auto/coding`, `auto/cheap`, `auto/fast`, `auto/offline`,
`auto/smart`, `auto/lkgp` — each backed by a tuned weight profile.
- **4 mode packs**: coding, fast, cheap, smart — shipped as preset weight
configurations callable from the dashboard.
- **6 mode packs**: `ship-fast`, `cost-saver`, `quality-first`, `offline-friendly`,
`reliability-first` and `chaos-mode` — preset weight configurations callable from
the dashboard. (Not to be confused with the `auto/*` prefixes above, which are
request-time variants.)

For full algorithmic detail (factor formulas, weight tuning), see
[`docs/routing/AUTO-COMBO.md`](../routing/AUTO-COMBO.md).
Expand Down
36 changes: 18 additions & 18 deletions docs/architecture/REPOSITORY_MAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -403,24 +403,24 @@ open-sse/

### Subsystem deep-dives

| Doc | Purpose |
| -------------------------- | ------------------------------------------------------------------- |
| `MCP-SERVER.md` | MCP server: 110 tools, 3 transports, 33 scopes, REST endpoints |
| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 6 skills, REST helpers, agent card |
| `AGENT_PROTOCOLS_GUIDE.md` | Unified guide: A2A vs ACP vs Cloud Agents |
| `CLOUD_AGENT.md` | Codex Cloud / Devin / Jules orchestration |
| `SKILLS.md` | Skills framework (built-in + marketplace + SkillsSH + sandbox) |
| `RADAR.md` | Radar free-model catalog overlay (`RADAR_ENABLED`, off by default) |
| `MEMORY.md` | Memory system (SQLite FTS5 + Qdrant) |
| `EVALS.md` | Eval framework (suites, runs, rubrics) |
| `GUARDRAILS.md` | PII masker, prompt injection, vision bridge |
| `COMPLIANCE.md` | Audit log, retention, noLog opt-out |
| `WEBHOOKS.md` | HMAC-signed webhook delivery |
| `REASONING_REPLAY.md` | Hybrid memory/SQLite cache for `reasoning_content` |
| `AUTHZ_GUIDE.md` | Authorization pipeline (`classify` → `policies` → `enforce`) |
| `RESILIENCE_GUIDE.md` | Circuit breaker + cooldown + model lockout |
| `STEALTH_GUIDE.md` | TLS fingerprinting (JA3/JA4), Claude Code CCH, MITM cert |
| `AUTO-COMBO.md` | Auto Combo engine (9-factor scoring, 4 mode packs, virtual factory) |
| Doc | Purpose |
| -------------------------- | -------------------------------------------------------------------- |
| `MCP-SERVER.md` | MCP server: 110 tools, 3 transports, 33 scopes, REST endpoints |
| `A2A-SERVER.md` | A2A v0.3: JSON-RPC, 6 skills, REST helpers, agent card |
| `AGENT_PROTOCOLS_GUIDE.md` | Unified guide: A2A vs ACP vs Cloud Agents |
| `CLOUD_AGENT.md` | Codex Cloud / Devin / Jules orchestration |
| `SKILLS.md` | Skills framework (built-in + marketplace + SkillsSH + sandbox) |
| `RADAR.md` | Radar free-model catalog overlay (`RADAR_ENABLED`, off by default) |
| `MEMORY.md` | Memory system (SQLite FTS5 + Qdrant) |
| `EVALS.md` | Eval framework (suites, runs, rubrics) |
| `GUARDRAILS.md` | PII masker, prompt injection, vision bridge |
| `COMPLIANCE.md` | Audit log, retention, noLog opt-out |
| `WEBHOOKS.md` | HMAC-signed webhook delivery |
| `REASONING_REPLAY.md` | Hybrid memory/SQLite cache for `reasoning_content` |
| `AUTHZ_GUIDE.md` | Authorization pipeline (`classify` → `policies` → `enforce`) |
| `RESILIENCE_GUIDE.md` | Circuit breaker + cooldown + model lockout |
| `STEALTH_GUIDE.md` | TLS fingerprinting (JA3/JA4), Claude Code CCH, MITM cert |
| `AUTO-COMBO.md` | Auto Combo engine (15-factor scoring, 6 mode packs, virtual factory) |

### Compression

Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/RESILIENCE_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -652,4 +652,4 @@ default `test:integration`, chaos and heap self-skip (without `RUN_CHAOS_INT`/`-

- [Architecture Guide](./ARCHITECTURE.md) — System architecture and internals
- [User Guide](../guides/USER_GUIDE.md) — Providers, combos, CLI integration
- [Auto-Combo Engine](../routing/AUTO-COMBO.md) — 13-factor scoring, mode packs
- [Auto-Combo Engine](../routing/AUTO-COMBO.md) — 15-factor scoring, mode packs
61 changes: 33 additions & 28 deletions docs/getting-started/AUTO-COMBO-GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,14 +46,14 @@ model: "auto/cheap" # Cheapest option

## Which "auto" Should I Use?

| If you want... | Use this | Best for | How it works |
|----------------|----------|----------|--------------|
| **Best overall** | `auto` | General questions, chat | Balances speed, cost, and quality |
| **Best code** | `auto/coding` | Writing code, debugging | Picks models good at coding tasks |
| **Fastest response** | `auto/fast` | Quick answers, low latency | Prioritizes speed over everything |
| **Cheapest option** | `auto/cheap` | Saving money | Picks the cheapest provider |
| **Smartest model** | `auto/smart` | Complex tasks | Quality-first + explores new models |
| **Most available** | `auto/offline` | When providers are busy | Picks providers with most capacity |
| If you want... | Use this | Best for | How it works |
| -------------------- | -------------- | -------------------------- | ----------------------------------- |
| **Best overall** | `auto` | General questions, chat | Balances speed, cost, and quality |
| **Best code** | `auto/coding` | Writing code, debugging | Picks models good at coding tasks |
| **Fastest response** | `auto/fast` | Quick answers, low latency | Prioritizes speed over everything |
| **Cheapest option** | `auto/cheap` | Saving money | Picks the cheapest provider |
| **Smartest model** | `auto/smart` | Complex tasks | Quality-first + explores new models |
| **Most available** | `auto/offline` | When providers are busy | Picks providers with most capacity |

### Examples

Expand Down Expand Up @@ -81,7 +81,7 @@ curl http://localhost:20128/v1/chat/completions \
When you send a request with `model: "auto"`, OmniRoute:

1. **Looks at all your connected providers** — Every provider you've added (OpenAI, Anthropic, Google, etc.)
2. **Scores each one** on 5 factors:
2. **Scores each one**, weighing among other things:
- Is it working? (health)
- Does it have capacity? (quota)
- How much does it cost? (price)
Expand All @@ -94,29 +94,29 @@ When you send a request with `model: "auto"`, OmniRoute:

Each provider gets a score from 0 to 1. The higher the score, the better the fit.

| Factor | Weight | What it means |
|--------|--------|---------------|
| Health | 20% | Is the provider working? (circuit breaker state) |
| Quota | 15% | Does it have capacity remaining? |
| Cost | 15% | How expensive is it? (cheaper = higher score) |
| Speed | 12% | How fast is it? (lower latency = higher score) |
| Task Fit | 8% | Is it good at this type of task? |
| Stability | 5% | Is it consistent? (low error rate) |
| Tier | 5% | Account tier (Ultra > Pro > Free) |
| Other | 20% | Context affinity, connection density, etc. |
| Factor | Weight | What it means |
| --------- | ------ | ------------------------------------------------ |
| Health | 20% | Is the provider working? (circuit breaker state) |
| Quota | 15% | Does it have capacity remaining? |
| Cost | 15% | How expensive is it? (cheaper = higher score) |
| Speed | 12% | How fast is it? (lower latency = higher score) |
| Task Fit | 8% | Is it good at this type of task? |
| Stability | 5% | Is it consistent? (low error rate) |
| Tier | 5% | Account tier (Ultra > Pro > Free) |
| Other | 20% | Context affinity, connection density, etc. |

### How Variants Change the Scoring

Each variant uses different weights:

| Variant | Prioritizes | Key Weights |
|---------|-------------|-------------|
| `auto` | Balanced | health=20%, quota=15%, cost=15% |
| `auto/coding` | Quality | taskFit=37%, stability=15% |
| `auto/fast` | Speed | latency=32%, health=28% |
| `auto/cheap` | Cost | cost=37% |
| `auto/smart` | Quality + Explore | taskFit=37%, exploration=10% |
| `auto/offline` | Capacity | quota=37%, health=28% |
| Variant | Prioritizes | Key Weights |
| -------------- | ----------------- | ------------------------------- |
| `auto` | Balanced | health=20%, quota=15%, cost=15% |
| `auto/coding` | Quality | taskFit=37%, stability=15% |
| `auto/fast` | Speed | latency=32%, health=28% |
| `auto/cheap` | Cost | cost=37% |
| `auto/smart` | Quality + Explore | taskFit=37%, exploration=10% |
| `auto/offline` | Capacity | quota=37%, health=28% |

---

Expand All @@ -125,15 +125,19 @@ Each variant uses different weights:
OmniRoute has **three layers of protection**:

### 1. Auto-Fallback

If the best provider fails, OmniRoute automatically tries the next one. You don't need to do anything.

### 2. Self-Healing

If a provider keeps failing:

- **Score < 0.2** → Excluded for 5 minutes
- **Circuit breaker open** → Auto-excluded
- **More than 50% providers down** → Incident mode (no exploration)

### 3. Emergency Fallback

If all providers fail, OmniRoute routes to stable free providers (like Kiro or Qoder) as a last resort.

---
Expand Down Expand Up @@ -209,7 +213,8 @@ Round-robin cycles through providers in order. Auto-combo **scores each provider
## Learn More

For developers and contributors, see the [Auto-Combo Technical Reference](../routing/AUTO-COMBO.md) for:
- Full 13-factor scoring algorithm

- Full 15-factor scoring algorithm
- Mode pack weight tables
- Implementation file paths
- API endpoints
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/FEATURES.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Visual guide to every section of the OmniRoute dashboard.

The v3.7.x → v3.8.0 cycle added zero-config auto routing, new providers, OAuth flows, deeper resilience, and a much richer CLI experience. Headline features below — full details further in the document and in linked specs.

- 🤖 **Auto Combo / Zero-config auto-routing** — use prefixes `auto/coding`, `auto/fast`, `auto/cheap`, `auto/offline`, `auto/smart`, `auto/lkgp`, `auto/chaos`. Backed by a 15-factor scoring engine and 6 curated **mode packs** (ship-fast, cost-saver, quality-first, offline-friendly)
- 🤖 **Auto Combo / Zero-config auto-routing** — use prefixes `auto/coding`, `auto/fast`, `auto/cheap`, `auto/offline`, `auto/smart`, `auto/lkgp`, `auto/chaos`. Backed by a 15-factor scoring engine and 6 curated **mode packs** (ship-fast, cost-saver, quality-first, offline-friendly, reliability-first, chaos-mode)
- 🆕 **Command Code provider** (#2199) — first-class registration with model catalog and quota tracking
- 🆕 **Z.AI provider** — new free-tier provider with quota labels
- 🎬 **KIE media expansion** — extended catalog including video generation models
Expand Down
Loading