feat: add iii-llm-router worker

llm-router/.gitignore

@@ -0,0 +1,2 @@
+target/
+Cargo.lock

llm-router/Cargo.toml

@@ -0,0 +1,25 @@
+[workspace]
+
+[package]
+name = "iii-llm-router"
+version = "0.1.0"
+edition = "2021"
+publish = false
+
+[[bin]]
+name = "iii-llm-router"
+path = "src/main.rs"
+
+[dependencies]
+iii-sdk = "=0.11.2"
+tokio = { version = "1", features = ["rt-multi-thread", "macros", "sync", "signal"] }
+serde = { version = "1", features = ["derive"] }
+serde_json = "1"
+serde_yaml = "0.9"
+anyhow = "1"
+thiserror = "2"
+tracing = "0.1"
+tracing-subscriber = { version = "0.3", features = ["fmt", "env-filter"] }
+clap = { version = "4", features = ["derive"] }
+uuid = { version = "1", features = ["v4"] }
+rand = "0.8"

llm-router/README.md

@@ -0,0 +1,127 @@
+# iii-llm-router
+
+Policy-based LLM routing brain. **Unopinionated** — ships with zero built-in model names, zero hardcoded pricing, zero provider assumptions. Wraps any gateway (LiteLLM, Bifrost, OpenRouter, a local vLLM, your own proxy) by sitting *in front* of it: gateway asks `router::decide` before every call, router returns a model ID, gateway forwards.
+
+## Why unopinionated matters
+
+Every existing LLM router (RouteLLM, Portkey, LiteLLM's routing block) bakes a specific catalog of models and a specific rank ordering into the library. That catalog is wrong the day you read it — new models ship weekly, pricing moves, quality tiers shift. This worker doesn't know what "Opus" or "GPT" is. You register what you actually use at runtime. The only thing the router enforces is its own logic: match → classify → budget → health → fallback.
+
+## Functions (18)
+
+| id | shape |
+|----|-------|
+| `router::decide` | hot path — returns `{model, reason, policy_id?, ab_test_id?, fallback?, confidence, request_id}` |
+| `router::policy_create` / `update` / `delete` / `list` / `test` | CRUD + dry-run |
+| `router::classify` | run the prompt heuristic only; returns `{complexity, confidence, suggested_model}` (suggested_model respects your classifier map) |
+| `router::classifier_config` | register `{id, thresholds: {simple/moderate/complex/expert → <your model id>}}` |
+| `router::ab_create` / `ab_record` / `ab_report` / `ab_conclude` | A/B tests with weighted variants + quality/latency/cost aggregation |
+| `router::health_update` / `health_list` | per-model availability + error rate; feeds fallback path |
+| `router::model_register` / `model_unregister` / `model_list` | you tell the router what models exist; used only by the budget-downgrade path and stats |
+| `router::stats` | usage by model, by policy, over a day window |
+
+## HTTP triggers (18)
+
+```
+POST /api/router/decide
+POST /api/router/policy/{create,update,delete,test}
+GET  /api/router/policy/list
+POST /api/router/classify
+POST /api/router/classifier
+POST /api/router/ab/{create,record,report,conclude}
+POST /api/router/health/update
+GET  /api/router/health/list
+POST /api/router/model/{register,unregister}
+GET  /api/router/model/list
+GET  /api/router/stats
+```
+
+## Decide logic
+
+```
+match policies (by tenant, feature, tags) and pick highest priority
+  if matching A/B test is running → sample a variant → return
+  else if policy.action.model == "auto" → classify → look up user mapping
+  if chosen model is unhealthy → use policy.fallback
+  if policy.max_cost_per_request > budget_remaining → search registered
+     models for a cheaper one meeting min_quality (if none: return original,
+     flag reason)
+  return {model, reason, policy_id, fallback, confidence}
+no policy matched:
+  if classifier exists → classify → map
+  else → return empty model + reason (caller should handle)
+```
+
+Router **never** invents a model name. If you ask it to pick "auto" without a classifier registered, it tells you so in `reason` and returns the policy's fallback (or empty).
+
+## State (engine-managed)
+
+All stored in `state_scope: "llm-router"` (configurable).
+
+```
+policies:<id>         — policy definitions
+ab_tests:<id>         — A/B test definitions
+ab_events:<test>:…    — recorded outcomes
+routing_log:<ts>:<id> — decision audit trail
+model_health:<name>   — availability + latency + error_rate
+classifier:<id>       — category → model mapping
+models:<name>         — registered models (quality, pricing, provider)
+```
+
+## Example
+
+```bash
+# 1. register two models you actually use
+curl -X POST localhost:3111/api/router/model/register -d '{
+  "model": "gw/cheap-fast", "quality": "low",
+  "input_per_1m": 0.1, "output_per_1m": 0.4
+}'
+curl -X POST localhost:3111/api/router/model/register -d '{
+  "model": "gw/strong", "quality": "high",
+  "input_per_1m": 15, "output_per_1m": 75
+}'
+
+# 2. configure the classifier (category → model is YOUR choice)
+curl -X POST localhost:3111/api/router/classifier -d '{
+  "id": "default",
+  "thresholds": {
+    "simple": "gw/cheap-fast",
+    "moderate": "gw/cheap-fast",
+    "complex": "gw/strong",
+    "expert": "gw/strong"
+  }
+}'
+
+# 3. write a policy
+curl -X POST localhost:3111/api/router/policy/create -d '{
+  "name": "support-auto",
+  "match": { "feature": "support-chat" },
+  "action": { "model": "auto", "fallback": "gw/cheap-fast" },
+  "priority": 100
+}'
+
+# 4. ask before every call
+curl -X POST localhost:3111/api/router/decide -d '{
+  "feature": "support-chat",
+  "prompt": "How do I reset my password?"
+}'
+# → {"model":"gw/cheap-fast", "reason":"policy: support-auto + classifier: simple", ...}
+```
+
+Your gateway (LiteLLM/Bifrost/OpenRouter/your-own) takes `model` and forwards. The router doesn't make any LLM call itself.
+
+## What this is NOT
+
+- Not a gateway — no LLM traffic passes through it, no API keys stored.
+- Not an observability platform — `routing_log` is for audit, use iii's OTel for real telemetry.
+- Not a training-based classifier — the shipped classifier is a cheap prompt heuristic (length, code markers, math markers). Swap it by calling `router::classifier_config` with your own mapping, or wrap a stronger classifier as a separate worker and call `router::decide` after you've called it.
+
+## SDK + stack
+
+- `iii-sdk 0.11.0` stable
+- State via `state::get`/`set`/`delete`/`list` against scope `llm-router`
+- `rand` for A/B variant weighted sampling
+- `serde_json` everywhere — all state blobs are JSON
+
+## Tests
+
+17 passing — policy matching, priority ordering, A/B weighted sampling, classifier mapping, auto-without-classifier, unhealthy-fallback, budget-downgrade with and without registered models, health skip thresholds, heuristic category classification.

llm-router/build.rs

@@ -0,0 +1,6 @@
+fn main() {
+    println!(
+        "cargo:rustc-env=TARGET={}",
+        std::env::var("TARGET").unwrap_or_default()
+    );
+}

llm-router/config.yaml

@@ -0,0 +1,4 @@
+state_scope: "llm-router"
+classifier_default_id: "default"
+stats_default_days: 7
+health_skip_threshold_error_rate: 0.3

llm-router/src/config.rs

@@ -0,0 +1,107 @@
+use anyhow::{Context, Result};
+use serde::{Deserialize, Serialize};
+use std::fs;
+
+#[derive(Debug, Clone, Serialize, Deserialize)]
+#[serde(deny_unknown_fields)]
+pub struct RouterConfig {
+    #[serde(default = "default_state_scope")]
+    pub state_scope: String,
+
+    #[serde(default = "default_classifier_id")]
+    pub classifier_default_id: String,
+
+    #[serde(default = "default_stats_days")]
+    pub stats_default_days: u32,
+
+    #[serde(default = "default_health_skip_error_rate")]
+    pub health_skip_threshold_error_rate: f64,
+}
+
+fn default_state_scope() -> String {
+    "llm-router".to_string()
+}
+fn default_classifier_id() -> String {
+    "default".to_string()
+}
+fn default_stats_days() -> u32 {
+    7
+}
+fn default_health_skip_error_rate() -> f64 {
+    0.3
+}
+
+impl Default for RouterConfig {
+    fn default() -> Self {
+        Self {
+            state_scope: default_state_scope(),
+            classifier_default_id: default_classifier_id(),
+            stats_default_days: default_stats_days(),
+            health_skip_threshold_error_rate: default_health_skip_error_rate(),
+        }
+    }
+}
+
+pub fn load_config(path: &str) -> Result<RouterConfig> {
+    let content = fs::read_to_string(path).with_context(|| format!("read {}", path))?;
+    let cfg: RouterConfig =
+        serde_yaml::from_str(&content).with_context(|| format!("parse {}", path))?;
+    validate(&cfg)?;
+    Ok(cfg)
+}
+
+fn validate(cfg: &RouterConfig) -> Result<()> {
+    if cfg.state_scope.trim().is_empty() {
+        anyhow::bail!("config: state_scope must be non-empty");
+    }
+    if cfg.classifier_default_id.trim().is_empty() {
+        anyhow::bail!("config: classifier_default_id must be non-empty");
+    }
+    if cfg.stats_default_days == 0 {
+        anyhow::bail!("config: stats_default_days must be >= 1");
+    }
+    let rate = cfg.health_skip_threshold_error_rate;
+    if !(0.0..=1.0).contains(&rate) || rate.is_nan() {
+        anyhow::bail!(
+            "config: health_skip_threshold_error_rate must be within 0.0..=1.0 (got {})",
+            rate
+        );
+    }
+    Ok(())
+}
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn test_defaults() {
+        let c = RouterConfig::default();
+        assert_eq!(c.state_scope, "llm-router");
+        assert_eq!(c.classifier_default_id, "default");
+        assert_eq!(c.stats_default_days, 7);
+    }
+
+    #[test]
+    fn validate_rejects_out_of_range_error_rate() {
+        let mut c = RouterConfig::default();
+        c.health_skip_threshold_error_rate = 2.0;
+        assert!(validate(&c).is_err());
+        c.health_skip_threshold_error_rate = -0.1;
+        assert!(validate(&c).is_err());
+    }
+
+    #[test]
+    fn validate_rejects_empty_strings() {
+        let mut c = RouterConfig::default();
+        c.state_scope = "".into();
+        assert!(validate(&c).is_err());
+    }
+
+    #[test]
+    fn validate_rejects_zero_stats_days() {
+        let mut c = RouterConfig::default();
+        c.stats_default_days = 0;
+        assert!(validate(&c).is_err());
+    }
+}