Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
242 changes: 242 additions & 0 deletions docs/plugins/PLUGIN_SDK.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,242 @@
# OmniRoute Plugin SDK

## Quick Start

```ts
import { definePlugin } from "omniroute/plugins/sdk";

export default definePlugin({
name: "my-plugin",
priority: 50,
onRequest: async (ctx) => {
console.log(`Request ${ctx.requestId} for ${ctx.model}`);
},
onResponse: async (ctx, response) => {
console.log(`Response for ${ctx.requestId}`);
return response;
},
onError: async (ctx, error) => {
console.error(`Error: ${error.message}`);
},
});
```

## API Reference

### `definePlugin(def: PluginDefinition): Plugin`

Factory function that creates a Plugin object with defaults.

**Parameters:**
- `name` (string, required) — Plugin name in kebab-case
- `priority` (number, optional, default: 100) — Lower runs first
- `enabled` (boolean, optional, default: true) — Start enabled?
- `onRequest` (function, optional) — Runs before chat handler
- `onResponse` (function, optional) — Runs after chat handler
- `onError` (function, optional) — Runs on handler error

### `blockRequest(response?): BlockingHookResult`

Block the request and optionally return a custom response.

```ts
onRequest: (ctx) => {
if (!ctx.headers["authorization"]) {
return blockRequest({ error: "Unauthorized", status: 401 });
}
};
```

### `modifyBody(body): PluginResult`

Modify the request body before it reaches the provider.

```ts
onRequest: (ctx) => {
return modifyBody({ ...ctx.body, temperature: 0.7 });
};
```

### `addMetadata(metadata): PluginResult`

Attach metadata to the request context.

```ts
onRequest: (ctx) => {
return addMetadata({ source: "my-plugin", version: "1.0.0" });
};
```

## Plugin Context (`PluginContext`)

| Field | Type | Description |
|---|---|---|
| `requestId` | `string` | Unique request identifier |
| `model` | `string` | Requested model name |
| `provider` | `string` | Target provider ID |
| `body` | `Record<string, unknown>` | Request body |
| `headers` | `Record<string, string>` | Request headers |
| `metadata` | `Record<string, unknown>` | Mutable metadata |
| `timestamp` | `number` | Request timestamp |

## Manifest (`plugin.json`)

```json
{
"name": "my-plugin",
"version": "1.0.0",
"description": "A sample plugin",
"author": "your-name",
"main": "index.js",
"hooks": {
"onRequest": { "enabled": true, "priority": 50 },
"onResponse": true,
"onError": false
},
"requires": {
"permissions": ["network", "file-read"]
},
"enabledByDefault": false,
"configSchema": {
"apiKey": { "type": "string", "description": "API key for external service" },
"maxRetries": { "type": "number", "min": 1, "max": 10, "default": 3 },
"debug": { "type": "boolean", "default": false },
"mode": { "type": "string", "enum": ["fast", "slow"], "default": "fast" }
}
}
```

### Hook Priority

Hooks can be configured with priority (lower = runs first):

```json
{
"hooks": {
"onRequest": { "enabled": true, "priority": 10 },
"onResponse": { "enabled": true, "priority": 100 }
}
}
```

Or as simple booleans (default priority 100):

```json
{
"hooks": {
"onRequest": true,
"onResponse": true
}
}
```

## Permission System

Plugins run in a sandboxed VM context. Access to external resources requires explicit permissions:

| Permission | Grants |
|---|---|
| `network` | `fetch`, `AbortController`, `Headers`, `Request`, `Response` |
| `file-read` | `fs.readFile`, `fs.readdir`, `fs.stat` |
| `file-write` | `fs.writeFile`, `fs.mkdir`, `fs.rm` |
| `env` | Read-only `process.env` proxy |
| `exec` | `child_process.exec`, `child_process.execSync` |

Without a permission, the corresponding globals are simply not available in the sandbox.

## Config Schema

Define configurable settings in `configSchema`:

```json
{
"configSchema": {
"apiKey": { "type": "string", "description": "External API key" },
"maxRetries": { "type": "number", "min": 1, "max": 10, "default": 3 },
"debug": { "type": "boolean", "default": false },
"mode": { "type": "string", "enum": ["fast", "slow"], "default": "fast" }
}
}
```

Field types: `string`, `number`, `boolean`, `select`

Field options: `default`, `min`, `max`, `enum`, `description`

Config values are persisted in the database and accessible via the dashboard config page.

## Built-in Events

| Event | When | Payload |
|---|---|---|
| `onRequest` | Before chat handler | Request context |
| `onResponse` | After chat handler | Response data |
| `onError` | On handler error | Error object |
| `onModelSelect` | Model selected for routing | Model info |
| `onComboResolve` | Combo routing resolved | Combo targets |
| `onRateLimit` | Rate limit hit | Limit info |
| `onQuotaExhaust` | Quota exhausted | Quota info |
| `onProviderError` | Provider returned error | Error details |
| `onStreamStart` | SSE stream started | Stream info |
| `onStreamEnd` | SSE stream ended | Stream stats |

## Examples

### Request Logger

```ts
import { definePlugin } from "omniroute/plugins/sdk";

export default definePlugin({
name: "request-logger",
onRequest: async (ctx) => {
console.log(`[${new Date().toISOString()}] ${ctx.method} ${ctx.model} -> ${ctx.provider}`);
},
});
```

### Rate Limiter

```ts
import { definePlugin, blockRequest } from "omniroute/plugins/sdk";

const requests = new Map<string, number[]>();

export default definePlugin({
name: "rate-limiter",
priority: 10,
onRequest: async (ctx) => {
const key = ctx.headers["x-api-key"] || "anonymous";
const now = Date.now();
const window = 60000; // 1 minute
const maxRequests = 100;

const timestamps = (requests.get(key) || []).filter(t => t > now - window);
timestamps.push(now);
requests.set(key, timestamps);

if (timestamps.length > maxRequests) {
return blockRequest({ error: "Rate limit exceeded", status: 429 });
}
},
});
```

### Response Transformer

```ts
import { definePlugin } from "omniroute/plugins/sdk";

export default definePlugin({
name: "response-transformer",
onResponse: async (ctx, response) => {
if (response.choices) {
response.choices = response.choices.map((c: any) => ({
...c,
message: { ...c.message, content: c.message.content.trim() },
}));
}
return response;
},
});
```
1 change: 1 addition & 0 deletions node_modules
20 changes: 18 additions & 2 deletions open-sse/handlers/chatCore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1565,8 +1565,10 @@ export async function handleChatCore({
),
};
}
if (pluginResult?.ctx && "body" in pluginResult.ctx) {
body = (pluginResult.ctx as unknown as Record<string, unknown>).body;
// hooks.ts emitHookBlocking returns the (possibly modified) body directly,
// not nested under a ctx — read it from pluginResult.body.
if (pluginResult?.body !== undefined) {
body = pluginResult.body as typeof body;
}
} catch (pluginErr) {
log?.debug?.(
Expand Down Expand Up @@ -5755,6 +5757,20 @@ export async function handleChatCore({
}
}

// ── Plugin onResponse hook (fire-and-forget notification) ──
try {
const { runOnResponse } = await import("@/lib/plugins/index");
await runOnResponse(
{ requestId: traceId, body, model, provider, apiKeyInfo, metadata: {} },
{ stream: finalStream, headers: responseHeaders }
);
Comment on lines +5760 to +5766

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The return value of runOnResponse is currently ignored in the streaming path. If a plugin modifies the stream or response headers during the onResponse hook, those modifications will be lost. Reassigning finalStream and merging the modified headers back into responseHeaders ensures that stream/header transformations applied by plugins are correctly propagated to the client.

  // ── Plugin onResponse hook ──
  try {
    const { runOnResponse } = await import("@/lib/plugins/index");
    const pluginResult = await runOnResponse(
      { requestId: traceId, body, model, provider, apiKeyInfo, metadata: {} },
      { stream: finalStream, headers: responseHeaders }
    ) as { stream?: any; headers?: Record<string, string> };
    if (pluginResult) {
      if (pluginResult.stream) finalStream = pluginResult.stream;
      if (pluginResult.headers) {
        Object.assign(responseHeaders, pluginResult.headers);
      }
    }
  } catch (pluginErr) {

} catch (pluginErr) {
log?.debug?.(
"PLUGIN",
`onResponse hook error (non-fatal): ${pluginErr instanceof Error ? pluginErr.message : String(pluginErr)}`
);
}

return {
success: true,
response: new Response(finalStream, {
Expand Down
Loading