Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,25 @@

## [Unreleased]

---

## [3.2.8] - 2026-03-29

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The release date 2026-03-29 is set to a future date. This seems to be a mistake and should be updated to the actual release date.


### ✨ Enhancements & Refactoring

- **Docker Auto-Update UI** β€” Integrated a detached background update process for Docker Compose deployments. The Dashboard UI now seamlessly tracks update lifecycle events combining JSON REST responses with SSE streaming progress overlays for robust cross-environment reliability.
- **Cache Analytics** β€” Repaired zero-metrics visualization mapping by migrating Semantic Cache telemetry logs directly into the centralized tracking SQLite module.

### πŸ› Bug Fixes

- **Authentication Logic** β€” Fixed a bug where saving dashboard settings or adding models failed with a 401 Unauthorized error when `requireLogin` was disabled. API endpoints now correctly evaluate the global authentication toggle. Resolved global redirection by reactivating `src/middleware.ts`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Incorrect file reference - mentions src/middleware.ts but the file was renamed to src/proxy.ts. Should be updated to reflect the actual file name.

- **CLI Tool Detection (Windows)** β€” Prevented fatal initialization exceptions during CLI environment detection by catching `cross-spawn` ENOENT errors correctly. Adds explicit detection paths for `\AppData\Local\droid\droid.exe`.
- **Codex Native Passthrough** β€” Normalized model translation parameters preventing context poisoning in proxy pass-through mode, enforcing generic `store: false` constraints explicitly for all Codex-originated requests.
- **SSE Token Reporting** β€” Normalized provider tool-call chunk `finish_reason` detection, fixing 0% Usage analytics for stream-only responses missing strict `<DONE>` indicators.
- **DeepSeek <think> Tags** β€” Implemented an explicit `<think>` extraction mapping inside `responsesHandler.ts`, ensuring DeepSeek reasoning streams map equivalently to native Anthropic `<thinking>` structures.

---

## [3.2.7] - 2026-03-29

### Fixed
Expand Down
2 changes: 1 addition & 1 deletion docs/openapi.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
openapi: 3.1.0
info:
title: OmniRoute API
version: 3.2.7
version: 3.2.8
description: |
OmniRoute is a local-first AI API proxy router. It provides an OpenAI-compatible
endpoint that routes requests to multiple AI providers with load balancing,
Expand Down
74 changes: 18 additions & 56 deletions open-sse/handlers/chatCore.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,13 +43,8 @@ import {
} from "@/lib/localDb";
import { getExecutor } from "../executors/index.ts";
import { getCacheControlSettings } from "@/lib/cacheControlSettings";
import {
shouldPreserveCacheControl,
trackCacheMetrics,
recordCacheHit,
type CacheControlMetrics,
} from "../utils/cacheControlPolicy.ts";
import { getCacheMetrics, updateCacheMetrics } from "@/lib/db/settings.ts";
import { shouldPreserveCacheControl, recordCacheHit } from "../utils/cacheControlPolicy.ts";
import { getCacheMetrics } from "@/lib/db/settings.ts";
Comment on lines +46 to +47

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore removed cache metrics symbols

This import change removed the symbols that the non-streaming path still relies on: later in handleChatCore it calls updateCacheMetrics(currentMetrics) before returning. For any stream=false chat request, that now raises a ReferenceError after the upstream response is processed, turning successful non-stream completions into server errors. Please either remove the stale call or keep the metrics variable/import lifecycle consistent.

Useful? React with πŸ‘Β / πŸ‘Ž.


import {
parseCodexQuotaHeaders,
Expand Down Expand Up @@ -701,27 +696,6 @@ export async function handleChatCore({
settings: { alwaysPreserveClientCache: cacheControlMode },
});

// Track cache metrics for this request
let currentMetrics = await getCacheMetrics().catch(() => ({
totalRequests: 0,
requestsWithCacheControl: 0,
totalInputTokens: 0,
totalCachedTokens: 0,
totalCacheCreationTokens: 0,
tokensSaved: 0,
estimatedCostSaved: 0,
byProvider: {},
byStrategy: {},
lastUpdated: new Date().toISOString(),
}));

currentMetrics = trackCacheMetrics({
preserved: preserveCacheControl,
provider,
strategy: comboStrategy,
metrics: currentMetrics,
});

if (preserveCacheControl) {
log?.debug?.(
"CACHE",
Expand Down Expand Up @@ -1473,18 +1447,6 @@ export async function handleChatCore({
(usage as any).prompt_tokens_details?.cache_creation_tokens
);

if (cachedTokens > 0 || cacheCreationTokens > 0) {
currentMetrics = updateCacheTokenMetrics({
metrics: currentMetrics,
provider,
strategy: comboStrategy,
inputTokens,
cachedTokens,
cacheCreationTokens,
costSaved: 0, // Will be calculated based on pricing
});
}

saveRequestUsage({
provider: provider || "unknown",
model: model || "unknown",
Expand Down Expand Up @@ -1649,17 +1611,22 @@ export async function handleChatCore({
(streamUsage as any).prompt_tokens_details?.cache_creation_tokens
);

if (cachedTokens > 0 || cacheCreationTokens > 0) {
currentMetrics = updateCacheTokenMetrics({
metrics: currentMetrics,
provider,
strategy: comboStrategy,
inputTokens,
cachedTokens,
cacheCreationTokens,
costSaved: 0,
});
}
saveRequestUsage({
provider: provider || "unknown",
model: model || "unknown",
tokens: streamUsage,
status: String(streamStatus || 200),
Comment on lines +1614 to +1618

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Avoid double-persisting streaming usage rows

onStreamComplete now writes saveRequestUsage(...), but the stream pipeline already persists usage via logUsage(...) in open-sse/utils/stream.ts (both passthrough and translated modes). This causes each successful streamed request to be recorded twice in usage_history, which inflates analytics/cost tracking and can misfire quota/budget decisions that depend on usage totals.

Useful? React with πŸ‘Β / πŸ‘Ž.

success: streamStatus === 200,
latencyMs: Date.now() - startTime,
timeToFirstTokenMs: ttft,
errorCode: null,
timestamp: new Date().toISOString(),
connectionId: connectionId || undefined,
apiKeyId: apiKeyInfo?.id || undefined,
apiKeyName: apiKeyInfo?.name || undefined,
}).catch((err) => {
console.error("Failed to save usage stats:", err.message);
});
}

persistAttemptLogs({
Expand All @@ -1673,11 +1640,6 @@ export async function handleChatCore({
claudeCacheUsageMeta: cacheUsageLogMeta,
});

// Persist cache metrics to database
updateCacheMetrics(currentMetrics).catch((err) => {
log?.debug?.("CACHE", `Failed to persist cache metrics: ${err?.message || "unknown"}`);
});

if (apiKeyInfo?.id && streamUsage) {
calculateCost(provider, model, streamUsage)
.then((estimatedCost) => {
Expand Down
18 changes: 16 additions & 2 deletions open-sse/handlers/embeddings.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,16 +80,24 @@ export async function handleEmbedding({
};
}

// Build upstream request
// Build upstream request β€” start with standard fields, then forward any extras
// the client sent (e.g. input_type, user, truncate for NVIDIA NIM asymmetric models).
const KNOWN_FIELDS = new Set(["model", "input", "dimensions", "encoding_format"]);

const upstreamBody: Record<string, unknown> = {
model: model,
input: body.input,
};

// Pass optional parameters
if (body.dimensions !== undefined) upstreamBody.dimensions = body.dimensions;
if (body.encoding_format !== undefined) upstreamBody.encoding_format = body.encoding_format;

for (const [key, value] of Object.entries(body)) {
if (!KNOWN_FIELDS.has(key) && value !== undefined) {
upstreamBody[key] = value;
}
}

// Build headers
const headers = {
"Content-Type": "application/json",
Expand All @@ -104,6 +112,12 @@ export async function handleEmbedding({
} else if (providerConfig.authHeader === "x-api-key") {
headers["x-api-key"] = token;
}
} else if (providerConfig.authType !== "none") {
return {
success: false,
status: 401,
error: `No valid authentication token for provider ${provider}. Check provider credentials.`,
};
}

if (log) {
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "omniroute",
"version": "3.2.7",
"version": "3.2.8",
"description": "Smart AI Router with auto fallback β€” route to FREE & cheap models, zero downtime. Works with Cursor, Cline, Claude Desktop, Codex, and any OpenAI-compatible tool.",
Comment on lines 2 to 4

Copilot AI Mar 29, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description calls out functional changes (Docker auto-update UI + cache analytics fixes), but the diff only bumps versions and updates the changelog. If the feature/code changes were intended to be part of this PR, they appear to be missing; otherwise, consider updating the PR description to reflect that this is a release/version bump only (with release notes).

Copilot uses AI. Check for mistakes.
"type": "module",
"bin": {
Expand Down
17 changes: 17 additions & 0 deletions release_notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
## [3.2.8] - 2026-03-29

### ✨ Enhancements & Refactoring

- **Docker Auto-Update UI** β€” Integrated a detached background update process for Docker Compose deployments. The Dashboard UI now seamlessly tracks update lifecycle events combining JSON REST responses with SSE streaming progress overlays for robust cross-environment reliability.
- **Cache Analytics** β€” Repaired zero-metrics visualization mapping by migrating Semantic Cache telemetry logs directly into the centralized tracking SQLite module.

### πŸ› Bug Fixes

- **Authentication Logic** β€” Fixed a bug where saving dashboard settings or adding models failed with a 401 Unauthorized error when `requireLogin` was disabled. API endpoints now correctly evaluate the global authentication toggle. Resolved global redirection by reactivating `src/middleware.ts`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WARNING: Incorrect file reference - mentions src/middleware.ts but the file was renamed to src/proxy.ts. Should be updated to reflect the actual file name.

- **CLI Tool Detection (Windows)** β€” Prevented fatal initialization exceptions during CLI environment detection by catching `cross-spawn` ENOENT errors correctly. Adds explicit detection paths for `\AppData\Local\droid\droid.exe`.
- **Codex Native Passthrough** β€” Normalized model translation parameters preventing context poisoning in proxy pass-through mode, enforcing generic `store: false` constraints explicitly for all Codex-originated requests.
- **SSE Token Reporting** β€” Normalized provider tool-call chunk `finish_reason` detection, fixing 0% Usage analytics for stream-only responses missing strict `<DONE>` indicators.
- **DeepSeek <think> Tags** β€” Implemented an explicit `<think>` extraction mapping inside `responsesHandler.ts`, ensuring DeepSeek reasoning streams map equivalently to native Anthropic `<thinking>` structures.

---

10 changes: 9 additions & 1 deletion src/app/api/v1/embeddings/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {
type EmbeddingProviderNodeRow,
type EmbeddingProvider,
} from "@omniroute/open-sse/config/embeddingRegistry.ts";
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
import { errorResponse, unavailableResponse } from "@omniroute/open-sse/utils/error.ts";
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
import * as log from "@/sse/utils/logger";
import { toJsonErrorPayload } from "@/shared/utils/upstreamError";
Expand Down Expand Up @@ -209,6 +209,14 @@ export async function POST(request) {
`No credentials for embedding provider: ${provider}`
);
}
if (credentials.allRateLimited) {
return unavailableResponse(
HTTP_STATUS.RATE_LIMITED,
`[${provider}] All accounts rate limited`,
credentials.retryAfter,
credentials.retryAfterHuman
);
}
}

const result = await handleEmbedding({
Expand Down
19 changes: 17 additions & 2 deletions src/app/api/v1/images/generations/route.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import {
getAllImageModels,
getImageProvider,
} from "@omniroute/open-sse/config/imageRegistry.ts";
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
import { errorResponse, unavailableResponse } from "@omniroute/open-sse/utils/error.ts";
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
import * as log from "@/sse/utils/logger";
import { toJsonErrorPayload } from "@/shared/utils/upstreamError";
Expand Down Expand Up @@ -156,15 +156,30 @@ export async function POST(request) {
`No credentials for image provider: ${provider}`
);
}
if (credentials.allRateLimited) {
return unavailableResponse(
HTTP_STATUS.RATE_LIMITED,
`[${provider}] All accounts rate limited`,
credentials.retryAfter,
credentials.retryAfterHuman
);
}
} else if (isCustomModel) {
// Custom models need credentials from the provider connection
credentials = await getProviderCredentials(provider);
if (!credentials) {
return errorResponse(
HTTP_STATUS.BAD_REQUEST,
`No credentials for custom image provider: ${provider}`
);
}
if (credentials.allRateLimited) {
return unavailableResponse(
HTTP_STATUS.RATE_LIMITED,
`[${provider}] All accounts rate limited`,
credentials.retryAfter,
credentials.retryAfterHuman
);
}
}

const result = await handleImageGeneration({
Expand Down
10 changes: 9 additions & 1 deletion src/app/api/v1/providers/[provider]/embeddings/route.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { CORS_ORIGIN } from "@/shared/utils/cors";
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
import { errorResponse, unavailableResponse } from "@omniroute/open-sse/utils/error.ts";
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
import { getRegistryEntry } from "@omniroute/open-sse/config/providerRegistry.ts";
import {
Expand Down Expand Up @@ -85,6 +85,14 @@ export async function POST(request, { params }) {
if (!credentials) {
return errorResponse(HTTP_STATUS.BAD_REQUEST, `No credentials for provider: ${rawProvider}`);
}
if (credentials.allRateLimited) {
return unavailableResponse(
HTTP_STATUS.RATE_LIMITED,
`[${rawProvider}] All accounts rate limited`,
credentials.retryAfter,
credentials.retryAfterHuman
);
}

const result = await handleEmbedding({ body, credentials, log });

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { CORS_ORIGIN } from "@/shared/utils/cors";
import { handleImageGeneration } from "@omniroute/open-sse/handlers/imageGeneration.ts";
import { errorResponse } from "@omniroute/open-sse/utils/error.ts";
import { errorResponse, unavailableResponse } from "@omniroute/open-sse/utils/error.ts";
import { HTTP_STATUS } from "@omniroute/open-sse/config/constants.ts";
import {
getProviderCredentials,
Expand Down Expand Up @@ -85,6 +85,14 @@ export async function POST(request, { params }) {
`No credentials for image provider: ${rawProvider}`
);
}
if (credentials.allRateLimited) {
return unavailableResponse(
HTTP_STATUS.RATE_LIMITED,
`[${rawProvider}] All accounts rate limited`,
credentials.retryAfter,
credentials.retryAfterHuman
);
}

const result = await handleImageGeneration({ body, credentials, log });

Expand Down
2 changes: 1 addition & 1 deletion src/proxy.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { isModelSyncInternalRequest } from "./shared/services/modelSyncScheduler

const SECRET = new TextEncoder().encode(process.env.JWT_SECRET || "");

export async function proxy(request) {
export async function proxy(request: any) {
const { pathname } = request.nextUrl;

// Pipeline: Add request ID header for end-to-end tracing
Expand Down
4 changes: 4 additions & 0 deletions src/shared/utils/apiAuth.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,10 @@ export async function verifyAuth(request: any): Promise<string | null> {
* need to conditionally skip auth should check that separately.
*/
export async function isAuthenticated(request: Request): Promise<boolean> {
// If settings say login/auth is disabled, treat all requests as authenticated
if (!(await isAuthRequired())) {
return true;
}
// 1. Check API key (for external clients)
const authHeader = request.headers.get("authorization");
if (authHeader?.startsWith("Bearer ")) {
Expand Down
Loading
Loading