fix(api): semantic-cache HIT bills incremental cost 0 + X-OmniRoute-Cost-Saved (PRD-2026-06-19)

CHANGELOG.md

@@ -18,6 +18,7 @@ _In development — bullets added per PR; finalized at release._
 
 ### 🐛 Fixed
 
+- **fix(api): cache-HIT `X-OmniRoute-Response-Cost` now reports the incremental cost (≈0), not the original** — on a semantic-cache HIT the gateway serves the stored response **without** an upstream call, but `X-OmniRoute-Response-Cost` was reporting the original call's full cost (recomputed from the cached `usage`). A consumer summing `response-cost` for billing was therefore charging for responses that cost ≈$0 to serve (and stale entries could inflate it). Cache hits now bill `X-OmniRoute-Response-Cost: 0.0000000000` (the real incremental cost), and the avoided cost is surfaced in a new **`X-OmniRoute-Cost-Saved`** header for cache analytics — mirroring the existing `tokens_saved` concept. The MISS path is unchanged. (PRD-2026-06-19-cache-hit-cost-reporting)
 - **fix(models): imported vision-capable models keep their vision capability** — after importing a provider key, vision-capable models (e.g. OpenRouter models whose `architecture` declares image input, and other synced providers) were listed as text-only in `/v1/models` and the dashboard — even though image requests actually worked. Synced model records never captured the vision flag, and the catalog's OpenRouter live-enrichment (which derives vision from `architecture.input_modalities`) is skipped once a provider has synced models. Discovery now captures `supportsVision` at sync time (from `architecture.input_modalities`, the string `architecture.modality`, or a top-level `input_modalities`), mirroring the existing `supportsThinking` capture, and the catalog surfaces `capabilities.vision` for synced models. ([#4264](https://github.com/diegosouzapw/OmniRoute/issues/4264) — thanks @FerLuisxd)
 - **fix(providers): Cloudflare Workers AI model discovery shows model names, not UUIDs** — importing a Cloudflare Workers AI key listed models with internal UUID identifiers (e.g. `429b9e8b-d99e-…`) instead of their usable slugs (`@cf/meta/llama-3.1-8b-instruct`). Cloudflare's `/ai/models/search` returns `{ id: "<uuid>", name: "@cf/…" }`, and discovery was passing the raw objects through — so the UUID `id` became the callable model id. The `cloudflare-ai` discovery now maps each result's `name` → id, surfacing the real `@cf/…` model ids. ([#4259](https://github.com/diegosouzapw/OmniRoute/issues/4259) — thanks @FerLuisxd)
 

docs/reference/API_REFERENCE.md

@@ -74,11 +74,14 @@ Content-Type: application/json
 | `X-OmniRoute-Session-Id` | Response  | Effective session ID used by OmniRoute           |
 | `X-OmniRoute-Request-Id` | Response  | Request correlation id (when known)              |
 | `X-OmniRoute-Version`    | Response  | OmniRoute build version (always present)         |
+| `X-OmniRoute-Cost-Saved` | Response  | USD the cache avoided on a HIT (cache hits only) |
 
 > Nginx note: if you rely on underscore headers (for example `x_session_id`), enable `underscores_in_headers on;`.
 
 > **Cost telemetry headers:** non-streaming success responses also carry the `X-OmniRoute-*` cost-telemetry set — `X-OmniRoute-Response-Cost` (USD, fixed 10 decimals; `0.0000000000` for free/unpriced), `X-OmniRoute-Tokens-In` / `X-OmniRoute-Tokens-Out`, `X-OmniRoute-Model`, `X-OmniRoute-Provider`, `X-OmniRoute-Latency-Ms`, `X-OmniRoute-Cache-Hit`, and `X-OmniRoute-Fallback-Attempts` (only when > 0), plus `X-OmniRoute-Request-Id` and `X-OmniRoute-Version`. These are emitted by chat completions, `/v1/responses`, `/v1/messages`, **and the media endpoints** — `/v1/embeddings`, `/v1/images/generations`, `/v1/audio/speech`, `/v1/audio/transcriptions`, `/v1/rerank`, `/v1/videos/generations`, `/v1/music/generations`, and `/v1/moderations` (always cost `0`). Media cost is computed per modality (per-image, per-second, per-character, per search-unit) when pricing is available, otherwise `0` (fail-open).
 
+> **Cache-hit cost semantics:** on a semantic-cache HIT (`X-OmniRoute-Cache-Hit: true`) no upstream call is made, so `X-OmniRoute-Response-Cost` is `0.0000000000` (the **incremental** cost of serving the hit). The original/would-have-been cost is reported separately in `X-OmniRoute-Cost-Saved`. Billing consumers should sum `X-OmniRoute-Response-Cost` (hits cost nothing); cache analytics can aggregate `X-OmniRoute-Cost-Saved`.
+
 ---
 
 ## Embeddings

docs/reference/openapi.yaml

@@ -1134,6 +1134,13 @@ paths:
               schema:
                 type: string
               description: OmniRoute build version (always present).
+            X-OmniRoute-Cost-Saved:
+              schema:
+                type: string
+              description: >-
+                On a semantic-cache HIT, the original (would-have-been) cost in USD that
+                the cache avoided (fixed 10 decimals). Present only on cache hits;
+                X-OmniRoute-Response-Cost is 0 for the same response (incremental cost).
           content:
             application/json:
               schema:

open-sse/handlers/chatCore/semanticCache.ts

@@ -74,13 +74,17 @@ export async function checkSemanticCache({
         "Content-Type": cachedSse ? "text/event-stream" : "application/json",
         [OMNIROUTE_RESPONSE_HEADERS.cache]: "HIT",
       };
+      // A cache HIT serves WITHOUT an upstream call, so the incremental cost billed to
+      // the client is 0 (consumers that sum X-OmniRoute-Response-Cost must not charge for
+      // hits). The original/would-have-been cost is surfaced via X-OmniRoute-Cost-Saved.
       attachOmniRouteMetaHeaders(headers, {
         provider,
         model,
         cacheHit: true,
         latencyMs: Date.now() - startTime,
         usage: cachedUsage,
-        costUsd: cachedCost,
+        costUsd: 0,
+        costSavedUsd: cachedCost,
       });
       return {
         success: true,

src/domain/omnirouteResponseMeta.ts

@@ -48,6 +48,7 @@ export function formatOmniRouteCost(costUsd: unknown): string {
 export function buildOmniRouteResponseMetaHeaders({
   cacheHit = false,
   costUsd = 0,
+  costSavedUsd = undefined,
   fallbackAttempts = 0,
   latencyMs = 0,
   model = null,
@@ -57,6 +58,14 @@ export function buildOmniRouteResponseMetaHeaders({
 }: {
   cacheHit?: boolean;
   costUsd?: unknown;
+  /**
+   * Cost the cache AVOIDED. A semantic-cache HIT serves at ≈0 incremental cost
+   * (`costUsd: 0`) but saved the original call's cost — surface it here so billing
+   * consumers don't charge for hits while analytics can still see what was saved.
+   * Emitted as `X-OmniRoute-Cost-Saved` only when provided (omitted on normal
+   * responses); pass `0` to explicitly mark a free-model HIT that saved nothing.
+   */
+  costSavedUsd?: unknown;
   fallbackAttempts?: number;
   latencyMs?: unknown;
   model?: string | null;
@@ -86,6 +95,12 @@ export function buildOmniRouteResponseMetaHeaders({
     headers[OMNIROUTE_RESPONSE_HEADERS.provider] = getProviderAlias(provider);
   }
 
+  // Cache-saved cost: emitted only when the caller passes a value (cache HITs), so
+  // non-cache responses keep their existing header shape. `0` is a valid saved cost.
+  if (costSavedUsd != null) {
+    headers[OMNIROUTE_RESPONSE_HEADERS.costSaved] = formatOmniRouteCost(costSavedUsd);
+  }
+
   const attempts = toNonNegativeInteger(fallbackAttempts);
   if (attempts > 0) {
     headers[OMNIROUTE_RESPONSE_HEADERS.fallbackAttempts] = String(attempts);

src/shared/constants/headers.ts

@@ -1,6 +1,7 @@
 export const OMNIROUTE_RESPONSE_HEADERS = {
   cache: "X-OmniRoute-Cache",
   cacheHit: "X-OmniRoute-Cache-Hit",
+  costSaved: "X-OmniRoute-Cost-Saved",
   fallbackAttempts: "X-OmniRoute-Fallback-Attempts",
   latencyMs: "X-OmniRoute-Latency-Ms",
   model: "X-OmniRoute-Model",

tests/unit/chatcore-semantic-cache.test.ts

@@ -18,6 +18,8 @@ const { generateSignature, setCachedResponse, clearCache } = await import(
   "../../src/lib/semanticCache.ts"
 );
 const { OMNIROUTE_RESPONSE_HEADERS } = await import("../../src/shared/constants/headers.ts");
+const { calculateCost } = await import("../../src/lib/usage/costCalculator.ts");
+const { formatOmniRouteCost } = await import("../../src/domain/omnirouteResponseMeta.ts");
 
 test.after(() => {
   core.resetDbInstance();
@@ -305,3 +307,88 @@ test("checkSemanticCache HITs even when the cached body has no usage (cost falls
   assert.equal(persistCalls.length, 1);
   assert.equal(persistCalls[0].tokens, undefined, "no usage -> persisted tokens is undefined");
 });
+
+// ─── Cache-HIT cost reporting (PRD-2026-06-19-cache-hit-cost-reporting) ───────
+// A HIT does NOT call upstream, so the INCREMENTAL cost of serving it is ≈0. The
+// X-OmniRoute-Response-Cost must therefore be 0 (so billing consumers don't charge
+// for cache hits), while the original cost is surfaced via X-OmniRoute-Cost-Saved.
+
+test("checkSemanticCache HIT bills 0 incremental cost and reports the original cost in X-OmniRoute-Cost-Saved", async () => {
+  clearCache();
+  const usage = { prompt_tokens: 1000, completion_tokens: 1000, total_tokens: 2000 };
+  const cached = {
+    id: "chatcmpl-cached-cost-saved",
+    choices: [
+      { index: 0, message: { role: "assistant", content: "cached answer" }, finish_reason: "stop" },
+    ],
+    usage,
+  };
+  const { args } = makeHitArgs({
+    body: {
+      model: "gpt-4o",
+      messages: [{ role: "user", content: "hit query cost-saved" }],
+      temperature: 0,
+    },
+    stream: false,
+  });
+  seedHit(args, cached);
+
+  // The original (would-have-been) cost — computed with the SAME calculator the handler
+  // uses, against the same fresh DATA_DIR, so the values match deterministically.
+  const expectedSaved = formatOmniRouteCost(
+    await calculateCost(args.provider, args.model, usage as Record<string, number>)
+  );
+  assert.notEqual(
+    expectedSaved,
+    "0.0000000000",
+    "sanity: gpt-4o must be priced for this regression to be meaningful"
+  );
+
+  const result = await checkSemanticCache(args as Parameters<typeof checkSemanticCache>[0]);
+  assert.ok(result, "HIT -> non-null result");
+  const res = result.response as Response;
+
+  assert.equal(res.headers.get(OMNIROUTE_RESPONSE_HEADERS.cache), "HIT");
+  // Incremental cost billed to the client on a HIT is 0 (no upstream call happened).
+  assert.equal(
+    res.headers.get(OMNIROUTE_RESPONSE_HEADERS.responseCost),
+    "0.0000000000",
+    "cache HIT must bill 0 incremental cost"
+  );
+  // The avoided cost is surfaced for cache analytics.
+  assert.equal(
+    res.headers.get(OMNIROUTE_RESPONSE_HEADERS.costSaved),
+    expectedSaved,
+    "X-OmniRoute-Cost-Saved reflects the original cost the cache avoided"
+  );
+});
+
+test("checkSemanticCache isolates HITs per apiKeyId (no cross-key cache sharing) [#3740 edge]", async () => {
+  clearCache();
+  const cached = {
+    id: "chatcmpl-cached-isolation",
+    choices: [
+      { index: 0, message: { role: "assistant", content: "key A answer" }, finish_reason: "stop" },
+    ],
+    usage: { prompt_tokens: 5, completion_tokens: 5, total_tokens: 10 },
+  };
+  const sharedBody = () => ({
+    model: "gpt-4o",
+    messages: [{ role: "user", content: "isolation probe" }],
+    temperature: 0,
+  });
+
+  // Seed the cache under apiKeyId "keyA".
+  const { args: argsA } = makeHitArgs({ body: sharedBody(), apiKeyId: "keyA" });
+  seedHit(argsA, cached);
+
+  // A different key with an IDENTICAL body must NOT see keyA's entry (namespaced signature).
+  const { args: argsB } = makeHitArgs({ body: sharedBody(), apiKeyId: "keyB" });
+  const missB = await checkSemanticCache(argsB as Parameters<typeof checkSemanticCache>[0]);
+  assert.equal(missB, null, "keyB must NOT resolve keyA's cached entry (per-key isolation)");
+
+  // keyA itself still gets a HIT.
+  const { args: argsA2 } = makeHitArgs({ body: sharedBody(), apiKeyId: "keyA" });
+  const hitA = await checkSemanticCache(argsA2 as Parameters<typeof checkSemanticCache>[0]);
+  assert.ok(hitA, "keyA must resolve its own cached entry");
+});

tests/unit/omniroute-response-meta.test.ts

@@ -120,3 +120,46 @@ test("buildOmniRouteSseMetadataComment emits comment lines compatible with SSE",
   assert.match(comment, /^: x-omniroute-tokens-out=2/m);
   assert.match(comment, /^: x-omniroute-response-cost=0\.0000000000/m);
 });
+
+test("buildOmniRouteResponseMetaHeaders emits X-OmniRoute-Cost-Saved only when costSavedUsd is provided", () => {
+  // Cache HIT: the incremental cost of serving the hit is 0, but the cache saved the
+  // original (would-have-been) cost — surfaced via the Cost-Saved header for analytics.
+  const hit = buildOmniRouteResponseMetaHeaders({
+    provider: "openai",
+    model: "gpt-4o",
+    cacheHit: true,
+    costUsd: 0,
+    costSavedUsd: 0.0125,
+  });
+  assert.equal(hit[OMNIROUTE_RESPONSE_HEADERS.responseCost], "0.0000000000");
+  assert.equal(hit[OMNIROUTE_RESPONSE_HEADERS.costSaved], "0.0125000000");
+
+  // A normal response (no costSavedUsd) omits the Cost-Saved header entirely.
+  const miss = buildOmniRouteResponseMetaHeaders({
+    provider: "openai",
+    model: "gpt-4o",
+    costUsd: 0.0125,
+  });
+  assert.equal(miss[OMNIROUTE_RESPONSE_HEADERS.costSaved], undefined);
+
+  // A free-model HIT still emits Cost-Saved (= 0) — it explicitly passed costSavedUsd.
+  const freeHit = buildOmniRouteResponseMetaHeaders({
+    cacheHit: true,
+    costUsd: 0,
+    costSavedUsd: 0,
+  });
+  assert.equal(freeHit[OMNIROUTE_RESPONSE_HEADERS.costSaved], "0.0000000000");
+});
+
+test("attachOmniRouteMetaHeaders forwards costSavedUsd onto a Headers bag", () => {
+  const headers = new Headers({ "Content-Type": "application/json" });
+  attachOmniRouteMetaHeaders(headers, {
+    provider: "openai",
+    model: "gpt-4o",
+    cacheHit: true,
+    costUsd: 0,
+    costSavedUsd: 0.0125,
+  });
+  assert.equal(headers.get(OMNIROUTE_RESPONSE_HEADERS.responseCost), "0.0000000000");
+  assert.equal(headers.get(OMNIROUTE_RESPONSE_HEADERS.costSaved), "0.0125000000");
+});