feat(routing): report why the zero-cost guard excluded a candidate

changelog.d/features/12319-strict-zero-cost-exclusion-reason.md

@@ -0,0 +1 @@
+- **feat(routing):** With `freeAccessPolicy: "strict"`, the read-only candidate listing (`GET /v1/auto-combo/{channel}/candidates`) no longer hides the candidates the zero-cost guard excludes — the same read-only transparency the resilience filter already honours (#9133). Each candidate now carries `freeAccessExclusion` saying why it would be kept out, and it tells an exhausted allowance apart from a quota reading that never arrived or went stale, which used to look identical from the outside. Routing is unchanged: the listing reports, it never enforces. The separate `excludeTosAvoid` guard still drops its candidates without a reason; that gap is now documented rather than closed ([#12319](https://github.com/diegosouzapw/OmniRoute/pull/12319))

docs/routing/STRICT_ZERO_COST.md

@@ -116,13 +116,37 @@ no waiting out the TTL after a 402/403/quota-exhausted response.
 `freeAccessPolicy`: a candidate can be economically `SAFE` and still excluded here for
 contractual reasons, or left in when this guard is off even with `freeAccessPolicy: "strict"` on.
 
-## What passes today
-
-Run `npx tsx scripts/ad-hoc/dry-run-strict-zero-cost.ts` against a live instance's
-`GET /v1/auto-combo/{channel}/candidates` output for a real before/after — the script reads each
-candidate's real `connectionId`, so it also proves the connection-safety fix live, not just in
-unit tests. Keyless candidates must arrive with the synthetic no-auth `connectionId`, never a
-real connection. The current built-in keyless auto path is OpenCode Free; exact candidate counts
+## Seeing what the guard excludes
+
+`GET /v1/auto-combo/{channel}/candidates` lists every candidate, including the ones this guard
+would keep out of dispatch, and each carries `freeAccessExclusion` — `null` when the guard is
+satisfied, otherwise the reason. The listing reports; it never enforces. Turning the policy off
+leaves the field `null` everywhere and costs nothing.
+
+| `freeAccessExclusion`  | What it means                                                                                                 | What to do about it                                                                                                                 |
+| :--------------------- | :------------------------------------------------------------------------------------------------------------ | :---------------------------------------------------------------------------------------------------------------------------------- |
+| `not-in-catalog`       | The provider/model pair is absent from `FREE_MODEL_BUDGETS`.                                                  | Add a curated entry, or accept that new pairs start excluded — that is the design.                                                  |
+| `regime-not-free`      | Catalogued, but its `freeType` is not one that grants free access (a discontinued tier, for instance).        | Nothing to fix. The model costs money.                                                                                              |
+| `no-hard-stop`         | Free regime, but `hardStopGuaranteed` is not `true`, so exceeding the allowance might silently start billing. | Verify the provider's terms and set the flag with the source in a comment — never to grow the catalog.                              |
+| `contradictory-noauth` | A no-auth candidate whose catalog entry is not `keyless`. Fail-closed on inconsistent metadata.               | Fix the catalog entry; the two facts disagree.                                                                                      |
+| `exhausted`            | A fresh reading says the allowance is used up.                                                                | Wait for the reset. This one resolves itself.                                                                                       |
+| `state-unknown`        | No quota reading, or one too old to trust.                                                                    | Go look: the provider may have no usage adapter registered, or the quota fetch is failing.                                          |
+| `no-connection`        | The candidate carries no account to check at all.                                                             | Not a quota problem: the candidate was built without a connection, so nothing was ever looked up. Check how the pool was assembled. |
+
+The last two are the pair worth separating. An exhausted allowance resets on its own; a reading
+that never arrives means the lookup itself is broken, and until now both looked identical from
+outside — the candidate simply vanished.
+
+**One gap remains, and it is deliberate.** `excludeTosAvoid` still removes candidates before the
+listing is built, so a model curated `tos: "avoid"` is absent with no reason given — the same
+invisibility this section just closed for the zero-cost guard. Closing it too means deciding what
+a ToS exclusion should report, which is a separate question from economic safety; this page names
+the gap rather than pretending it is not there.
+
+For an offline before/after, `npx tsx scripts/ad-hoc/dry-run-strict-zero-cost.ts` still works
+against a live instance's candidates output; it reads each candidate's real `connectionId`, so it
+also exercises the connection-safety path. Keyless candidates must arrive with the synthetic
+no-auth `connectionId`, never a real connection. The current built-in keyless auto path is OpenCode Free; exact candidate counts
 still depend on live model discovery and should be measured on the target deployment instead of
 copied from an older run. A `recurring-*` candidate passes only when it has both a registered
 usage adapter and `hardStopGuaranteed: true`; incomplete metadata remains fail-closed.

open-sse/handlers/autoComboCandidates.ts

@@ -30,6 +30,7 @@ import { buildErrorBody } from "@omniroute/open-sse/utils/error.ts";
 import { getCircuitBreaker } from "@/shared/utils/circuitBreaker";
 import { isModelLocked } from "@omniroute/open-sse/services/accountFallback.ts";
 import { parseModel } from "@omniroute/open-sse/services/model.ts";
+import type { StrictZeroCostExclusionReason } from "@omniroute/open-sse/services/autoCombo/strictZeroCostFilter.ts";
 import { getProviderConnectionById } from "@/lib/db/providers";
 import { getExcludedConnectionIds } from "@/lib/db/autoCandidateOverrides";
 
@@ -51,6 +52,13 @@ export interface AutoComboCandidateView {
   breakerState: string;
   connectionCooldown: boolean;
   modelLocked: boolean;
+  /**
+   * Why STRICT_ZERO_COST would exclude this candidate from dispatch, or null
+   * when it would not — and null as well when the policy is off, which is the
+   * default. Reported, never enforced: this listing shows the candidate either
+   * way, the routing path is what acts on it.
+   */
+  freeAccessExclusion: StrictZeroCostExclusionReason | null;
 }
 
 export interface AutoComboCandidatesResult {
@@ -69,6 +77,7 @@ async function decorateCandidate(candidate: {
   connectionId: string;
   model: string;
   modelStr: string;
+  freeAccessExclusion?: StrictZeroCostExclusionReason | null;
 }): Promise<AutoComboCandidateView> {
   const breaker = getCircuitBreaker(candidate.provider);
   const breakerStatus = breaker.getStatus();
@@ -111,6 +120,7 @@ async function decorateCandidate(candidate: {
     breakerState: String(breakerStatus.state),
     connectionCooldown,
     modelLocked,
+    freeAccessExclusion: candidate.freeAccessExclusion ?? null,
   };
 }
 
@@ -160,6 +170,7 @@ export async function getAutoComboCandidates(
     connectionId: string | null;
     allowedConnectionIds?: string[];
     model: string;
+    freeAccessExclusion?: StrictZeroCostExclusionReason | null;
   }> = Array.isArray(virtualCombo?.models) ? virtualCombo.models : [];
   // Routing keeps one logical provider/model candidate, but the management API
   // remains account-oriented so operators can inspect and toggle each fallback.
@@ -178,6 +189,7 @@ export async function getAutoComboCandidates(
         connectionId: candidate.connectionId,
         model: candidate.model,
         modelStr: candidate.model,
+        freeAccessExclusion: candidate.freeAccessExclusion,
       });
       return { ...decorated, excluded: excludedConnectionIds.has(candidate.connectionId) };
     })

open-sse/services/autoCombo/strictZeroCostFilter.ts

@@ -129,25 +129,52 @@ export function findBudgetEntry(
   return catalog.find((m) => m.provider === candidate.provider && m.modelId === candidate.model);
 }
 
-function isConnectionStateSafe(
+/** Why the guard cannot trust a candidate right now. */
+export type StrictZeroCostExclusionReason =
+  | "not-in-catalog"
+  | "regime-not-free"
+  | "no-hard-stop"
+  | "contradictory-noauth"
+  | "exhausted"
+  | "state-unknown"
+  | "no-connection";
+
+export type StrictZeroCostVerdict =
+  { outcome: "safe"; safeConnectionIds: string[] } | { outcome: StrictZeroCostExclusionReason };
+
+/**
+ * Why one connection cannot be trusted right now. Splitting "exhausted" from
+ * "state-unknown" is the whole point: an exhausted allowance resets on its own
+ * and the operator waits, while a missing or stale reading means the quota
+ * lookup itself is not working and the operator has to go fix something.
+ * Opposite actions, and until now the same silence.
+ *
+ * Freshness is checked before status, so a stale EXHAUSTED reading reports
+ * "state-unknown" rather than asserting an exhaustion nobody has confirmed
+ * lately. The exclusion verdict is identical either way -- only the reason
+ * shown to the operator differs.
+ */
+export function classifyConnectionState(
   provider: string,
   connectionId: string,
   resolveFreeAccessState: StrictZeroCostOptions["resolveFreeAccessState"],
   options: Pick<StrictZeroCostOptions, "minRemainingAllowance" | "maxStateAgeMs" | "now">
-): boolean {
+): "safe" | "exhausted" | "state-unknown" {
   const state = resolveFreeAccessState(provider, connectionId);
-  if (!state) return false; // no usage adapter for this provider, or lookup never ran/is stale
-  if (state.status !== "SAFE") return false;
+  if (!state) return "state-unknown"; // no usage adapter for this provider, or lookup never ran
 
   const now = (options.now ?? Date.now)();
   const checkedAtMs = Date.parse(state.checkedAt);
-  if (!Number.isFinite(checkedAtMs) || now - checkedAtMs > options.maxStateAgeMs) return false;
+  if (!Number.isFinite(checkedAtMs) || now - checkedAtMs > options.maxStateAgeMs)
+    return "state-unknown";
 
-  if (state.remainingFreeAllowance === null) return false;
+  if (state.status === "EXHAUSTED") return "exhausted";
+  if (state.status !== "SAFE") return "state-unknown";
+  if (state.remainingFreeAllowance === null) return "state-unknown";
   // A negative threshold would let a negative/garbage reading pass; a caller
   // that genuinely wants "any allowance greater than zero" should pass 0.
-  if (options.minRemainingAllowance < 0) return false;
-  return state.remainingFreeAllowance > options.minRemainingAllowance;
+  if (options.minRemainingAllowance < 0) return "state-unknown";
+  return state.remainingFreeAllowance > options.minRemainingAllowance ? "safe" : "exhausted";
 }
 
 /**
@@ -168,7 +195,28 @@ export function evaluateCandidateConnections(
   resolveFreeAccessState: StrictZeroCostOptions["resolveFreeAccessState"],
   options: Pick<StrictZeroCostOptions, "minRemainingAllowance" | "maxStateAgeMs" | "now">
 ): string[] {
-  if (!budgetEntry) return []; // not in the catalog at all → paid, or genuinely unknown
+  const verdict = classifyStrictZeroCostCandidate(
+    candidate,
+    budgetEntry,
+    resolveFreeAccessState,
+    options
+  );
+  return verdict.outcome === "safe" ? verdict.safeConnectionIds : [];
+}
+
+/**
+ * Same decision as `evaluateCandidateConnections`, but it says why instead of
+ * answering with an empty list. The read-only candidate listing needs the why;
+ * the pool filter only needs the list, so it reads this one and throws the
+ * reason away.
+ */
+export function classifyStrictZeroCostCandidate(
+  candidate: StrictZeroCostCandidate,
+  budgetEntry: FreeModelBudget | undefined,
+  resolveFreeAccessState: StrictZeroCostOptions["resolveFreeAccessState"],
+  options: Pick<StrictZeroCostOptions, "minRemainingAllowance" | "maxStateAgeMs" | "now">
+): StrictZeroCostVerdict {
+  if (!budgetEntry) return { outcome: "not-in-catalog" }; // paid, or genuinely unknown
 
   const isGenuineNoAuthCandidate = candidate.connectionId === SYNTHETIC_NOAUTH_CONNECTION_ID;
   if (allowsNoAuthShortcut(budgetEntry.freeType)) {
@@ -180,30 +228,47 @@ export function evaluateCandidateConnections(
     // any other freeType, and is excluded there unless hardStopGuaranteed is
     // also set for it (which the curated catalog does not do for keyless
     // entries today, so it will correctly exclude).
-    if (isGenuineNoAuthCandidate) return [SYNTHETIC_NOAUTH_CONNECTION_ID];
+    if (isGenuineNoAuthCandidate)
+      return { outcome: "safe", safeConnectionIds: [SYNTHETIC_NOAUTH_CONNECTION_ID] };
   }
-  if (!grantsFreeAccess(budgetEntry.freeType)) return [];
-  if (isGenuineNoAuthCandidate) return []; // no-auth path but a non-keyless catalog entry: contradictory metadata, fail closed
+  if (!grantsFreeAccess(budgetEntry.freeType)) return { outcome: "regime-not-free" };
+  // no-auth path but a non-keyless catalog entry: contradictory metadata, fail closed
+  if (isGenuineNoAuthCandidate) return { outcome: "contradictory-noauth" };
 
   // Every remaining freeType (recurring-*, one-time-initial, a keyless entry
   // reached via a real connection, and any future type this module doesn't
   // special-case) requires a documented hard stop before any live check even
   // runs — no point burning a quota lookup on a connection we could never
   // trust regardless of its answer.
-  if (budgetEntry.hardStopGuaranteed !== true) return [];
+  if (budgetEntry.hardStopGuaranteed !== true) return { outcome: "no-hard-stop" };
 
   const candidateConnectionIds = candidate.connectionId
     ? [candidate.connectionId]
     : (candidate.allowedConnectionIds ?? []);
 
+  // No account at all to check. Reporting `state-unknown` here would send the
+  // operator hunting a quota lookup that was never attempted; this is a wiring
+  // problem, not a quota one.
+  if (candidateConnectionIds.length === 0) return { outcome: "no-connection" };
+
   const safe: string[] = [];
+  // An observed exhaustion outranks a missing reading: one is a fact, the other
+  // is the absence of one, and the operator needs the fact. Without this rule the
+  // reason would depend on the order the connections happen to be listed in.
+  let sawExhausted = false;
   for (const connectionId of candidateConnectionIds) {
     if (connectionId === SYNTHETIC_NOAUTH_CONNECTION_ID) continue; // never reachable here, defensive
-    if (isConnectionStateSafe(candidate.provider, connectionId, resolveFreeAccessState, options)) {
-      safe.push(connectionId);
-    }
+    const state = classifyConnectionState(
+      candidate.provider,
+      connectionId,
+      resolveFreeAccessState,
+      options
+    );
+    if (state === "safe") safe.push(connectionId);
+    else if (state === "exhausted") sawExhausted = true;
   }
-  return safe;
+  if (safe.length > 0) return { outcome: "safe", safeConnectionIds: safe };
+  return { outcome: sawExhausted ? "exhausted" : "state-unknown" };
 }
 
 /**

open-sse/services/autoCombo/virtualFactory.ts

@@ -37,7 +37,12 @@ import {
   orderPoolByRung,
   type LadderOptions,
 } from "./subscriptionLadder";
-import { filterStrictZeroCostCandidates, filterTosAvoidCandidates } from "./strictZeroCostFilter";
+import {
+  classifyStrictZeroCostCandidate,
+  filterStrictZeroCostCandidates,
+  filterTosAvoidCandidates,
+  findBudgetEntry,
+} from "./strictZeroCostFilter";
 import { resolveFreeAccessState } from "./freeAccessQuota";
 import { isModelExcludedByConnection } from "@/domain/connectionModelRules";
 import { resolveProviderAlias } from "../model.ts";
@@ -106,6 +111,13 @@ export interface VirtualAutoComboCandidate {
   resolvedSupportsVision?: boolean;
   resolvedReasoning?: boolean;
   resolvedSupportsThinking?: boolean;
+  /**
+   * Why STRICT_ZERO_COST would exclude this candidate, or null when it would
+   * not. Only populated for the read-only inspector build (`skip`), where the
+   * guard is deliberately not applied — dispatch builds leave it undefined and
+   * do no extra work.
+   */
+  freeAccessExclusion?: import("./strictZeroCostFilter").StrictZeroCostExclusionReason | null;
 }
 
 type VirtualAutoCombo = AutoComboConfig & {
@@ -119,6 +131,9 @@ type VirtualAutoCombo = AutoComboConfig & {
     allowedConnectionIds?: string[];
     weight: number;
     label: string;
+    /** Carried through from the candidate for the read-only inspector; absent
+     * on every dispatch build. */
+    freeAccessExclusion?: import("./strictZeroCostFilter").StrictZeroCostExclusionReason | null;
   }>;
   /** MAX of candidates' context windows — safe to advertise because the
    * auto-combo context pre-filter routes oversized requests to large-window
@@ -749,17 +764,42 @@ export async function prepareVirtualAutoComboInputs(
     // per-candidate: `resolveFreeAccessState` here is a raw pass-through of the real
     // per-(provider,connectionId) resolver; the filter itself decides which connection(s)
     // on each candidate to check and rewrites `allowedConnectionIds` to the SAFE subset.
-    const strictFilteredPool = filterStrictZeroCostCandidates(pool, {
-      enabled: settings.freeAccessPolicy === "strict",
-      resolveFreeAccessState,
+    const strictZeroCostThresholds = {
       // 1 percentage point of headroom, not 0: `freeAccessQuota.ts` reports
       // remaining allowance as a percentage, and a raw ">0" comparison would
       // let a reading of e.g. 0.3% (rounding noise, not real headroom) pass.
       minRemainingAllowance: 1,
       maxStateAgeMs: toNumber(settings.autoRefreshProviderQuotaInterval, 180) * 1000,
+    };
+    const strictZeroCostOn = settings.freeAccessPolicy === "strict";
+    const strictFilteredPool = filterStrictZeroCostCandidates(pool, {
+      // The read-only candidate inspector (#9133) must be able to see what the
+      // guard would exclude, and why — the same opt-out the resilience filter
+      // already honours through `skip`. Dispatch (`skip === false`) is unaffected.
+      enabled: strictZeroCostOn && !skip,
+      resolveFreeAccessState,
+      ...strictZeroCostThresholds,
     });
     if (strictFilteredPool !== pool) pool = strictFilteredPool;
 
+    // Annotate here rather than in the handler: this is where the thresholds and
+    // `resolveFreeAccessState` already live. Doing it downstream would mean a second
+    // copy of both, with nothing to keep them in agreement.
+    if (strictZeroCostOn && skip) {
+      pool = pool.map((candidate) => {
+        const verdict = classifyStrictZeroCostCandidate(
+          candidate,
+          findBudgetEntry(candidate),
+          resolveFreeAccessState,
+          strictZeroCostThresholds
+        );
+        return {
+          ...candidate,
+          freeAccessExclusion: verdict.outcome === "safe" ? null : verdict.outcome,
+        };
+      });
+    }
+
     // Separate, optional ToS guard — independent of economic safety on purpose.
     const tosFilteredPool = filterTosAvoidCandidates(pool, settings.excludeTosAvoid === true);
     if (tosFilteredPool !== pool) pool = tosFilteredPool;
@@ -1058,6 +1098,9 @@ export async function createVirtualAutoComboFromPrepared(
       : {}),
     weight: snapshotScores.get(candidate.modelStr) ?? 1,
     label: candidate.provider,
+    ...(candidate.freeAccessExclusion === undefined
+      ? {}
+      : { freeAccessExclusion: candidate.freeAccessExclusion }),
   }));
   const autoConfig = {
     candidatePool: providerPool,