fix(providers): one payload free-evidence predicate for every /models reader - #14930
Merged
diegosouzapw merged 3 commits intoSep 29, 2026
Conversation
… tags Model discovery only read OpenRouter-style `pricing.prompt` / `pricing.completion`. Vercel AI Gateway's /v1/models publishes `pricing.input` / `pricing.output` and a `tags: ["free"]` array, so its zero-priced models synced without `isFree` and never appeared under the provider page's "Free only" filter. - Read `input`/`output` as fallbacks for `prompt`/`completion`. - Accept a `free` tag as live evidence unless the same payload publishes a non-zero price for that model. - Let the default (non-strict) provider-page badge honor explicit `isFree === true` on every provider, as strict mode already does, so live discovery evidence is not dropped for providers outside the static free catalog.
… reader Discovery, the import-only-free filter and the OpenRouter catalog each re-implemented "is this /models entry free", and each read only OpenRouter's pricing.prompt/pricing.completion. Measured on 2026-09-27 across the 47 distinct public provider catalogs reachable from the registry's /models URLs, that dropped real evidence in both directions: - input/output pricing and `free` tags were ignored (Vercel AI Gateway, xKiro): 34 zero-priced models never marked free; - a provider's explicit `isFree: false` / `free: false` was overridden by a 0/0 token price (Kilo, LLM Gateway): 48 models wrongly marked free; - a 0/0 token price marked models billed per request, image, second or minute as free (EUrouter, FastRouter, MegaNova, OFox): 23 models; - a top-level `free: true` with no pricing object was ignored (g4f): 1. hasPayloadFreeEvidence (src/shared/utils/payloadFreeEvidence.ts) applies one precedence to every reader: explicit boolean flag (top-level or in pricing) > `:free` suffix > zero prompt/input and completion/output with no positive price anywhere else in pricing > `free` tag under the same condition. Ratio fields (discount, multiplier, coefficient, thresholds) are not prices. Across those 47 catalogs the free set goes from 277 to 241 models: +35 detected, -71 false positives removed, no other differences.
aldoeliacim
changed the title
diegosouzapw
merged commit Sep 29, 2026
6955fdd
into
diegosouzapw:release/v3.8.51
11 of 16 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every place that reads a provider's live
/modelsentry to decide "is this free" now goes through a single predicate,hasPayloadFreeEvidence(src/shared/utils/payloadFreeEvidence.ts):normalizeDiscoveredModels: theisFreepersisted at model sync, which feeds the provider-page Free only filter and badgeisFreeModel/selectModelsForImport: the "import only free models" connection optionisOpenRouterFreeModel: the OpenRouter entries in/v1/modelsBefore this PR each one re-implemented the check, and all of them read only OpenRouter's
pricing.prompt/pricing.completion.Precedence, strongest first:
isFree/is_free/free, top-level or insidepricing) is the provider's own verdict and wins both ways.:freeid suffix.prompt/inputandcompletion/outputprice, with no positive price anywhere else inpricing(per request, image, second, minute, …). Ratio fields (discount,multiplier,coefficient, thresholds) are not prices.freetag, under the same no-positive-price condition.The default Free-badge rule also honours explicit
isFree === trueon every provider, as strict mode already did. Without that, discovery evidence for providers outsideFREE_MODEL_BUDGETS(e.g.vercel-ai-gateway) was dropped at render time.Measured bug class (not a single instance)
On 2026-09-27 I fetched every public
/modelsURL referenced by the provider registry (104 URLs, 47 distinct catalogs returned a model list without auth). I then ran the realnormalizeDiscoveredModelsfrom the base (a58000c76) and from this branch over the raw payloads:input/outputpricing or afreetag ignored → free model never marked freeisFree: false/free: falseoverridden by a 0/0 token price → marked freefree: truewith nopricingobject ignoredAcross the 47 catalogs the free set goes from 277 to 241 models: 35 added, 71 false positives removed, no other differences. The Vercel case that surfaced this (
poolside/laguna-s-2.1-free,inclusionai/ling-3.0-flash-sante[-free],stealth/pixel-canary) is 4 of the 34.Related Issues / prior art
isFreeModelrequiresproviderHasFreeModels) are unchanged.Validation
payload-free-evidence.test.ts(8), plus 2 new cases inmodel-discovery-free-metadata.test.tsand 1 infree-badge-provider-gate.test.tsnode --import tsx/esm --test): 858 pass, 1 fail. The failure,noauth-catalog-only-providers, fails identically on the clean base and is in the inherited base-red set. The Vitest suite (autoCombo/free-regime-not-read-by-predicate) passes 7/7.npm run typecheck:core→ 0;npm run lint:json -- --max-warnings 0→ 0;prettier --checkcleanFast Quality Gateslist, run locally on this branch and on a clean base worktree: identical results (onlymutation-test-coverageanddashboard-typecheckfail, on both, for files this PR does not touch)release/v3.8.51(0 commits behind)npm run test:scopedreports "hub file changed — run the full suite", so the full shards are left to CI per the golden pathReviewer Notes
hidePaidModelsgate is loosened: those still require a documented provider free tier.isZeroPricestays exported fromcatalogOpenrouter.ts(covered bycatalog-helpers-extraction.test.ts). The private copies inmodelDiscovery.tsandfreeModels.tsare gone.