Skip to content

fix(catalog): re-audit free-tier quotas against official pages - #12649

Merged
diegosouzapw merged 15 commits into
release/v3.8.51from
fix/free-tier-quota-reaudit
Sep 4, 2026
Merged

diegosouzapw merged 15 commits into
release/v3.8.51from
fix/free-tier-quota-reaudit

Conversation

@diegosouzapw

@diegosouzapw diegosouzapw commented Sep 3, 2026 •

Copy link
Copy Markdown
Owner

Summary

Re-audits five free-tier entries of the per-model catalog (open-sse/config/freeModelCatalog.data.ts) against the providers' own pages. On top of the current base (release/v3.8.51 @ 2265ce7, which reclassified Cerebras as a one-time $5 signup credit in #12591, −30M) the public headline lands at ~1.47B documented free tokens/month: steady 1,473,225,000, first month with signup credits 2,099,725,000 (~2.10B).

Isolated effect of this PR on the base: −60M gemini, −20M ollama-cloud, +15M groq, +60M nara → base 1,438,225,000 → 1,473,225,000. The move from the ~1.50B quoted in the first revision of this PR to ~1.47B is entirely the upstream Cerebras reclassification, not a change of this audit.

  • gemini −60M → uncapped (Google removed the per-model free-tier table on 2025-12-23; nothing left to sum)
  • ollama-cloud −20M → uncapped (Free plan is "starter usage credits", no token figure)
  • groq +15M → five per-model 200K-TPD caps (6M each, independent — not a shared pool); three IDs retired from the free tier dropped
  • nara +60M → one 7M/day account bucket = 210M, the 8 models of the Free plan
  • mistral 1B kept — the figure only exists inside the account console; console-verified on 2026-09-02 by the owner (Limits → Free mode → "Tokens per month" = 1,000,000,000) and recorded next to the entry

Every re-audited block now carries an // evidence: comment with its source; the docs, README, free-tiers guide, budget SVG and the CI-gated counts (442 entries / 34 recurring pool keys / 16 quantified pools + 5 Groq caps) follow the catalog.

Provider table

Provider Before After Evidence
gemini 60M (pooled gemini-free, 4 models) uncapped, never summed (4 models) https://ai.google.dev/gemini-api/docs/rate-limits — per-model table removed 2025-12-23; limits "can be viewed in Google AI Studio"
ollama-cloud 20M (pooled, 8 models) uncapped, never summed (8 models) https://ollama.com/pricing — Free: "Starter usage credits included"; https://docs.ollama.com/cloud — "usage resets monthly"; no token figure
groq 15M (one groq pool, 5 models incl. llama-3.3-70b-versatile, llama-4-scout, qwen3-32b) 5 × 6M per-model caps (gpt-oss-120b, gpt-oss-20b, gpt-oss-safeguard-20b, qwen3.6-27b, qwen3.8-27b), hardStopGuaranteed https://console.groq.com/docs/rate-limits — Free Plan Limits: 200K TPD per model, 429 on limit; https://console.groq.com/docs/billing-faqs — no payment method on file; https://console.groq.com/docs/deprecations — retirements 2026-07-17 / 2026-08-16
nara 150M (3 models) 210M (7M/day × 30, one bucket, 8 models) https://router.bynara.id/api/plans — plan free: token_cap_daily=7000000, rpm_limit=15, model list (public, unauthenticated)
mistral 1B (pooled, 5 models) 1B kept (pooled, 5 models) console-verified 2026-09-02 (console.mistral.ai → Limits, Free mode, "Tokens per month" = 1,000,000,000); public pages only confirm the cap exists: https://docs.mistral.ai/admin/billing-usage/usage-limits, https://help.mistral.ai/en/articles/698531

Headline: ~1.47B steady on the current base · pools 37 → 34 (gemini/ollama-cloud left the steady set; Groq's caps are not a shared pool) · entries 437 → 442 vs the base (nara +5, groq retired IDs replaced).

Evidence classes

Every number added or changed by this PR cites its source in a comment next to the entry (FREE_TIERS.md → Methodology → Evidence classes):

  • public-page — a provider page anyone can read (gemini, groq, ollama-cloud);
  • api-public — an unauthenticated endpoint of the provider (nara /api/plans);
  • console-verified <date> por <who> — the figure is only visible inside an account console; the comment records who saw it and when, plus the public page that says the cap exists. Today only mistral is console-verified. The regression test tests/unit/free-tier-reaudit-2026-09.test.ts pins this: the 1B pool is accepted only while a dated console-verified line is present in the source; otherwise every Mistral row must be recurring-uncapped (0).

Entries that predate the 2026-09-02 re-audit inherit the earlier (2026-06-17) research until they are touched; the catalog header now says the file is hand-curated (there never was a generator) and states the rule.

Collateral changes

  • open-sse/config/freeTierCatalog.ts (legacy provider-level map): gemini/ollama-cloud removed, nara 210M, groq 30M (map kept in descending order; cerebras stays removed as fix(providers): reclassify Cerebras as a one-time $5 signup credit #12591 left it — 17 legacy keys, 1,475,025,000).
  • open-sse/config/providers/registry/{groq,nara}: Groq registry gains openai/gpt-oss-safeguard-20b; NaraRouter registry lists the 8 Free-plan models.
  • src/app/(dashboard)/dashboard/combos/page.tsx: FREE_STACK_PRESET_MODELS swaps the retired groq/llama-3.3-70b-versatile for groq/openai/gpt-oss-120b.
  • src/shared/constants/providers/apikey/{frontier-labs,gateways}.ts: Groq freeNote ("30 RPM / 14.4K RPD" was stale) and NaraRouter onboarding note re-written; docs/reference/PROVIDER_REFERENCE.md regenerated (npm run gen:provider-reference).
  • i18n (src/i18n/messages/*.json): providers.onboardingProviderDescriptions.groq and .gemini rewritten in en and pt-BR; the other locales carry the documented __MISSING__:<en> sentinel (runtime falls back to English, queued for the translation pipeline) — except vi, which must not hold placeholders (tests/unit/i18n-vi-completeness.test.ts) and received a real translation that needs a native-speaker review.
  • Docs: README.md (headline, counts, hero/budget alt texts), docs/reference/FREE_TIERS.md (TL;DR, dated re-audit note, contributors, per-provider table, Evidence classes, resolved Groq finding), docs/diagrams/free-tier-budget.svg (bar/grid regenerated for 16 pools + Groq caps; Gemini/Ollama Cloud moved to the "un-countable" row), docs/diagrams/readme-hero.svg, docs/getting-started/FREE-TIERS-GUIDE.md, docs/diagrams/README.md.
  • Tests: tests/unit/free-tier-reaudit-2026-09.test.ts (new, pins every re-audited fact incl. retired Groq IDs and registry routability), free-tier-catalog.test.ts, free-providers-batch-2026-07.test.ts, autoCombo/strict-zero-cost-filter.test.ts (fixture re-pointed to groq/openai/gpt-oss-120b).
  • Base merge: origin/release/v3.8.51 merged in (no rebase); the duplicate in locale removed upstream by feat(i18n): locale-expansion tooling — add-locale orchestrator, aliases, translation-ratio gate; retire duplicate 'in' locale #12496 stays removed.

Validation

Base merged: origin/release/v3.8.51 @ 2265ce7 (8 conflicts resolved — freeTierCatalog.ts, free-tier-catalog.test.ts, README.md, FREE_TIERS.md, FREE-TIERS-GUIDE.md, docs/diagrams/README.md, free-tier-budget.svg, readme-hero.svg). Rule applied: keep the base's Cerebras reclassification (one-time $5 credit — rows, legacy-map removal, cerebras === undefined assertion, per-provider table row) and every re-audited row/prose from this branch, then let npm run check:docs-counts decide every published number. docs/screenshots/free-tier-budget-card.svg was taken from the base unchanged (this branch owns no generator change for it). Totals on the merged head: steady 1,473,225,000 · first month 2,099,725,000 · 34 pool keys · 442 entries · legacy FREE_TIER_BUDGETS 17 keys / 1,475,025,000.

Local gates run on the merged head (this devbox is saturated and the full unit suite times out here; the full matrix runs in CI on this PR):

Check Result
npm run check:docs-counts ✓ All checks pass
npm run check:docs-sync PASS (41 llm.txt mirrors, 41 CHANGELOG translations)
npm run typecheck:core exit 0
npm run check:dashboard-typecheck exit 0 (run in Task 1.4 after the combos/page.tsx preset change)
npm run check:provider-consistency OK (273 REGISTRY entries, 356 canonical providers)
Focused unit tests (free-tier-reaudit-2026-09, free-providers-batch-2026-07, free-tier-catalog, free-model-catalog, free-tier-summary-route, check-docs-counts-sync, removed-providers-blocklist) 83 ok / 0 failed (re-run after the 2265ce7 merge)
Vitest tests/unit/autoCombo/strict-zero-cost-filter.test.ts 15/15
ESLint on the changed .ts/.tsx/.mjs files (project suppressions applied) exit 0, 0 errors
i18n:check-ui-coverage / i18n:check-value-drift:warn / i18n-vi-completeness PASS / PASS / 5/5 (Task 1.4 fix round)

⚠️ base-red inherited: #12581 (🔴 Release branch not green: release/v3.8.51, opened 2026-09-03; the failing gates there do not touch this PR's files)

Refs: internal research 2026-09-02 (primary-source quota audit).

CI triage (first run on 09942a5)

Check Verdict
Docs Gates (fast-path) — check:fabricated-docs flagged /api/plans in FREE_TIERS.md ours, fixed in 113b608 (the NaraRouter example no longer looks like an OmniRoute route)
Fast Quality Gates — file-size gateways.ts 1462 > 1459 ours, rebaselined in 113b608 (+3: nara authHint wrap + Prettier reflow of two pre-existing over-long lines; same rationale as the seekai/logfare/freebuff entries)
Fast Quality Gates — open-sse-typecheck open-sse/executors/glm.ts TS2554 and API Route Typecheck (same error) inherited: glm.ts was last changed on the base by #12179; this PR does not touch it
Unit Tests fast-path shards — stream-failure sanitization / error-boundary / redaction tests (#12506 campaign), sanitizeErrorMessage ReDoS timing inherited: none of those modules are in this diff
Unit Tests fast-path — #7253 release-green: docs contain zero fabricated API/file-ref drift ours, same /api/plans cause, fixed in 113b608

CI triage (final run on 113b608)

Check Verdict
Docs Gates (fast-path), Merge integrity, Vitest (fast-path), Change Classification green
Fast Quality Gates / API Route Typecheck — open-sse/executors/glm.ts TS2554 inherited (#12179 on the base; not in this diff)
Unit Tests fast-path (1–4/4) — 16 failing tests (stream-failure sanitization, credential redaction, HuggingChat/Kiro/codex-image error boundaries, model-sync route) inherited: none of the 24 failing test files are in this diff, and the identical 16 tests fail on unrelated branches fix/reset-aware-model-family (run 33825690606) and fix/overloaded-not-provider-breaker (run 33825612505)

…ier-quota-reaudit

# Conflicts:
#	README.md
#	docs/diagrams/readme-hero.svg
#	src/i18n/messages/in.json
…ier-quota-reaudit

# Conflicts:
#	README.md
#	docs/diagrams/free-tier-budget.svg
#	docs/diagrams/readme-hero.svg
#	open-sse/config/freeModelCatalog.data.ts
…ier-quota-reaudit

# Conflicts:
#	README.md
#	docs/diagrams/README.md
#	docs/diagrams/free-tier-budget.svg
#	docs/diagrams/readme-hero.svg
#	docs/getting-started/FREE-TIERS-GUIDE.md
#	docs/reference/FREE_TIERS.md
#	open-sse/config/freeTierCatalog.ts
#	tests/unit/free-tier-catalog.test.ts
@diegosouzapw
diegosouzapw merged commit 16b0d4e into release/v3.8.51 Sep 4, 2026
14 of 21 checks passed
diegosouzapw added a commit to PixmaNts/OmniRoute that referenced this pull request Sep 4, 2026
… merge

check-file-size counts split('\n') = 1477 for the merged gateways.ts
(origin diegosouzapw#12649 nara wrap at 1462 + Lyceum catalog entry). Keep the
god-file cap in lock-step; do not weaken catalog asserts.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
muhamadgalihsaputra pushed a commit to niyatna/NiyatnaRoute that referenced this pull request Sep 27, 2026
…souzapw#12649)

* test(catalog): pin the 2026-09-02 free-tier re-audit facts for gemini, ollama-cloud, groq, nara and mistral

* fix(catalog): re-audit gemini, ollama-cloud, groq, nara and mistral against official pages

* fix(catalog): restore the console-verified Mistral 1B pool and harden its regression test

* docs(free-tiers): move headline to the re-audited ~1.50B and refresh pool counts

* chore(free-tiers): retire stale Groq free-tier text and preset model; fix catalog header

* docs(free-tiers): state the evidence-comment rule honestly and retire the last "14.4K RPD" Groq texts

* docs(free-tiers): retire the stale Gemini onboarding quota text

* docs(free-tier): refresh catalog-entry counts to 442 after base sync

* docs(free-tiers): restore README spacing lost in the merge and re-sync the guide counts

* docs(free-tiers): re-sync numbers after merging release/v3.8.51 (Cerebras reclassified upstream)

* fix(docs): keep the NaraRouter plans endpoint out of the API-path checker; rebaseline gateways.ts (+3)

---------

Co-authored-by: diegosouzapw <diegosouzapw@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant