Skip to content

fix(usage): bill credit-metered providers by their reported cost - #16087

Merged
diegosouzapw merged 4 commits into
cheaper-inference:release/v3.8.52from
thotypous:feat/credit-metered-provider-cost
Oct 19, 2026
Merged

diegosouzapw merged 4 commits into
cheaper-inference:release/v3.8.52from
thotypous:feat/credit-metered-provider-cost

Conversation

@thotypous

@thotypous thotypous commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • Kiro bills each request in credits, reported in meteringEvent frames ({ unit: "credit", usage }). OmniRoute ignored them and priced Kiro requests from per-token catalog rows that describe no real tariff, so quotas, budgets and the costs dashboard showed an invented amount.
  • The Kiro executor now sums a response's credits and converts them with KIRO_CREDIT_PRICE_USD (default 0.02 USD/credit) into the existing exact-cost field cost_in_usd_ticks, carrying provider_credits alongside. A malformed metering frame fails the stream. KIRO_REQUIRE_CREDITS=true (default off) also fails a response that reports no credits.
  • provider_credits / cost_in_usd_ticks now pass through usage normalization and extraction (usageTracking, usageExtractor), the response sanitizer allow-lists, and the OpenAI Responses translator, so they reach saveRequestUsage.
  • Migration 206_usage_history_provider_reported_cost.sql adds usage_history.provider_cost_usd and usage_history.provider_credits. provider_cost_usd is stored for any provider that reports an exact cost (xAI's cost_in_usd_ticks included). provider_credits is stored only for credit-metered providers.
  • New src/lib/usage/creditMeteredProviders.ts (mirrors flatRateProviders.ts) lists kiro / kr. computeCostFromPricing returns 0 for them, so their token rows are never a cost basis.
  • Every cost surface reads the stored cost and prices only the tokens of rows without one:
    • per-key USD quotas (apiKeyUsageLimits): keeps calculateCostDetailed / priced / hasUnpricedUsage; a group whose rows all carry a stored cost never counts as unpriced;
    • usageStats (stats + connection spend);
    • providerWindowCosts;
    • the analytics queries and route (keeps is_aggregated and includeFlatRateEstimates);
    • the retention rollup (rollupUsageHistoryBeforeDate), which archives a request's stored cost into total_cost and prices the other requests per request, as before.
  • The combo forecast marks credit-metered rows as unpriced, because call_logs carries no cost for them. Pricing coverage then reports the gap instead of a misleading $0.
  • The metered budget needed no change. The chat handlers charge meteredBudgetCost(provider, calculateCost(...)), and calculateCost already short-circuits on cost_in_usd_ticks. A test pins this.

An earlier, deployment-specific version of this change ran in production at UFSCar from September 2026 as a patch on v3.8.50. This version has been running in production on release/v3.8.52 since 2026-10-09.

Related Issues

  • None known.

Validation

Ran in a clean node:26.9.0-trixie container with npm ci --no-audit --no-fund + node scripts/ci/verify-ci-install.mjs (same install as .github/actions/npm-ci-retry), on this PR's head rebased onto release/v3.8.52 @ 2591176.

  • Change type: provider / DB (usage accounting)
  • Focused tests: node --max-old-space-size=8192 --import tsx/esm --import ./open-sse/utils/setupPolyfill.ts --import ./tests/_setup/isolateDataDir.ts --test --test-force-exit --test-concurrency=4 tests/unit/kiro-credits.test.ts tests/unit/usage-cost-accounting.test.ts tests/unit/db/usage-cost-migrations.test.ts tests/unit/usage-analytics-route.test.ts tests/unit/combo-forecast.test.ts tests/unit/executor-kiro.test.ts tests/unit/chatcore-streaming-cost.test.ts tests/unit/metered-budget-must-not-disable-flat-rate.test.ts tests/unit/api-key-usage-limits.test.ts: 96 pass, 0 fail
  • npm run lint (full repo): pass
  • npm run typecheck:core: pass
  • node scripts/check/check-api-typecheck.mjs: pass
  • npm run check:provider-consistency, npm run check:provider-assets, npm run check:known-symbols: pass
  • npm run check:migration-numbering, npm run check:db-rules: pass
  • npm run check:file-size, npm run check:env-doc-sync, npm run check:changelog-integrity: pass
  • Reconciled with the current active release base: rebased onto release/v3.8.52 @ 2591176 (no conflicts); every check above was rerun on the rebased head
  • Production-code changes include a new or updated automated test in this PR

Tests Added Or Updated

  • tests/unit/kiro-credits.test.ts (new): credit accumulation and conversion, invalid metering and an invalid price both throw, normalization and extraction keep credits, persistence and dedup, quota, stats and rollup read the stored cost and a later price change does not rewrite it, a failed unmetered Kiro request costs $0, and the real createSSEStream pipeline (OpenAI passthrough and Responses translate) hands credits to onComplete.
  • tests/unit/usage-cost-accounting.test.ts (new): exact cost persisted for xAI alongside token-priced rows, credit-metered rows never priced by tokens (quota, stats, connection spend), credits stored only for credit-metered providers, a failed token-billed request still costs its tokens, the rollup archives stored cost plus per-request token pricing, and the metered budget is charged the exact cost.
  • tests/unit/db/usage-cost-migrations.test.ts (new): fresh DB has the columns and the 206 marker; upgrade replay leaves existing rows NULL; negative costs are rejected.
  • tests/unit/usage-analytics-route.test.ts (updated): stored cost is used by summary, byProvider, byAccount, byApiKey, byModel and the daily trend; measured and token-priced usage stay apart after rollup; a failed credit-metered request is billed only by its measurement. The existing rollupUsageHistoryBeforeDate prices each request test still passes unchanged.
  • tests/unit/combo-forecast.test.ts (updated): credit-metered rows are unpriced (coverage 0%, cost $0).
  • tests/unit/executor-kiro.test.ts (updated): credits summed across metrics frames, credit-only response, strict mode rejects missing and malformed metering, and fragmented toolUseEvent arguments keep required fields while the response is billed (expected tokens derived from the registry context length).

Coverage Notes

  • open-sse/executors/kiro.ts, kiroCredits.ts: executor-kiro.test.ts, kiro-credits.test.ts.
  • open-sse/utils/usageTracking.ts, handlers/usageExtractor.ts, handlers/responseSanitizer.ts, translator/response/openai-responses.ts: kiro-credits.test.ts (normalize, extract, stream pipeline in both formats).
  • src/lib/usage/* and src/lib/db/usageAnalytics*, src/app/api/usage/analytics/route.ts: usage-cost-accounting.test.ts, kiro-credits.test.ts, usage-analytics-route.test.ts, combo-forecast.test.ts, plus the existing suites that import those modules.
  • Migration 206: db/usage-cost-migrations.test.ts.
  • Frozen-size files stay under their ceilings without touching file-size-baseline.json: the sanitizer usage allow-lists moved to open-sse/handlers/responseSanitizer/usageFields.ts, the reported-cost parsing is readReportedCost() in creditMeteredProviders.ts, and the two identical usage_history row mappers in usageHistory.ts became one mapUsageHistoryRow() in usageHistory/helpers.ts (usageHistory.ts 1227 → 1183, responseSanitizer.ts 1215 → 1194).

Reviewer Notes

  • Migration: 206_usage_history_provider_reported_cost.sql, two nullable REAL columns with CHECK (>= 0). NULL means "no reported cost, price from tokens", so existing rows behave exactly as before. No backfill; already-stored costs are not rewritten.
  • No daily_usage_summary migration. Upstream's retention rollup already stores a priced total_cost per day, and analytics, stats and the preset source read it. A request's reported cost now just goes into that total_cost in place of its token price. A separate cost_tokens_* split on summaries is no longer needed: no reader re-prices a rollup that has a non-zero total_cost, and legacy zero-cost summaries keep their existing input/output fallback.
  • Behavior change for Kiro: a request with no credit metering now costs $0 on every surface. Previously it was priced from estimated tokens. Operators who want such responses rejected can set KIRO_REQUIRE_CREDITS=true.
  • Per-key USD quotas now also count failed requests that carry a reported cost (success = 1 OR provider_cost_usd IS NOT NULL), since the provider billed them. Failed requests without one are still excluded, as before.
  • Analytics flex-tier savings: a raw row's reported cost counts the same on both sides of the comparison, so it contributes no savings.
  • Adding another credit-metered provider is a one-line change to creditMeteredProviders.ts plus that provider's executor metering. Cost surfaces key on the stored provider_cost_usd, not on provider ids.
  • tests/unit/provider-translate-path-golden.test.ts fails on the release tip itself (@ 2591176, without this PR): the committed snapshot still lists claude-code-20250219 in Anthropic-Beta, which the provider service no longer sends. This PR leaves the snapshot alone.

@thotypous
thotypous force-pushed the feat/credit-metered-provider-cost branch from 16dc579 to 9ed1de5 Compare October 9, 2026 18:35
Kiro bills each request in credits, reported in `meteringEvent` frames
(`{ unit: "credit", usage }`). OmniRoute ignored them and priced Kiro
requests from per-token catalog rows that describe no real tariff, so
quotas, budgets and analytics showed an invented amount.

- The Kiro executor sums a response's credits, converts them with
  KIRO_CREDIT_PRICE_USD (default 0.02) into the existing exact-cost field
  `cost_in_usd_ticks`, and carries `provider_credits`. Malformed metering
  fails the stream; KIRO_REQUIRE_CREDITS=true also fails a response that
  reports no credits.
- `provider_credits`/`cost_in_usd_ticks` survive usage normalization,
  extraction, the response sanitizer and the Responses translator.
- Migration 206 adds `usage_history.provider_cost_usd` (any provider that
  reports an exact cost, xAI included) and `provider_credits` (credit-metered
  providers only).
- `creditMeteredProviders.ts` lists kiro/kr; `computeCostFromPricing`
  returns 0 for them, so their token rows are never a cost basis.
- Per-key USD quotas, usage stats, connection spend, provider window
  costs, the analytics route and retention rollups use the stored cost and
  price only the tokens of rows without one. A group whose rows all carry a
  stored cost is never reported as unpriced.
- The combo forecast reports credit-metered rows as unpriced, since
  call_logs carries no cost for them.
@thotypous
thotypous force-pushed the feat/credit-metered-provider-cost branch from 9ed1de5 to c928a0d Compare October 9, 2026 18:47
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Hi @thotypous — closing this pull request to clear the open queue. The branch feat/credit-metered-provider-cost is kept, so the commits stay. Comment here if it should be reopened.

@thotypous

Copy link
Copy Markdown
Contributor Author

Could you reopen this one and mark it for the next cycle (e.g. backlog)? No rush, it doesn't need to make v3.8.52, since it's a larger feature. We have been running this in production since September, though, so it's relatively well tested. I'll retarget and rebase it once the next release branch is cut.

@diegosouzapw

Copy link
Copy Markdown
Collaborator

Hi @thotypous — reopening this pull request. The close was a mistake, and it is back in the merge queue. The branch feat/credit-metered-provider-cost was kept.

@diegosouzapw diegosouzapw reopened this Oct 10, 2026
@thotypous

Copy link
Copy Markdown
Contributor Author

Note on the failing Docs Gates (fast-path) check: it's the migration count. Migration 206 makes check:docs-counts see 203 SQL files while README.md, AGENTS.md, llm.txt and the docs/i18n/* READMEs still say 202. I left it out of this PR so it doesn't touch ~70 translated files that the next count-sync commit will update anyway; happy to add the bump here if you prefer.

The other failing checks (unit fast-path shards, integration 2/2, file-size on executeTargetAttempt.ts) also fail on other PRs against release/v3.8.52 and aren't related to this change.

Brings the PR up to date with the release tip (780 commits) and adapts it:

- Migration renumbered 206 -> 212_usage_history_provider_reported_cost.sql
  (206-211 are taken on the tip), with a version-keyed idempotency guard in
  migrationRunner so a DB that applied the old 206 file skips the re-run.
- The tip moved the usage_history INSERT from usageHistory.saveRequestUsage into
  usageWriteRetry.insertEntry (deferred lock retry); provider_credits and
  provider_cost_usd are now written there.
- readReportedCost also reads the tip's normalized OpenRouter `cost_usd`
  (cheaper-inference#16640), with the same precedence as extractExactCostUsd (ticks first).
- The tip's lightweight API-key totals view (getApiKeyTotalsRows, cheaper-inference#17056) carries
  the stored cost and priceable-token sums like the other raw-row queries.
- costCalculator keeps the credit-metered guard ahead of the tip's long-context
  tier pricing; comboForecast, kiro executor, usageExtractor and usageTracking
  keep both sides' additions.
- usage-analytics-route test clears the analytics response cache per test (the
  route caches any build >= 1 s keyed by query string, which leaked data between
  tests on a loaded runner).
- Migration-count docs 208 -> 209 (README, AGENTS.md, llm.txt and i18n mirrors).
# Conflicts:
#	docs/i18n/am/README.md
#	docs/i18n/ar/README.md
#	docs/i18n/az/README.md
#	docs/i18n/bg/README.md
#	docs/i18n/bn/README.md
#	docs/i18n/bs/README.md
#	docs/i18n/cs/README.md
#	docs/i18n/da/README.md
#	docs/i18n/de/README.md
#	docs/i18n/el/README.md
#	docs/i18n/et/README.md
#	docs/i18n/fa/README.md
#	docs/i18n/fi/README.md
#	docs/i18n/ga/README.md
#	docs/i18n/gu/README.md
#	docs/i18n/ha/README.md
#	docs/i18n/he/README.md
#	docs/i18n/hi/README.md
#	docs/i18n/hr/README.md
#	docs/i18n/hy/README.md
#	docs/i18n/id/README.md
#	docs/i18n/ig/README.md
#	docs/i18n/ja/README.md
#	docs/i18n/ka/README.md
#	docs/i18n/km/README.md
#	docs/i18n/kn/README.md
#	docs/i18n/ko/README.md
#	docs/i18n/lt/README.md
#	docs/i18n/lv/README.md
#	docs/i18n/ml/README.md
#	docs/i18n/mr/README.md
#	docs/i18n/ms/README.md
#	docs/i18n/mt/README.md
#	docs/i18n/my/README.md
#	docs/i18n/ne/README.md
#	docs/i18n/nl/README.md
#	docs/i18n/no/README.md
#	docs/i18n/or/README.md
#	docs/i18n/pa/README.md
#	docs/i18n/pl/README.md
#	docs/i18n/pt-BR/README.md
#	docs/i18n/pt/README.md
#	docs/i18n/ro/README.md
#	docs/i18n/si/README.md
#	docs/i18n/sl/README.md
#	docs/i18n/sr/README.md
#	docs/i18n/sw/README.md
#	docs/i18n/ta/README.md
#	docs/i18n/te/README.md
#	docs/i18n/th/README.md
#	docs/i18n/tr/README.md
#	docs/i18n/uk-UA/README.md
#	docs/i18n/ur/README.md
#	docs/i18n/uz/README.md
#	docs/i18n/vi/README.md
#	docs/i18n/yo/README.md
#	docs/i18n/zh-CN/README.md
#	docs/i18n/zh-TW/README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants