Skip to content

feat: report cached input tokens - #16452

Open
KiaroSama wants to merge 45 commits into
cheaper-inference:release/v3.8.53from
KiaroSama:feature/cached-input-reporting
Open

KiaroSama wants to merge 45 commits into
cheaper-inference:release/v3.8.53from
KiaroSama:feature/cached-input-reporting

Conversation

@KiaroSama

@KiaroSama KiaroSama commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Report provider-recorded cache-read input tokens separately in analytics KPIs, model/account/key breakdowns and legacy statistics. Preserve input/output and billing arithmetic. Retain archive measurements and explicitly mark unknown legacy history or partial identity subtotals. Legacy rows without the new field retain their existing display.

Target and current source

  • Target: release/v3.8.53; integrated freshly fetched official da1e6c79fb51bf4615e4f1db03a49cc8a09196fb through a normal merge, including the now-landed mutation test registration repair.
  • Current published head: acc6289349f787d6ca3fcbd3562f906d5bf45ddd; exact local/fork/live PR equality verified; current-head hosted acceptance pending.

Analytics fixture timing repair

The preceding head's completed UI job125537716328 failed only the active analytics case:1059ms against a1000ms case override;4107other tests passed and1was skipped. The mounted case now uses the existing repository20000ms UI budget, documented for shared-runner React rendering. Its payload, translated60/Partial assertion, exactly-one-fetch checks, fake-timer control and cleanup are unchanged; the three static-render cases retain1000ms. Scoped format/ESLint and native commit checks passed. Local runtime was not rerun because @radix-ui/react-select is absent; no dependency install or substitute mock was used. Current-head hosted Vitest job125584593712 now passes all4cases in this file (995ms total); the active analytics case passed in722ms. The whole job passes588core tests and4108UI tests, with1UI skip and no failed files. This validates the fixture within the existing budget, not a measured rendering-performance improvement. Final coverage/admission remains pending.

Hosted storage repair

The previous head's Unit4 job https://github.com/cheaper-inference/OmniRoute/actions/runs/41405772081/job/125149337918 stopped with runner ENOSPC, not a completed test result. Seven other raw coverage artifacts each occupied 1.37–1.56 GB compressed. This repair reclaims only the unused Android SDK on disposable GitHub-hosted Linux unit runners before dependency restoration. Self-hosted machines, toolcache, Node, browsers, Docker and swap remain untouched. The collection EXIT diagnostic records free disk/raw coverage size while preserving the test's exit status. No test, coverage source map, shard, threshold or raw-upload requirement was removed.

Persistent workflow contract test: before repair 0/2 pass; afterward 2/2 pass, 0 failures/skips, 355.6942 ms. Native formatting/lint/commit checks passed. The action has not been executed on a hosted runner yet: actual recovered space and elimination of ENOSPC remain pending current-head CI. No SDK cleanup was executed locally and no dependencies/application were installed.

Current migration repair

The official release now owns 223_call_logs_queue_wait_ms.sql. Cache SQL moves byte-identically to 224_daily_cached_input.sql; no dummy SQL, gap allowlist or runner-policy change.

  • Exact-name compatibility for published206/213/214/215/217/218/223 targets224, preserving measured values and timestamps while canonical migrations execute in their own slots.
  • The real-runner regression covers256ledger combinations, legacy NULL versus measured60, canonical223 replacement, repeated-startup no-op and fail-closed unknown target occupancy.
  • A separate case preserves the already-published224history alongside canonical223.

Evidence

Focused isolated migration command:

node --import tsx/esm --test --test-concurrency=1 --test-timeout=15000 --test-reporter=spec tests/unit/cached-input-migration.test.ts

Recorded output for the migration repair retained in the current head:

ℹ tests 3
ℹ pass 3
ℹ fail 0
ℹ cancelled 0
ℹ skipped 0
ℹ duration_ms 1005.6845

The compatibility-data snapshot passed8/8 on its earlier223target; its updated224target/46entry shape is source-reviewed, not rerun. These are focused results, not final hosted acceptance.

  • Current isolated migration regression:3passed,0failed/skipped/cancelled,1005.6845ms;256combination case705.5011ms.
  • Numbering gate passed:221SQL files,3established gaps,0duplicates. Count validation has zero strict failures/two soft warnings.135numeric-only documents synchronized; existing provenance not fabricated. No baseline or assertion weakening.
  • Focused formatting/lint and native commit controls passed; optional private UI-kit reference check skipped because its clone is absent.
  • Existing analytics, archive, pricing and translated UI fixtures remain. No application activation, dependency installation, real provider request or operator database access.

Remaining acceptance

Current hosted checkpoint — checked 2026-11-03

Current-source Vitest job125584593712 in CI run41531615752 completed successfully:588core tests and4108UI tests passed,1UI skip. The repaired analytics file passes4/4, including its actual-payload mounted case. Quality run41531615667 has successful Fast Quality and Gate / Quality results. The CI run API reports head acc6289349f787d6ca3fcbd3562f906d5bf45ddd; it was still in progress at this checkpoint, so final coverage/ratchet/admission is not claimed complete.

Historical pending statements below remain scoped to those earlier checkpoints. Combined migration compatibility, authenticated-server/browser and package/platform acceptance are separate outstanding obligations. No new CI run, installation or paid request was initiated for this update.

Shared repair #18658 is now MERGED and included through the canonical release history. All 68 conflicts were count-only: retain 220 SQL files and incoming provider/source repairs. The automatic UsageAnalytics merge retains both upstream animated counters and this contribution's cached-input subvalue; migration SQL, compatibility data and regression fixtures are unchanged by the merge.

Fresh merged-tree documentation mirror/version sync, counts and migration numbering passed (220 SQL, 3 established gaps, no duplicates); native formatting/lint/commit controls passed. The focused analytics UI test could not collect tests locally because the existing checkout lacks @radix-ui/react-select: 0 tests ran, so UI success is not claimed. No dependency was installed or mock added to hide that gap; hosted CI must validate the merged UI and full candidate.

Shared guard repair #18693 is now MERGED as 075d6e5bcbe87a21355e0b2aac7bc0f5eb96101c and included through a fresh conflict-free canonical merge. This addresses all three inherited unit guards without loosening assertions. The focused provider-banner regression now passes 6/6, 0 failures/skips, 1769.4506 ms; native formatting/lint/commit controls passed. Cache SQL/history and feature seams remain unchanged. Earlier local UI dependency limitation remains recorded above; the c842 hosted Vitest run subsequently passed, but final 6160275 hosted acceptance is still pending. No full local suite or duplicate CI dispatch.

Migration and reviewer notes

  • The new column remains nullable: unknown differs from measured zero. Provider ingestion, pricing and billing arithmetic are unchanged by this repair.
  • Independent feat: report proxy payload traffic #16455 now uses standalone224against this release. Choose merge order and reconcile the second contribution's slot and historical-name compatibility before both land. Standalone checks do not establish combined compatibility.
  • No upstream merge, deployment, installation or other session's work was changed.

@KiaroSama
KiaroSama marked this pull request as draft October 13, 2026 07:58
@KiaroSama
KiaroSama marked this pull request as ready for review October 13, 2026 10:45
@diegosouzapw diegosouzapw added the deferred-v3.8.53 Feat PRs held for v3.8.53. Keep open. Do not merge into v3.8.52. label Oct 13, 2026
@diegosouzapw diegosouzapw changed the title feat: report cached input tokens [defer] feat: report cached input tokens Oct 13, 2026
@diegosouzapw
diegosouzapw changed the base branch from release/v3.8.52 to release/v3.8.53 October 19, 2026 13:28
@diegosouzapw

Copy link
Copy Markdown
Collaborator

Re-homed to release/v3.8.53: v3.8.52 entered its release freeze, so the branch now belongs to the release captain and development continues on the next cycle. Nothing is wrong with this PR — it just needed a live base. No action needed from you; CI will re-run against the new base.

@diegosouzapw diegosouzapw removed the deferred-v3.8.53 Feat PRs held for v3.8.53. Keep open. Do not merge into v3.8.52. label Oct 20, 2026
@diegosouzapw diegosouzapw changed the title [defer] feat: report cached input tokens feat: report cached input tokens Oct 20, 2026
KiaroSama and others added 25 commits October 24, 2026 00:23
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
… took 217

release/v3.8.53 landed 217_quota_recovery_tickets.sql (cheaper-inference#14982), so the merged tree carried two 217 migrations and check-migration-numbering failed. Move daily_cached_input to 218, rehome the earlier 217 ledger entry like the other published slots, and update the migration tests and doc.

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
…th release 217)

Co-authored-by: diegosouzapw <8016841+diegosouzapw@users.noreply.github.com>
@diegosouzapw diegosouzapw added the deferred-v3.8.54 Feat PRs held for v3.8.54. Keep open. Do not merge into v3.8.53. label Nov 2, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deferred-v3.8.54 Feat PRs held for v3.8.54. Keep open. Do not merge into v3.8.53.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants