fix(web): include cache token buckets in analytics

[stefanpieter]

Summary

Fix dashboard analytics token accounting so the Token Usage views include all persisted token buckets, not just input/output.

This updates /api/analytics/usage and /api/analytics/models to aggregate and return:

• cache_read_tokens
• cache_write_tokens
• reasoning_tokens
• total_tokens = input + output + cache read + cache write

The Analytics and Models pages now display/sort using the corrected totals, including cache tokens. Reasoning tokens remain exposed separately and are not double-counted into total_tokens.

Related context: #23270 and #25400 cover adjacent token visibility gaps, but this PR is specifically about persisted dashboard analytics undercounting cache buckets.

Changes

• Add cache write and total token fields to usage/model analytics API responses.
• Sort model analytics by total_tokens, not only input/output.
• Include cache read/write in model average token calculations.
• Show cache token usage in Analytics daily chart/table/model table.
• Show cache write in Models token bars.
• Keep separately reported reasoning metadata outside the proportional Models token bar so its four additive segments use the same canonical denominator as the displayed card total.
• Update TypeScript API interfaces for the new fields.
• Add API regressions plus an adversarial UI-contract regression with every token bucket nonzero.

Semantic contract

CanonicalUsage.total_tokens is authoritative: input + cache read + cache write + output. The API totals, sorting key, card total/fallback, and proportional token-bar denominator route through that same four-bucket contract. reasoning_tokens is separately reported metadata and does not consume stacked-bar width.

Verification

Current reviewed head: a71bd8bf79384b46aeb1d1737a52c1b66d1aef41

• RED reproduced on the reviewed implementation: adversarial fixture produced bar total 240 instead of canonical 200 when reasoning was nonzero.
• npm --workspace web test — 34 passed.
• npm --workspace web run typecheck — passed.
• npm --workspace web run build — passed.
• python -m pytest -q tests/hermes_cli/test_web_server.py -k 'analytics_usage_accounts_for_cache_token_buckets or analytics_models_accounts_for_cache_token_buckets or models_analytics' — 3 passed, 356 deselected.
• git diff --cached --check and added-line secret/danger scan — clean.
• Independent embedded-diff re-review — PASS; no security concerns or logic errors.
• Repository-wide ESLint remains at the exact current-head baseline of 28 errors and 3 warnings; the only touched-file finding is the pre-existing ModelsPage effect at line 1185. No new lint finding was introduced.
• A repository-wide Python run was attempted but not claimed as a passing gate: the old branch's broad suite entered live network paths and exhausted file descriptors before completion. The affected Python and complete web gates above passed.

No public API or schema compatibility change is introduced by the review remediation; it only corrects the Models proportional visual and labels reasoning as separately reported.

[alt-glitch]

Related to open #16658 (same cache token undercounting issue in dashboard analytics SQL). Also note: #25438 (merged) hides token analytics behind a config flag (default off), reducing the user-facing impact of this bug.

[teknium1]

Thanks for the focused cache-accounting fix. The premise is confirmed on current main: /api/analytics/usage omits cache_write_tokens at hermes_cli/web_server.py:13943-13979, and /api/analytics/models omits it from model totals at hermes_cli/web_server.py:14017-14148.

Problems

• web/src/pages/ModelsPage.tsx:100 includes reasoning in the TokenBar denominator. The PR's card total at web/src/pages/ModelsPage.tsx:382 uses total_tokens, while the canonical definition at agent/usage_pricing.py:40-46 is input + cache read + cache write + output, explicitly excluding reasoning. The bar therefore double-counts reasoning relative to the displayed total.

Suggested changes

• Keep the TokenBar's proportional total to input + output + cache read + cache write. Show reasoning as non-additive metadata, or otherwise make its relationship to output explicit. Add a regression case with nonzero reasoning tokens.

Automated hermes-sweeper review.

[teknium1]

CanonicalUsage.total_tokens is prompt plus output (agent/usage_pricing.py:40-46), so reasoning is intentionally not additive. Including it here makes the stacked bar disagree with the entry.total_tokens card at line 382 and double-counts reasoning. Keep this denominator to input + output + cache read + cache write; show reasoning without adding it as another total segment.

[stefanpieter]

@teknium1 the reasoning-token proportional-bar blocker is fixed on current head a71bd8bf79384b46aeb1d1737a52c1b66d1aef41; please re-review this exact SHA.

• CanonicalUsage.total_tokens remains input + cache read + cache write + output across producer/API/sort/card fallback.
• The Models stacked bar now contains only those four additive buckets; reasoning_tokens is labeled as separately reported metadata and consumes no bar width.
• Added an adversarial regression with all five buckets nonzero: canonical total 200, additive segments sum to 200, widths sum to 100%, reasoning 40 remains separate. The pre-fix implementation reproduced RED at 240.

Verification:

• full web tests: 34 passed
• web typecheck + production build: passed
• affected analytics API tests: 3 passed
• staged diff/added-line security scan: clean
• independent final-diff review: PASS, no security or logic blockers
• exact-head GitHub checks: none configured/reported

The PR body has been refreshed with the semantic contract, exact SHA, baseline lint status, and current evidence.