Skip to content

fix(hourly): polish cache schema, dispatch, sort defaults, and float guards - #359

Merged
junhoyeo merged 1 commit into
junhoyeo:mainfrom
crhan:feat/hourly-report
Apr 26, 2026
Merged

fix(hourly): polish cache schema, dispatch, sort defaults, and float guards#359
junhoyeo merged 1 commit into
junhoyeo:mainfrom
crhan:feat/hourly-report

Conversation

@crhan

@crhan crhan commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds tokscale hourly CLI subcommand with --json, --light, date filters, and all client flags (including the newly added --kilo)
  • Adds a Hourly TUI tab (between Daily and Stats) with the same keyboard navigation
  • Overview page gains a toggle (h) to switch between daily and hourly breakdown
  • Cache× column: cache hit multiplier (cache_read / input_tokens), renamed from Cache% to clarify it's a multiplier not a percentage
  • Turn column: counts distinct user-turn timestamps per hour/day — shows actual interaction turns, not just assistant messages
  • Msgs column in TUI hourly and daily views: raw message count per period

Details

The hourly report aggregates token usage, cost, turn count, and message count by hour. The Turn count uses a 10-second dedup window to collapse rapid consecutive user messages into a single logical turn (matching CC's session behavior).

Cache× is defined as cache_read_tokens / input_tokens — a value > 1 means more tokens were served from cache than paid for as fresh input, indicating good cache utilization.

Test notes

Three pre-existing scanner tests (test_scan_all_clients_claude, test_scan_all_clients_multiple, test_scan_all_clients_headless_paths) fail on upstream main as well — they scan the developer's real ~/.claude/projects instead of the temp dir due to an unrelated env/path issue. Not introduced by this PR.

🤖 Generated with Claude Code


Open in Devin Review

@vercel

vercel Bot commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

@crhan is attempting to deploy a commit to the Inevitable Team on Vercel.

A member of the Team first needs to authorize it.

cubic-dev-ai[bot]

This comment was marked as resolved.

@crhan

crhan commented Mar 26, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the thorough review. Addressed in 974b28c:

Fixed:

  • lib.rs:1213clients/models now sorted before collecting from HashSet, ensuring deterministic JSON output
  • app.rs:511/512 — Extracted apply_tab_sort_defaults() called only from tab-switch handlers (Tab/BackTab/Left/Right). reset_selection() no longer overrides sort field, so Cost/Tokens sorting persists while on the Hourly tab

Won't fix / false positives:

  • cache.rs:57hourly field already has #[serde(default)] (line 56), so old cache files deserialize correctly with an empty hourly vec. No schema bump needed.

  • mod.rs:39 (is_turn_start serde default) — The message cache (message_cache.rs) handles bincode deserialization errors gracefully: Err(_) => return Self::default(). Old cache files will simply miss on first run after upgrade and be re-populated. Not a correctness issue.

  • claudecode.rs:318 (< prefix heuristic) — Intentional tradeoff. CC system-injected messages (environment details, tool results, context summaries) consistently start with XML-style tags. User messages starting with literal < are rare in practice and are a known limitation of the heuristic approach.

  • overview.rs:76 — Hourly dataset is at most 24 × history_days entries. Sort on every render is negligible at that scale.

  • app.rs:511 initial sort — reset_selection() is called from tab-switch handlers, so switching to Hourly correctly applies Date/Descending on arrival. This was only a bug when set_sort() also called reset_selection(), which is now fixed.

  • main.rs:790 TUI dispatch — By design. The hourly subcommand outputs a table; the Hourly TUI tab is available via the main tokscale command. Adding full TUI dispatch to the subcommand is a separate enhancement.

cubic-dev-ai[bot]

This comment was marked as resolved.

cubic-dev-ai[bot]

This comment was marked as resolved.

@christian-taillon

Copy link
Copy Markdown

This worked great for me. Should be merged.

I also built #395 based on it for a need I had to assess my token utilization.
Thank you @crhan

@junhoyeo
junhoyeo force-pushed the feat/hourly-report branch from 92930f1 to 1bd3758 Compare April 26, 2026 03:38
@vercel

vercel Bot commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
tokscale Ignored Ignored Preview Apr 26, 2026 4:19am

Request Review

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 3 additional findings.

Open in Devin Review

@junhoyeo junhoyeo left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the work on this, @crhan! The hourly tab landed via #422 in the meantime, so this PR is now the polish layer addressing the cubic findings (cache schema bumps, dispatch routing, NaN guards, tag allowlist). Approving — will squash merge after #464 and #454. ⏰

…defaults, and float guards

- bump TUI cache schema to 6 and core message_cache schema to 7 so old caches
  with empty hourly/turn fields don't deserialize as Fresh
- restrict is_human_turn's '<' filter to a known set of internal tool tags
  so user prompts containing HTML/XML are no longer dropped from turn counts
- route 'tokscale hourly' through the standard TUI/CLI dispatch so the TUI
  Hourly tab is reachable interactively and --light is honored
- guard format_cost_per_million against NaN/Inf and non-positive denominators
- stop re-sorting the entire hourly/daily dataset every render in
  overview chart; consume already-sorted data
- match CLI fallback behavior in TUI hourly bucketing for messages with
  missing/zero timestamps
@junhoyeo junhoyeo changed the title feat(cli/tui): add hourly usage report — CLI subcommand + TUI tab fix(hourly): polish cache schema, dispatch, sort defaults, and float guards Apr 26, 2026
@junhoyeo
junhoyeo force-pushed the feat/hourly-report branch from 1bd3758 to 94af15e Compare April 26, 2026 04:19
@junhoyeo
junhoyeo merged commit f8eb692 into junhoyeo:main Apr 26, 2026
13 of 15 checks passed
junhoyeo added a commit that referenced this pull request Apr 26, 2026
…nd client filter UX (#467)

Pre-release correctness and security fixes across the four PRs landed this cycle (#464, #454, #359, #355).

## Antigravity

Trust boundary around the local language-server RPC was too loose:

- Bound RPC body sizes at 16 MiB (Content-Length, chunked, and read-to-end paths)
- Verify process identity by checking the executable path (`lsof` on macOS, `/proc/<pid>/exe` on Linux); accept paths containing either `antigravity` or `language_server` since some Antigravity-flavored servers launch from generic `language_server` binaries with `--app_data_dir antigravity`
- Probe candidate endpoints with a real RPC call and JSON-shape check (probe body capped at 4 KiB) instead of trusting any 200 response; consume HTTP headers before reading body so the cap applies to the JSON body alone
- Lock concurrent syncs on a per-cache PID lock file with bounded retry (3 attempts) instead of unbounded recursion. Eviction is gated on PID liveness only — long-running syncs no longer get stomped on by age-based timeouts.
- Enforce manifest version on load: future versions abort, older versions start fresh
- Recover corrupted manifests by moving them aside as `manifest.json.corrupt-<ts>` instead of failing every subsequent sync

## Codebuff

Parser correctness around silent data loss:

- Accumulate run-state usage across the full reverse `messageHistory` walk instead of returning on the first signal-bearing entry. Previously a newest assistant entry carrying only a model id would short-circuit the walk and silently drop real token counts on earlier entries.
- Include the source-array ordinal in the fallback dedup key so two id-less assistant messages with identical session/timestamp/model/tokens no longer collapse into a single record
- Reject non-positive numeric timestamps in the shared `parse_timestamp_value`/`parse_timestamp_str` helpers so messages with `timestamp: 0` or negative epochs fall through to chat-id / file-mtime fallback chain

## TUI + parsing

User-visible client-filter UX:

- Move `SYNTHETIC_HOTKEY` from `'x'` to `'n'`. `'x'` collided with Mux's hotkey, and the dispatch order made the displayed `[x]` for Synthetic purely cosmetic
- Enable `ignore_case` on `--client/-c` so `OPENCODE`, `Codebuff`, and `antigravity` all parse as the same canonical filter

## Test results

- `cargo test -p tokscale-cli` — 415 unit + 83 integration, all pass
- `cargo test -p tokscale-core` — 566 unit + 10 codebuff + 3 hermes, all pass
- `cargo check --workspace` — clean

Each fix has a regression test where applicable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants