Skip to content

feat: add token usage visualization frontend - #1

Merged
junhoyeo merged 23 commits into
mainfrom
junhoyeo/app
Dec 1, 2025
Merged

feat: add token usage visualization frontend#1
junhoyeo merged 23 commits into
mainfrom
junhoyeo/app

Conversation

@junhoyeo

@junhoyeo junhoyeo commented Dec 1, 2025

Copy link
Copy Markdown
Owner

Summary

Add a complete frontend application for visualizing LLM API token usage with GitHub-style contribution graphs.

CLI Enhancements

  • Add graph command to export contribution data as JSON
  • Support timestamp extraction for Claude Code and Gemini messages
  • Generate daily contribution data with cost/token breakdowns

Frontend Features

  • 2D Contribution Graph: Canvas-based GitHub-style heatmap
  • 3D Isometric Graph: Interactive 3D view using obelisk.js
  • Multiple Color Themes: 9 palettes (green, halloween, teal, blue, pink, purple, orange, monochrome, YlGnBu)
  • System Theme Support: UI automatically follows system dark/light mode preference
  • Detailed Breakdowns: Click any day to see per-source, per-model token usage
  • Statistics Panel: Total cost, tokens, active days, streaks, and more
  • Source Filtering: Filter by source (OpenCode, Claude, Gemini, Codex)
  • Year Selection: Navigate between years of data

Technical Details

  • Next.js 16 with App Router
  • TypeScript with strict type definitions
  • CSS variables using GitHub Primer design system
  • Responsive design with Tailwind CSS

Screenshots

image

- Update themes to use GitHub Primer design system colors
- Add GitHub-style BtnGroup toggle for 2D/3D views
- Add stats overlays on 3D canvas (contributions panel, streaks)
- Reorganize controls with year selector and theme dropdown
- Pass stats data from GraphContainer to TokenGraph3D
- UI colors now use CSS variables that follow system dark/light mode
- Color palettes (green, halloween, teal, etc.) only affect graph cells
- Added GitHub Primer design system CSS variables
- Updated all components to use CSS variables for backgrounds, text, borders
- Renamed theme/themeName to palette/paletteName for clarity
@junhoyeo
junhoyeo merged commit 056571e into main Dec 1, 2025
@junhoyeo
junhoyeo deleted the junhoyeo/app branch December 1, 2025 17:06
junhoyeo added a commit that referenced this pull request Jun 17, 2026
#713)

* feat(sessions): read Antigravity CLI usage from local SQLite databases

The Antigravity CLI (the terminal agent that stores its data under `~/.gemini/antigravity-cli/`) was never counted. tokscale only knew two Gemini-family sources: the Gemini CLI (scans `~/.gemini/tmp/*.{json,jsonl}`) and Antigravity (pulls usage from a running IDE language server over RPC and caches it under the config dir). The Antigravity CLI fell into neither bucket, so its on-disk usage was invisible — `tokscale antigravity sync` found the filesystem candidates but cached zero because its only artifact path still requires a live language-server RPC connection.

This adds Antigravity CLI as a first-class local scan source so its usage updates automatically like every other file-based source — no RPC, no `antigravity sync`. A new `antigravity-cli` client globs `~/.gemini/antigravity-cli/conversations/*.db` (honoring `GEMINI_CLI_HOME`) and a new parser reads each conversation database directly.

Each `gen_metadata` row is one generation encoded as the same `GeneratorMetadata` protobuf the IDE returns over `GetCascadeTrajectoryGeneratorMetadata`. The repository has no `.proto`/prost decoder (the IDE path receives JSON because the language server does the proto-to-JSON conversion), so the parser ships a tiny dependency-free wire-format reader and pulls only the fields it needs. The field numbers were reverse-engineered from real databases and cross-checked across 6 sessions / 140 turns: `chatModel.#19` is the response model, `usage.#5`/`#9`/`#10` are cacheRead/output/thinking (verified by the invariant `#9 + #10 == #3`, the stored total output), `#11` is the responseId used for dedup, and input combines the fixed system-prompt count `#1` with the newly-processed input `#2`. The session timestamp and workspace come from `trajectory_metadata_blob`.

Adding the new `ClientId` variant fans out to the usual registration points: the scanner gains a `*.db` glob arm (which naturally rejects `.db-wal`/`.db-shm` sidecars), both local-parse dispatch paths gain a branch, and the CLI `ClientFilter`, client labels, TUI picker, and frontend source maps gain entries. The deprecated per-client boolean flags intentionally do not, since `antigravity-cli` is reachable only via the canonical `--client antigravity-cli`.

Closes #712.

* fix(sessions): handle file:// authority/UNC paths and test Antigravity CLI wiring

Addresses the cubic review on #713.

`file_uri_to_path` previously stripped `file://` and only special-cased the leading slash before a Windows drive letter, so a non-empty authority (`file://host/share/...`, the UNC form) lost its host and collapsed into a bare path. It now treats an empty-authority remainder as before (`/C:/x` → `C:/x`, `/home/x` kept) and reconstructs a non-empty authority as a UNC path (`host/share/x` → `//host/share/x`) so `normalize_workspace_key` preserves the `//` prefix. A unit test covers the Windows-drive, POSIX, UNC, and percent-encoded-CJK cases.

The new `AntigravityCli` client wiring is now asserted in `test_client_as_str`, `test_client_key`, and `test_client_from_key` (display name "Antigravity CLI", hotkey `f`, and the reverse hotkey mapping).

* style: rustfmt antigravity_cli.rs

* fix(antigravity-cli): add gemini-3-flash-a pricing alias and harden parser tests

Map the raw #19 responseModel `gemini-3-flash-a` onto the priced
`gemini-3-flash-preview` so Antigravity CLI cost no longer resolves to 0.
Add alias-resolution, #9/#10==#3 field-mapping invariant, and
malformed-protobuf bounds tests.

Constraint: must not weaken the #707 brand-token fuzzy-match guard in lookup.rs

Confidence: high

Scope-risk: narrow

---------

Co-authored-by: Junho Yeo <i@junho.io>
junhoyeo added a commit that referenced this pull request Aug 25, 2026
…bed surfaces (#1192)

The leaderboard ranks a finite period with a sequential ROW_NUMBER over
`total_tokens DESC, total_cost DESC, LOWER(username) ASC, user_id ASC`
(cost sort swaps the two metrics), and ranks all-time with a shared
`RANK() OVER (ORDER BY <metric> DESC)`.

The public profile's period rank and the embed card's period rank each
used a bare `RANK() OVER (ORDER BY total_tokens DESC)` instead, so two
users on the same period total read #1 and #2 on the leaderboard's week
or month tab but #1 and #1 on their own profile and embed card.

Move both finite-period queries onto the leaderboard's expression and
tiebreak columns. The CTEs now carry `u.username` and, on the profile
side, `SUM(CAST(d.cost AS DECIMAL(18,4)))` so the tiebreak has its
columns in scope; the `leaderboard_hidden = false` filter, the date
bounds, and the embed's `of N` denominator are unchanged, so the same
rows are ranked and only the order within a tie moves.

Both all-time queries stay on shared RANK, which is what the
leaderboard's all-time tab does — switching them would be the same
divergence mirrored.

Constraint: profile period rank must keep its unstable_cache key and the empty-window skip
Rejected: sequential ranks everywhere | all-time surfaces would then disagree with the leaderboard's all-time tab
Rejected: shared RANK everywhere | loses the leaderboard's deterministic pagination order
Confidence: high
Scope-risk: narrow
Directive: the finite and lifetime windows rank differently on purpose; each mirrors the leaderboard tab it sits beside
Not-tested: no database integration test — the regression tests assert the emitted SQL, not Postgres' ordering of a real tie
junhoyeo added a commit that referenced this pull request Aug 25, 2026
…1.1.18 gen_metadata layout (#1196)

* fix(antigravity-cli): read the per-generation timestamp from the agy 1.1.18 gen_metadata layout

agy 1.1.18 dropped `chatModel.#9.#4`, the `{#1: seconds, #2: nanos}`
Timestamp this parser used to date each turn. `#9` now carries `#2` =
u64::MAX (an int64 -1 "unset" sentinel) plus a new `#10` holding 8
length-delimited bytes. With `#4` gone every row fell through to the
session-created stamp, so on a long-running session every turn was
bucketed to the session start date and `--today` reported zero.

`#9.#4` is still read first and unchanged, so pre-1.1.18 databases and
older installs keep their exact behaviour. When it is absent, `#9.#10`
is decoded as a nested Timestamp, a nested message holding the scalar in
field 1 (varint or fixed64), or the payload itself as 8 raw
fixed64-style bytes in either byte order. Every one of those readings is
unit-detected by magnitude and range-checked against 2020-01-01..now+5y
before it is accepted; anything outside that window is discarded and the
session-created fallback takes over. `#9.#2` is never consulted, and
u64::MAX is rejected explicitly so no path can promote the sentinel into
a date.

Constraint: no agy 1.1.18 install or gen_metadata database available, so `#10`'s encoding is inferred from a field dump in the issue, not observed
Rejected: read `#9.#2` as the new timestamp | the only value ever seen there is the u64::MAX unset sentinel
Rejected: decode the 8 bytes as an IEEE-754 f64 | any double in the 2^30-ish exponent range reads as a plausible epoch-second count, so it is the one candidate with a non-trivial false-positive rate against a non-timestamp payload
Confidence: high that pre-1.1.18 parsing is unchanged; medium that the 1.1.18 reading fires on real data
Scope-risk: narrow
Directive: keep every inferred reading behind `plausible_epoch_ms` — a wrong date silently corrupts day buckets and the monotonic ratchet, which is worse than the session-start fallback this degrades to
Not-tested: a real agy 1.1.18 `gen_metadata` row; if `#10` is neither a timestamp nor decodes in range, behaviour is identical to today's

* fix(antigravity-cli): bound inferred generation timestamps to the session window

The agy 1.1.18 `chatModel.#9.#10` payload is 8 bytes whose encoding was
never confirmed against a real database, so every candidate reading of it
is a guess that has to earn acceptance. The only gate on those guesses was
an absolute "is this a believable date" window running from 2020-01-01 to
five years out. That is not a meaningful test for a raw integer: read as a
nanosecond count the window alone covers ~2% of the u64 range, so trying
both byte orders leaves an arbitrary payload — an id, a hash, a duration —
a few percent chance of passing as a date. A false accept silently buckets
a turn into the wrong day and feeds the server-side monotonic ratchet,
which has no correction path, making it strictly worse than the known-wrong
session-start dating it replaces.

Require every inferred reading to land inside the containing session's own
lifetime as well: at or after the session-created stamp less one hour, and
at or before now plus one hour. A turn cannot predate its conversation nor
happen after we read the file, and that pair of bounds is hours or days
wide instead of a decade. When there is no positive anchor to corroborate
against, decline inference entirely and let the caller fall back as before.

The explicit `#9.#4` Timestamp is untouched: it is a confirmed
representation read off real pre-1.1.18 databases, keeps its `ms > 0`
filter, takes no session bound, and still outranks the inferred reading.

Constraint: `#9.#10`'s encoding is inferred from a field dump, not observed
Constraint: mis-dating is uncorrectable downstream; under-dating is not
Rejected: tightening only the absolute window | no absolute date range is
  narrow enough to make a raw 8-byte integer a safe timestamp
Rejected: day-wide tolerances | hands back the integer space the session
  window exists to remove
Confidence: high
Scope-risk: narrow
Directive: the one-hour tolerances are load-bearing and pinned by tests;
  widening them re-opens the false-accept surface this closes
Not-tested: a real agy 1.1.18 database — none was available, which is why
  the reading is inferred in the first place
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant