Skip to content

fix(submit): make usage merges device-aware - #517

Merged
junhoyeo merged 1 commit into
junhoyeo:mainfrom
IvGolovach:codex/device-aware-submit-merge
May 24, 2026
Merged

fix(submit): make usage merges device-aware#517
junhoyeo merged 1 commit into
junhoyeo:mainfrom
IvGolovach:codex/device-aware-submit-merge

Conversation

@IvGolovach

@IvGolovach IvGolovach commented May 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Attach a stable CLI submit device id to tokscale submit payloads.
  • Store submitted devices per user and merge daily rows by (submission, device, date) instead of only (submission, date).
  • Recalculate submission totals across all device rows while counting active days by distinct date to avoid multi-device regressions.

Why

Submitting from multiple machines currently shares one daily row per user submission. A later submit for the same date can replace another device's client breakdown because the merge cannot distinguish which local dataset is being refreshed.

Diff scope

  • crates/tokscale-cli/src/device.rs: resolves stable submit device identity from env/config, with TOKSCALE_DEVICE_ID and TOKSCALE_DEVICE_NAME overrides.
  • crates/tokscale-cli/src/main.rs: includes the device only in submit payloads, not graph JSON output.
  • packages/frontend/src/lib/validation/submission.ts: validates optional device metadata and folds the device id into the submission hash.
  • packages/frontend/src/lib/db/schema.ts and packages/frontend/src/lib/db/migrations/0006_add_submitted_devices.sql: add submitted_devices, backfill legacy rows, and change the daily uniqueness boundary.
  • packages/frontend/src/app/api/submit/route.ts: upserts submit devices, locks/merges rows for the current device, and recalculates aggregate totals across all devices.
  • packages/frontend/src/app/api/settings/submitted-data/route.ts: deletes submit device metadata together with submitted usage data.

Migration notes

  • Adds submitted_devices with a per-user unique device_key.
  • Backfills existing daily rows into a legacy-default device before making daily_breakdown.submitted_device_id required.
  • Replaces the old (submission_id, date) uniqueness constraint with (submission_id, submitted_device_id, date).
  • No submitted usage rows are deleted by the migration.

Safety

  • Legacy payloads without device remain valid and use the legacy-default bucket.
  • Same-device resubmits still replace that device's daily data; different devices on the same date retain separate rows.
  • Submission totals continue to sum all daily rows, while activeDays counts distinct active dates so it does not inflate across devices.
  • Device identity is generated locally for submit only; graph/report JSON output remains unchanged unless submitting.

Validation

  • cargo fmt --all -- --checkPASS.
  • cargo test -p tokscale-cliPASS: 473 passed, 1 ignored; cli_tests: 88 passed.
  • /opt/homebrew/bin/cargo-clippy --package=tokscale-cli --all-features -- -D warningsPASS.
  • bunx vitest run packages/frontend/__tests__PASS: 21 files, 182 tests.
  • bun run lint in packages/frontendPASS, 0 errors; 7 existing warnings in untouched files.
  • git diff --check origin/main...HEADPASS, no output.

Rollback

Revert the merge commit. If the database migration has already been applied, rollback also needs to remove daily_breakdown.submitted_device_id, drop submitted_devices, and restore the previous (submission_id, date) uniqueness constraint.

@vercel

vercel Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
tokscale Ready Ready Preview, Comment May 24, 2026 3:00pm

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 11 files

@BenLampson

Copy link
Copy Markdown
Contributor

So when can I use this feature?

@IvGolovach

Copy link
Copy Markdown
Collaborator Author

So when can I use this feature?

Thanks for asking! This is not available in latest yet — it will be usable once this PR is merged and included in the next deployment/release. After that, updating to the latest tokscale and submitting from each device normally should preserve the device-aware totals :)

@junhoyeo junhoyeo left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — migration is well-designed, COUNT(DISTINCT) fix is correct. Follow-up: add a one-line comment on schemaVersion: 2 semantics.

Validation:
- cargo fmt --all -- --check
- cargo test -p tokscale-cli
- /opt/homebrew/bin/cargo-clippy --package=tokscale-cli --all-features -- -D warnings
- bunx vitest run packages/frontend/__tests__
- bun run lint (0 errors; existing warnings only)
- git diff --check && git diff --cached --check
- git diff --check origin/main...HEAD

Rollback:
- Revert this commit.
@junhoyeo
junhoyeo force-pushed the codex/device-aware-submit-merge branch from 74b5308 to e4f6098 Compare May 24, 2026 14:59
@junhoyeo
junhoyeo merged commit 360eaef into junhoyeo:main May 24, 2026
12 of 14 checks passed
junhoyeo added a commit that referenced this pull request May 25, 2026
…rift (#593)

Schema:
- Add 0010_submit_count_safety.sql to ALTER TABLE submissions ADD COLUMN
  IF NOT EXISTS submit_count. The column is referenced by schema.ts and
  written on every /api/submit call, but no earlier .sql migration adds
  it; prod has it because somebody once ran drizzle-kit push directly,
  so a fresh drizzle-kit migrate on CI/staging/dev currently breaks
  /api/submit. Verified locally against postgres:16.

API:
- GET /api/users/[username]/devices: aggregated per-device totals
  (tokens, cost, input/output, active days) for a user's submission
  devices, sourced from submitted_devices LEFT JOIN daily_breakdown.
- GET /api/users/[username]/devices/[deviceId]: per-device detail with
  day-by-day contributions. Cross-user device ids silently 404.
- PATCH /api/settings/devices/[deviceId]: session-authenticated rename
  (or clear, via name: null). Validates length and rejects control
  characters; ownership enforced in the same WHERE.

Helpers:
- src/lib/devices/shared.ts: deviceDisplayLabel() and toIsoString()
  shared across the three routes so the public label format cannot drift.

Local dev:
- scripts/seed-dev.ts: idempotent synthetic seed (3 users × 2 devices
  × 14 days + 1 group). Refuses any non-localhost DATABASE_URL.

Ported from #389 onto main's submitted_devices model (#517) rather than
that PR's submissions.source_id schema rewrite. #389's CLI source-id
lockfile and profile Devices tab UI are deliberately not in this PR;
they want separate decisions (CLI lock vs main's existing device.id
payload; UI as a styled-components review pass).

Constraint: do not regress main's submitted_devices schema or rehash the migrated multi-machine model
Rejected: include #389's index-cleanup migration (drop idx_submissions_user_id et al.) | needs a fresh pg_stat_user_indexes audit on current prod before dropping anything in a single transaction
Rejected: bundle #389's CLI source-id lockfile | main already sends device.id via #517/#545; the lockfile is a behavior change, not a port
Confidence: high
Scope-risk: narrow
Directive: /api/users/[username]/devices is public — match it to /api/users/[username]'s visibility model when extending; the rename endpoint is the only auth-gated piece
Not-tested: interaction with the existing rank cache (rankTotal in embed routes is unaffected)
@junhoyeo

Copy link
Copy Markdown
Owner

@IvGolovach @BenLampson this has been merged to v3.0.0: https://github.com/junhoyeo/tokscale/releases/tag/v3.0.0 thanks for the contribution!

junhoyeo added a commit to blpeng2/tokscale that referenced this pull request Jun 9, 2026
Resolution policy: take main wholesale for everything superseded by the
relational device model that landed via junhoyeo#517/junhoyeo#593/junhoyeo#609 — device.rs,
main.rs, schema.ts, helpers.ts, submit route + tests, and the whole TUI
(app.rs, tui/mod.rs, footer.rs). Keep only the PR's remote-stats half
(tui/remote.rs, /api/me/stats route, meStats tests) to be rebuilt on
top of main in follow-up commits. Drop the next-server.d.ts shim that
the PR added as a workaround for the PR-era Next version.

Refs junhoyeo#699
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants