chore: sync upstream junhoyeo/tokscale (66 commits) + server-side TZ normalization (R2)

[donghyun-mp]

Summary

• Sync: 66 upstream commits since PR chore: sync upstream junhoyeo/tokscale (137 commits) — adds Hermes/Crush/Copilot etc. #1, including the Kiro/Trae clients, the Groups feature, multiple new embed templates, Codex/Hermes parser tightening, and the active-time-ms schema additions.
• Root fix for the KST daily-leaderboard bug (R2): server-side normalization of `daily_breakdown.date` from the CLI's earliest-message `timestampMs` when `LEADERBOARD_TIMEZONE` is set. PR fix(cli): drop UTC-today cap so KST same-day submits land on daily leaderboard #2 only removed the CLI-side cap; this closes the loop so the bucket no longer depends on the submitter's machine TZ.
• Regression coverage: 5 new unit tests for `normalizeContributionDate` + 1 CLI test pinning that the submit payload never drops a future-dated contribution.

Two commits for clarity:

1. `b05d378` — chore: sync upstream junhoyeo/tokscale (66 commits)
2. `a8d9fd9` — feat(submit): normalize daily_breakdown.date by LEADERBOARD_TIMEZONE (R2)

Why this PR exists (beyond a routine sync)

PR #2 fixed the CLI side: it no longer drops contributions whose date is past UTC today. But the server still trusted the CLI's date string (`chrono::Local` on the submitter's machine). So a teammate submitting at KST 00:30 from a UTC-7 laptop would still write a row with `date = 2026-05-25`, and KST 5/26 "Today" would not see it.

Upstream PR junhoyeo#187 had already added `timestampMs` to every contribution payload — they store the earliest-message timestamp end-to-end, but explicitly keep `daily_breakdown.date` in the CLI's local-day format so each user sees their own local day on the global SaaS. That's the wrong default for a self-hosted, single-TZ deployment.

This PR introduces a single env-gated server-side rebucket. When `LEADERBOARD_TIMEZONE` is set, the server recomputes the row's day from `timestampMs`; when it is unset, the server keeps upstream behavior. So this change is also a clean candidate for upstream contribution — it does not break their default.

Sync conflict notes (worth a careful look during review)

Spelled out in the merge commit body. Highlights:

Area	Decision
Client enums (`main.rs`, `clients.rs`, `tui/data`, types/validation)	Union: fork's `AnthropicApi` + upstream's `Kiro`/`Trae`. `ClientId::COUNT` 22 → 25.
`getLeaderboard.ts`	Fork's `LEADERBOARD_TIMEZONE` as base; new `custom` and `last-month` periods folded in TZ-aware (not upstream's UTC).
`api/submit/route.ts`	Union of fork's `modelBreakdown` + `deviceContributions` and upstream's `active_time_ms` in INSERT VALUES + UPDATE SET.
`leaderboard/types.ts`	`Period = all|month|last-month|week|day|custom`; `SortBy = tokens|cost|time`.
`api/leaderboard/route.ts`	Kept fork's `NextRequest` signature (4ea3289), absorbed upstream's expanded period/sortBy lists.
`(main)/leaderboard/page.tsx`	Kept fork's auth redirect, absorbed upstream's Groups `ViewSelector` + searchParams plumbing.
`(main)/leaderboard/LeaderboardClient.tsx`	Kept fork (HEAD) verbatim — upstream's URL-state sync + custom-period UI is large enough that mixing it with fork's refresh-interval + self-hosted-URL feature in this PR was judged too risky. See follow-up.
`(main)/page.tsx`	Kept fork's simplified landing (a9b4276). Upstream's leaderboard preview is intentionally not surfaced internally.
`profile/ProfileEmbedDialog.tsx`	Stays deleted (fork bf6b9c8).
`SourceLogo.tsx`	Local `/assets/logos/*` paths kept over upstream's raw GitHub URLs; `kiro.ico` added.
`validation/submission.ts`	Source list unioned.

`c3c7844` ("scope client-slot wipe to each day's incoming clients") flagged the risk of tighter upstream parsers regressing device-aware merges. QA should pay extra attention to multi-device submits and daily aggregation around midnight in Asia/Seoul before promoting.

R2 design (a8d9fd9)

```ts
// packages/frontend/src/lib/leaderboard/normalizeContributionDate.ts
export function normalizeContributionDate(
rawDate: string,
timestampMs: number | null,
timezoneOverride: string | null,
): string {
if (!timezoneOverride) return rawDate; // upstream default
if (timestampMs == null) return rawDate; // legacy CLI (schemaVersion = 0)
return new Date(timestampMs).toLocaleDateString("en-CA", {
timeZone: timezoneOverride,
});
}
```

Used in `/api/submit` per incoming day; the normalized string drives both `existingDaysMap` lookup and the INSERT `date`.

Test plan

• `cargo build --workspace` (clean, 14s)
• `cargo test -p tokscale-cli` — 543 passed, 1 ignored, 0 failed + 91 passed integration
• `bun vitest run tests/lib/normalizeContributionDate.test.ts` — 5 passed
• CI: lint, clippy, full vitest, Rust tests
• Manual (per a8d9fd9):
  • KST machine submit at KST 00:30 → daily "Today" shows the new row immediately
  • UTC-7 machine submit at the same instant → row lands on the same KST day, not the Pacific local day
  • Legacy CLI (no `timestampMs`) → row date follows the CLI string (back-compat)

Follow-ups (not in this PR)

• Weave upstream's URL-state sync + custom-period UI into `LeaderboardClient.tsx` on top of the fork's refresh-interval + self-hosted-URL feature.
• Consider upstreaming the R2 helper (env-gated, so upstream default unchanged).

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
tokscale-frontend	Ready	Preview, Comment	May 26, 2026 11:32am

[gemini-code-assist]

Code Review

This pull request introduces several major features and integrations, including support for the Trae (ByteDance AI) and Kiro IDEs, a new subscription usage and quota command/TUI tab, a high-granularity Minutely view, and local custom pricing overrides. It also adds session time metrics and refines parsing logic across multiple existing clients. The review feedback highlights key areas for improvement: ensuring cross-platform path resolution for Trae's global storage, making the Amp total-quota parser robust against missing trailing whitespace, replacing a custom base64 decoder in Copilot with the existing base64 crate dependency, and preventing dummy zero-token messages in Gemini by returning None when all token fields are missing.

[gemini-code-assist]

The path to the Trae global storage directory is hardcoded using the macOS-specific Library/Application Support path. This will cause automatic credential discovery to fail on Windows and Linux platforms. Using the dirs crate's cross-platform directories ensures correct path resolution on all supported operating systems.

        let app_dir = if cfg!(target_os = "macos") {
            home.join("Library/Application Support").join(variant.app_dir_name())
        } else if cfg!(target_os = "windows") {
            dirs::data_dir()
                .context("could not determine AppData directory")?
                .join(variant.app_dir_name())
        } else {
            dirs::config_dir()
                .context("could not determine config directory")?
                .join(variant.app_dir_name())
        };

[gemini-code-assist]

If there is no whitespace after the total amount (e.g., "$4.50/$20.00"), after.find will return None, causing the parsing of the total amount to be skipped entirely. Using unwrap_or(after.len()) ensures that the entire remaining slice is parsed as the total amount when no trailing whitespace is present.

                let space_pos = after.find(|c: char| c.is_ascii_whitespace()).unwrap_or(after.len());
                if let Ok(total) = after[..space_pos].replace(',', "").parse::<f64>() {
                    if total > 0.0 && total.is_finite() && remaining.is_finite() {
                        let used = (total - remaining).max(0.0);
                        let used_pct = if used.is_finite() {
                            (used / total * 100.0).clamp(0.0, 100.0)
                        } else {
                            0.0
                        };
                        let remaining_pct = (100.0 - used_pct).clamp(0.0, 100.0);
                        let mut resets_at = None;

                        // Estimate reset time from hourly replenish rate
                        if let Some(rate) = parse_dollar_after(text, "+$") {
                            if rate > 0.0 && used > 0.0 && rate.is_finite() {
                                let secs = (used / rate * 3600.0) as i64;
                                let resets =
                                    chrono::Utc::now() + chrono::Duration::seconds(secs);
                                resets_at = Some(resets.to_rfc3339());
                            }
                        }

                        metrics.push(UsageMetric {
                            label: "Free".into(),
                            used_percent: used_pct,
                            remaining_percent: remaining_pct,
                            remaining_label: Some(format!("${remaining:.2}/${total:.2}")),
                            resets_at,
                        });
                    }
                }

[gemini-code-assist]

The base64 crate is already a dependency of tokscale-cli. Re-implementing a custom base64 decoder is redundant and prone to edge-case bugs (such as handling padding or alternative alphabets). It is highly recommended to reuse the standard base64 crate.

fn base64_decode(input: &str) -> Result<String> {
    use base64::prelude::BASE64_STANDARD;
    use base64::Engine;
    let bytes = BASE64_STANDARD.decode(input.trim())?;
    Ok(String::from_utf8(bytes)?)
}

[gemini-code-assist]

The deserialize_tokens function currently always returns Some(GeminiTokens { ... }) even if all token fields are None. This can lead to dummy zero-token messages being parsed and recorded. Returning None when no valid token fields are present prevents this.

fn deserialize_tokens(value: &Value) -> Option<GeminiTokens> {
    let input = first_i64(value, &["input", "prompt", "input_tokens", "prompt_tokens", "promptTokenCount"]);
    let output = first_i64(value, &["output", "candidates", "output_tokens", "completion_tokens", "candidatesTokenCount"]);
    let cached = first_i64(value, &["cached", "cached_tokens", "cachedContentTokenCount"]);
    let thoughts = first_i64(value, &["thoughts", "reasoning", "thoughts_tokens"]);
    let tool = first_i64(value, &["tool", "tool_tokens"]);
    let total = first_i64(value, &["total", "totalTokenCount", "total_tokens"]);

    if input.is_none() && output.is_none() && cached.is_none() && thoughts.is_none() && tool.is_none() && total.is_none() {
        return None;
    }

    Some(GeminiTokens {
        input,
        output,
        cached,
        thoughts,
        tool,
        total,
    })
}

[donghyun-mp]

@gemini review

[gemini-code-assist]

Code Review

This pull request introduces integration with Trae (ByteDance's AI IDE) and Kiro, adding credential decryption, token refresh, and usage synchronization. It also adds a new "Subscription Usage" feature to track real-time quotas across AI providers, a "Minutely" tab in the TUI, and session time metrics. Additionally, it implements custom pricing overrides, improves model grouping strategies (adding session-level grouping), and refactors the TUI theme handling for better terminal compatibility. The reviewer provided several high-quality suggestions to improve robustness, including handling OAuth refresh token non-rotation in Claude and Kimi providers, resolving platform-specific paths for Trae's global storage using dirs::data_dir(), making whitespace parsing more robust in the Amp provider, refactoring the daily active time loop in sessionize.rs, and restoring the which/where binary fallback in the Node.js wrapper while avoiding self-referential recursion.

[gemini-code-assist]

If the OAuth server does not rotate the refresh token (which is common and valid in OAuth2), refreshed.refresh_token will be None. In this case, the refreshed access_token is never saved, leading to repeated refreshes or authentication failures. Fall back to the existing refresh_token (rt) if a new one is not returned.

                let new_rt = refreshed.refresh_token.as_deref().unwrap_or(rt.as_str());
                save_credentials(
                    &new,
                    new_rt,
                    oauth.subscription_type.as_deref(),
                    oauth.rate_limit_tier.as_deref(),
                );

[gemini-code-assist]

If the OAuth server does not rotate the refresh token, refreshed.refresh_token will be None. In this case, the refreshed access_token is never saved. Fall back to the existing refresh_token (rt_str) if a new one is not returned.

                        if let Some(expires_in) = refreshed.expires_in {
                            let new_rt = refreshed.refresh_token.as_deref().unwrap_or(rt_str.as_str());
                            stored_refresh_token = Some(new_rt.to_string());
                            save_credentials(&access_token, new_rt, expires_in);
                        }

[gemini-code-assist]

If the OAuth server does not rotate the refresh token, refreshed.refresh_token will be None. In this case, the refreshed access_token is never saved. Fall back to the existing refresh_token (rt_str) if a new one is not returned.

                if let Some(expires_in) = refreshed.expires_in {
                    let new_rt = refreshed.refresh_token.as_deref().unwrap_or(rt_str.as_str());
                    save_credentials(&new, new_rt, expires_in);
                }

[gemini-code-assist]

The path Library/Application Support is macOS-specific. On Windows, Trae's global storage is located in %APPDATA%. Using dirs::data_dir() provides a cross-platform way to resolve this directory on both macOS and Windows.

        let app_dir = dirs::data_dir()
            .context("could not determine data directory")?
            .join(variant.app_dir_name());

[gemini-code-assist]

Using after.find(is_ascii_whitespace) will fail to parse the total if there is no trailing whitespace after the number (e.g., if the string ends with the number). Using split_whitespace().next() is more robust and handles both cases gracefully.

Suggested change

	if let Some(space_pos) = after.find(|c: char| c.is_ascii_whitespace()) {
	if let Ok(total) = after[..space_pos].replace(',', "").parse::<f64>() {
	if let Some(total_str) = after.split_whitespace().next() {
	if let Ok(total) = total_str.replace(',', "").parse::<f64>() {

[gemini-code-assist]

Using a while day <= end_date loop is safer and more idiomatic than a loop with manual breaks. It also completely avoids the redundant day.succ_opt() calls by reusing next_day.

        while day <= end_date {
            let day_key = day.format("%Y-%m-%d").to_string();
            let Some(day_start) = local_day_start(day, timezone) else {
                break;
            };

            let Some(next_day) = day.succ_opt() else {
                break;
            };
            let Some(next_day_start) = local_day_start(next_day, timezone) else {
                break;
            };

            let overlap_start = interval.start_ts.max(day_start);
            let overlap_end = interval.end_ts.min(next_day_start);
            let overlap = (overlap_end - overlap_start).max(0);
            let proportion = overlap as f64 / wall as f64;
            let active_for_day = (interval.active_duration_ms as f64 * proportion) as i64;

            if active_for_day > 0 {
                *daily.entry(day_key).or_default() += active_for_day;
            }

            day = next_day;
        }

[gemini-code-assist]

Removing the which/where fallback entirely might break environments where the native binary is installed globally but the wrapper is run from elsewhere. We can safely restore the fallback by checking !isSelfReference(found) to prevent infinite recursion.

if (!binary) {
  try {
    const whichCmd = process.platform === "win32" ? "where" : "which";
    const found = execSync(`${whichCmd} ${binaryName}`, {
      encoding: "utf-8",
      stdio: ["pipe", "pipe", "pipe"],
    })
      .trim()
      .split("\n")[0];
    if (found && existsSync(found) && !isSelfReference(found)) {
      binary = found;
    }
  } catch {}
}

if (!binary) {

[donghyun-mp]

Thanks for the second pass. Walked through all 7 new inline comments and the merits/scope are:

All seven flag code that arrived in this PR from the upstream sync (junhoyeo/tokscale), not changes authored by this fork. This PR's purpose is to absorb upstream's 66 commits + add the fork-only R2 server-side TZ normalization. Fixing upstream-owned code inside the sync diff would:

• diverge the fork from upstream on every subsequent sync (more conflicts each round),
• hide real bugs from every other downstream consumer of junhoyeo/tokscale, and
• blur the review boundary between "what this fork is doing" and "what upstream is doing."

So the right home for these is upstream PRs/issues, not this sync. Concrete dispositions:

Genuine bugs worth reporting upstream

• OAuth refresh_token rotation (claude.rs:203, kimi.rs:194, kimi.rs:215): if the provider doesn't rotate the refresh token (spec-allowed and common), refreshed.refresh_token is None and the new access_token is dropped on the floor. The refreshed.refresh_token.as_deref().unwrap_or(rt) pattern is correct. Real bug, should be filed at junhoyeo/tokscale.
• Trae macOS-only path (trae.rs:329, also :248): Library/Application Support is hardcoded — Win/Linux discovery silently breaks. dirs::data_dir() / dirs::config_dir() fixes it. Should be filed upstream as a follow-up to PR junhoyeo#555. Low impact for us since the company runs macOS; high impact for any upstream user on Win/Linux.
• Amp parser edge case (amp.rs:64, also :94): after.find(is_ascii_whitespace) returns None when the number ends the string; split_whitespace().next() is more robust. Real but narrow.
• Gemini dummy zero tokens (gemini.rs:81, prior round): deserialize_tokens should return None when every field is None instead of an all-None Some. Avoids recording dummy zero-token rows.

Refinements (preference, not bugs)

• Hand-rolled base64 (copilot.rs:155, prior round): the base64 crate is already a dep; reusing it is cleaner. Upstream cleanup.
• while loop over loop+break (sessionize.rs:298): idiomatic; not a behavior change.
• Restore which/where fallback with !isSelfReference (packages/cli/src/index.ts:171): upstream PR junhoyeo#585 deliberately removed the fallback to escape the bun-optionalDependencies fork-bomb. Gemini's !isSelfReference(found) guard is a fair refinement that recovers the original UX without re-opening the loop — but the decision is still upstream's, since junhoyeo#585's author already weighed safety vs. fallback coverage.

This PR

Stays focused on (a) the upstream sync as-is and (b) the fork-only R2 normalization + the type-check fixes in ab4d8ed5. Internal QA energy in this PR is best spent on R2 (KST same-day buckets) and the multi-device merge interaction with the new upstream parsers — that's where the fork's risk surface actually is.

[donghyun-mp]

@woongeun-mp
Upstream 갱신했는데, 갱신을 하자니 팔로업으로 DB Migration 적용 등 추가 액션이 필요한 부분이 있습니다.
tokscale-post-merge-checklist.html

그리고 Upstream의 CLI를 그대로 쓰면 이 문제가 해소가 안되고 지속된다네요... 😭 이쪽에서 릴리즈를 하도록 해서 팀원들에게 공유하거나, 그냥 단점을 인지하고 쓰거나 해야할 것 같습니다.
tokscale-upstream-cli-problem.html