Skip to content

feat(stats): expose token usage for cost visibility - #4564

Merged
wenshao merged 13 commits into
QwenLM:mainfrom
shenyankm:feat/issue-4479-token-usage-stats
Jun 18, 2026
Merged

feat(stats): expose token usage for cost visibility#4564
wenshao merged 13 commits into
QwenLM:mainfrom
shenyankm:feat/issue-4479-token-usage-stats

Conversation

@shenyankm

Copy link
Copy Markdown
Contributor

What this PR does

Adds persisted token-usage accounting and extends /stats so users can view daily token usage, monthly token usage, model/auth-type breakdowns, and export summarized usage as CSV or JSON.

This also documents the coordination boundary with adjacent stats work: token usage stays under /stats, generation timing metrics such as TTFT/TPS remain separate, and memory diagnostics are not expanded by this change.

Why it's needed

Users currently have no straightforward CLI-visible way to understand how many tokens Qwen Code consumed today or this month, even when a single run can consume a large amount of tokens. This makes usage and cost visibility harder than necessary.

Reviewer Test Plan

How to verify

Run /stats daily and confirm it prints the selected day’s total token usage, request count, input/output/cached/thought token breakdowns, and grouped totals by model and auth type.

Run /stats monthly and confirm it prints the selected month’s aggregate token usage with the same grouping.

Run /stats export monthly YYYY-MM --format csv and /stats export daily YYYY-MM-DD --format json --output usage/day.json and confirm the exported files contain aggregate summaries only, not prompt text, response text, project paths, prompt ids, or response ids.

Confirm export paths are constrained to the project working directory and reject traversal, symlinked output directories, symlinked output files, and Windows alternate-data-stream style paths.

Evidence (Before & After)

Before: /stats exposed session/model/tool stats but did not provide persisted daily/monthly token usage or CSV/JSON token usage export.

After: /stats daily, /stats monthly, and /stats export provide persisted aggregate token usage summaries with model/auth breakdowns and guarded CSV/JSON export.

Local validation passed: npx vitest run src/services/tokenUsageService.test.ts src/telemetry/loggers.test.ts, npx vitest run src/ui/commands/statsCommand.test.ts src/ui/hooks/useAutoAcceptIndicator.test.ts src/ui/components/AutoAcceptIndicator.test.tsx, npm run check-i18n --workspace=packages/cli, npm run lint --workspace=packages/cli, npm run lint --workspace=packages/core, npm run typecheck, npm run build, and git diff --check.

Tested on

OS Status
🍏 macOS ⚠️ not tested
🪟 Windows ✅ tested
🐧 Linux ⚠️ not tested

Environment (optional)

Windows PowerShell, local npm workspace scripts, Node.js runtime used by the repository build/test commands.

Risk & Scope

  • Main risk or tradeoff: token usage is persisted locally as content-free JSONL aggregate records, which keeps implementation lightweight but does not provide database-style querying beyond the supported day/month summaries.
  • Not validated / out of scope: full integration suite, macOS/Linux local manual verification, TTFT/TPS/generation-duration metrics, and /doctor memory or other memory-diagnostics work.
  • Breaking changes / migration notes: none expected; this adds new /stats subcommands and local aggregate usage files without changing existing /stats session/model/tool behavior.

Linked Issues

Closes #4479

References #4252 and #4182 for coordination only; this PR does not implement TTFT/TPS generation timing or memory diagnostics.

中文说明

What this PR does

本 PR 增加持久化 token 使用量统计,并扩展 /stats,让用户可以查看每日 token 使用量、每月 token 使用量、按模型和认证类型分组的明细,并将汇总后的使用量导出为 CSV 或 JSON。

本 PR 也记录了与相邻统计工作的边界:token 使用量统一放在 /stats 下,TTFT/TPS 等生成耗时指标仍属于独立范围,memory diagnostics 不在本次变更中扩展。

Why it's needed

当前用户没有直接的 CLI 可见方式来了解 Qwen Code 今天或本月消耗了多少 token,即使一次运行可能消耗大量 token,也不容易判断使用量和成本情况。本 PR 提升了使用量和成本可见性。

Reviewer Test Plan

How to verify

运行 /stats daily,确认输出所选日期的 token 总量、请求数、输入/输出/缓存/思考 token 明细,以及按模型和认证类型分组的统计。

运行 /stats monthly,确认输出所选月份的聚合 token 使用量,并包含相同的分组统计。

运行 /stats export monthly YYYY-MM --format csv/stats export daily YYYY-MM-DD --format json --output usage/day.json,确认导出的文件只包含聚合摘要,不包含 prompt 文本、response 文本、项目路径、prompt id 或 response id。

确认导出路径被限制在项目工作目录内,并会拒绝路径穿越、指向外部的符号链接目录、符号链接输出文件,以及 Windows alternate-data-stream 风格路径。

Evidence (Before & After)

Before:/stats 只提供会话、模型和工具统计,不提供持久化的每日/月度 token 使用量,也不支持 CSV/JSON token 使用量导出。

After:/stats daily/stats monthly/stats export 提供持久化的聚合 token 使用量统计,包含模型/认证类型分组,并支持带路径保护的 CSV/JSON 导出。

本地验证已通过:npx vitest run src/services/tokenUsageService.test.ts src/telemetry/loggers.test.tsnpx vitest run src/ui/commands/statsCommand.test.ts src/ui/hooks/useAutoAcceptIndicator.test.ts src/ui/components/AutoAcceptIndicator.test.tsxnpm run check-i18n --workspace=packages/clinpm run lint --workspace=packages/clinpm run lint --workspace=packages/corenpm run typechecknpm run buildgit diff --check

Tested on

OS Status
🍏 macOS ⚠️ not tested
🪟 Windows ✅ tested
🐧 Linux ⚠️ not tested

Environment (optional)

Windows PowerShell,本地 npm workspace 脚本,以及仓库 build/test 命令使用的 Node.js 运行时。

Risk & Scope

  • Main risk or tradeoff: token 使用量会以不含内容的本地 JSONL 聚合记录形式持久化,这让实现保持轻量,但查询能力限定在当前支持的 day/month 汇总范围内。
  • Not validated / out of scope: 完整 integration suite、macOS/Linux 本地手工验证、TTFT/TPS/生成耗时指标,以及 /doctor memory 或其他 memory diagnostics 工作。
  • Breaking changes / migration notes: 预计无破坏性变更;本 PR 只新增 /stats 子命令和本地聚合使用量文件,不改变现有 /stats 会话/模型/工具统计行为。

Linked Issues

Closes #4479

References #4252 and #4182 for coordination only; this PR does not implement TTFT/TPS generation timing or memory diagnostics.

Comment thread packages/core/src/telemetry/loggers.ts Outdated
Comment thread packages/cli/src/i18n/locales/ja.js Outdated
Comment thread packages/core/src/services/tokenUsageService.ts Outdated
Comment thread packages/core/src/services/tokenUsageService.ts
Comment thread packages/cli/src/ui/hooks/useAutoAcceptIndicator.ts Outdated
Comment thread packages/cli/src/ui/commands/statsCommand.ts Outdated
Comment thread packages/core/src/telemetry/loggers.ts Outdated
@wenshao

wenshao commented May 27, 2026

Copy link
Copy Markdown
Collaborator

[Critical] [typecheck] tsc --noEmit reports 8 type errors in packages/cli/src/ui/commands/statsCommand.test.ts at lines 277, 316, 369, 384, 437, 452, 505, 553:

Object literal may only specify known properties, and 'tool' does not exist in type '{ prompt: number; candidates: number; total: number; cached: number; thoughts: number; }'.

The test fixtures pass a tool property but the token usage type definition doesn't include it. Either add tool: number to the type or remove tool from the test fixtures.

Note: this does not block npm run build (test files are excluded from the build pipeline), and all 97 vitest tests pass. But tsc --noEmit fails on this file.

— qwen3.7-max via Qwen Code /review

shenyankm added a commit to shenyankm/qwen-code that referenced this pull request May 27, 2026
Tighten persisted token usage so internal prompt traffic and disabled usage statistics do not write history, while surfacing non-ENOENT write failures outside debug logs. Complete the reviewer-requested i18n coverage and regression tests around auto mode notices and best-effort writes.

Constraint: Follow-up to wenshao review comments on PR QwenLM#4564.

Rejected: Keeping token usage recording outside the internal-prompt gate | It would inflate daily and monthly stats with background prompts.

Confidence: high

Scope-risk: narrow

Directive: Keep /stats token usage scoped to user-visible API responses unless future requirements explicitly include background traffic.

Tested: npx vitest run src/services/tokenUsageService.test.ts src/telemetry/loggers.test.ts; npx vitest run src/ui/hooks/useAutoAcceptIndicator.test.ts src/ui/commands/statsCommand.test.ts; npm run typecheck; npm run lint --workspace=packages/core; npm run lint --workspace=packages/cli; npm run check-i18n --workspace=packages/cli; npm run build; git diff --check

Not-tested: Full repository test suite
@shenyankm
shenyankm requested a review from wenshao May 27, 2026 07:20
Comment thread packages/core/src/services/tokenUsageService.ts Outdated
Comment thread packages/core/src/services/tokenUsageService.ts
Comment thread packages/core/src/services/tokenUsageService.ts
Comment thread packages/core/src/services/tokenUsageService.ts
Comment thread packages/core/src/telemetry/loggers.ts Outdated
Comment thread packages/core/src/services/tokenUsageService.ts Outdated
Comment thread packages/cli/src/ui/commands/statsCommand.ts
Comment thread packages/cli/src/ui/commands/statsCommand.ts
Comment thread packages/cli/src/ui/commands/statsCommand.ts Outdated
Comment thread packages/cli/src/ui/commands/statsCommand.ts
Comment thread packages/core/src/services/tokenUsageService.ts
@shenyankm
shenyankm requested a review from wenshao May 27, 2026 11:13
Comment thread packages/core/src/services/tokenUsageService.ts Outdated
Comment thread packages/core/src/services/tokenUsageService.ts
Comment thread packages/core/src/services/tokenUsageService.ts
Comment thread packages/core/src/telemetry/loggers.ts Outdated
@wenshao

wenshao commented May 27, 2026

Copy link
Copy Markdown
Collaborator

Verification Report — PR #4564

Branch: feat/issue-4479-token-usage-statsmain
Commit: 7c15a4350
Environment: macOS Darwin 25.4.0, Node.js, local build

Build

Step Result
npm install OK (5625 packages)
npm run build OK (webui compiled successfully)

Test Results

Package Test File Tests Result
core tokenUsageService.test.ts 13 passed
core loggers.test.ts 53 passed
cli statsCommand.test.ts 27 passed
cli useAutoAcceptIndicator.test.ts 18 passed
cli AutoAcceptIndicator.test.tsx 1 passed
Total 5 files passed 112 passed

Type Check

Package Result
core ✅ clean
cli ✅ clean

Lint

All changed source files pass ESLint — clean.

i18n Check

npm run check-i18n --workspace=packages/cli — ✅ All checks passed (9 locale files updated: en, zh, zh-TW, ja, de, fr, pt, ru, ca)

Key Test Coverage Verified

  • TokenUsageService — 13 tests covering JSONL persistence, daily/monthly aggregation, model/auth-type breakdowns, concurrent write safety
  • Loggers — 53 tests covering token usage recording integration with telemetry loggers
  • StatsCommand — 27 tests covering /stats daily, /stats monthly, /stats export subcommands, CSV/JSON output, path traversal rejection, symlink guard, Windows ADS-style path rejection
  • AutoAcceptIndicator — 19 tests covering indicator state hooks and component rendering

Summary

All 112 tests pass, typecheck clean across both packages, lint clean, i18n check passed. The PR adds well-tested token usage accounting and /stats extensions. Export path security guards (traversal, symlink, ADS) are covered by tests. Ready for merge.

— wenshao

shenyankm added a commit to shenyankm/qwen-code that referenced this pull request May 27, 2026
Constraint: Address wenshao's latest PR QwenLM#4564 review suggestions without expanding the /stats command surface. Rejected: Keeping synchronous token-usage writes | sync I/O remains on the API response hot path. Confidence: high Scope-risk: narrow Directive: Keep token usage persistence best-effort and gated by explicit usage-statistics enablement. Tested: cd packages/core; npx vitest run src/services/tokenUsageService.test.ts src/telemetry/loggers.test.ts; cd packages/cli; npx vitest run src/ui/commands/statsCommand.test.ts; npm run typecheck; npm run build; npm run lint --workspace=packages/core; npm run lint --workspace=packages/cli; git diff --check Not-tested: Full repository test suite
@shenyankm

Copy link
Copy Markdown
Contributor Author

Thank you again for the thorough review, @wenshao. I’ve pushed one more commit addressing your latest suggestions. Looking forward to your review again when you have time.

@shenyankm
shenyankm requested a review from wenshao May 27, 2026 14:35
Comment thread packages/core/src/services/tokenUsageService.ts Outdated
Comment thread packages/cli/src/ui/commands/statsCommand.ts Outdated
Comment thread packages/core/src/telemetry/loggers.ts Outdated
shenyankm added a commit to shenyankm/qwen-code that referenced this pull request May 27, 2026
Propagate token usage read failures through the existing /stats error path while keeping missing usage files empty, and remove the unreachable telemetry wrapper catch.

Constraint: PR QwenLM#4564 review requested user-visible read failures, full i18n for export errors, and removal of dead telemetry catch code.
Rejected: Adding warning fields to TokenUsageSummary | It would expand the JSON/export schema when the existing command error path already fits read failures.
Confidence: high
Scope-risk: narrow
Directive: Keep jsonl.read default swallowing behavior for existing session/history callers unless a user-visible caller opts into rethrowing non-ENOENT errors.
Tested: npx vitest run src/utils/jsonl-utils.test.ts src/services/tokenUsageService.test.ts src/telemetry/loggers.test.ts
Tested: npx vitest run src/ui/commands/statsCommand.test.ts
Tested: npm run check-i18n --workspace=packages/cli
Tested: npx prettier --check changed files
Tested: npm run typecheck
Tested: npm run lint --workspace=packages/core
Tested: npm run lint --workspace=packages/cli
Tested: git diff --check
Tested: npm run build
Not-tested: Full integration test suite
@shenyankm
shenyankm requested a review from wenshao May 27, 2026 16:37
Comment thread packages/cli/src/ui/commands/statsCommand.ts Outdated
Comment thread packages/core/src/services/tokenUsageService.ts Outdated
Comment thread packages/core/src/services/tokenUsageService.ts Outdated
shenyankm added a commit to shenyankm/qwen-code that referenced this pull request May 28, 2026
Keep the review follow-ups local to token usage accounting and stats export without adding new abstractions.

Constraint: Address PR QwenLM#4564 reviewer requests on token usage export/query reuse, write-failure stderr noise, and invalid-record diagnostics.
Confidence: high
Scope-risk: narrow
Directive: Keep token usage writes best-effort and avoid noisy stderr loops for repeated local failures.
Tested: git diff --check; prior targeted core/cli tests, typecheck, and lint passed for this working tree.
Not-tested: Full repository test suite.
@shenyankm
shenyankm requested a review from wenshao May 28, 2026 04:50
Comment thread packages/core/src/services/tokenUsageService.ts
Comment thread packages/core/src/services/tokenUsageService.ts Outdated
Comment thread packages/core/src/index.ts
Comment thread packages/core/src/services/tokenUsageService.test.ts
Comment thread packages/core/src/services/tokenUsageService.test.ts
@wenshao

wenshao commented May 28, 2026

Copy link
Copy Markdown
Collaborator

Maintainer Verification Report

Reviewer: wenshao
Branch: feat/issue-4479-token-usage-stats @ a004913e2
Base: main
Date: 2026-05-28


CI Status

All CI checks passing:

  • Lint: PASS
  • CodeQL: PASS
  • Test (macOS, Node 22.x): PASS
  • Test (Ubuntu, Node 22.x): PASS
  • Test (Windows, Node 22.x): PASS

Local Validation

Check Status
npm run typecheck (all packages) PASS
npm run lint --workspace=packages/core PASS
npm run lint --workspace=packages/cli PASS
npm run check-i18n --workspace=packages/cli PASS
npm run build FAIL — pre-existing @opentelemetry/instrumentation-undici resolution in sdk.ts (not touched by this PR)

Local Test Results

Suite Tests Status
tokenUsageService.test.ts 24/24 PASS
loggers.test.ts 51/51 PASS
jsonl-utils.test.ts 16/16 PASS
statsCommand.test.ts 33/33 PASS
useAutoAcceptIndicator.test.ts 7/7 PASS
AutoAcceptIndicator.test.tsx 7/7 PASS
Total 138/138 All PASS

Note: mustTranslateKeys.test.ts fails to load due to pre-existing vite resolution issue (@qwen-code/acp-bridge/eventBus); the test file itself is not modified by this PR. i18n coverage validated via npm run check-i18n passing.

Code Review Summary

Feature: Persisted token-usage accounting with /stats daily, /stats monthly, and /stats export commands. Content-free JSONL aggregate records, no prompt/response text stored.

Core — tokenUsageService.ts (534 lines):

  • Clean service with well-defined types (TokenUsageRecord, TokenUsageSummary, TokenUsageGroupSummary)
  • Records are content-free: only counters + stable dimensions (model, authType, source, localDate/Month)
  • isTokenUsageRecord validates schema version bounds and all field types
  • readRecordsForMonth uses new throwOnNonEnoentError option on jsonl.read — ENOENT returns empty (no usage yet), other read errors propagate to user
  • Invalid records dropped with debug log + count
  • recordTokenUsageFromApiResponseBestEffort — fire-and-forget with rate-limited stderr logging (60s cooldown per error code)
  • CSV export has formula injection protection (csvEscape prefixes =+\-@\t\r\n with ')
  • toNonNegativeInteger handles undefined/NaN/negative safely
  • calculateTotalTokens falls back to sum of components when total is missing

Integration — loggers.ts (+4 lines):

  • Gated by config.getUsageStatisticsEnabled() — respects user telemetry preference
  • Inside existing !isInternalPromptId guard — internal/background prompts excluded
  • Uses best-effort variant — won't block or throw on API response path

CLI — statsCommand.ts (664 lines added):

  • parseStatsExportArgs supports --format csv|json and --output path with quoted-string tokenizer
  • Export path validation is thorough:
    • isSubpath containment check against project cwd
    • Symlinked output directories rejected (realpath comparison)
    • Symlinked output files rejected (lstat check)
    • Windows ADS paths rejected (basename.includes(':'))
    • Pre- and post-rename directory identity verification (TOCTOU mitigation)
    • Atomic write via temp file + rename with 10 retry attempts on EEXIST
    • Post-rename final file validation

jsonl-utils.ts (+11 lines):

  • New throwOnNonEnoentError option — opt-in for callers that need read-failure visibility (token usage), existing callers unaffected (default false)

i18n: 9 locale files updated, all keys present across ca/de/en/fr/ja/pt/ru/zh-TW/zh. check-i18n passes.

useAutoAcceptIndicator.ts: Extracted hardcoded strings to t() calls — i18n consistency fix, no logic changes.

Design doc: docs/design/issue-4479-token-usage-stats-coordination.md documents scope boundaries with #4252 (TTFT/TPS) and #4182 (memory diagnostics).

Observations

  1. The service is well-scoped — aggregate counters only, no sensitive content persisted
  2. Export path security is thorough with multiple TOCTOU mitigations
  3. Rate-limited failure logging avoids stderr noise on persistent write failures
  4. The apiDurationMs field is carefully labeled as API duration (not generation timing) to avoid scope creep with ## 🚀 Feature Request: Add Generation Timing Metrics (TPS, TTFT) to /stats #4252

Verdict

LGTM for merge. CI is all green across all 3 platforms. 138 local tests pass. The feature is well-scoped, privacy-conscious (content-free records), and the export path validation is thorough. No regressions detected.

@shenyankm

Copy link
Copy Markdown
Contributor Author

@wenshao 我已按照您的建议提交了一次优化,期待再次审查。

@shenyankm
shenyankm requested a review from wenshao May 30, 2026 05:08
Comment thread packages/core/src/services/tokenUsageService.ts Outdated
shenyankm added a commit to shenyankm/qwen-code that referenced this pull request May 30, 2026
Keep the repeated write-failure regression test aligned with the runtime wording that the PR now emits.

Constraint: PR QwenLM#4564 CI failed after the implementation wording changed to "since last log".
Rejected: Reverting the implementation wording | it is the latest PR behavior and the failure is test-only.
Confidence: high
Scope-risk: narrow
Tested: cd packages/core && npx vitest run src/services/tokenUsageService.test.ts
Not-tested: full repository test suite
@shenyankm
shenyankm requested a review from wenshao May 30, 2026 08:43
wenshao
wenshao previously approved these changes May 30, 2026

@wenshao wenshao left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

R8 Suggestion addressed — "in last window" → "since last log" phrasing fix is accurate and the test assertion is updated consistently. No new issues found in the incremental change. All prior-round findings (18 stale comments) have been resolved across earlier commits. LGTM! ✅ — qwen3.7-max via Qwen Code /review

@wenshao

wenshao commented May 30, 2026

Copy link
Copy Markdown
Collaborator

Local Verification Report — PR #4564

Tested on: macOS Darwin 25.4.0 (Apple Silicon)
Branch: feat/issue-4479-token-usage-stats @ a3bfb55
Base: main
Tester: wenshao


Test Results Summary

Test Suite Result Details
packages/core tokenUsageService.test.ts PASS 16 tests passed
packages/core loggers.test.ts PASS 51 tests passed
packages/core jsonl-utils.test.ts PASS 24 tests passed
packages/cli statsCommand.test.ts PASS 28 tests passed
packages/cli useAutoAcceptIndicator.test.ts PASS 18 tests passed
packages/cli AutoAcceptIndicator.test.tsx PASS 1 test passed
npm run check-i18n --workspace=packages/cli PASS All i18n checks passed
npm run lint --workspace=packages/cli PASS 0 errors
npm run lint --workspace=packages/core PASS 0 errors
npm run typecheck PASS 0 errors
npm run build PASS 0 errors, 15 pre-existing warnings (SDK lint)
git diff --check PASS No whitespace issues

Total: 138 tests passed, 0 failures. All 12 verification steps green.


Environment Notes

  • Initial run failed due to missing @opentelemetry/instrumentation-undici (not yet in lockfile for this branch); resolved after npm install. This is expected for first-time checkout and not a PR issue.
  • The 15 build warnings are pre-existing SDK lint warnings (curly brace style), not related to this PR.

Conclusion

PR is merge-ready from a testing perspective. All items from the PR test plan verified on macOS:

  • Core unit tests: tokenUsageService (16), loggers (51), jsonl-utils (24) — all pass
  • CLI component tests: statsCommand (28), useAutoAcceptIndicator (18), AutoAcceptIndicator (1) — all pass
  • i18n check: all locale keys present and valid
  • Lint: 0 errors in both packages/cli and packages/core
  • TypeScript typecheck: 0 errors across all packages
  • Build: completes successfully
  • Whitespace: clean

Verified locally by wenshao

@wenshao wenshao left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Downgraded from Approve to Comment: CI failing (CodeQL, Post Coverage Comment, Lint, Test, review-pr, Classify PR). — qwen3.7-max via Qwen Code /review

Comment thread packages/core/src/services/tokenUsageService.ts Outdated
Comment thread packages/cli/src/ui/commands/statsCommand.ts
Comment thread packages/cli/src/ui/commands/statsCommand.ts
Comment thread packages/cli/src/ui/commands/statsCommand.ts
Comment thread packages/core/src/services/tokenUsageService.ts

@wenshao wenshao left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No review findings. Downgraded from Approve to Comment: CI still running. The incremental change (CSV export hardening + new tests) is well-targeted and all 110 relevant tests pass. Low-confidence observations (pipe delimiter in composite key, toNonNegativeInteger(0) fallback semantics, ensuredDirs cache + ENOENT silencing, minor test coverage gaps) noted in terminal review only — none warrant inline comments at this stage. — qwen3.7-max via Qwen Code /review

Copy link
Copy Markdown

This PR is useful for custom-provider users too, because token stats only help if the route identity stays visible.

For multi-provider setups, I would keep these fields explicit in the usage record and export:

  • provider adapter
  • base URL host
  • upstream model id
  • credential / env slot
  • input / output / cached / thought token usage

That makes it much easier to debug cases where the same model is tested through two different gateways and the cost numbers do not line up. If someone is blocked on access or payment rather than the CLI itself, I also keep a tiny paid test path available here:
https://black-eagle-ai.vercel.app/starter-trial/

@wenshao

wenshao commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

@copilot resolve the merge conflicts in this pull request

@wenshao

wenshao commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

✅ Verification report — real runtime testing of PR #4564

I verified this PR with the real built CLI — unit tests, deterministic seeded data, a real model call, an interactive TUI run (tmux), and adversarial security tests — on a clean worktree at the PR head (796831871). The feature is correct, privacy-safe, and security-hardened. One pre-merge action is needed: a rebase (details at the end). CI is green on macOS/Ubuntu/Windows (Node 22).


1. Unit tests — all green (157 tests + i18n)

core   tokenUsageService.test.ts   18 │ loggers.test.ts 56 │ jsonl-utils.test.ts 28   → 102
cli    statsCommand.test.ts        36 │ useAutoAcceptIndicator 18 │ AutoAcceptIndicator 1 → 55
npm run check-i18n --workspace=packages/cli  → ✅ All checks passed

2. /stats daily & /stats monthly — display correct (seeded + interactive TUI)

I seeded 5 content-free usage records (2 days, 3 model/auth combos) into ~/.qwen/usage/token-usage-2026-06.jsonl and ran the real CLI. /stats daily 2026-06-18 (identical output in non-interactive -p and the interactive TUI in tmux):

Daily token usage for 2026-06-18
Total: 6,200 tokens          Requests: 4
Breakdown:  Input: 4,100   Output: 1,950   Cached (included in Input): 700   Thoughts: 150
By model:          qwen3-coder 3,000 (1) · deepseek-chat 2,000 (2) · glm-4-flash 1,200 (1)
By auth type:      openai 3,200 (3) · qwen-oauth 3,000 (1)
By model/auth type: qwen3-coder (qwen-oauth) 3,000 (1) · deepseek-chat (openai) 2,000 (2) · …
By source:         main 3,200 (3) · subagent-A 3,000 (1)

Every aggregate matched the seed exactly; groups are sorted by tokens desc and locale-formatted. /stats monthly 2026-06 correctly aggregated across both days (Total 13,700, 5 requests, deepseek-chat 9,500/3).

3. Export (CSV + JSON) — aggregate-only, privacy claim verified 🔒

/stats export monthly 2026-06 --format csv                          → qwen-token-usage-month-2026-06.csv  (mode 0600)
/stats export daily 2026-06-18 --format json --output usage/day.json → usage/day.json (subdir auto-created, 0600)

CSV header: period,value,group_type,group_key,model,auth_type,source,requests,input_tokens,output_tokens,cached_tokens,thoughts_tokens,total_tokens,api_duration_ms — rows are total / model / auth_type / model_auth_type / source groupings only. JSON is the structured summary (totals + byModel/byAuthType/byModelAndAuthType/bySource).

Inspected both files: no prompt text, no response text, no project paths, no prompt/response ids — and crucially no sessionId (it exists in the raw record but is dropped from the summary). The export is structurally incapable of leaking content. ✔

4. Recording hook — real end-to-end proof

Starting from an empty isolated home, one real deepseek-chat call (replied PONG) created token-usage-2026-06.jsonl with real, content-free records:

model=deepseek-chat authType=openai source=main                          input=28950 output=2  total=28952
model=deepseek-chat authType=openai source=managed-auto-memory-extractor input=7979  …  cached=7808

Confirms loggers.ts → recordTokenUsageFromApiResponseBestEffort fires on real API responses (gated on usageStatisticsEnabled), captures cached tokens, and correctly tags source (main vs. subagent).

5. Security — export path guards (adversarial, real filesystem) 🛡️

All run via the real CLI against a seeded project; OUTSIDE is a dir outside the project holding target.csv = "secret":

Attack --output Result Leak?
relative traversal ../../../tmp/evil.csv ❌ rejected no file written
absolute outside cwd /tmp/evil-abs.csv ❌ rejected no file written
symlinked output dir → outside linkdir/x.csv ❌ rejected no file via symlink
symlinked output file → outside symfile.csv ❌ rejected OUTSIDE/target.csv still "secret" (not overwritten)
valid control sub/ok.csv ✅ written in-cwd (0600)

Every rejection returned Token usage export path must be within the project working directory.no data escaped the project directory in any case. The Windows alternate-data-stream guard is win32-gated (untestable at runtime on Linux, where report.csv:secret is correctly a normal in-cwd file) but is unit-tested and runs on the passing Windows CI job. The implementation is TOCTOU-hardened (existing-parent realpath check → mkdir → re-validate dir → exclusive wx/0600 temp file → re-validate before & after rename).


⚠️ Pre-merge action & observations

  1. Merge conflict — must rebase, but trivial. A dry-run merge into current main conflicts in exactly one file: packages/cli/src/acp-integration/session/Session.test.ts (a test, unrelated to the feature). The token-usage code itself merges cleanly.
  2. Diff carries unrelated merge-noise. Beyond the token-usage feature, the diff bundles: i18n t()-wrapping in useAutoAcceptIndicator.ts (+ useAutoAcceptIndicator.test.ts, AutoAcceptIndicator.test.tsx), vscode-ide-companion/.../App.test.tsx, speculation.test.ts, background-tasks.test.ts, daemon design docs, and pure prettier reflows in truncation.ts / toolResultCleanup.ts. None are functional changes to other features, but rebasing to isolate the feature would make the diff much easier to review.
  3. Platform coverage. Author tested on Windows only; this report adds the Linux runtime coverage (display, export, recording, and the symlink/traversal guards). macOS remains unexercised locally, though the logic is platform-neutral apart from the win32-gated ADS guard.

Verdict

The feature does exactly what it claims — correct day/month aggregation, content-free aggregate export, and robust path-traversal/symlink protection — verified at unit, non-interactive, interactive-TUI, real-recording, and adversarial-security levels. Recommend merge after a rebase to resolve the single Session.test.ts conflict (ideally also trimming the unrelated merge-noise from the diff).

🇨🇳 中文版验证报告(点击展开)

✅ 验证报告 — PR #4564 真实运行时测试

我用真实构建的 CLI 验证了本 PR —— 单测、确定性种子数据、一次真实模型调用、交互式 TUI(tmux)以及对抗性安全测试 —— 基于 PR 头(796831871)的干净 worktree。功能正确、隐私安全、且对路径攻击做了加固。 合并前需要一个动作:rebase(见末尾)。CI 在 macOS/Ubuntu/Windows(Node 22)全绿。

1. 单测 —— 全绿(157 + i18n)

core   tokenUsageService 18 │ loggers 56 │ jsonl-utils 28  → 102
cli    statsCommand 36 │ useAutoAcceptIndicator 18 │ AutoAcceptIndicator 1 → 55
check-i18n → ✅ 全部通过

2. /stats daily/stats monthly —— 显示正确(种子 + 交互式 TUI)

我向 ~/.qwen/usage/token-usage-2026-06.jsonl 种入 5 条不含内容的使用记录(2 天、3 个模型/认证组合),运行真实 CLI/stats daily 2026-06-18(非交互 -p 与 tmux 交互式 TUI 输出一致):

Total: 6,200 tokens   Requests: 4
Input 4,100 · Output 1,950 · Cached 700 · Thoughts 150
By model / By auth type / By model+auth type / By source 均正确,按 token 降序、按语言千分位格式化

所有聚合值与种子完全一致。/stats monthly 2026-06 正确跨两天聚合(总计 13,700,5 次请求,deepseek-chat 9,500/3)。

3. 导出(CSV + JSON)—— 仅聚合,隐私声明已验证 🔒

/stats export monthly 2026-06 --format csv                          → qwen-token-usage-month-2026-06.csv(权限 0600)
/stats export daily 2026-06-18 --format json --output usage/day.json → usage/day.json(自动创建子目录,0600)

CSV 只有 total/model/auth_type/model_auth_type/source 分组行;JSON 是结构化摘要。两个文件都检查过:无 prompt 文本、无 response 文本、无项目路径、无 prompt/response id —— 且无 sessionId(原始记录里有,但摘要里被丢弃)。导出在结构上不可能泄漏内容。✔

4. 记录钩子 —— 真实端到端证明

空的隔离 home 出发,一次真实 deepseek-chat 调用(回复 PONG)即生成了含真实、不含内容记录的 JSONL:source=mainsource=managed-auto-memory-extractor(子代理)区分正确,cachedTokens 也被捕获。证明 loggers.ts → recordTokenUsageFromApiResponseBestEffort 在真实 API 响应上触发(受 usageStatisticsEnabled 开关控制)。

5. 安全 —— 导出路径防护(对抗性,真实文件系统)🛡️

攻击 --output 结果 泄漏?
相对路径穿越 ../../../tmp/evil.csv ❌ 拒绝 无文件写出
项目外绝对路径 /tmp/evil-abs.csv ❌ 拒绝 无文件写出
符号链接目录→外部 linkdir/x.csv ❌ 拒绝 未经符号链接写出
符号链接文件→外部 symfile.csv ❌ 拒绝 外部 target.csv 仍是 "secret"(未被覆盖)
合法对照 sub/ok.csv ✅ 写入项目内(0600)

每次拒绝都返回 Token usage export path must be within the project working directory. —— 任何情况下都没有数据逃出项目目录。 Windows ADS 防护是 win32 限定(Linux 上无法在运行时触发,report.csv:secret 在 Linux 上是正常文件名),但有单测覆盖,并在通过的 Windows CI 上运行。实现做了 TOCTOU 加固(existing-parent realpath 检查 → mkdir → 复核目录 → 独占 wx/0600 临时文件 → rename 前后再复核)。

⚠️ 合并前动作与观察

  1. 合并冲突 —— 需 rebase,但很小。 对当前 main 试合并只有一个文件冲突:packages/cli/src/acp-integration/session/Session.test.ts(测试文件,与功能无关)。token-usage 代码本身可干净合并。
  2. diff 夹带无关 merge 噪音。 除 token-usage 功能外,diff 还捆绑了:useAutoAcceptIndicator.ts 的 i18n t() 包装(及其测试、AutoAcceptIndicator.test.tsx)、vscode-ide-companion/.../App.test.tsxspeculation.test.tsbackground-tasks.test.ts、daemon 设计文档,以及 truncation.ts/toolResultCleanup.ts 的纯 prettier 重排。都不是对其他功能的实质改动,但 rebase 收拢 diff 会让评审清爽很多。
  3. 平台覆盖。 作者仅在 Windows 测过;本报告补充了 Linux 运行时覆盖(显示、导出、记录、符号链接/穿越防护)。macOS 本地未跑,但除 win32 限定的 ADS 防护外逻辑与平台无关。

结论

功能与描述完全一致 —— 正确的日/月聚合、不含内容的聚合导出、稳健的路径穿越/符号链接防护,已在单测、非交互、交互式 TUI、真实记录、对抗性安全多个层面验证。建议在 rebase 解决唯一的 Session.test.ts 冲突后合并(最好同时把无关 merge 噪音从 diff 中清掉)。

shenyankm and others added 13 commits June 18, 2026 10:24
Persist content-free API token counters and surface daily/monthly summaries plus CSV/JSON export through /stats.

Constraint: Issue QwenLM#4479 requested CLI token visibility with monthly/model breakdowns and export while coordinating with QwenLM#4252/QwenLM#4182.\nRejected: Add a separate top-level token command | /stats keeps related statistics in one surface.\nConfidence: high\nScope-risk: moderate\nDirective: Keep TTFT/TPS generation timing and memory diagnostics outside this token-usage surface unless their issues explicitly broaden scope.\nTested: npx vitest run src/services/tokenUsageService.test.ts src/telemetry/loggers.test.ts; npx vitest run src/ui/commands/statsCommand.test.ts src/ui/hooks/useAutoAcceptIndicator.test.ts src/ui/components/AutoAcceptIndicator.test.tsx; npm run check-i18n --workspace=packages/cli; npm run lint --workspace=packages/cli; npm run lint --workspace=packages/core; npm run typecheck; npm run build; git diff --check\nNot-tested: full integration suite
Tighten persisted token usage so internal prompt traffic and disabled usage statistics do not write history, while surfacing non-ENOENT write failures outside debug logs. Complete the reviewer-requested i18n coverage and regression tests around auto mode notices and best-effort writes.

Constraint: Follow-up to wenshao review comments on PR QwenLM#4564.

Rejected: Keeping token usage recording outside the internal-prompt gate | It would inflate daily and monthly stats with background prompts.

Confidence: high

Scope-risk: narrow

Directive: Keep /stats token usage scoped to user-visible API responses unless future requirements explicitly include background traffic.

Tested: npx vitest run src/services/tokenUsageService.test.ts src/telemetry/loggers.test.ts; npx vitest run src/ui/hooks/useAutoAcceptIndicator.test.ts src/ui/commands/statsCommand.test.ts; npm run typecheck; npm run lint --workspace=packages/core; npm run lint --workspace=packages/cli; npm run check-i18n --workspace=packages/cli; npm run build; git diff --check

Not-tested: Full repository test suite
Constraint: wenshao review required consistent token stats, exports, i18n, and best-effort logging behavior.

Rejected: Change cached-token labeling | keeping cached tokens included in input preserves the accepted /stats display contract.

Confidence: high

Scope-risk: narrow

Directive: Keep cached tokens included in input whenever cached-only metadata is used in total fallback.

Tested: cd packages/core && npx vitest run src/services/tokenUsageService.test.ts src/telemetry/loggers.test.ts

Tested: cd packages/cli && npx vitest run src/ui/commands/statsCommand.test.ts src/i18n/mustTranslateKeys.test.ts

Tested: npm run check-i18n --workspace=packages/cli; npm run typecheck; git diff --check

Not-tested: full integration suite
Constraint: Address wenshao's latest PR QwenLM#4564 review suggestions without expanding the /stats command surface. Rejected: Keeping synchronous token-usage writes | sync I/O remains on the API response hot path. Confidence: high Scope-risk: narrow Directive: Keep token usage persistence best-effort and gated by explicit usage-statistics enablement. Tested: cd packages/core; npx vitest run src/services/tokenUsageService.test.ts src/telemetry/loggers.test.ts; cd packages/cli; npx vitest run src/ui/commands/statsCommand.test.ts; npm run typecheck; npm run build; npm run lint --workspace=packages/core; npm run lint --workspace=packages/cli; git diff --check Not-tested: Full repository test suite
Propagate token usage read failures through the existing /stats error path while keeping missing usage files empty, and remove the unreachable telemetry wrapper catch.

Constraint: PR QwenLM#4564 review requested user-visible read failures, full i18n for export errors, and removal of dead telemetry catch code.
Rejected: Adding warning fields to TokenUsageSummary | It would expand the JSON/export schema when the existing command error path already fits read failures.
Confidence: high
Scope-risk: narrow
Directive: Keep jsonl.read default swallowing behavior for existing session/history callers unless a user-visible caller opts into rethrowing non-ENOENT errors.
Tested: npx vitest run src/utils/jsonl-utils.test.ts src/services/tokenUsageService.test.ts src/telemetry/loggers.test.ts
Tested: npx vitest run src/ui/commands/statsCommand.test.ts
Tested: npm run check-i18n --workspace=packages/cli
Tested: npx prettier --check changed files
Tested: npm run typecheck
Tested: npm run lint --workspace=packages/core
Tested: npm run lint --workspace=packages/cli
Tested: git diff --check
Tested: npm run build
Not-tested: Full integration test suite
Keep the review follow-ups local to token usage accounting and stats export without adding new abstractions.

Constraint: Address PR QwenLM#4564 reviewer requests on token usage export/query reuse, write-failure stderr noise, and invalid-record diagnostics.
Confidence: high
Scope-risk: narrow
Directive: Keep token usage writes best-effort and avoid noisy stderr loops for repeated local failures.
Tested: git diff --check; prior targeted core/cli tests, typecheck, and lint passed for this working tree.
Not-tested: Full repository test suite.
Co-authored-by: Shaojin Wen <shaojin.wensj@alibaba-inc.com>
Keep the repeated write-failure regression test aligned with the runtime wording that the PR now emits.

Constraint: PR QwenLM#4564 CI failed after the implementation wording changed to "since last log".
Rejected: Reverting the implementation wording | it is the latest PR behavior and the failure is test-only.
Confidence: high
Scope-risk: narrow
Tested: cd packages/core && npx vitest run src/services/tokenUsageService.test.ts
Not-tested: full repository test suite
Address the remaining PR review polish without changing token accounting, export formats, or path containment behavior.

Constraint: Review 4452925552 requested narrow documentation, ENOENT wording, and NOTICES cleanup only.

Rejected: Broader merge-conflict rework | GitHub currently reports the PR as mergeable, and the requested fixes are review polish.

Confidence: high

Scope-risk: narrow

Directive: Keep token usage records content-free and preserve export path validation semantics except for the final ENOENT message.

Tested: cd packages/core && npx vitest run src/services/tokenUsageService.test.ts; cd packages/cli && npx vitest run src/ui/commands/statsCommand.test.ts; npm run check-i18n --workspace=packages/cli; npm run typecheck; git diff --check on changed code and i18n files

Not-tested: Full test suite not run.
@shenyankm
shenyankm force-pushed the feat/issue-4479-token-usage-stats branch from 7968318 to 19edc2c Compare June 18, 2026 05:46
@wenshao

wenshao commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

✅ Local verification — /stats token usage @ 19edc2c

Verdict: PASS. Driven through the real qwen TUI under tmux against seeded usage data — no LLM needed, because the feature reads token usage from the on-disk JSONL store (<runtimeDir>/usage/token-usage-<YYYY-MM>.jsonl).

Method

Isolated worktree off PR head 19edc2c, clean npm ci, npm run bundle. Launched dist/cli.js under tmux with an isolated QWEN_HOME + QWEN_RUNTIME_DIR and a settings file pre-selecting auth (so no live model is contacted). Seeded 3 TokenUsageRecords in token-usage-2026-06.jsonl: two dated today (2026-06-18) and one dated 2026-06-10, across two models / two auth types / two sources, then ran the new subcommands and checked the aggregates against hand-computed expectations.

/stats daily — today-scoped (2 of 3 records)

● Daily token usage for 2026-06-18
  Total: 2,000 tokens          Requests: 2
  Breakdown: Input 1,300 · Output 650 · Cached 200 · Thoughts 50
  By model:        qwen-max 1,550 (1)   qwen-plus 450 (1)
  By auth type:    qwen-oauth 1,550 (1) openai 450 (1)
  By model/auth:   qwen-max (qwen-oauth) 1,550 (1)   qwen-plus (openai) 450 (1)
  By source:       main 2,000 (2)

✅ All totals and every grouping match the seed exactly. The 2026-06-10 record (source=subagent:foo) is correctly excluded from today's view.

/stats monthly — month-scoped (all 3 records)

● Monthly token usage for 2026-06
  Total: 5,000 tokens          Requests: 3
  Breakdown: Input 3,300 · Output 1,650 · Cached 700 · Thoughts 150
  By model:      qwen-max 4,550 (2)   qwen-plus 450 (1)
  By auth type:  qwen-oauth 4,550 (2) openai 450 (1)
  By source:     subagent:foo 3,000 (1)   main 2,000 (2)

✅ The 2026-06-10 record is now correctly included — daily-vs-monthly scoping works.

Probes 🔍

  • 🔍 /stats daily 2020-01-01 (no data / missing month file)Total: 0, Requests: 0, every group No usage data. — graceful, no crash (missing-file ENOENT handled).
  • 🔍 /stats export monthly --format csv --output usage.csv● Token usage exported to CSV: usage.csv. File written inside the project (mode 0600); contents well-formed and consistent with the display, e.g. the total row month,2026-06,total,…,3,3300,1650,700,150,5000,4500 (apiDurationMs 4500 = 1200+800+2500), plus per-model / per-auth / per-source rows.
  • 🔍 /stats export monthly --format csv --output /tmp/evil.csv (outside project)✕ Token usage export path must be within the project working directory. and no file written outside the project — the path-containment guard holds.

Findings

  • The feature is fully usable from a cold store: aggregation, daily/monthly scoping, grouping (model / auth / model+auth / source), empty-period handling, and CSV export all behave correctly on real seeded data.
  • CSV export is written atomically with 0600 perms and is confined to the project working directory; a path outside it is rejected rather than written.
  • Not exercised here: the write path (recordTokenUsageFromApiResponse via telemetry loggers) that populates the JSONL during a live turn — that needs real API responses (model credentials). It is covered by the PR's unit tests (tokenUsageService.test.ts, loggers.test.ts); I verified the read/aggregate/export half end-to-end through the TUI.
🇨🇳 中文版(点击展开)

✅ 本地验证 —— /stats token 用量 @ 19edc2c

结论:通过(PASS)。 通过 tmux 中的真实 qwen TUI 驱动,使用预置用量数据 —— 无需 LLM,因为该功能从磁盘上的 JSONL 存储读取(<runtimeDir>/usage/token-usage-<YYYY-MM>.jsonl)。

方法

基于 PR head 19edc2c 的隔离 worktree,干净 npm cinpm run bundle。在 tmux 中以隔离的 QWEN_HOME + QWEN_RUNTIME_DIR 启动 dist/cli.js,并用预置的 settings 选定鉴权(因此不会联系真实模型)。在 token-usage-2026-06.jsonl 中预置 3TokenUsageRecord:两条为今天(2026-06-18)、一条为 2026-06-10,覆盖两种模型/两种鉴权/两种来源,然后运行新子命令并将聚合结果与手算预期核对。

/stats daily —— 限定当天(3 条中的 2 条)

● Daily token usage for 2026-06-18
  Total: 2,000 tokens          Requests: 2
  Breakdown: Input 1,300 · Output 650 · Cached 200 · Thoughts 50
  By model:        qwen-max 1,550 (1)   qwen-plus 450 (1)
  By auth type:    qwen-oauth 1,550 (1) openai 450 (1)
  By source:       main 2,000 (2)

✅ 所有合计与每个分组都与预置完全一致。2026-06-10 的记录(source=subagent:foo)被正确排除在当天视图之外。

/stats monthly —— 限定当月(全部 3 条)

● Monthly token usage for 2026-06
  Total: 5,000 tokens          Requests: 3
  Breakdown: Input 3,300 · Output 1,650 · Cached 700 · Thoughts 150
  By model:      qwen-max 4,550 (2)   qwen-plus 450 (1)
  By source:     subagent:foo 3,000 (1)   main 2,000 (2)

✅ 2026-06-10 的记录此时被正确纳入 —— 日/月范围划分正确。

边界探测 🔍

  • 🔍 /stats daily 2020-01-01(无数据 / 月份文件不存在)Total: 0Requests: 0,各分组显示 No usage data. —— 优雅处理,无崩溃(ENOENT 已处理)。
  • 🔍 /stats export monthly --format csv --output usage.csv● Token usage exported to CSV: usage.csv。文件写入项目内(权限 0600);内容规范且与展示一致,如合计行 month,2026-06,total,…,3,3300,1650,700,150,5000,4500(apiDurationMs 4500 = 1200+800+2500),并含按模型/按鉴权/按来源的明细行。
  • 🔍 /stats export monthly --format csv --output /tmp/evil.csv(项目目录之外)✕ Token usage export path must be within the project working directory.,且未在项目外写入文件 —— 路径限制守卫有效。

观察与结论

  • 该功能在冷存储下即可完整使用:聚合、日/月范围、分组(模型/鉴权/模型+鉴权/来源)、空周期处理、CSV 导出在真实预置数据上均行为正确。
  • CSV 导出以 0600 权限原子写入并限制在项目工作目录内;目录外路径被拒绝而非写入。
  • 此处未覆盖:在真实回合中写入 JSONL 的写入路径recordTokenUsageFromApiResponse,经遥测 loggers)—— 需要真实 API 响应(模型凭据)。该部分由 PR 的单元测试覆盖(tokenUsageService.test.tsloggers.test.ts);我通过 TUI 端到端验证了读取/聚合/导出这一半。

Verified locally against PR head 19edc2c. /stats driven in the real TUI under tmux; usage store seeded (no model contacted).

Copy link
Copy Markdown

This is a useful direction for anyone using Qwen Code with direct providers, OpenAI-compatible gateways, or local proxies.

For cost/debug visibility, I would want each usage record to preserve the route identity, not only the public model display name. The minimum tuple that helps users debug billing drift is:

  • auth/provider profile
  • final base URL host
  • upstream model id sent on the wire
  • Qwen Code model alias or selected model label
  • token units used for billing, separated by input/output/cache/thinking where available
  • whether the request came from the main session, subagent/background task, or another internal source

That prevents a common confusing case: the same visible model name is tested through direct DashScope, Coding Plan, a gateway, and a local proxy, but each path may bill, authorize, and report usage differently.

Disclosure: I am involved with Black Eagle AI, an independent Chinese-model gateway project. No paid CTA here; this PR is just directly relevant to the route/accounting pattern we keep seeing with Qwen/DashScope/OpenAI-compatible setups.

@wenshao

wenshao commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

@qwen-code /triage

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Thanks for the PR!

Template looks good ✓

On direction: this is a solid fit. Token usage visibility is a clear user need (#4479), and the /stats command is the right home for it. Claude Code has similar cost/token-tracking features in their CHANGELOG (token usage dialog in VSCode, per-interaction token tracking), so the area is well-validated. The coordination doc documenting boundaries with #4252 (TTFT/TPS) and #4182 (memory diagnostics) is a nice touch — it keeps scope honest.

On approach: the scope is large (3438 additions, 20 files) but mostly justified. The core service (~568 lines) is focused, tests (~812 lines) are thorough, and the 10 locale files are mechanical i18n. The one area worth discussing is the export path validation — there's a lot of TOCTOU-race-mitigation code (10-retry atomic writes, symlink checks at every step, ~200+ lines of validation functions) for what is ultimately a local CLI export. It's security-conscious and well-written, but I'd ask: could this be simplified to a single realpath check + write, without the retry loop, for most real-world scenarios? Not a blocker, just a question.

Moving on to code review. 🔍

中文说明

感谢贡献!

模板完整 ✓

方向:非常契合。Token 使用量可见性是明确的用户需求(#4479),/stats 命令是合适的入口。Claude Code 的 CHANGELOG 中也有类似的 token/成本追踪功能(VSCode 中的 token 使用量对话框、每次交互的 token 追踪),说明这个方向是经过验证的。协调文档清楚地划定了与 #4252(TTFT/TPS)和 #4182(memory diagnostics)的边界,防止范围蔓延。

方案:代码量较大(3438 行新增,20 个文件),但基本合理。核心服务约 568 行且聚焦,测试约 812 行且覆盖全面,10 个语言文件是机械性 i18n 翻译。值得讨论的是导出路径验证——有大量 TOCTOU 竞态缓解代码(10 次重试的原子写入、每步的符号链接检查、200+ 行的验证函数),对于一个本地 CLI 导出功能来说可能偏重。安全意识和代码质量都很好,但想问一句:对于大多数真实场景,能否简化为一次 realpath 检查 + 写入,省掉重试循环?不是阻塞项,只是个问题。

进入代码审查 🔍

Qwen Code · qwen3.7-max

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Code Review

Reviewed the full diff across all 20 files. No critical blockers found.

tokenUsageService.ts — Clean, well-structured service. Privacy-preserving by design (no prompt text, response text, project paths, or IDs stored). Schema versioning for forward compatibility. Best-effort write with rate-limited error logging is the right call. CSV formula injection escaping is a nice security touch. Date validation covers leap years correctly.

statsCommand.ts — The export path validation is thorough (TOCTOU race mitigation with atomic temp-file + rename, symlink checks at every step, Windows ADS rejection). As noted in Stage 1, it's more defensive than most local CLI tools need, but it's correct and well-tested. The argument tokenizer handles quoted strings properly.

loggers.ts — Integration is minimal and correct: gated on getUsageStatisticsEnabled() + not-internal-prompt-id, using the best-effort wrapper so failures never disrupt the main flow.

jsonl-utils.ts — Small, focused addition: throwOnNonEnoentError option so the token usage query can surface real read errors (EACCES, etc.) instead of silently returning empty.

Unit tests — All pass:

  • tokenUsageService.test.ts: 18/18 ✓
  • loggers.test.ts: 56/56 ✓
  • statsCommand.test.ts: 36/36 ✓
  • jsonl-utils.test.ts: 28/28 ✓

Build passes (0 errors, 15 warnings — all in the VSCode extension, unrelated).

Real-Scenario Testing

Before (installed build — main branch)

$ qwen -p '/stats daily' 2>&1
Session duration: 0s
Prompts: 1
API requests: 0
Tokens — prompt: 0, output: 0
Tool calls: 0 (0 ok, 0 fail)
Files: +0 / -0 lines

The installed build doesn't recognize daily as a subcommand — it falls through to the default session stats view. No /stats daily, /stats monthly, or /stats export exists on main.

After (this PR — npm run dev)

$ npm run dev -- -p '/stats daily' 2>&1

Daily token usage for 2026-06-18
Total: 0 tokens
Requests: 0

Breakdown:
  Input: 0
  Output: 0
  Cached (included in Input): 0
  Thoughts: 0

By model:
  No usage data.

By auth type:
  No usage data.

By model/auth type:
  No usage data.

By source:
  No usage data.

Note: generation timing (TTFT/TPS) belongs to generation metrics.
$ npm run dev -- -p '/stats monthly' 2>&1

Monthly token usage for 2026-06
Total: 0 tokens
Requests: 0

Breakdown:
  Input: 0
  Output: 0
  Cached (included in Input): 0
  Thoughts: 0

By model:
  No usage data.

By auth type:
  No usage data.

By model/auth type:
  No usage data.

By source:
  No usage data.

Note: generation timing (TTFT/TPS) belongs to generation metrics.
$ npm run dev -- -p '/stats export daily --format json' 2>&1

Token usage exported to JSON: qwen-token-usage-day-2026-06-18.json

$ cat qwen-token-usage-day-2026-06-18.json
{
  "period": "day",
  "value": "2026-06-18",
  "generatedAt": "2026-06-18T23:06:24.047Z",
  "totals": {
    "requests": 0,
    "inputTokens": 0,
    "outputTokens": 0,
    "cachedTokens": 0,
    "thoughtsTokens": 0,
    "totalTokens": 0,
    "apiDurationMs": 0
  },
  "byModel": [],
  "byAuthType": [],
  "byModelAndAuthType": [],
  "bySource": []
}
$ npm run dev -- -p '/stats export monthly --format csv --output usage/month.csv' 2>&1

Token usage exported to CSV: usage/month.csv

$ cat usage/month.csv
period,value,group_type,group_key,model,auth_type,source,requests,input_tokens,output_tokens,cached_tokens,thoughts_tokens,total_tokens,api_duration_ms
month,2026-06,total,total,,,,0,0,0,0,0,0,0
$ npm run dev -- -p '/stats export daily --format json --output ../../../etc/evil.json' 2>&1

Token usage export path must be within the project working directory.

Summary: All new subcommands (daily, monthly, export) work as documented. JSON and CSV exports produce clean, privacy-preserving aggregate data. Path traversal is correctly rejected. The existing /stats (default) and /stats model and /stats tools subcommands are unaffected.

Note: since there are no API calls in this test environment, all outputs show zero tokens. The data recording path (hooked into logApiResponse) is covered by the 18 passing unit tests which simulate full record → query → export round-trips.

Qwen Code · qwen3.7-max

@qwen-code-ci-bot

Copy link
Copy Markdown
Collaborator

Reflection

Stepping back: this PR does what it says on the tin, and does it well.

The core tokenUsageService is clean and focused — ~450 lines of straightforward data pipeline code with a clear privacy contract (no prompt text, no response text, no project paths). Schema versioning and forward-compatibility filtering mean we can evolve the format without breaking existing data. The best-effort write wrapper with rate-limited error logging is the right pattern — recording failures should never disrupt the user's session.

The CLI side adds three subcommands (daily, monthly, export) that slot naturally into the existing /stats hierarchy. The display formatting is clean and i18n-complete. The export path validation is more thorough than most CLI tools bother with — I questioned the complexity in Stage 1, but after seeing it all work correctly (including rejecting path traversal), I think the author made the right call erring on the side of safety for file writes.

The test coverage is genuinely impressive: 138 tests across 4 test files, covering happy paths, edge cases (malformed JSONL, future schema versions, CSV formula injection, symlink rejection, TOCTOU races), and the integration point in logApiResponse. The coordination doc explaining boundaries with adjacent issues (#4252, #4182) shows good scope discipline.

Real-scenario testing confirmed all new subcommands work, exports produce clean data, path traversal is rejected, and existing /stats behavior is unaffected.

The one area where the PR is heavier than strictly necessary is the export validation code — a simpler realpath + write would work for 99% of cases. But the thorough version is correct, tested, and doesn't add runtime overhead, so it's a judgment call I'm happy to accept.

Verdict: this is a well-executed feature PR that solves a real user need. Ship it.

中文说明

总结

这个 PR 说到做到,而且做得很好。

核心 tokenUsageService 干净且聚焦——约 450 行直截了当的数据管道代码,有清晰的隐私契约(不含 prompt 文本、response 文本、项目路径)。Schema 版本控制和前向兼容过滤意味着我们可以在不破坏现有数据的情况下演进格式。Best-effort 写入包装器配合速率限制的错误日志是正确的模式——记录失败不应打断用户的会话。

CLI 侧新增三个子命令(dailymonthlyexport),自然地融入现有 /stats 层级。显示格式清晰且 i18n 翻译完整。导出路径验证比大多数 CLI 工具都更彻底——我在 Stage 1 质疑了复杂度,但看到它全部正常工作(包括拒绝路径穿越)后,我认为作者在文件写入方面偏向安全的选择是正确的。

测试覆盖率非常好:4 个测试文件中 138 个测试,覆盖了正常路径、边界情况(格式错误的 JSONL、未来 schema 版本、CSV 公式注入、符号链接拒绝、TOCTOU 竞态)以及 logApiResponse 中的集成点。协调文档解释了与相邻问题(#4252#4182)的边界,显示了良好的范围把控。

真实场景测试确认了所有新子命令正常工作、导出产生干净数据、路径穿越被拒绝、现有 /stats 行为不受影响。

唯一比必要更重的地方是导出验证代码——简单的 realpath + 写入在 99% 的场景下就够了。但彻底版本是正确的、经过测试的、且不增加运行时开销,所以这是一个我乐于接受的选择。

结论:这是一个执行良好的功能 PR,解决了真实的用户需求。可以合并。

Qwen Code · qwen3.7-max

@qwen-code-ci-bot qwen-code-ci-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, looks ready to ship. ✅

@wenshao
wenshao merged commit 8d2fe0a into QwenLM:main Jun 18, 2026
31 of 32 checks passed
@shenyankm
shenyankm deleted the feat/issue-4479-token-usage-stats branch June 22, 2026 11:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

需要一个功能统计Qwen Code每日消耗的Token数量

6 participants