fix(pricing): require source-backed model prices

README.md

@@ -151,7 +151,7 @@ In the age of AI-assisted development, **tokens are the new energy**. They power
   - Real-time filtering and sorting
   - Zero flicker rendering
 - **Multi-platform support** - Track usage across OpenCode, Claude Code, Codex CLI, Copilot CLI, Cursor IDE, Gemini CLI, Amp, Codebuff, Command Code, Droid, OpenClaw, Hermes Agent, Pi, OMP, Kimi, Qwen CLI, Roo Code, Kilo, Mux, Kilo CLI, Goose, Antigravity, Zed, Kiro, Trae, and Cline
-- **Real-time pricing** - Fetches current pricing from LiteLLM with 1-hour disk cache; automatic OpenRouter fallback and Cursor model pricing for newly released models
+- **Real-time pricing** - Fetches current pricing from LiteLLM, OpenRouter, and models.dev with 1-hour disk cache
 - **Detailed breakdowns** - Input, output, cache read/write, and reasoning token tracking
 - **Native Rust core** - All parsing and aggregation done in Rust for 10x faster processing
 - **Web visualization** - Interactive contribution graph with 2D and 3D views
@@ -407,11 +407,9 @@ The pricing lookup uses a multi-step resolution strategy:
 
 1. **Custom Pricing Overrides** - Exact user-defined entries from `~/.config/tokscale/custom-pricing.json`
 2. **Exact Match** - Direct lookup in LiteLLM/OpenRouter databases
-3. **Alias Resolution** - Resolves friendly names (e.g., `big-pickle` → `glm-4.7`)
-4. **Version Normalization** - Handles version formats (`claude-3-5-sonnet` ↔ `claude-3.5-sonnet`)
-5. **Provider Prefix Matching** - Tries common prefixes (`anthropic/`, `openai/`, etc.)
-6. **Cursor Model Pricing** - Hardcoded pricing for models not yet in LiteLLM/OpenRouter (e.g., `gpt-5.3-codex`)
-7. **Fuzzy Matching** - Word-boundary matching for partial model names
+3. **Version Normalization** - Handles deterministic catalog version formats (`claude-3-5-sonnet` ↔ `claude-3.5-sonnet`)
+4. **Provider Prefix Matching** - Tries catalog provider prefixes (`anthropic/`, `openai/`, etc.)
+5. **Fuzzy Matching** - Word-boundary matching for partial model names
 
 Standalone pricing lookup does not infer arbitrary route prefixes, source prefixes, or reasoning-tier suffixes. Source-specific model decoding belongs in the parser that produced the usage row; `tokscale pricing <model>` is a catalog query.
 
@@ -445,7 +443,7 @@ Create `custom-pricing.json` in Tokscale's config directory (`~/.config/tokscale
 
 Override prices are entered in dollars per million tokens, matching how most API providers publish pricing; Tokscale converts them to per-token rates internally. At least one of `input_cost_per_million_tokens` or `output_cost_per_million_tokens` must be present and positive, and cache-read/cache-creation fields are optional. LiteLLM-style per-token field names such as `input_cost_per_token`, `output_cost_per_token`, and `cache_read_input_token_cost` are also accepted for copy/paste compatibility, but the per-million names are the recommended user-facing form. To omit a tier or cache price, leave the field out; negative or non-finite values are treated as invalid and the whole model entry is skipped so typos do not silently alter accounting. Optional `source` and `notes` fields are ignored by Tokscale and can be used for your own bookkeeping.
 
-Overrides are exact-only and case-insensitive. Tokscale checks the full model ID exactly as written, then falls through to LiteLLM, OpenRouter, Cursor pricing, and fuzzy matching if no override matches. Gateway paths such as `accounts/fireworks/models/kimi-k2p6` or `accounts/fireworks/routers/kimi-k2p6-turbo` must be written as full keys when you want to price that exact route. Overrides are loaded once at startup; restart the command after editing the file. This is the recommended local fix for wrong-model pricing bugs while waiting on upstream LiteLLM pricing updates.
+Overrides are exact-only and case-insensitive. For local reports, exact overrides match the canonical model ID emitted by the source parser; standalone `tokscale pricing <model>` queries match the command argument literally. Tokscale then falls through to LiteLLM, OpenRouter, models.dev, and direct catalog matching if no override matches. Gateway paths such as `accounts/fireworks/models/kimi-k2p6` or `accounts/fireworks/routers/kimi-k2p6-turbo` must be written as full keys when you want to price that exact route. Overrides are loaded once at startup; restart the command after editing the file. This is the recommended local fix for wrong-model pricing bugs while waiting on upstream pricing updates.
 
 **Provider Preference:**
 
@@ -1556,15 +1554,14 @@ Codebuff (formerly Manicode) writes per-chat JSON files. Tokscale parses token u
 
 ## Pricing
 
-Tokscale fetches real-time pricing from [LiteLLM's pricing database](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json).
+Tokscale fetches real-time pricing from [LiteLLM's pricing database](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json), OpenRouter, and models.dev.
 
-**Dynamic Fallback**: For models not yet available in LiteLLM (e.g., recently released models), Tokscale automatically fetches pricing from [OpenRouter's endpoints API](https://openrouter.ai/docs/api/api-reference/endpoints/list-endpoints). This ensures you get accurate pricing from the model's author provider (e.g., Z.AI for glm-4.7) without waiting for LiteLLM updates.
-
-**Cursor Model Pricing**: For very recently released models not yet in either LiteLLM or OpenRouter (e.g., `gpt-5.3-codex`), Tokscale includes hardcoded pricing sourced from [Cursor's model docs](https://cursor.com/en-US/docs/models). These overrides are checked after all upstream sources but before fuzzy matching, so they automatically yield once real upstream pricing becomes available.
+If a model does not match custom pricing or an upstream pricing source, Tokscale leaves its derived cost at `$0.00` rather than using built-in prices.
 
 **Caching**: Pricing data is cached to disk with 1-hour TTL for fast startup:
 - LiteLLM cache: `~/.config/tokscale/cache/pricing-litellm.json`
 - OpenRouter cache: `~/.config/tokscale/cache/pricing-openrouter.json` (caches author pricing for models from supported providers)
+- models.dev cache: `~/.config/tokscale/cache/pricing-models-dev.json`
 
 Pricing includes:
 - Input tokens

README.zh-cn.md

@@ -376,13 +376,11 @@ tokscale pricing "claude-3-5-sonnet" --provider litellm
 
 1. **自定义价格覆盖** - `~/.config/tokscale/custom-pricing.json` 中大小写不敏感的完整 key 精确匹配
 2. **精确匹配** - 在 LiteLLM/OpenRouter 数据库中直接查找
-3. **别名解析** - 解析可信显式别名（例如：`big-pickle` → `glm-4.7`）
-4. **版本标准化** - 处理版本格式（`claude-3-5-sonnet` ↔ `claude-3.5-sonnet`）
-5. **提供商前缀匹配** - 尝试常见前缀（`anthropic/`、`openai/` 等）
-6. **Cursor 模型定价** - LiteLLM/OpenRouter 中尚未收录的模型的硬编码定价（例如：`gpt-5.3-codex`）
-7. **模糊匹配** - 部分模型名称的词边界匹配
+3. **版本标准化** - 处理确定性的目录版本格式（`claude-3-5-sonnet` ↔ `claude-3.5-sonnet`）
+4. **提供商前缀匹配** - 尝试目录中的提供商前缀（`anthropic/`、`openai/` 等）
+5. **模糊匹配** - 部分模型名称的词边界匹配
 
-独立价格查询不会推断任意 route 前缀、source 前缀或 reasoning-tier 后缀。来源特定的模型解码应在对应 parser 中完成；`tokscale pricing <model>` 是目录查询，不是原始 route 清洗工具。自定义价格覆盖也只匹配完整 key；如果要为 `accounts/fireworks/models/...` 这类 gateway 路径定价，需要在 custom pricing 文件中写入完整路径。
+独立价格查询不会推断任意 route 前缀、source 前缀或 reasoning-tier 后缀。来源特定的模型解码应在对应 parser 中完成；`tokscale pricing <model>` 是目录查询，不是原始 route 清洗工具。对本地报告而言，自定义价格覆盖匹配的是 parser 输出的 canonical model ID；独立价格查询则按命令参数本身匹配。如果要为 `accounts/fireworks/models/...` 这类 gateway 路径定价，需要在 custom pricing 文件中写入完整路径。
 
 **提供商优先级：**
 
@@ -1324,15 +1322,14 @@ Codebuff（前身 Manicode）按聊天写入 JSON 文件。Tokscale 从 `metadat
 
 ## 定价
 
-Tokscale 从 [LiteLLM 的价格数据库](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)获取实时价格。
+Tokscale 从 [LiteLLM 的价格数据库](https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json)、OpenRouter 和 models.dev 获取实时价格。
 
-**动态回退**：对于 LiteLLM 中尚未收录的模型（例如最近发布的模型），Tokscale 会自动从 [OpenRouter 的端点 API](https://openrouter.ai/docs/api/api-reference/endpoints/list-endpoints) 获取定价。
-
-**Cursor 模型定价**：对于 LiteLLM 和 OpenRouter 中都尚未收录的最新模型（例如 `gpt-5.3-codex`），Tokscale 使用从 [Cursor 模型文档](https://cursor.com/en-US/docs/models)获取的硬编码定价。这些覆盖在所有上游来源之后、模糊匹配之前检查，因此当真正的上游定价可用时会自动让步。
+如果模型没有命中自定义价格或上游价格来源，Tokscale 会保留 `$0.00` 派生成本，不再使用内置价格。
 
 **缓存**：价格数据以 1 小时 TTL 缓存到磁盘，确保快速启动：
 - LiteLLM 缓存：`~/.config/tokscale/cache/pricing-litellm.json`
 - OpenRouter 缓存：`~/.config/tokscale/cache/pricing-openrouter.json`（缓存支持提供商的模型作者定价信息）
+- models.dev 缓存：`~/.config/tokscale/cache/pricing-models-dev.json`
 
 定价包括：
 - 输入 Token

crates/tokscale-cli/src/tui/cache.rs

@@ -22,7 +22,7 @@ use super::data::{
 
 /// Cache staleness threshold: 5 minutes (matches TS implementation)
 const CACHE_STALE_THRESHOLD_MS: u64 = 5 * 60 * 1000;
-const CACHE_SCHEMA_VERSION: u32 = 22;
+const CACHE_SCHEMA_VERSION: u32 = 24;
 
 #[derive(Debug, Clone, Default, PartialEq, Eq, Serialize, Deserialize)]
 #[serde(rename_all = "camelCase")]
@@ -970,7 +970,7 @@ mod tests {
         fs::write(
             &cache_path,
             r#"{
-  "schemaVersion": 22,
+  "schemaVersion": 24,
   "timestamp": 9999999999999,
   "enabledClients": ["claude"],
   "groupBy": "model",
@@ -1220,7 +1220,7 @@ mod tests {
         fs::create_dir_all(cache_path.parent().unwrap()).unwrap();
         let mut cached: serde_json::Value = serde_json::from_str(
             r#"{
-  "schemaVersion": 22,
+  "schemaVersion": 24,
   "timestamp": 0,
   "enabledClients": ["claude", "cursor"],
   "groupBy": "model",

crates/tokscale-core/src/lib.rs

@@ -3104,7 +3104,7 @@ mod tests {
     }
 
     #[test]
-    fn test_cursor_parse_path_reprices_zero_cost_composer_1_5_rows() {
+    fn test_cursor_parse_path_keeps_zero_cost_for_unpriced_composer_rows() {
         let temp_dir = tempfile::TempDir::new().unwrap();
         let cursor_cache_dir = temp_dir.path().join(".config/tokscale/cursor-cache");
         std::fs::create_dir_all(&cursor_cache_dir).unwrap();
@@ -3124,7 +3124,11 @@ mod tests {
         assert_eq!(messages.len(), 1);
         assert_eq!(messages[0].client.as_ref(), "cursor");
         assert_eq!(messages[0].model_id.as_ref(), "Composer 1.5");
-        assert!(messages[0].cost > 0.0);
+        assert_eq!(messages[0].tokens.input, 1000);
+        assert_eq!(messages[0].tokens.output, 2000);
+        assert_eq!(messages[0].tokens.cache_read, 5000);
+        assert_eq!(messages[0].tokens.cache_write, 200);
+        assert_eq!(messages[0].cost, 0.0);
     }
 
     fn write_kimi_code_usage_fixture(source_home: &std::path::Path) {
@@ -5624,7 +5628,17 @@ model = "gpt-5.5"
 
     #[test]
     fn test_apply_token_pricing_prices_claude_code_gpt_5_3_codex() {
-        let pricing = pricing::PricingService::new(HashMap::new(), HashMap::new());
+        let mut litellm = HashMap::new();
+        litellm.insert(
+            "gpt-5.3-codex".into(),
+            pricing::ModelPricing {
+                input_cost_per_token: Some(0.00000175),
+                output_cost_per_token: Some(0.000014),
+                cache_read_input_token_cost: Some(0.000000175),
+                ..Default::default()
+            },
+        );
+        let pricing = pricing::PricingService::new(litellm, HashMap::new());
 
         let mut msg = UnifiedMessage::new(
             "claude",

crates/tokscale-core/src/message_cache.rs

@@ -35,7 +35,7 @@ use std::time::UNIX_EPOCH;
 // namespace, and nonzero usage with unknown display labels is preserved.
 // 31: OpenCode/Qwen and shared source model canonicalization moved raw
 // decorated model ids out of report grouping and pricing cleanup.
-const CACHE_SCHEMA_VERSION: u32 = 31;
+const CACHE_SCHEMA_VERSION: u32 = 32;
 const CACHE_FILENAME: &str = "source-message-cache.bin";
 const CACHE_LOCK_FILENAME: &str = "source-message-cache.lock";
 const SHARDS_DIRNAME: &str = "shards";

crates/tokscale-core/src/model_aliases.rs

@@ -2,6 +2,8 @@ pub(crate) const DEEPSEEK_V4_PRO_BETA_ALIAS: &str = "model1";
 pub(crate) const DEEPSEEK_V4_FLASH_BETA_ALIAS: &str = "model2";
 
 const CLAUDE_FAMILIES: &[&str] = &["opus", "sonnet", "haiku", "fable"];
+const OPENAI_REASONING_TIERS: &[&str] =
+    &["minimal", "low", "medium", "high", "xhigh", "auto", "none"];
 
 pub(crate) fn is_deepseek_v4_beta_alias(model: &str) -> bool {
     let lower = model.trim().to_lowercase();
@@ -24,7 +26,7 @@ pub(crate) fn canonicalize_source_model_id(model: &str) -> Option<String> {
     }
 
     canonicalize_modern_claude_source_model(&lower)
-        .or_else(|| canonicalize_openai_source_model(&lower).map(str::to_string))
+        .or_else(|| canonicalize_openai_source_model(&lower))
         .or_else(|| canonicalize_glm_source_model(&lower).map(str::to_string))
         .or_else(|| canonicalize_qwen_source_model(&lower))
         .or_else(|| canonicalize_kimi_source_model(&lower).map(str::to_string))
@@ -40,13 +42,71 @@ fn canonical_model_segment(model: &str) -> &str {
         .unwrap_or(model)
 }
 
-fn canonicalize_openai_source_model(model: &str) -> Option<&'static str> {
-    match canonical_model_segment(model) {
-        "gpt-5.5-fast" => Some("gpt-5.5"),
-        _ => None,
+fn canonicalize_openai_source_model(model: &str) -> Option<String> {
+    let model = canonical_model_segment(model);
+
+    if let Some(base) = strip_parenthesized_openai_reasoning_tier(model) {
+        return Some(base.to_string());
+    }
+
+    if let Some((base, tier)) = model.rsplit_once('-') {
+        if (tier == "fast" || OPENAI_REASONING_TIERS.contains(&tier))
+            && is_openai_gpt_source_base_model(base)
+        {
+            return Some(base.to_string());
+        }
+    }
+
+    None
+}
+
+fn strip_parenthesized_openai_reasoning_tier(model: &str) -> Option<&str> {
+    let (base, tier) = model.rsplit_once('(')?;
+    let tier = tier.strip_suffix(')')?;
+    let base =
+        base.trim_end_matches(|ch: char| ch.is_ascii_whitespace() || matches!(ch, '-' | '_'));
+    if OPENAI_REASONING_TIERS.contains(&tier) && is_openai_gpt_source_base_model(base) {
+        Some(base)
+    } else {
+        None
+    }
+}
+
+fn is_openai_gpt_source_base_model(model: &str) -> bool {
+    let rest = match model.strip_prefix("gpt-") {
+        Some(rest) => rest,
+        None => return false,
+    };
+
+    let (version, suffix) = match rest.split_once('-') {
+        Some((version, suffix)) => (version, Some(suffix)),
+        None => (rest, None),
+    };
+    if !is_openai_gpt_version(version) {
+        return false;
+    }
+
+    match suffix {
+        None => true,
+        Some("nano" | "mini" | "pro" | "codex" | "codex-max" | "codex-spark") => true,
+        Some(_) => false,
     }
 }
 
+fn is_openai_gpt_version(value: &str) -> bool {
+    if value == "5" {
+        return true;
+    }
+
+    matches!(
+        value.split_once('.'),
+        Some((major, minor))
+            if major == "5"
+                && !minor.is_empty()
+                && minor.bytes().all(|byte| byte.is_ascii_digit())
+    )
+}
+
 fn canonicalize_glm_source_model(model: &str) -> Option<&'static str> {
     let model = canonical_model_segment(model);
     if matches!(model, "glm-4.7-free" | "glm-4.7:free" | "glm-4.7 (free)") {
@@ -253,6 +313,21 @@ mod tests {
         let cases = [
             ("gpt-5.5-fast", "gpt-5.5"),
             ("openai/gpt-5.5-fast", "gpt-5.5"),
+            ("gpt-5.5(high)", "gpt-5.5"),
+            ("gpt-5.5 (high)", "gpt-5.5"),
+            ("gpt-5.5-(high)", "gpt-5.5"),
+            ("gpt-5.5_(high)", "gpt-5.5"),
+            ("openai/gpt-5.5(xhigh)", "gpt-5.5"),
+            ("openai/gpt-5.5 (xhigh)", "gpt-5.5"),
+            ("gpt-5.5-high", "gpt-5.5"),
+            ("gpt-5.5-xhigh", "gpt-5.5"),
+            ("gpt-5.4-mini-xhigh", "gpt-5.4-mini"),
+            ("gpt-5.4-mini(high)", "gpt-5.4-mini"),
+            ("gpt-5.4-nano-xhigh", "gpt-5.4-nano"),
+            ("gpt-5.4-pro(high)", "gpt-5.4-pro"),
+            ("gpt-5.3-codex-xhigh", "gpt-5.3-codex"),
+            ("gpt-5.3-codex-spark-high", "gpt-5.3-codex-spark"),
+            ("gpt-5.1-codex-max-xhigh", "gpt-5.1-codex-max"),
             ("glm-4.7-free", "glm-4.7"),
             ("glm-4.7:free-fast", "glm-4.7"),
             ("glm-4.7 (free)-medium", "glm-4.7"),
@@ -267,8 +342,10 @@ mod tests {
 
         assert_eq!(canonicalize_source_model_id("qwen3.7-max-2605"), None);
         assert_eq!(canonicalize_source_model_id("qwen3.7-max-05-20"), None);
+        assert_eq!(canonicalize_source_model_id("gpt-5.3-codex-spark"), None);
+        assert_eq!(canonicalize_source_model_id("gpt-4o-high"), None);
         assert_eq!(
-            canonicalize_source_model_id("gpt-5.1-codex-max-xhigh"),
+            canonicalize_source_model_id("gpt-5.3-codex-spark-lite"),
             None
         );
     }