Rust core only - #64
Merged
Merged
Conversation
…ng CLI - Add fetchWithRetry utility with configurable retries (TOKSCALE_FETCH_RETRIES) - Apply same retry logic to both LiteLLM and OpenRouter fetches - Retry-After header: integer seconds only (no HTTP-date) - Add robust cache validation: reject arrays, empty data, corrupt JSON - Delete corrupt/empty cache files on load to prevent poisoning - Add parsePrice helper: reject NaN, Infinity, negative values - Cache sortedPricingKeys for faster lookups - Strip provider prefixes in getOpenRouterPricing for correct mapping - Add openrouter/ prefix to all prefix lists for consistency - Only persist cache when data is non-empty - Add glm-4.7-free mapping (same pricing as glm-4.7) - Add 'tokscale pricing <model-id>' command with --provider and --json options
- Split pricing.ts into pricing/ directory structure
- Create LiteLLMProvider and OpenRouterProvider classes
- Unified cache filename pattern: pricing-{provider}.json
- Move mappings to respective provider files
- Extract shared utils (fetchWithRetry, normalizeModelName, etc.)
- Add new pricing/ module with async HTTP fetching (reqwest + tokio) - Implement LiteLLM pricing fetcher with 1-hour disk cache - Implement OpenRouter fallback for models not in LiteLLM - Add model alias resolution (e.g., 'big-pickle' -> 'glm-4.7') - Add unified PricingLookup with fuzzy matching and provider prefixes - Add lookup_pricing NAPI export for CLI pricing command - Rename old pricing.rs to pricing_legacy.rs for backward compatibility New dependencies: reqwest, tokio, dirs, futures, once_cell All 52 existing tests pass. Build verified in release mode.
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
- Delete TypeScript pricing module (use Rust lookupPricing instead) - Delete TypeScript session parsers (Rust handles all parsing) - Keep sessions/types.ts for shared type definitions - Update consumers to import from @tokscale/core - Add pricing-stub.ts for compatibility during transition - ~2900 lines of TypeScript removed BREAKING CHANGE: Native module is now required. Run `bun run build:core` before using CLI.
- Add lookupPricing to nativeBinding destructure in index.js - Add module.exports.lookupPricing export - Update handlePricingCommand to use core.lookupPricing directly - Enables 'tokscale pricing <model>' command to work with Rust backend Tested: pricing commands for claude-3-5-sonnet-20241022 and big-pickle work correctly
- pricing-stub.ts: Add fetchPricingForModels() using Rust lookupPricing - TUI: Fetch pricing after parsing to get correct costs (not $0) - submit.ts: Remove misleading 'Using TypeScript fallback' message - cli.ts: Standardize core-loading pattern with mod.default ?? mod - cli.ts: Improve error handling (distinguish module/model/network errors) - graph.ts: Include original error cause in native module error message Fixes issues found in Oracle review iteration 1.
- pricing-stub.ts: Add ESM/CJS interop (mod.default ?? mod) - pricing-stub.ts: Capture errors instead of swallowing (add pricingError field) - pricing-stub.ts: Fix provider filtering to check result.source === provider - cli.ts: Mark --provider flag as deprecated with runtime warning - README.md: Update to reflect native module is required (remove fallback claims) Fixes issues found in Oracle review iteration 2.
- cli.ts: Remove logNativeStatus() and all fallback messages - README.md: Fix TOC link to match section header - README.ja.md: Update to reflect native module is required - README.ko.md: Update to reflect native module is required - README.zh-cn.md: Update to reflect native module is required - pricing-stub.ts: Clear pricingError at start of fetchPricingForModels() All documentation now consistently states native module is required. Fixes issues found in Oracle review iteration 3.
- Remove outdated Architecture section from all READMEs - Fix cache file paths (pricing-litellm.json, pricing-openrouter.json) - Add --no-spinner option to models, monthly, graph, pricing commands - Keeps stdout clean for AI agents and scripts
TUI now shows actual progress through loading phases: - Syncing Cursor data... (if Cursor enabled) - Parsing session files... (local sources) - Loading pricing data... - Finalizing report... Previously only showed 'Loading pricing data...' the entire time.
CLI spinner now shows granular phases: - Syncing Cursor data... (if cursor enabled) - Parsing session files... (local sources) - Loading pricing data... - Finalizing report... / Generating graph data... Consistent with TUI loading phases.
- Add scripts/cli.sh with perl-based millisecond timing (macOS compatible) - Update package.json cli script to use the wrapper - Running `bun cli` now shows execution time at the end
…tion The ring crate used by rustls has cross-compilation issues on ARM Linux targets. Switching to native-tls uses the system's OpenSSL which is available on CI runners and works with the napi-rs cross-compilation toolchain.
The native-tls feature requires system OpenSSL which isn't available when cross-compiling for aarch64-linux-gnu. Using native-tls-vendored compiles OpenSSL from source using the cross toolchain.
Cursor sync and local parsing run in parallel, so showing a separate 'Syncing Cursor data...' phase was misleading. Now shows unified 'Scanning session data...' for the initial loading phase.
junhoyeo
marked this pull request as ready for review
December 29, 2025 08:32
Use temp file + rename pattern instead of direct fs::write() to prevent: - Partial writes on process crash/kill - Corrupted JSON from concurrent CLI invocations - Race conditions between multiple tokscale processes The atomic rename is guaranteed by POSIX for same-filesystem operations.
Use saturating_sub and explicit future-check to handle clock skew scenarios: - NTP adjustments moving clock backward - VM/container clock drift - Malicious cache file modification Previously: now - cached.timestamp would underflow to u64::MAX when timestamp > now, causing cache to appear stale forever and forcing constant network refetches.
Replace per-call PricingService::fetch() with process-wide singleton using tokio::sync::OnceCell. This eliminates redundant cache reads and HTTP fetches when looking up pricing for multiple models: Before: 50 models → 50 cache reads + 50 JSON parses + potential 50 HTTP calls After: 50 models → 1 cache read + 1 JSON parse + at most 1 HTTP call The singleton is lazily initialized on first lookup and reused for all subsequent calls within the same process.
Add missing droid configuration to match validation schema: - SOURCE_DISPLAY_NAMES: 'Droid' - SOURCE_COLORS: '#1F1D1C' (dark charcoal) - SOURCE_TEXT_COLORS: '#FFFFFF' (white for contrast on dark background) This ensures UI components display proper labels and colors for droid source instead of falling back to raw 'droid' string with default styling.
Add exponential backoff retry (3 attempts, 200ms/400ms/800ms) for: - Network errors (DNS, timeout, connection refused) - Server errors (5xx) - Rate limiting (429) Add strict pricing validation: - Reject NaN and Infinity values (previously passed through) - Reject negative prices - Trim whitespace before parsing - Validate cache pricing fields too Add diagnostic logging for debugging: - Log HTTP status codes for non-success responses - Log JSON parse failures with context - Log missing provider endpoints - Log invalid pricing values with details - Log retry exhaustion with final error
Add exponential backoff retry (3 attempts, 200ms/400ms/800ms) for: - Network errors (DNS, timeout, connection refused) - Server errors (5xx) - Rate limiting (429) Add connect_timeout (10s) separate from total timeout (30s). Add diagnostic logging for debugging network issues. Consistent retry behavior with OpenRouter fetch.
… reuse Cache atomicity improvements: - Use PID-based unique temp filename to prevent concurrent writer conflicts - Clean up temp file on write failure - Hidden temp file (dot prefix) to avoid clutter HTTP retry improvements: - Consume response body before retry sleep to enable connection reuse - Prevents connection pool exhaustion under load
- Delete pricing_legacy.rs (~360 lines) - Remove PricingEntry and pricing parameter from all report APIs - All report functions now async and fetch pricing via PricingService::get_or_init() - Add NaN/Inf validation in PricingLookup::calculate_cost - Use nanosecond timestamp for atomic cache writes (thread-safe) - Remove unused fetch_missing, PricingService::fetch - Simplify CLI: remove PricingFetcher, toPricingEntries usage - Simplify pricing-stub.ts to just type exports BREAKING CHANGE: Report APIs no longer accept pricing parameter; pricing fetched internally by Rust
…it tests - Add tier suffix stripping for -low, -high, -medium, -free variants - Fix source display bug (case-sensitive LiteLLM comparison) - Restore --provider flag to force litellm or openrouter source - Rewrite lookup strategy with family-aware fuzzy matching - Add blocklist for generic terms (auto, mini, chat, base) - Add 18 unit tests with inline mock data for lookup logic - OpenRouter fetcher now retrieves all models from /api/v1/models
- Add date pattern preservation to normalize_version_separator() to avoid mangling model IDs like claude-3-5-sonnet-20241022 (preserves date suffix) - Add comprehensive tests for OpenCode Zen models (glm-4.7, big-pickle, etc.) - Remove unused pricing-stub.ts (types now provided by @tokscale/core) - Fix misleading comment in native.ts about direct vs subprocess calls - Fix type assertion for dynamic import in native.ts Tests: 96 passed (was 95, +1 for date pattern test)
OpenCode Zen uses -xhigh, -high, -low suffixes for quality tiers. These now correctly resolve to base model pricing: - gpt-5.2-xhigh → gpt-5.2 - gpt-5.1-codex-max-xhigh → gpt-5.1-codex-max Tests: 98 passed (+2 for xhigh)
When multiple models match (e.g., grok-code), prefer the original model creator over cloud resellers: Original providers (preferred): - x-ai, xai, anthropic, openai, google, meta-llama, mistralai, deepseek, z-ai, qwen, cohere, perplexity, moonshotai Resellers (deprioritized): - azure, azure_ai, bedrock, vertex_ai, together, fireworks_ai, groq Example: grok-code now matches xai/grok-code-fast-1-0825 ($0.20/$1.50) instead of azure_ai/grok-code-fast-1 ($3.50/$17.50) Tests: 102 passed (+4 provider preference tests)
…er docs Documents the fix for preferring original providers over resellers: - Before: grok-code → azure_ai/grok-code-fast-1 ($3.50/$17.50) ❌ - After: grok-code → xai/grok-code-fast-1-0825 ($0.20/$1.50) ✅ Tests: 103 passed
- Document tokscale pricing command with examples - Explain 6-step lookup strategy (exact, alias, tier suffix, version, prefix, fuzzy) - Document provider preference (original providers over resellers) - Add example showing grok-code matching xai/ instead of azure_ai/
- Add lookup cache (RwLock<HashMap>) for memoization of repeated lookups - Pre-compute lowercase key maps for O(1) exact matching - Pre-compute OpenRouter model part index for direct model name lookups - Replace O(n) linear scans with O(1) HashMap lookups Estimated speedup: ~1500x for typical sessions (100K messages, 10 unique models)
This was referenced Dec 29, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tokscale pricingto fetch model pricing (and to debug how it's resolved) #63