Skip to content

refactor(metrics): Replace tiktoken with gpt-tokenizer - #1245

Closed
yamadashy wants to merge 8 commits into
mainfrom
refactor/replace-tiktoken-with-gpt-tokenizer
Closed

refactor(metrics): Replace tiktoken with gpt-tokenizer#1245
yamadashy wants to merge 8 commits into
mainfrom
refactor/replace-tiktoken-with-gpt-tokenizer

fix(metrics): Guard against TOCTOU race in async getTokenCounter

37f632b
Select commit
Loading
Failed to load commit list.
Codecov / codecov/project succeeded Mar 20, 2026 in 0s

87.24% (target 75.00%)

View this Pull Request on Codecov

87.24% (target 75.00%)

Details

Codecov Report

❌ Patch coverage is 94.44444% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 87.24%. Comparing base (ad7abc9) to head (37f632b).

Files with missing lines Patch % Lines
src/core/metrics/tokenCounterFactory.ts 87.50% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1245      +/-   ##
==========================================
+ Coverage   87.18%   87.24%   +0.05%     
==========================================
  Files         115      116       +1     
  Lines        4324     4328       +4     
  Branches     1002     1004       +2     
==========================================
+ Hits         3770     3776       +6     
+ Misses        554      552       -2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.