Fix regression from update to theseer/tokenizer 2.x

[staabm]

as discussed in #1125 (comment)

see also theseer/tokenizer#44

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.42%. Comparing base (c467c59) to head (50fea36).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files

@@            Coverage Diff            @@
##               main    #1129   +/-   ##
=========================================
  Coverage     88.42%   88.42%           
  Complexity     1390     1390           
=========================================
  Files           105      105           
  Lines          4700     4700           
=========================================
  Hits           4156     4156           
  Misses          544      544           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.