perf(lexer): inline handle_byte into read_next_token

[overlookmotel]

Preparatory step for #15513. That PR adds a 2nd callsite for handle_byte. Mark it as #[inline(always)] to make sure it gets inlined in both places.

This was originally part of #15513, but have split it out into a separate PR so that Codspeed's results on #15513 measure the actual substantive change in isolation - to check that change is having the effect I think it is, and that the gain wasn't actually coming from adding #[inline(always)] here. This PR has no effect on performance, so the gain is in #15513.

[overlookmotel]

• perf(lexer): skip single space in read_next_token #15513
• perf(lexer): inline handle_byte into read_next_token #15520 👈 (View in Graphite)
• ci(benchmarks/lexer): ensure next_token is inlined into lexer benchmark #15519
• ci(benchmarks/lexer): fix lexer benchmark #15517
• main

---

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• 0-merge - adds this PR to the back of the merge queue
• hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

CodSpeed Performance Report

Merging #15520 will not alter performance

_{Comparing 11-09-perf_lexer_inline_handle_byte_into_read_next_token_ (ff4461f) with 11-09-ci_benchmarks_lexer_ensure_next_token_is_inlined_into_lexer_benchmark (fade858)}

Summary

✅ 37 untouched

[graphite-app]

Merge activity

• Nov 9, 4:45 PM UTC: Boshen added this pull request to the Graphite merge queue.
• Nov 9, 4:53 PM UTC: Merged by the Graphite merge queue.