This repository was archived by the owner on Jul 30, 2026. It is now read-only.
Add CodSpeed performance benchmarks - #71
Closed
codspeed-hq[bot] wants to merge 1 commit into
Closed
Conversation
Author
Congrats! CodSpeed is installed 🎉
You will start to see performance impacts in the reports once the benchmarks are run from your default branch.
|
Owner
|
Closing this CodSpeed wizard PR because PR #70 has already integrated the useful benchmark/profile/badge pieces while preserving the fork-specific workflow hardening and personal/local-clients triggers. The generated workflow here should not be merged. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR sets up continuous performance measurement with CodSpeed for the
tokscale-corecrate, which holds the CPU-bound hot paths of the project (session parsing and model aggregation).Changes
codspeed-criterion-compatas a dev-dependency oftokscale-core(imported under thecriterionname), so existing criterion-style benchmarks integrate directly with CodSpeed.crates/tokscale-core/benches/core_benchmarks.rscovering hot paths exercised on every scanned message:normalize_model_for_groupingover a representative batch of model identifiers (Anthropic/GPT canonicalization, custom prefixes, date suffixes, reasoning tiers, free markers).parse_jsonl_file) at 100 / 1000 / 5000 lines to capture scaling behavior.parse_json_file) over 1000 objects.[profile.bench]to the workspaceCargo.toml. Benchmark builds inherit fromrelease, which enables full LTO and a single codegen unit; on this dependency tree (bundled SQLite, vendored TLS, zstd) that made benchmark compilation prohibitively slow. The bench profile keepsopt-level = 3while disabling cross-crate LTO and allowing parallel codegen units. Release binaries are unaffected..github/workflows/codspeed.ymlrunning benchmarks in CPU simulation mode on pushes tomain, pull requests, and manual dispatch, authenticated via OpenID Connect (id-token: write).README.md.Notes on runners
The workflow uses CPU simulation mode on
ubuntu-latest, which is deterministic and hardware-agnostic, so standard GitHub-hosted runners are appropriate here.Verification
Benchmarks were built and run locally in simulation mode (
cargo codspeed build+codspeed run --mode simulation -- cargo codspeed run); all five benchmarks were measured and results uploaded successfully.Next steps
main.Summary by cubic
Adds CodSpeed performance benchmarks and CI to track
tokscale-corehot paths (model normalization and JSON/JSONL parsing). Bench builds are faster with a custom profile; release builds are unchanged.New Features
normalize_model_for_grouping,parse_jsonl_file(100/1000/5000 lines), andparse_json_file(1000 objects).cargo-codspeedon push, PR, and manual trigger in deterministic simulation mode.[profile.bench]to speed bench compilation (keepsopt-level = 3, disables LTO, setscodegen-units = 16).Dependencies
codspeed-criterion-compat(aliased ascriterion) and a bench target fortokscale-core.Written for commit 1bec0ee. Summary will update on new commits.