Skip to content
This repository was archived by the owner on Jul 30, 2026. It is now read-only.

Add CodSpeed performance benchmarks - #71

Closed
codspeed-hq[bot] wants to merge 1 commit into
personal/local-clientsfrom
codspeed/wizard-1781806378475
Closed

Add CodSpeed performance benchmarks#71
codspeed-hq[bot] wants to merge 1 commit into
personal/local-clientsfrom
codspeed/wizard-1781806378475

Conversation

@codspeed-hq

@codspeed-hq codspeed-hq Bot commented Jun 18, 2026

Copy link
Copy Markdown

Summary

This PR sets up continuous performance measurement with CodSpeed for the tokscale-core crate, which holds the CPU-bound hot paths of the project (session parsing and model aggregation).

Changes

  • Added codspeed-criterion-compat as a dev-dependency of tokscale-core (imported under the criterion name), so existing criterion-style benchmarks integrate directly with CodSpeed.
  • Added crates/tokscale-core/benches/core_benchmarks.rs covering hot paths exercised on every scanned message:
    • normalize_model_for_grouping over a representative batch of model identifiers (Anthropic/GPT canonicalization, custom prefixes, date suffixes, reasoning tiers, free markers).
    • SIMD-accelerated JSONL parsing (parse_jsonl_file) at 100 / 1000 / 5000 lines to capture scaling behavior.
    • JSON array parsing (parse_json_file) over 1000 objects.
  • Added a dedicated [profile.bench] to the workspace Cargo.toml. Benchmark builds inherit from release, which enables full LTO and a single codegen unit; on this dependency tree (bundled SQLite, vendored TLS, zstd) that made benchmark compilation prohibitively slow. The bench profile keeps opt-level = 3 while disabling cross-crate LTO and allowing parallel codegen units. Release binaries are unaffected.
  • Added .github/workflows/codspeed.yml running benchmarks in CPU simulation mode on pushes to main, pull requests, and manual dispatch, authenticated via OpenID Connect (id-token: write).
  • Added the CodSpeed badge to README.md.

Notes on runners

The workflow uses CPU simulation mode on ubuntu-latest, which is deterministic and hardware-agnostic, so standard GitHub-hosted runners are appropriate here.

Verification

Benchmarks were built and run locally in simulation mode (cargo codspeed build + codspeed run --mode simulation -- cargo codspeed run); all five benchmarks were measured and results uploaded successfully.

Next steps

  • Merge to establish the baseline on main.
  • Consider adding benchmarks for the aggregation/sessionize pipelines once representative fixtures are wired up.

Summary by cubic

Adds CodSpeed performance benchmarks and CI to track tokscale-core hot paths (model normalization and JSON/JSONL parsing). Bench builds are faster with a custom profile; release builds are unchanged.

  • New Features

    • Added CodSpeed-compatible benches for normalize_model_for_grouping, parse_jsonl_file (100/1000/5000 lines), and parse_json_file (1000 objects).
    • Added GitHub Action to build and run cargo-codspeed on push, PR, and manual trigger in deterministic simulation mode.
    • Added [profile.bench] to speed bench compilation (keeps opt-level = 3, disables LTO, sets codegen-units = 16).
    • Added CodSpeed badge to the README.
  • Dependencies

    • Added dev-dependency codspeed-criterion-compat (aliased as criterion) and a bench target for tokscale-core.

Written for commit 1bec0ee. Summary will update on new commits.

Review in cubic

@codspeed-hq

codspeed-hq Bot commented Jun 18, 2026

Copy link
Copy Markdown
Author

Congrats! CodSpeed is installed 🎉

🆕 5 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


Open in CodSpeed

@codspeed-hq
codspeed-hq Bot marked this pull request as ready for review June 18, 2026 19:01
@makoMakoGo

Copy link
Copy Markdown
Owner

Closing this CodSpeed wizard PR because PR #70 has already integrated the useful benchmark/profile/badge pieces while preserving the fork-specific workflow hardening and personal/local-clients triggers. The generated workflow here should not be merged.

@makoMakoGo makoMakoGo closed this Jun 18, 2026
@makoMakoGo
makoMakoGo deleted the codspeed/wizard-1781806378475 branch June 18, 2026 21:42
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants