Skip to content

chore: refresh benchmarks for v2.2.3 - #1417

Merged
jdx merged 1 commit into
mainfrom
bench-refresh
Aug 31, 2026
Merged

jdx merged 1 commit into
mainfrom
bench-refresh

Conversation

@jdx

@jdx jdx commented Aug 31, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 2.2.0; the workspace is now 2.2.3. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Versions:

  • aube: 2.2.0 -> 2.2.3
  • deno: 2.9.5 -> 2.9.6
  • pnpm: 11.24.0 -> 11.25.0

Public ratios: warm installs vs Bun 2x -> 3x; warm installs vs pnpm 6x -> 9x; repeat test vs Bun 1.9x -> 2x; repeat test vs pnpm 21x -> 27x.

Benchmark aube bun deno pnpm npm yarn
Fresh install (warm cache) 246ms -> 214ms (-13%) 593ms -> 600ms (+1%) 657ms -> 616ms (-6%) 1495ms -> 1918ms (+28%) 3709ms -> 3695ms (0%) 2994ms -> 2906ms (-3%)
Fresh install (cold cache) 7189ms -> 7083ms (-1%) 1605ms -> 1523ms (-5%) 1703ms -> 1679ms (-1%) 8708ms -> 8397ms (-4%) 5951ms -> 5901ms (-1%) 6573ms -> 6877ms (+5%)
npm install && npm run test 17ms -> 13ms (-24%) 33ms -> 31ms (-6%) 105ms -> 140ms (+33%) 351ms -> 352ms (+0%) 647ms -> 655ms (+1%) 1247ms -> 1240ms (-1%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.


Generated by the bench-refresh workflow.


Note

Low Risk
Documentation and benchmark artifact updates only; no runtime or install logic changes.

Overview
Refreshes published benchmark numbers for aube 2.2.3 (from 2.2.0) by re-running the benchmark suite and updating benchmarks/results.json with a new timestamp, tool versions (deno 2.9.6, pnpm 11.25.0), and revised timings across warm/cold install and repeat install && test scenarios.

The README BENCH_RATIOS block is regenerated so marketing ratios match the new data: warm installs are now cited as ~9× vs pnpm and ~3× vs Bun (up from 6× and 2×), and repeat test commands up to 27× vs pnpm and 2× vs Bun (up from 21× and 1.9×). Aube’s own warm-install and repeat-test numbers improved modestly in this run; pnpm warm-install regressed in the benchmark matrix, which drives the higher relative ratios.

Reviewed by Cursor Bugbot for commit 66db4d7. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Documentation

    • Updated benchmark figures in the README to reflect improved reported performance for warm installs and repeat test commands.
  • Chores

    • Refreshed benchmark results, timestamps, tool versions, and measurements across warm-cache, cold-cache, and install-test scenarios.

@coderabbitai

coderabbitai Bot commented Aug 31, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 6a052f3a-7f99-4181-97c1-4923c5d3adfe

📥 Commits

Reviewing files that changed from the base of the PR and between 840845a and 66db4d7.

📒 Files selected for processing (2)
  • README.md
  • benchmarks/results.json

Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review.


📝 Walkthrough

Walkthrough

The benchmark results metadata, tool versions, and measurements were refreshed. The README now reflects the updated warm-install and repeat-test speedup figures.

Changes

Benchmark Refresh

Layer / File(s) Summary
Refresh benchmark data
benchmarks/results.json
The timestamp and selected tool versions were updated. Warm-cache, cold-cache, and install-test measurements and statistics were replaced.
Update README benchmark claims
README.md
The documented speedups were updated for warm installs and repeat test commands.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 66db4

This PR refreshes benchmark artifacts and README performance ratios without changing runtime or installation behavior; no actionable merge-blocking risk remains beyond normal checks and review.

Possibly related PRs

  • jdx/aube#907: Refreshes the same README benchmark claims and benchmark measurements.
  • jdx/aube#962: Updates the same benchmark data and README performance ratios.
  • jdx/aube#1080: Refreshes benchmark measurements, timestamps, tool versions, and README claims.

Suggested reviewers: mise-en-dev, greptile-apps

Poem

A rabbit checks the numbers bright

Warm and cold now match just right
Versions hop from old to new
README speedups follow through
Fresh benchmarks sparkle in the light

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: refreshing benchmark data for version 2.2.3.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (2 skipped: 2 unsupported.)


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Refreshes published benchmark data for aube 2.2.3 and updates the README’s derived performance claims.

  • Updates benchmark timestamps, tool versions, timing values, and statistics.
  • Regenerates the README benchmark ratios from the refreshed results.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness or security issues identified.

The refreshed benchmark values agree with their raw statistics, the README ratios are correctly derived from those values, and the data remains compatible with existing consumers and validation.

Important Files Changed

Filename Overview
README.md Updates four benchmark ratios; each value matches the repository’s derivation and rounding rules.
benchmarks/results.json Refreshes benchmark metadata and measurements; aggregate values, statistics, schema, and ordering are internally consistent.

Reviews (1): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

@github-actions

Copy link
Copy Markdown

Instruction counts

benchmark trend instructions Δ wall (min) Δ
graph ▅▅█▅▁▁▅▁▁▄▅█▅▁▁█▅▅█▅ 13,403,569 → 13,351,539 -0.39% 4.71 → 4.64ms -1.33%
install ███████▁▁▂▂▂▂▂▂▂▂▂▂▂ 108,139,173 → 108,154,815 +0.01% 27.36 → 26.97ms -1.41%
run ▁▂▄▃▄▃▃▆▄▅█▄▆▅▄▄▆▅▄▆ 3,058,095 → 3,062,867 +0.16% 4.07 → 4.10ms +0.66%
startup ▁▁▁▁▁▁▂▇▅▂███▂▂▃▅▂▂▆ 2,380,335 → 2,383,558 +0.14% 3.10 → 2.88ms -7.26%
tree ▅▅▁▅▅▁▁▁▂▅▅▁▂▁▁▅▂▁▅█ 12,672,254 → 12,744,695 +0.57% 4.49 → 4.12ms -8.19%

No instruction-count regression above 1%.

Only instruction counts gate. Wall clock is shown for context — on identical hardware it moves 4-20% run to run.

Measured by tak — instruction-counted CLI benchmarks, stored in this repository's git notes.

66db4d7ce4b2 vs 840845a865f9 · measured on this runner, not pushed to the history.

@jdx
jdx merged commit d73312d into main Aug 31, 2026
22 checks passed
@jdx
jdx deleted the bench-refresh branch August 31, 2026 11:09
@cursor cursor Bot mentioned this pull request Aug 31, 2026
@cursor cursor Bot mentioned this pull request Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant