Skip to content

chore: refresh benchmarks for v1.35.0 - #1172

Merged
jdx merged 1 commit into
mainfrom
bench-refresh
Jul 29, 2026
Merged

jdx merged 1 commit into
mainfrom
bench-refresh

Conversation

@jdx

@jdx jdx commented Jul 29, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 1.34.0; the workspace is now 1.35.0. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Versions:

  • aube: 1.34.0 -> 1.35.0

Public ratios: warm installs vs Bun 2x -> 3x; warm installs vs pnpm 5x -> 6x; repeat test vs Bun 8x -> 4x; repeat test vs pnpm 40x -> 51x.

Benchmark aube bun deno pnpm npm yarn
Fresh install (warm cache) 495ms -> 419ms (-15%) 1127ms -> 1342ms (+19%) 875ms -> 1397ms (+60%) 2613ms -> 2616ms (+0%) 6856ms -> 7331ms (+7%) 8270ms -> 8501ms (+3%)
Fresh install (cold cache) 14357ms -> 14885ms (+4%) 5118ms -> 5033ms (-2%) 8340ms -> 9879ms (+18%) 10954ms -> 17222ms (+57%) 9171ms -> 10808ms (+18%) 13182ms -> 13648ms (+4%)
npm install && npm run test 9ms -> 10ms (+11%) 69ms -> 41ms (-41%) 44ms -> 43ms (-2%) 359ms -> 510ms (+42%) 694ms -> 729ms (+5%) 1513ms -> 1586ms (+5%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.


Generated by the bench-refresh workflow.


Note

Low Risk
Documentation-only benchmark artifact refresh with no runtime or install logic changes.

Overview
Refreshes published benchmark data for aube 1.35.0 after re-running the hermetic bench:bump pipeline.

benchmarks/results.json updates the run timestamp, pins versions.aube to 1.35.0, and replaces timing values (and stats) for warm/cold fresh installs and the repeat install && test scenario across all package managers.

README.md regenerates the BENCH_RATIOS block so marketing copy matches the new numbers: warm install vs pnpm 5x → 6x, vs Bun 2x → 3x; repeat test vs pnpm 40x → 51x, vs Bun 8x → 4x.

Reviewed by Cursor Bugbot for commit a9ea96a. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Documentation

    • Updated the “Fast installs” benchmark figures, including warm-install and repeat-test speedup ratios.
  • Chores

    • Refreshed benchmark results and metadata for the latest version, including warm, cold, and install-test measurements.

@coderabbitai

coderabbitai Bot commented Jul 29, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: fecbeb49-f340-41e0-ac8b-a86adc64aa68

📥 Commits

Reviewing files that changed from the base of the PR and between fff2bb6 and a9ea96a.

📒 Files selected for processing (2)
  • README.md
  • benchmarks/results.json

📝 Walkthrough

Walkthrough

Updated benchmark metadata and measurements for aube 1.35.0, then revised the README’s Fast installs ratios to match the refreshed results.

Changes

Benchmark refresh

Layer / File(s) Summary
Regenerated benchmark results
benchmarks/results.json
Updated the timestamp and aube version, and replaced measurements and statistics for gvs-warm, gvs-cold, and install-test.
Updated README benchmark ratios
README.md
Revised the documented warm-install and repeat-test speedup ratios for pnpm and Bun.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • jdx/aube#937: Refreshes the same benchmark artifacts and README ratios.
  • jdx/aube#947: Regenerates the same benchmark results and documented figures.
  • jdx/aube#962: Updates the same benchmark entries and Fast installs metrics.

Suggested reviewers: mise-en-dev

Poem

I’m a bunny with benchmarks bright,
Numbers refreshed in morning light.
Warm installs hop, repeat tests race,
New results neatly take their place.
aube’s little chart now wears a grin!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: refreshing benchmarks for version 1.35.0.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Refreshes the published benchmarks for aube 1.35.0.

  • Regenerates benchmark values and statistics across three scenarios.
  • Updates the README performance ratios to match the refreshed results.

Confidence Score: 5/5

The PR appears safe to merge because the refreshed benchmark data and corresponding README claims are consistent.

Every displayed ratio matches the updated benchmark values, and each summarized millisecond value and formatted statistic agrees with the benchmark generator’s derivation.

Important Files Changed

Filename Overview
README.md Updates all four public performance ratios consistently with the refreshed benchmark values and generator rounding behavior.
benchmarks/results.json Updates the benchmark timestamp, aube version, measurements, and statistics; all derived values and formatted statistics are internally consistent.

Reviews (1): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

@github-actions

Copy link
Copy Markdown

Instruction counts

benchmark trend instructions Δ wall (min) Δ
graph ▁▂▁▁▁▁▁▁▁▁▁▁▁█████▇█ 16,714,414 → 16,714,726 +0.00% 4.44 → 3.81ms -14.13%
install ▁▁▁▇▅▅▆▆▅▆▅▇▅█▆▇█▇▇█ 103,097,562 → 103,140,626 +0.04% 31.70 → 28.98ms -8.60%
startup ▁▂▁▂▁▂▂▂▁▁▂▁▁▇▇▇█▇▆█ 7,665,680 → 7,680,131 +0.19% 2.95 → 2.64ms -10.45%
tree ▂▂▂▁▁▁▁▂▁▁▁▁▁██▇█▇▇▇ 17,729,964 → 17,732,895 +0.02% 4.35 → 3.91ms -10.15%

No instruction-count regression above 1%.

Only instruction counts gate. Wall clock is shown for context — on identical hardware it moves 4-20% run to run.

Measured by tak — instruction-counted CLI benchmarks, stored in this repository's git notes.

a9ea96aed55f vs fff2bb681add · measured on this runner, not pushed to the history.

@jdx
jdx merged commit ba6b67a into main Jul 29, 2026
20 checks passed
@jdx
jdx deleted the bench-refresh branch July 29, 2026 14:24
@cursor cursor Bot mentioned this pull request Jul 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant