Skip to content

chore: refresh benchmarks for v1.31.0 - #1072

Merged
jdx merged 1 commit into
mainfrom
bench-refresh
Jul 21, 2026
Merged

jdx merged 1 commit into
mainfrom
bench-refresh

Conversation

@jdx

@jdx jdx commented Jul 21, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 1.29.1; the workspace is now 1.31.0. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Versions:

  • aube: 1.29.1 -> 1.31.0
  • pnpm: 11.13.1 -> 11.15.1

Public ratios: warm installs vs Bun 1.4x -> 1.5x; warm installs vs pnpm 3x -> 3x; repeat test vs Bun 4x -> 4x; repeat test vs pnpm 40x -> 32x.

Benchmark aube bun deno pnpm npm yarn
Fresh install (warm cache) 952ms -> 989ms (+4%) 1367ms -> 1457ms (+7%) 1999ms -> 2434ms (+22%) 3262ms -> 2477ms (-24%) 9475ms -> 7150ms (-25%) 8860ms -> 8139ms (-8%)
Fresh install (cold cache) 15519ms -> 15024ms (-3%) 6795ms -> 4827ms (-29%) 7464ms -> 7911ms (+6%) 11930ms -> 11536ms (-3%) 10078ms -> 9866ms (-2%) 13347ms -> 13150ms (-1%)
npm install && npm run test 9ms -> 12ms (+33%) 40ms -> 45ms (+13%) 45ms -> 47ms (+4%) 359ms -> 380ms (+6%) 709ms -> 736ms (+4%) 1554ms -> 1576ms (+1%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.


Generated by the bench-refresh workflow.


Note

Low Risk
Documentation and benchmark data only; no runtime or application code changes.

Overview
Re-runs the hermetic benchmark suite and updates pinned results for aube 1.31.0 (from 1.29.1) and pnpm 11.15.1 (from 11.13.1) in benchmarks/results.json, with a new updated timestamp and refreshed means/stddev for warm/cold install and repeat install && test scenarios.

The README BENCH_RATIOS block is regenerated to match: warm install vs Bun is now ~1.5x (was ~1.4x), repeat test vs pnpm is up to 32x (was 40x); the 3x vs pnpm warm-install and 4x vs Bun repeat-test claims are unchanged.

Reviewed by Cursor Bugbot for commit ed370bb. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Documentation

    • Updated installation benchmark comparisons, including warm-install and repeat-test command speedups.
  • Chores

    • Refreshed benchmark results, timestamps, tool versions, and performance statistics.

Copilot AI review requested due to automatic review settings July 21, 2026 06:28
@coderabbitai

coderabbitai Bot commented Jul 21, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: d929ce7b-017a-465e-a6e5-6920794759f3

📥 Commits

Reviewing files that changed from the base of the PR and between 55f1a9f and ed370bb.

📒 Files selected for processing (2)
  • README.md
  • benchmarks/results.json

📝 Walkthrough

Walkthrough

Benchmark metadata and timing results were regenerated for three benchmark scenarios, and the README’s Fast installs speedup figures were updated accordingly.

Changes

Benchmark refresh

Layer / File(s) Summary
Refresh benchmark measurements
benchmarks/results.json
Updated benchmark timestamp, pnpm version, per-manager values, and statistics for warm, cold, and install-test runs.
Update published speedup ratios
README.md
Adjusted the Fast installs speedup figures for Bun and pnpm while retaining the Bun repeat-command comparison.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • jdx/aube#1060 — Refreshes the same benchmark rows, metadata, and README figures.
  • jdx/aube#1050 — Regenerates benchmark statistics and updates the Fast installs claims.
  • jdx/aube#1041 — Updates the same benchmark artifacts and published ratios.

Suggested reviewers: copilot, mise-en-dev

Poem

I’m a rabbit with numbers to spare,
Refreshing benchmark rows with care.
Warm, cold, and tests hop anew,
README speedups now match the view.
Fresh stats nibble the clock all day.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly conveys that benchmark data was refreshed for the v1.31.0 release.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refreshes the repo’s published benchmark artifacts to reflect the current workspace release (aube v1.31.0) and updated pnpm version, keeping README performance claims aligned with the latest hermetic benchmark run.

Changes:

  • Updated the README BENCH_RATIOS block to reflect new “warm install” and “install && test” relative-performance ratios.
  • Regenerated benchmarks/results.json with a new updated timestamp, pinned tool versions, and refreshed benchmark values/stats.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
README.md Updates the public benchmark ratio statement in the BENCH_RATIOS block.
benchmarks/results.json Updates pinned versions, timestamp, and benchmark measurements/statistics for the latest hermetic run.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md

<!-- BENCH_RATIOS:START -->
**[Fast installs](https://aube.jdx.dev/benchmarks).** Warm installs are about 3x faster than pnpm and ~1.4x faster than Bun in the current benchmarks. Repeat test commands run up to 40x faster than pnpm and up to 4x faster than Bun.
**[Fast installs](https://aube.jdx.dev/benchmarks).** Warm installs are about 3x faster than pnpm and ~1.5x faster than Bun in the current benchmarks. Repeat test commands run up to 32x faster than pnpm and up to 4x faster than Bun.
@greptile-apps

greptile-apps Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR refreshes the published benchmarks for aube 1.31.0. The main changes are:

  • Updates benchmark versions, timings, and statistics.
  • Regenerates the README performance ratios.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.
  • The README ratios match the refreshed values and generator rules.
  • The benchmark data remains consistent with its consumers.

Important Files Changed

Filename Overview
README.md Updates the generated performance claims to match the refreshed benchmark values and configured rounding rules.
benchmarks/results.json Refreshes manager versions, timings, and statistics while preserving the expected data shape and internal consistency.

Reviews (1): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

@jdx
jdx merged commit 5d6f24c into main Jul 21, 2026
19 checks passed
@jdx
jdx deleted the bench-refresh branch July 21, 2026 17:23
@cursor cursor Bot mentioned this pull request Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants