Skip to content

chore: refresh benchmarks for v1.38.1 - #1257

Merged
jdx merged 1 commit into
mainfrom
bench-refresh
Aug 10, 2026
Merged

jdx merged 1 commit into
mainfrom
bench-refresh

Conversation

@jdx

@jdx jdx commented Aug 10, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 1.38.0; the workspace is now 1.38.1. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Versions:

  • aube: 1.38.0 -> 1.38.1

Public ratios: warm installs vs Bun 3x -> 3x; warm installs vs pnpm 7x -> 7x; repeat test vs Bun 4x -> 4x; repeat test vs pnpm 47x -> 42x.

Benchmark aube bun deno pnpm npm yarn
Fresh install (warm cache) 202ms -> 216ms (+7%) 525ms -> 541ms (+3%) 567ms -> 495ms (-13%) 1450ms -> 1423ms (-2%) 3523ms -> 3451ms (-2%) 2835ms -> 2832ms (0%)
Fresh install (cold cache) 6939ms -> 6935ms (0%) 1469ms -> 1477ms (+1%) 1607ms -> 1674ms (+4%) 6600ms -> 6633ms (+1%) 5731ms -> 5794ms (+1%) 6153ms -> 6391ms (+4%)
npm install && npm run test 7ms -> 8ms (+14%) 27ms -> 28ms (+4%) 79ms -> 90ms (+14%) 327ms -> 332ms (+2%) 644ms -> 659ms (+2%) 1183ms -> 1215ms (+3%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.


Generated by the bench-refresh workflow.


Note

Low Risk
Documentation and benchmark data only; no runtime or application code changes.

Overview
Re-runs the hermetic benchmark suite and updates benchmarks/results.json for aube 1.38.1 (from 1.38.0), including refreshed timings and stats for warm/cold installs and the repeat install && test scenario across all package managers.

The auto-generated README.md BENCH_RATIOS block is updated to match; the only public ratio change is repeat test vs pnpm (47x → 42x); warm vs pnpm/Bun claims stay at 7x and 3x.

Reviewed by Cursor Bugbot for commit db11a34. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Documentation

    • Updated the published benchmark claim for repeat test commands from up to 47× faster to up to 42× faster than pnpm.
  • Benchmarks

    • Refreshed installation and install/test performance results across package managers.
    • Updated benchmark metadata and version information.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 03349918-8b3c-469f-b902-80733c9b90c7

📥 Commits

Reviewing files that changed from the base of the PR and between 88292e7 and db11a34.

📒 Files selected for processing (2)
  • README.md
  • benchmarks/results.json

📝 Walkthrough

Walkthrough

The benchmark metadata, Aube version, and measurements were refreshed in benchmarks/results.json. The README repeat-test performance claim was changed from up to 47x faster to up to 42x faster than pnpm.

Changes

Benchmark refresh

Layer / File(s) Summary
Benchmark metadata and measurements
benchmarks/results.json
The timestamp and Aube version were updated. Warm-cache, cold-cache, and install-and-test measurements and statistics were replaced.
README benchmark claim
README.md
The repeat-test speedup claim changed from 47x to 42x faster than pnpm.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • jdx/aube#1244: Refreshes benchmark results and the README repeat-test speed claim.
  • jdx/aube#1206: Refreshes benchmark data and README speed ratios.
  • jdx/aube#1124: Replaces benchmark measurements and updates README benchmark claims.

Suggested reviewers: mise-en-dev

Poem

A rabbit refreshed the numbers with care,
New timings now bloom in the benchmark lair.
The README claim hops from forty-seven to forty-two,
While warm, cold, and test runs take their cue.
Squeak! The results are ready to view.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the benchmark refresh and identifies the updated Aube version.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR refreshes benchmark data for aube 1.38.1 and synchronizes the README’s generated performance ratios.

  • Updates benchmark timestamps, versions, measurements, and statistics.
  • Adjusts the repeat-test comparison against pnpm from 47x to 42x.
  • Keeps all rounded values and public ratios consistent with the repository’s benchmark generation logic.

Confidence Score: 5/5

The PR appears safe to merge, with benchmark data and README claims remaining synchronized and internally consistent.

The refreshed measurements match their full-precision statistics, the displayed ratios match the repository’s generation logic, and no additional coordinated updates are required.

Important Files Changed

Filename Overview
README.md Updates the generated benchmark ratio from 47x to 42x, matching the refreshed install-test values.
benchmarks/results.json Refreshes aube 1.38.1 benchmark metadata and measurements; values and formatted statistics are internally consistent with the generator.

Reviews (1): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

@github-actions

Copy link
Copy Markdown

Instruction counts

benchmark trend instructions Δ wall (min) Δ
graph ▁▂▁▂▁▁▂▂▁▂▂▂▃█▃▃▃ 17,569,341 → 17,570,739 +0.01% 4.71 → 4.46ms -5.31%
install ▇█▅▆▄▃▅▄▆▄▃▅▁▁▁▁▁ 108,467,095 → 108,463,417 -0.00% 25.02 → 23.96ms -4.26%
startup ▂▂▂▁▁▁▂▂▂▁▁▂█▇▇▇▇ 7,477,720 → 7,478,800 +0.01% 3.36 → 3.23ms -3.61%
tree ▂▂▂▁▂▃▃▂▃▃▃▃▇▇███ 17,753,742 → 17,752,228 -0.01% 4.65 → 4.56ms -1.87%

No instruction-count regression above 1%.

Only instruction counts gate. Wall clock is shown for context — on identical hardware it moves 4-20% run to run.

Measured by tak — instruction-counted CLI benchmarks, stored in this repository's git notes.

db11a34deb3d vs 88292e72be5a · measured on this runner, not pushed to the history.

@jdx
jdx merged commit 5de9b3b into main Aug 10, 2026
20 checks passed
@jdx
jdx deleted the bench-refresh branch August 10, 2026 13:28
@cursor cursor Bot mentioned this pull request Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant