Skip to content

chore: refresh benchmarks for v1.39.0 - #1285

Merged
jdx merged 1 commit into
mainfrom
bench-refresh
Aug 13, 2026
Merged

jdx merged 1 commit into
mainfrom
bench-refresh

Conversation

@jdx

@jdx jdx commented Aug 13, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 1.38.1; the workspace is now 1.39.0. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Versions:

  • aube: 1.38.1 -> 1.39.0
  • pnpm: 11.20.0 -> 11.21.0

Public ratios: warm installs vs Bun 3x -> 3x; warm installs vs pnpm 7x -> 7x; repeat test vs Bun 4x -> 4x; repeat test vs pnpm 42x -> 43x.

Benchmark aube bun deno pnpm npm yarn
Fresh install (warm cache) 216ms -> 212ms (-2%) 541ms -> 536ms (-1%) 495ms -> 566ms (+14%) 1423ms -> 1433ms (+1%) 3451ms -> 3681ms (+7%) 2832ms -> 2827ms (0%)
Fresh install (cold cache) 6935ms -> 6987ms (+1%) 1477ms -> 1489ms (+1%) 1674ms -> 1637ms (-2%) 6633ms -> 8415ms (+27%) 5794ms -> 5785ms (0%) 6391ms -> 6365ms (0%)
npm install && npm run test 8ms -> 8ms (0%) 28ms -> 28ms (0%) 90ms -> 92ms (+2%) 332ms -> 344ms (+4%) 659ms -> 639ms (-3%) 1215ms -> 1217ms (+0%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.


Generated by the bench-refresh workflow.


Note

Low Risk
Docs and benchmark artifact updates only; no runtime or package-manager logic changes.

Overview
Refreshes published benchmark numbers after bumping the pinned aube version from 1.38.1 to 1.39.0 (and pnpm to 11.21.0) via mise run bench:bump, updating benchmarks/results.json with a new run timestamp and revised timings across warm/cold install and repeat install && test scenarios.

The README BENCH_RATIOS block is regenerated so marketing copy matches the new data—the only ratio change called out is repeat test commands vs pnpm 42x → 43x; warm-install vs pnpm/Bun ratios stay at 7x and 3x.

Reviewed by Cursor Bugbot for commit 1885724. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Documentation

    • Updated the benchmark claim to report repeat test commands as up to 43× faster than pnpm.
  • Benchmarks

    • Refreshed benchmark metadata, tool versions, measurements, and statistical summaries for warm-cache, cold-cache, and install/test runs.

@coderabbitai

coderabbitai Bot commented Aug 13, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: 43970c4e-4030-4e16-9dd4-0463f69064f1

📥 Commits

Reviewing files that changed from the base of the PR and between d52f0ab and 1885724.

📒 Files selected for processing (2)
  • README.md
  • benchmarks/results.json

📝 Walkthrough

Walkthrough

The benchmark timestamp, tool versions, warm-cache results, cold-cache results, and install/test measurements were refreshed. The README now reports repeat tests as up to 43x faster than pnpm.

Changes

Benchmark refresh

Layer / File(s) Summary
Refresh benchmark metadata and measurements
benchmarks/results.json
The benchmark timestamp, Aube and pnpm versions, cache measurements, and install/test statistics were replaced with newer results.
Update documented speed claim
README.md
The repeat-test benchmark claim changed from 42x to 43x faster than pnpm.

Estimated code review effort: 2 (Simple) | ~10 minutes

Mergeability Score: ⚪ Minimal · up to 18857

This PR refreshes benchmark outputs and README ratios for the new release; no actionable merge-blocking risk remains beyond normal checks and review.

Possibly related PRs

  • jdx/aube#1244: Refreshes benchmark results and the README repeat-test speedup claim.
  • jdx/aube#1257: Updates benchmark measurements and the README repeat-test speed claim.
  • jdx/aube#1206: Regenerates benchmark results and updates README performance claims.

Suggested reviewers: mise-en-dev, greptile-apps

Poem

A rabbit checks the numbers bright,
New benchmark results take flight.
Forty-three hops past pnpm’s pace,
Fresh timings settle into place.
The README tells the measured tale.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the benchmark refresh and identifies the target version, matching the main changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR refreshes published benchmark artifacts for aube 1.39.0 and updates the derived README comparison.

  • Updates benchmark versions, measurements, statistics, and timestamp in benchmarks/results.json.
  • Regenerates the README repeat-test comparison from 42x to 43x versus pnpm.
  • Keeps the generated README ratios consistent with the refreshed benchmark values.

Confidence Score: 5/5

The PR appears safe to merge because the refreshed benchmark artifacts are internally consistent and no actionable defect was identified.

The README ratios match the generated benchmark values, and the refreshed measurements follow the repository’s benchmark-generation and validation conventions.

Important Files Changed

Filename Overview
README.md Updates the generated pnpm repeat-test ratio to 43x, consistent with the refreshed 344ms and 8ms results.
benchmarks/results.json Refreshes tool versions and benchmark measurements; the aggregate values and detailed statistics are internally consistent.

Fix All in Greploop

Reviews (1): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

@github-actions

Copy link
Copy Markdown

Instruction counts

benchmark trend instructions Δ wall (min) Δ
graph █▂▁▁▁▂▇▇▁▇▁▂▂█▃▁▂▁▁█ 17,562,527 → 17,663,084 +0.57% 4.72 → 4.62ms -2.27%
install ▁▁▁▁▁▂▁▁▁▁▁▇█▇▇▇█▇▇█ 108,617,049 → 108,630,598 +0.01% 24.81 → 24.93ms +0.47%
startup ▂▃▂▂▂▂▃▃▃▁▁▄▄▅▂▂▅▆▇█ 7,483,038 → 7,484,226 +0.02% 3.42 → 3.28ms -4.02%
tree ▇▄▆▃▃▄▇▄▇▆▆▄▇▃▆▁█▅▅▇ 17,747,748 → 17,749,561 +0.01% 4.70 → 4.59ms -2.39%

No instruction-count regression above 1%.

Only instruction counts gate. Wall clock is shown for context — on identical hardware it moves 4-20% run to run.

Measured by tak — instruction-counted CLI benchmarks, stored in this repository's git notes.

18857240d979 vs d52f0ab89594 · measured on this runner, not pushed to the history.

@jdx
jdx merged commit 741b815 into main Aug 13, 2026
21 checks passed
@jdx
jdx deleted the bench-refresh branch August 13, 2026 13:16
@cursor cursor Bot mentioned this pull request Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant