Skip to content

chore: refresh benchmarks for v1.38.0 - #1244

Merged
jdx merged 1 commit into
mainfrom
bench-refresh
Aug 9, 2026
Merged

jdx merged 1 commit into
mainfrom
bench-refresh

Conversation

@jdx

@jdx jdx commented Aug 8, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 1.37.0; the workspace is now 1.38.0. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Versions:

  • aube: 1.37.0 -> 1.38.0
  • deno: 2.9.4 -> 2.9.5
  • pnpm: 11.18.0 -> 11.20.0

Public ratios: warm installs vs Bun 3x -> 3x; warm installs vs pnpm 7x -> 7x; repeat test vs Bun 4x -> 4x; repeat test vs pnpm 48x -> 47x.

Benchmark aube bun deno pnpm npm yarn
Fresh install (warm cache) 203ms -> 210ms (+3%) 519ms -> 528ms (+2%) 461ms -> 518ms (+12%) 1417ms -> 1372ms (-3%) 3590ms -> 3466ms (-3%) 2847ms -> 2811ms (-1%)
Fresh install (cold cache) 6712ms -> 6937ms (+3%) 1448ms -> 1468ms (+1%) 1658ms -> 1639ms (-1%) 6649ms -> 8582ms (+29%) 5689ms -> 5747ms (+1%) 6229ms -> 6156ms (-1%)
npm install && npm run test 7ms -> 7ms (0%) 27ms -> 28ms (+4%) 104ms -> 96ms (-8%) 337ms -> 332ms (-1%) 629ms -> 632ms (+0%) 1194ms -> 1187ms (-1%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.


Generated by the bench-refresh workflow.


Note

Low Risk
Documentation and benchmark artifacts only; no runtime or product code changes.

Overview
Refreshes published benchmark data for aube 1.38.0 (was 1.37.0) by re-running the hermetic bench:bump pipeline and committing the new benchmarks/results.json snapshot, including updated tool versions (deno 2.9.5, pnpm 11.20.0) and revised timings/stats for warm install, cold install, and install+test scenarios.

The auto-generated README BENCH_RATIOS block is updated so the repeat-test claim vs pnpm drops from 48× to 47×; warm vs pnpm/Bun headline ratios stay at 7× and 3×.

Reviewed by Cursor Bugbot for commit f5b5a1c. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Documentation

    • Updated the README benchmark claim to report repeated test commands as up to 47× faster than pnpm.
  • Benchmarks

    • Refreshed benchmark timestamps, tool versions, measurements, and statistics across warm-cache, cold-cache, and install/test scenarios.
    • Updated reported results to reflect the latest performance comparisons and installation/testing measurements.

@coderabbitai

coderabbitai Bot commented Aug 8, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The PR refreshes benchmark metadata, tool versions, measurements, and statistics in benchmarks/results.json. It updates the README repeat-test speedup claim from 48x to 47x.

Changes

Benchmark refresh

Layer / File(s) Summary
Refresh benchmark data and claim
benchmarks/results.json, README.md
The benchmark timestamp, tool versions, warm-cache, cold-cache, and install/test results are updated. The README maximum repeat-test speedup claim changes from 48x to 47x.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • jdx/aube#975: Refreshes the same benchmark artifacts and README speedup claims.
  • jdx/aube#1041: Updates benchmarks/results.json and the README benchmark claim.
  • jdx/aube#1211: Regenerates benchmark results and updates the same README claim.

Suggested reviewers: greptile-apps

Poem

A rabbit checks the numbers bright,
New timings hop into the light.
Forty-seven leads the way,
Fresh results are here to stay.
The benchmark burrow rests tonight.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the primary change: refreshing benchmark data for version 1.38.0.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Aug 8, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

Refreshes the published benchmark artifacts for aube 1.38.0.

  • Updates recorded tool versions, timestamps, measurements, and statistics in benchmarks/results.json.
  • Synchronizes the README repeat-command comparison with the regenerated pnpm ratio.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
README.md Updates the generated benchmark ratio from 48x to 47x, consistent with the refreshed results.
benchmarks/results.json Refreshes benchmark metadata, package-manager versions, timing values, and corresponding statistics.

Reviews (2): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

greptile-apps[bot]
greptile-apps Bot previously approved these changes Aug 8, 2026
@github-actions

github-actions Bot commented Aug 8, 2026 •

Copy link
Copy Markdown

Instruction counts

benchmark trend instructions Δ wall (min) Δ
graph ▁▃██▁▂▂▃▂▂▂▂▂▂▂▂▂▂▂ 17,551,400 → 17,551,689 +0.00% 4.76 → 4.36ms -8.48%
install ▁▂▃▁▃▆▃▆▆█▄▅▃▁▄▃▅▃▁ 127,467,302 → 120,216,859 -5.69% 33.44 → 30.58ms -8.55%
startup ▇▄▇█▁▄▅▅███▄▄▄▇▄▆▂▄ 7,458,603 → 7,459,987 +0.02% 3.82 → 3.18ms -16.72%
tree ▁█▇▇▃▅▅▅▆▅▅▃▅▇▇▆▇█▆ 17,735,995 → 17,733,180 -0.02% 4.88 → 4.47ms -8.42%

No instruction-count regression above 1%.

Only instruction counts gate. Wall clock is shown for context — on identical hardware it moves 4-20% run to run.

Measured by tak — instruction-counted CLI benchmarks, stored in this repository's git notes.

f5b5a1c69144 vs 3d5fc64086db · measured on this runner, not pushed to the history.

@greptile-apps
greptile-apps Bot dismissed their stale review August 9, 2026 05:37

Dismissed because a newer commit was pushed; Greptile will re-review the current head.

@jdx
jdx merged commit b03f664 into main Aug 9, 2026
20 checks passed
@jdx
jdx deleted the bench-refresh branch August 9, 2026 06:52
@cursor cursor Bot mentioned this pull request Aug 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant