Skip to content

chore: refresh benchmarks for v2.2.4 - #1434

Merged
jdx merged 1 commit into
mainfrom
bench-refresh
Aug 31, 2026
Merged

jdx merged 1 commit into
mainfrom
bench-refresh

Conversation

@jdx

@jdx jdx commented Aug 31, 2026 •

Copy link
Copy Markdown
Collaborator

🤖 Refreshed benchmarks

benchmarks/results.json was pinned to aube 2.2.3; the workspace is now 2.2.4. Re-ran mise run bench:bump on the hermetic Verdaccio registry (500mbit / 50ms per the mise task) and regenerated benchmarks/results.json plus the README BENCH_RATIOS block. The benchmark matrix pins aube's GVS mode via npm_config_enable_global_virtual_store=true|false (the auto-synthesized env alias for the enableGlobalVirtualStore setting), so GitHub Actions' inherited CI=true environment does not change whether aube runs with GVS enabled or disabled.

Benchmark changes

Versions:

  • aube: 2.2.3 -> 2.2.4

Public ratios: warm installs vs Bun 3x -> 3x; warm installs vs pnpm 9x -> 8x; repeat test vs Bun 2x -> 2x; repeat test vs pnpm 27x -> 24x.

Benchmark aube bun deno pnpm npm yarn
Fresh install (warm cache) 214ms -> 187ms (-13%) 600ms -> 607ms (+1%) 616ms -> 657ms (+7%) 1918ms -> 1524ms (-21%) 3695ms -> 3701ms (+0%) 2906ms -> 3112ms (+7%)
Fresh install (cold cache) 7083ms -> 1424ms (-80%) 1523ms -> 1515ms (-1%) 1679ms -> 1761ms (+5%) 8397ms -> 8963ms (+7%) 5901ms -> 5978ms (+1%) 6877ms -> 7412ms (+8%)
npm install && npm run test 13ms -> 15ms (+15%) 31ms -> 33ms (+6%) 140ms -> 118ms (-16%) 352ms -> 362ms (+3%) 655ms -> 663ms (+1%) 1240ms -> 1248ms (+1%)

Review the numbers before merging — if anything looks wildly off vs. the previous release, investigate before landing. Hermetic proxy jitter or an npmjs uplink hiccup can occasionally skew results.

Once merged to main, the updated bench results flow into the next release-plz-pr run automatically.


Generated by the bench-refresh workflow.


Note

Low Risk
Documentation and benchmark data only; no application or install logic changes in the diff.

Overview
Re-runs the benchmark suite and pins benchmarks/results.json to aube 2.2.4 (from 2.2.3), with a new updated timestamp and refreshed timings across warm/cold install and repeat install && test scenarios.

The auto-generated README.md BENCH_RATIOS block is updated to match: warm install vs pnpm 9x → 8x, repeat test vs pnpm 27x → 24x (vs Bun stays 3x / 2x). Notable in the raw numbers: aube warm install improves slightly (~214ms → ~187ms) and the cold-cache aube row drops sharply (~7.1s → ~1.4s), so reviewers should sanity-check that run before treating cold numbers as a product regression/improvement—this PR only changes published bench artifacts, not runtime code.

Reviewed by Cursor Bugbot for commit 9a28c1b. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Documentation

    • Updated benchmark comparisons in the README to reflect the latest measured performance.
  • Benchmark Results

    • Refreshed benchmark data and metadata for warm installs, cold installs, and repeat install tests.
    • Updated the recorded package version and benchmark timestamp.

@coderabbitai

coderabbitai Bot commented Aug 31, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited), Organization UI (inherited)

Review profile: CHILL

Plan: Pro Plus

Run ID: ef37fe32-019d-45c6-aba4-fad4563147b5

📥 Commits

Reviewing files that changed from the base of the PR and between 36dc1f0 and 9a28c1b.

📒 Files selected for processing (2)
  • README.md
  • benchmarks/results.json

Included review availability: Your plan provides up to 4 included reviews per hour; 2 remain after this review.


📝 Walkthrough

Walkthrough

The PR refreshes benchmark metadata and measurements for Aube 2.2.4, then updates the README benchmark ratios for warm installs and repeat test commands.

Changes

Benchmark refresh

Layer / File(s) Summary
Recorded benchmark results
benchmarks/results.json
The benchmark timestamp and Aube version are updated. Measurements and statistics are refreshed for the gvs-warm, gvs-cold, and install-test scenarios.
Published benchmark claims
README.md
The README changes the pnpm comparisons to about 8x faster for warm installs and up to 24x faster for repeat test commands. Bun comparisons remain unchanged.

Estimated code review effort: 2 (Simple) | ~10 minutes

Merge Risk: ⚪ Minimal · up to 9a28c

This PR refreshes benchmark data for aube 2.2.4 and updates the README ratios without changing application or install logic; no actionable merge-blocking risk remains beyond normal checks and review.

Possibly related PRs

  • jdx/aube#937: Refreshes benchmark data and matching README performance claims.
  • jdx/aube#962: Updates the same benchmark results file and README figures.
  • jdx/aube#1172: Mirrors this benchmark data and README refresh pattern.

Suggested reviewers: mise-en-dev, greptile-apps

Poem

A rabbit checks the numbers bright
Fresh benchmark stats hop into sight
Aube’s version rises two point four
README ratios match once more
Ears up, the refreshed results run true

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: refreshing benchmark results for version 2.2.4.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (2 skipped: 2 unsupported.)


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR refreshes benchmark data for aube 2.2.4 and synchronizes the README’s generated performance ratios.

  • Updates benchmark versions, measurements, and statistics for all package managers.
  • Revises the README’s pnpm comparisons from 9x/27x to 8x/24x.
  • Keeps generated values, statistics, and public ratios internally consistent.

Confidence Score: 5/5

The PR appears safe to merge with no actionable correctness or security issues in the refreshed benchmark artifacts.

The README ratios match the generator’s rounding rules, the benchmark values agree with their underlying means and formatted statistics, and the unusually large cold-cache change is consistent with the benchmark’s deliberate script-policy alignment.

Important Files Changed

Filename Overview
README.md Updates generated benchmark claims to ratios that correctly match the refreshed warm-install and repeat-test values.
benchmarks/results.json Refreshes the benchmark matrix for aube 2.2.4; rounded values, raw statistics, and formatted measurements are internally consistent.

Reviews (1): Last reviewed commit: "chore: refresh benchmarks" | Re-trigger Greptile

@github-actions

Copy link
Copy Markdown

Instruction counts

benchmark trend instructions Δ wall (min) Δ
graph ▄▅█▅▁▁█▅▅█▁▅▅▅▅▅▅▅ 13,341,207 → 13,339,348 -0.01% 4.49 → 4.26ms -5.15%
install ▅█▆▇▇▁▂▃▆▃▅▆▅█▆▁▂▄ 108,134,486 → 108,146,000 +0.01% 26.82 → 26.47ms -1.31%
run ▅█▄▆▅▄▄▆▅▄▅█▂▂▃▂▁▄ 3,052,343 → 3,059,087 +0.22% 3.96 → 4.32ms +9.28%
startup ▃███▃▃▃▆▃▃▆▆▁▃▃▄▁▂ 2,379,074 → 2,379,954 +0.04% 2.77 → 2.80ms +1.14%
tree ▇█▁▂▁▁▇▁▁▇▁▁▇▁▇▁▁▁ 12,603,621 → 12,605,082 +0.01% 3.94 → 4.09ms +3.65%

No instruction-count regression above 1%.

Only instruction counts gate. Wall clock is shown for context — on identical hardware it moves 4-20% run to run.

Measured by tak — instruction-counted CLI benchmarks, stored in this repository's git notes.

9a28c1b159a6 vs 36dc1f0a3ec4 · measured on this runner, not pushed to the history.

@jdx
jdx merged commit c53b514 into main Aug 31, 2026
22 checks passed
@jdx
jdx deleted the bench-refresh branch August 31, 2026 19:34
@cursor cursor Bot mentioned this pull request Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant