Skip to content

chore: refresh benchmarks for v1.21.0 #890

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jdx merged 1 commit into from
Jun 14, 2026
Merged

chore: refresh benchmarks for v1.21.0 #890

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
## Why Try It

<!-- BENCH_RATIOS:START -->
**[Fast installs](https://aube.jdx.dev/benchmarks).** Warm installs are about 8x faster than pnpm and about 6x faster than Bun in the current benchmarks. Repeat test commands run up to 34x faster than pnpm and up to 5x faster than Bun.
**[Fast installs](https://aube.jdx.dev/benchmarks).** Warm installs are about 3x faster than pnpm and about 2x faster than Bun in the current benchmarks. Repeat test commands run up to 30x faster than pnpm and up to 5x faster than Bun.
<!-- BENCH_RATIOS:END -->

**[Existing lockfiles](https://aube.jdx.dev/package-manager/lockfiles).** Reads and writes `pnpm-lock.yaml`, `package-lock.json`, `npm-shrinkwrap.json`, `yarn.lock`, and `bun.lock` in place.
Expand Down
220 changes: 110 additions & 110 deletions benchmarks/results.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"updated": "2026-06-12T06:58:57.644Z",
"updated": "2026-06-14T06:56:35.450Z",
"unit": "ms",
"managers": [
"aube",
Expand All @@ -10,7 +10,7 @@
"yarn"
],
"versions": {
"aube": "1.19.0",
"aube": "1.21.0",
"bun": "1.3.14",
"deno": "2.8.3",
"pnpm": "11.6.0",
Expand All @@ -23,167 +23,167 @@
"key": "gvs-warm",
"label": "Fresh install (warm cache)",
"values": {
"aube": 308,
"bun": 1889,
"deno": 1254,
"pnpm": 2515,
"npm": 7080,
"yarn": 8282
"aube": 886,
"bun": 1813,
"deno": 1616,
"pnpm": 2512,
"npm": 7136,
"yarn": 7708
},
"stats": {
"aube": {
"text": "0.308s ± 0.077s",
"mean": 0.30761542092000005,
"stddev": 0.07720214241262749,
"min": 0.24460349472,
"max": 0.43811340672000004
"text": "0.886s ± 0.260s",
"mean": 0.8857092395799999,
"stddev": 0.2603188798119799,
"min": 0.41187857048000004,
"max": 1.13239212248
},
"bun": {
"text": "1.889s ± 0.326s",
"mean": 1.88935886912,
"stddev": 0.3258903880951513,
"min": 1.5542772249199999,
"max": 2.31304034192
"text": "1.813s ± 0.395s",
Comment on lines +30 to +42

@greptile-apps greptile-apps Bot Jun 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 npm warm-install stddev spiked 28x — possible noisy environment

npm's gvs-warm stddev went from 0.041s to 1.138s (a 28x increase), with a min/max spread of 6.36s–8.82s vs the previous run's tight 7.02s–7.12s range. That level of variance strongly suggests OS or network interference during this scenario's run. A noisy environment can silently drag other tools' numbers in either direction too. Worth re-running the warm-cache scenario in isolation to confirm the aube 308ms→886ms jump is real and not an artifact of a loaded machine.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Claude Code

"mean": 1.81348942708,
"stddev": 0.39494729345877344,
"min": 0.88972477258,
"max": 2.2266976135800003
},
"deno": {
"text": "1.254s ± 0.253s",
"mean": 1.2537232119800001,
"stddev": 0.2525640329659767,
"min": 0.57540808678,
"max": 1.4063693907800001
"text": "1.616s ± 0.223s",
"mean": 1.61627348468,
"stddev": 0.22337633131482657,
"min": 1.17143179098,
"max": 1.92658235198
},
"pnpm": {
"text": "2.515s ± 0.055s",
"mean": 2.5149517476599996,
"stddev": 0.055134595175826966,
"min": 2.44608261366,
"max": 2.58747086766
"text": "2.512s ± 0.092s",
"mean": 2.51226510468,
"stddev": 0.09171619060915105,
"min": 2.41284999048,
"max": 2.62676730848
},
"npm": {
"text": "7.080s ± 0.041s",
"mean": 7.080204031020001,
"stddev": 0.04148102404255947,
"min": 7.023763257020001,
"max": 7.12161811302
"text": "7.136s ± 1.138s",
"mean": 7.13554796277,
"stddev": 1.1377408959840538,
"min": 6.35875688352,
"max": 8.82148937252
},
"yarn": {
"text": "8.282s ± 0.117s",
"mean": 8.282211578160002,
"stddev": 0.1171433876864422,
"min": 8.118846342160001,
"max": 8.39647272416
"text": "7.708s ± 0.022s",
"mean": 7.70834003805,
"stddev": 0.02209149754351933,
"min": 7.682411184299999,
"max": 7.7271990753
}
}
},
{
"key": "gvs-cold",
"label": "Fresh install (cold cache)",
"values": {
"aube": 6209,
"bun": 5205,
"deno": 8135,
"pnpm": 13454,
"npm": 12234,
"yarn": 14123
"aube": 5860,
"bun": 4856,
"deno": 6748,
"pnpm": 12925,
"npm": 8758,
"yarn": 11079
},
"stats": {
"aube": {
"text": "6.209s ± 0.232s",
"mean": 6.2094188492199995,
"stddev": 0.2320404471552325,
"min": 5.79256539032,
"max": 6.4963373433200005
"text": "5.860s ± 0.121s",
"mean": 5.859741182980001,
"stddev": 0.12119234254897016,
"min": 5.61643841598,
"max": 6.0312778609799995
},
"bun": {
"text": "5.205s ± 0.320s",
"mean": 5.2046229224,
"stddev": 0.31976685130271,
"min": 4.753269405899999,
"max": 5.674859554899999
"text": "4.856s ± 0.254s",
"mean": 4.85556895118,
"stddev": 0.2535700347982356,
"min": 4.55070336988,
"max": 5.41695267588
},
"deno": {
"text": "8.135s ± 0.413s",
"mean": 8.13510543482,
"stddev": 0.4128883280212506,
"min": 7.34951930622,
"max": 8.78693202522
"text": "6.748s ± 0.517s",
"mean": 6.7482074805400005,
"stddev": 0.5165418273616005,
"min": 5.54867566784,
"max": 7.370266415840001
},
"pnpm": {
"text": "13.454s ± 0.491s",
"mean": 13.454090231760002,
"stddev": 0.4914180058834111,
"min": 12.608312429560002,
"max": 13.89179368756
"text": "12.925s ± 0.584s",
"mean": 12.92499362018,
"stddev": 0.5841709076654438,
"min": 12.381140631780001,
"max": 13.61272642378
},
"npm": {
"text": "12.234s ± 0.313s",
"mean": 12.233646502660001,
"stddev": 0.31338432045280123,
"min": 11.81489622166,
"max": 12.544362276660001
"text": "8.758s ± 0.117s",
"mean": 8.7577026709,
"stddev": 0.11736487531287457,
"min": 8.5878052594,
"max": 8.8536562954
},
"yarn": {
"text": "14.123s ± 1.146s",
"mean": 14.122906365970001,
"stddev": 1.1461030052359442,
"min": 12.83407510872,
"max": 15.38274136772
"text": "11.079s ± 0.198s",
"mean": 11.078784169630001,
"stddev": 0.19786503360306565,
"min": 10.82311241738,
"max": 11.306064046380001
}
}
},
{
"key": "install-test",
"label": "npm install && npm run test",
"values": {
"aube": 12,
"bun": 54,
"deno": 95,
"pnpm": 405,
"npm": 780,
"yarn": 1351
"aube": 11,
"bun": 60,
"deno": 59,
"pnpm": 331,
"npm": 681,
"yarn": 1158
},
"stats": {
"aube": {
"text": "0.012s ± 0.001s",
"mean": 0.01192989048,
"stddev": 0.001344083485004542,
"min": 0.01029210528,
"max": 0.01418888428
"text": "0.011s ± 0.001s",
"mean": 0.01057162638,
"stddev": 0.0010285670311284533,
"min": 0.00899837858,
"max": 0.012653757580000001
},
"bun": {
"text": "0.054s ± 0.003s",
"mean": 0.05396974016,
"stddev": 0.0027128146552191377,
"min": 0.05040210936,
"max": 0.05824936536
"text": "0.060s ± 0.002s",
"mean": 0.06023251586,
"stddev": 0.0018515189858099568,
"min": 0.05806497436,
"max": 0.06364068136
},
"deno": {
"text": "0.095s ± 0.016s",
"mean": 0.0950754448,
"stddev": 0.016093321804329604,
"min": 0.0721047861,
"max": 0.11395357310000001
"text": "0.059s ± 0.011s",
"mean": 0.05903903086000001,
"stddev": 0.011317417342944523,
"min": 0.048758849660000006,
"max": 0.07656082366
},
"pnpm": {
"text": "0.405s ± 0.012s",
"mean": 0.40534537866,
"stddev": 0.012441142633816371,
"min": 0.38350651526,
"max": 0.41408264226
"text": "0.331s ± 0.012s",
"mean": 0.33075239404,
"stddev": 0.01159884442284716,
"min": 0.31987814264000003,
"max": 0.34588973864
},
"npm": {
"text": "0.780s ± 0.038s",
"mean": 0.7801812600800001,
"stddev": 0.038201832304791336,
"min": 0.7487227095800001,
"max": 0.8334927725800001
"text": "0.681s ± 0.007s",
"mean": 0.6814556049600001,
"stddev": 0.007281756185120684,
"min": 0.67226852546,
"max": 0.6883501554600001
},
"yarn": {
"text": "1.351s ± 0.043s",
"mean": 1.3510298492300001,
"stddev": 0.04339636353517077,
"min": 1.2975966324800001,
"max": 1.40319157148
"text": "1.158s ± 0.024s",
"mean": 1.15774077333,
"stddev": 0.023721424121133275,
"min": 1.1309359440800002,
"max": 1.1887344340800001
}
}
}
Expand Down
Loading