Skip to content

Refresh engine comparison benchmark results#2621

Merged
lahma merged 1 commit into
sebastienros:mainfrom
lahma:benchmark-refresh-2026-07-10
Jul 10, 2026
Merged

Refresh engine comparison benchmark results#2621
lahma merged 1 commit into
sebastienros:mainfrom
lahma:benchmark-refresh-2026-07-10

Conversation

@lahma

@lahma lahma commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Full re-run of the EngineComparison suite on current main, certifying the 2026-07-10 batch (#2613, #2615#2620) on the published table. Same machine/SDK as the 2026-07-09 refresh; competitor versions unchanged, their rows served as thermal canaries (times within ±3%, allocations byte-identical).

Highlights vs the 2026-07-09 refresh

  • dromaeo-string-base64 allocation −30% (2.30 → 1.62 MB per op), -modern −4.3% time; the classic row is now ranked first outright (previously statistically tied with NiL.JS). This is Member-bound loop test lane for i < arr.length / i < s.length #2617's member-bound comparison lane: i < s.length / i < arr.length loop tests read the length unboxed instead of materializing a number per iteration.
  • The batch's engine-reuse wins are deliberately called out as not visible here — every row in this table creates a fresh engine per op. Re-evaluating prepared scripts on a long-lived engine gets Reuse hoisted function definitions across re-evaluations on an engine #2613/Reuse class member definitions across class re-evaluations #2615 (hoisted-function and class-member definition reuse, class re-evaluation −23% time / −34% alloc) and Re-resolve slot locations when a shared handler tree changes function instance #2616 (slot-cache fix keeping all unboxed lanes alive from the second re-evaluation onward, −72% time on a re-evaluated 100k-iteration loop). The README notes this so embedders don't under-estimate the reuse path.
  • Everything else moved within the table's documented noise bands (core-eval ±2% bimodality, regexp/string families ±3–7%). Rank changes: the base64 promotion above, and stopwatch where NiL.JS's own row drifted −3% and BDN now groups it one rank ahead (Jint 136 ms, within ~3%, still ahead of Jurassic — prose updated honestly).
  • Prose/derived numbers synced to the new table (base64 alloc ratios 8.5×→12×, field-max 332×→472×, cube 5.99→6.03 ms, core-eval 0.92→0.93 ms).

🤖 Generated with Claude Code

https://claude.ai/code/session_01BY8HoKam5H8vTPn1bMY2Hv

Full re-run after the 2026-07-10 batch (sebastienros#2613, sebastienros#2615-sebastienros#2620). Highlights vs
the 2026-07-09 refresh: dromaeo-string-base64 allocation -30% (2.30 -> 1.62
MB/op) and -modern -4.3% time, the classic row now ranked first outright
(previously tied with NiL.JS) - loop tests of the form i < s.length read the
length unboxed through the comparison lane (sebastienros#2617). The batch's engine-reuse
wins (sebastienros#2613/sebastienros#2615 definition reuse, sebastienros#2616 slot-cache lane restoration) target
long-lived-engine embeddings and are not visible in this fresh-engine-per-op
table. Everything else moved within documented noise bands; competitor rows
served as thermal canaries (times within +/-3%, allocations byte-identical).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BY8HoKam5H8vTPn1bMY2Hv
@lahma
lahma enabled auto-merge (squash) July 10, 2026 12:01
@lahma
lahma merged commit 177d336 into sebastienros:main Jul 10, 2026
4 checks passed
@lahma
lahma deleted the benchmark-refresh-2026-07-10 branch July 10, 2026 12:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant