Refresh engine comparison benchmark results#2572
Merged
Merged
Conversation
Full-suite EngineComparison re-run for the 4.11.0 release, on the branch rebased onto main @ 664d7f8 (includes sebastienros#2579). Competitor versions are unchanged (Jurassic 3.2.9, NiL.JS 2.6.1722, YantraJS.Core 1.2.406), so their rows act as a thermal canary confirming the window is comparable to the previous refresh. No regressions: every Jint row is flat or faster than the previous refresh, with allocation flat or lower. Headline wins from the loop-lane and environment work now in main: dromaeo-core-eval -27% (2.10 -> 1.53 ms prepared), dromaeo-3d-cube -13..15% (allocation also -16%), stopwatch -10%, dromaeo-string-base64 -5..12%. The core-eval gap to the closest competitor narrows from ~1.7x to ~1.3x and 3d-cube from ~1.4x to ~1.3x. Also retunes the prose to frame results against "the closest competitor" rather than singling out engines by name, and refers to Jint as 4.11.0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lahma
force-pushed
the
benchmark-readme-refresh-2026-07
branch
from
July 5, 2026 07:24
f19c9fb to
6b56d1b
Compare
lahma
enabled auto-merge (squash)
July 5, 2026 07:30
This was referenced Jul 6, 2026
This was referenced Jul 13, 2026
This was referenced Jul 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Full-suite re-run on
main@ f7935bb after the eval-execution wave landed (#2562, #2565–#2567, #2570, #2571).Highlights vs the 2026-06-30 refresh:
dromaeo-core-eval: −22.6% re-parse / −23.0% prepared measured same-window (worktree A/B against the pre-Slot-backed strict-eval environments with per-source pooling #2565 base: 2.51 → 1.94 ms / 2.48 → 1.91 ms); the wall-clock gap to NiL.JS narrows from ~2× to ~1.7× while Jint keeps allocating 4.7× less on the row. Modern variants −20.1%/−17.8%.Measurement hygiene: an earlier run of the suite was discarded — concurrent builds on the host inflated version-identical competitor rows by up to +80% with 10–22% StdDev. This table comes from a verified quiet window (competitor canary rows within ~5% of the previous refresh, StdDev back at historical levels). One documented exception is noted in the prose:
dromaeo-object-stringreads 13–22% higher for every engine uniformly on an identical OS/SDK/runtime — ambient drift on that GC-heavy workload; within-table comparisons are unaffected.🤖 Generated with Claude Code