Conversation
…al is not refined into a pattern
`{a = 0}` (a CoverInitializedName) is only valid when the object literal
containing it is refined into an ObjectAssignmentPattern; §13.2.5.1 requires
an early error otherwise. The recorded shorthand-assign error was discarded
whenever its position fell within the converted left-hand side of an
assignment - regardless of whether the object literal holding it was actually
refined. `ToAssignable` does not descend into the object of a member
expression (a member expression is a valid assignment target on its own), so
in `[{a = 0}.x] = []` the literal stays an ObjectLiteral while the pending
error was dropped anyway. The head of a for-in/of statement dropped the
recorded error silently for the same reason.
`ConsumeCoverInitializedNameError` now decides this by looking at the
converted node: the error is only discarded when no CoverInitializedName is
left in it, otherwise it is reported after the left-hand side itself has been
validated. The search runs only when an error is actually pending within the
converted node, so the common code paths cost one extra comparison per
assignment expression and nothing else.
The original acornjs implementation has the same defect (verified with acorn
8.18.0), so this is a deliberate deviation from upstream. V8 rejects all of
the affected forms.
Fixes adams85#46
|
Damn, the JS grammar is full of surprises...😅 Not only is this issue present in the upstream project, but the latest Back to the matter at hand, the fix seems correct to me (at least, I couldn't find a construct that breaks it so far), but it also feels kind of brute-force and suboptimal performance-wise. I suspect there is another way to approach this problem where you catch these errors on the fly, in So, let's stick with this solution for now. I reported the issue in the upstream project. In case they come up with something better, we'll get back to this. |
|
The proposed solution had one major shortcoming, though: this whole problem also applies to object expressions with duplicate So, I made an attempt at fixing also this here: af572dc Could you (make Claude) take a look at it? |
|
Thanks for extending this — the I ran a differential against V8 (node 24 / V8 13.6) over 692 cases: 72 hand-written
The one remaining is Two things I'd like to raise, one of which I think is a blocker. 1.
|
| thread stack | survives | crashes | source size at crash |
|---|---|---|---|
| 256 KB | 773 | 789 | ~1.6 kB |
| 512 KB | 1803 | 1819 | ~3.6 kB |
| 1 MB (typical pool/request thread) | 3848 | 3864 | ~7.7 kB |
| 1.5 MB (default main thread) | 5820 | 5859 | ~11.7 kB |
The same shape with {a: 0} — no pending error, so no walk — parses fine at 50,000 links (100 kB), so the depth itself is not a parser problem. Nested arrays (the shape from #46) do hit StackGuard cleanly at n≈1000; member chains sail straight past it. It reproduces through =, for-in, for-of and for await-of, for both the shorthand and the __proto__ variant.
That matters for embedders parsing untrusted script (Jint does), because it is a process kill rather than a catchable exception.
I have a verified iterative rewrite that keeps your reported positions exactly: byte-identical output on all 672 differential rows, survives 500,000 links, survives 50,000 links on a 256 KB stack, and keeps Acornima.Tests (all four TFMs) and test262 green. The shape is the obvious one — an explicit ArrayList<Node> stack, sawProto still per node, and the offending properties collected so the earliest-in-source one is raised at the end (which is what the pre-order recursion found first, and it removes the dependence on traversal order). Happy to push it to this branch or open it against yours, whichever you prefer.
2. RaiseAtNext — the goal is right, but Raise(property.Start, …) gets there more cheaply
Reporting at the actual location is a real improvement over our first-recorded-position approximation, and for duplicate __proto__ it lands perfectly: 184 of 185 rejected rows match V8's position exactly. But for the shorthand error it is 0 of 177 — V8 reports at the start of the offending property, and AssignmentPattern.Left.End re-scans to the =, two characters later. The two paths also disagree with each other: ({a = 0}) reports at } (via CheckExpressionErrors, since c3f1663 moved the ShorthandAssign recording past the initializer) while [{a = 0}.x] = [] reports at =.
I tried replacing that one call with Raise(property.Start, InvalidCoverInitializedName): 79 rows change, 75 of them to an exact V8 match, none move away, no error-class or accept/reject changes, exact V8 agreement goes 456/671 → 531/671, and the full suite passes unchanged on all four TFMs (the theories assert ex.Description only, so positions aren't pinned). It also matches what the sibling __proto__ branch already does — and RaiseAtNext has exactly one caller, so the whole helper goes away.
That's worth doing on its own, because RaiseAtNext has two side effects I don't think are intended:
- It clears collected errors in tolerant mode.
ResetInternalends with_options._errorHandler.Reset(), which onParseErrorCollectoris_errors.Clear(). WithTolerant = trueand a collector,"use strict"; delete x; [{a = 0}.x] = [];ends withErrors.Count == 0, where the same source ending in({a = 0});or a__proto__pair ends with 1. - It mutates a shared, user-owned
TokenizerOptions.Parsertakesoptions.GetTokenizerOptions()without cloning, andnew Parser()shares the process-wideParserOptions.Default, so nulling_onComment/_onToken/_onRegExpis visible to any other parser built from the same options. With one thread parsing[{a = 0}.x] = []in a loop, another observedOnToken == nullon 0.59% of 123 M reads.
Smaller notes
- The
NOTE:aboveCheckAssignmentLhsErrorsis now stale — it says the reported position is the first recorded one and that V8 agrees, but the walk deliberately reports the offending occurrence.[{__proto__: a, __proto__: a, x = 0}, {__proto__: a, __proto__: a}.x] = []reports @53 where V8 reports @16, which is the better answer and worth saying so explicitly. - Nothing asserts the reported position anywhere, so the improvement
RaiseAtNextexists for has no regression net. A handful ofAssert.Equal(n, ex.Error.Index)rows would lock it in. c3f1663restoring the early return inParseExprSubscripts(and droppingCheckExpressionErrorsfromParseMaybeUnary's prefix branch) moves ~10++/--rows away from V8's message where master matched it —++{a = 0},({a = 0}++),[{a = 0} ++x]and the__proto__equivalents now report the early error instead of "Invalid left-hand side expression in prefix/postfix operation". You updated four test rows for this, so I assume it's deliberate; it also reverses the deviation the removed comment recorded as intentional. Both are errors either way, so it's cosmetic — just checking it's the trade you meant.[{a = 0}, {__proto__:1, __proto__:2}.x] = []is another row where acornima is arguably more correct than V8 (V8 reports the shorthand from the object it did refine); might be worth a row alongside yourE-series ones.Debug.Assert(!Unsafe.IsNullRef(ref destructuringErrors))is genuinely unnecessary today, butCheckAssignmentLhsErrorsdereferences unconditionally on its first line now, so restoring it is cheap insurance against a future caller.
Everything else looks right to me — the >= convertedNode.Start gate correctly generalises acorn's oldDoubleProto save/restore, the CheckKeyName and KeywordOperator edits are behaviour-neutral, and all 88 rows of ShouldHandleCoverInitializedNameEdgeCases still pass verbatim.
72d10d9 to
ae3b66c
Compare
…anket permission The last five entries under the STAGING EXCLUSIONS banner were all blocked on the parser, not on Jint. Three of them - the derived-constructor arrow eval files and bug1587574 - needed Acornima to have a switch for a direct super() at all, which adams85/acornima#44 adds; the other two are closed by adams85/acornima#48 and sebastienros#49 with no Jint change beyond deleting their entries. The wiring is the interesting half. Jint already parses eval code with AllowSuperOutsideMethod, which covers SuperProperty only, so eval("super()") in a derived constructor failed to parse instead of running. PerformEval (https://tc39.es/ecma262/#sec-performeval) permits a direct super() exactly when inDerivedConstructor is true, and EvalFunction already computes that flag for its own post-parse early error - so the flag is passed through to the parser rather than the option being switched on for every eval. That is why the adjusted ParserOptions are memoized in two slots now: the adjustment is no longer constant, and the variant that admits a super() must never reach an eval the specification forbids one in. The eval cache already validates the ParserOptions an entry was parsed with, so one source evaluated from both contexts gets one parse each. The option is scoped to the eval'd unit's this binding, so it reaches arrow functions declared in the eval but not ordinary ones - which is what lets Jint enable it for the whole eval and still have the parser refuse eval("(function () { super() })"), as V8 does. Acornima 1.8.0 does not exist yet: the three pull requests are open, and the version here is the expected next one. Everything was verified against a local Acornima built from master plus those three, which takes test262 from 102,495 passed / 189 skipped to 102,505 passed / 179 skipped, 0 failed either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHnbgQGW5QgfatxL6DgGEd
|
Verified downstream, since this is one of three PRs Jint is waiting on. A local Acornima built from The stack-overflow blocker is gone: |
…anket permission The last five entries under the STAGING EXCLUSIONS banner were all blocked on the parser, not on Jint. Three of them - the derived-constructor arrow eval files and bug1587574 - needed Acornima to have a switch for a direct super() at all, which adams85/acornima#44 adds; the other two are closed by adams85/acornima#48 and sebastienros#49 with no Jint change beyond deleting their entries. The wiring is the interesting half. Jint already parses eval code with AllowSuperOutsideMethod, which covers SuperProperty only, so eval("super()") in a derived constructor failed to parse instead of running. PerformEval (https://tc39.es/ecma262/#sec-performeval) permits a direct super() exactly when inDerivedConstructor is true, and EvalFunction already computes that flag for its own post-parse early error - so the flag is passed through to the parser rather than the option being switched on for every eval. That is why the adjusted ParserOptions are memoized in two slots now: the adjustment is no longer constant, and the variant that admits a super() must never reach an eval the specification forbids one in. The eval cache already validates the ParserOptions an entry was parsed with, so one source evaluated from both contexts gets one parse each. The option is scoped to the eval'd unit's this binding, so it reaches arrow functions declared in the eval but not ordinary ones - which is what lets Jint enable it for the whole eval and still have the parser refuse eval("(function () { super() })"), as V8 does. Acornima 1.8.0 does not exist yet: the three pull requests are open, and the version here is the expected next one. Everything was verified against a local Acornima built from master plus those three, which takes test262 from 102,495 passed / 189 skipped to 102,505 passed / 179 skipped, 0 failed either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHnbgQGW5QgfatxL6DgGEd
…anket permission The last five entries under the STAGING EXCLUSIONS banner were all blocked on the parser, not on Jint. Three of them - the derived-constructor arrow eval files and bug1587574 - needed Acornima to have a switch for a direct super() at all, which adams85/acornima#44 adds; the other two are closed by adams85/acornima#48 and sebastienros#49 with no Jint change beyond deleting their entries. The wiring is the interesting half. Jint already parses eval code with AllowSuperOutsideMethod, which covers SuperProperty only, so eval("super()") in a derived constructor failed to parse instead of running. PerformEval (https://tc39.es/ecma262/#sec-performeval) permits a direct super() exactly when inDerivedConstructor is true, and EvalFunction already computes that flag for its own post-parse early error - so the flag is passed through to the parser rather than the option being switched on for every eval. That is why the adjusted ParserOptions are memoized in two slots now: the adjustment is no longer constant, and the variant that admits a super() must never reach an eval the specification forbids one in. The eval cache already validates the ParserOptions an entry was parsed with, so one source evaluated from both contexts gets one parse each. The option is scoped to the eval'd unit's this binding, so it reaches arrow functions declared in the eval but not ordinary ones - which is what lets Jint enable it for the whole eval and still have the parser refuse eval("(function () { super() })"), as V8 does. Acornima 1.8.0 does not exist yet: the three pull requests are open, and the version here is the expected next one. Everything was verified against a local Acornima built from master plus those three, which takes test262 from 102,495 passed / 189 skipped to 102,505 passed / 179 skipped, 0 failed either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHnbgQGW5QgfatxL6DgGEd
|
Closing in favor of #52 |
…anket permission The last five entries under the STAGING EXCLUSIONS banner were all blocked on the parser, not on Jint. Three of them - the derived-constructor arrow eval files and bug1587574 - needed Acornima to have a switch for a direct super() at all, which adams85/acornima#44 adds; the other two are closed by adams85/acornima#48 and sebastienros#49 with no Jint change beyond deleting their entries. The wiring is the interesting half. Jint already parses eval code with AllowSuperOutsideMethod, which covers SuperProperty only, so eval("super()") in a derived constructor failed to parse instead of running. PerformEval (https://tc39.es/ecma262/#sec-performeval) permits a direct super() exactly when inDerivedConstructor is true, and EvalFunction already computes that flag for its own post-parse early error - so the flag is passed through to the parser rather than the option being switched on for every eval. That is why the adjusted ParserOptions are memoized in two slots now: the adjustment is no longer constant, and the variant that admits a super() must never reach an eval the specification forbids one in. The eval cache already validates the ParserOptions an entry was parsed with, so one source evaluated from both contexts gets one parse each. The option is scoped to the eval'd unit's this binding, so it reaches arrow functions declared in the eval but not ordinary ones - which is what lets Jint enable it for the whole eval and still have the parser refuse eval("(function () { super() })"), as V8 does. Acornima 1.8.0 does not exist yet: the three pull requests are open, and the version here is the expected next one. Everything was verified against a local Acornima built from master plus those three, which takes test262 from 102,495 passed / 189 skipped to 102,505 passed / 179 skipped, 0 failed either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHnbgQGW5QgfatxL6DgGEd
…r the last five staging exclusions (#3277) * Eval: hand PerformEval's inDerivedConstructor to the parser, not a blanket permission The last five entries under the STAGING EXCLUSIONS banner were all blocked on the parser, not on Jint. Three of them - the derived-constructor arrow eval files and bug1587574 - needed Acornima to have a switch for a direct super() at all, which adams85/acornima#44 adds; the other two are closed by adams85/acornima#48 and #49 with no Jint change beyond deleting their entries. The wiring is the interesting half. Jint already parses eval code with AllowSuperOutsideMethod, which covers SuperProperty only, so eval("super()") in a derived constructor failed to parse instead of running. PerformEval (https://tc39.es/ecma262/#sec-performeval) permits a direct super() exactly when inDerivedConstructor is true, and EvalFunction already computes that flag for its own post-parse early error - so the flag is passed through to the parser rather than the option being switched on for every eval. That is why the adjusted ParserOptions are memoized in two slots now: the adjustment is no longer constant, and the variant that admits a super() must never reach an eval the specification forbids one in. The eval cache already validates the ParserOptions an entry was parsed with, so one source evaluated from both contexts gets one parse each. The option is scoped to the eval'd unit's this binding, so it reaches arrow functions declared in the eval but not ordinary ones - which is what lets Jint enable it for the whole eval and still have the parser refuse eval("(function () { super() })"), as V8 does. Acornima 1.8.0 does not exist yet: the three pull requests are open, and the version here is the expected next one. Everything was verified against a local Acornima built from master plus those three, which takes test262 from 102,495 passed / 189 skipped to 102,505 passed / 179 skipped, 0 failed either way. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHnbgQGW5QgfatxL6DgGEd * Eval: follow Acornima's rename of the option to AllowSuperCallOutsideConstructor adams85/acornima#44 merged as 09c08b3, and the option we contributed as ParserOptions.AllowDirectSuperOutsideMethod shipped under a different name: ParserOptions.AllowSuperCallOutsideConstructor. The new name says what the option admits (a direct super *call*) and where the code it is parsing came from (the *constructor* of a derived class, PerformEval's inDerivedConstructor) rather than naming the parser-internal AllowDirectSuper flag it seeds, which also reads better beside the AllowSuperOutsideMethod it implies. Nothing but the name changed: the option still defaults to false, still seeds the root scope with Super | DirectSuper, and so still follows the eval'd unit's this binding - reaching arrow functions declared in the eval but not ordinary ones. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FHnbgQGW5QgfatxL6DgGEd * Test262: move the pin to 14e8c908, the upstream tip Thirteen commits since 3655e746, eleven of them CI tables and tooling. The two that touch test/ are ac7b5f8c, which adds negative parse tests for a braced quantifier whose lower bound exceeds its upper bound (/a{2,1}/ and /a{2,1}/u), and 14e8c908, which lists testTypedArray.js once instead of twice in four includes lines. The parser already rejects the quantifier at parse time, and the RegExp constructor rejects it under every flag, so nothing in the engine moves: +4 passed, +4 total, skipped unchanged. features.txt is unchanged. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fixes #46.
{ a = 0 }is only legal as a cover for anObjectAssignmentPattern. When the enclosing left-hand side is refined into an assignment pattern the literal becomes a pattern and the shorthand-with-initializer is fine; when it is not refined — because it is the object of a member expression, which is already a valid assignment target on its own — the literal stays anObjectLiteraland §13.2.5.1's early error must fire. All nine forms in the issue parsed clean, where V8 rejects every one.Root cause
ReinterpretAssignmentTargetdiscarded the pendingShorthandAssignerror whenever its position fell inside the converted left-hand side — a position test standing in for a refinement test.ToAssignabledeliberately stops at aMemberExpression(it is already a validDestructuringAssignmentTarget) and never descends into its object, so in[{a = 0}.x] = []the literal stays anObjectExpressionwhile the pending error is dropped anyway.The for-in/of heads have the same hole for a different reason: they never consult the pending error at all, so it is silently discarded when
destructuringErrorsgoes out of scope.Why the reset line could not simply be deleted
I built that variant to check, and it is worth recording because it is the obvious first attempt: deleting the reset breaks legal code —
({a = 0} = {});and({a: {b = 0}} = {});, ordinary destructuring-with-defaults, start throwing — and it still misses five of the nine forms, because the=branch ofParseMaybeAssignreturns without ever callingCheckExpressionErrors, so an un-reset error is never reported, and the for-in/of heads are separate sites.So the fix has to decide refinement from the converted node rather than from a position, raise the error explicitly, and cover the for-in/of heads.
ConsumeCoverInitializedNameErrordoes the first two; the decision procedure rests on an invariant worth stating plainly:So "was it refined?" is answered by searching the converted node for such a node.
The error is raised after
CheckLValPattern/CheckLValSimple, so an invalid left-hand side still reports first — which preserves the existing({x = yield} += 1)→ Invalid left-hand side in assignment pin, and matches what V8 does in most of these cases.Cost
Nothing is added to per-node work.
ConsumeCoverInitializedNameErroropens with the sameShorthandAssign < node.Startcomparison the old reset used and returns immediately when nothing is pending, so the common path pays one compare-and-branch per assignment expression and per expression-headed for-in/of.ContainsCoverInitializedNameruns only when a CoverInitializedName was actually recorded inside the converted node — i.e. either an about-to-be-reported error, or a destructuring assignment genuinely using shorthand defaults — and is then bounded by that left-hand side's size. It is iterative with an explicitArrayList<Node>stack, so it adds no depth to the parser's stack-guarded recursion.Measured on the repo's own eight benchmark files (best-of-15, 5×8 parses, alternating pre/post processes): pre median 413.7 ms, post median 399.6 ms — the post median is lower, which the change cannot cause, so the effect is below run-to-run noise. An 800-deep nested array pattern resolves correctly both ways; depth 1600 raises
InsufficientExecutionStackExceptionon both builds, which is the pre-existingStackGuard.Verification
New theory
ShouldHandleCoverInitializedNameEdgeCases, 88 rows (44 sources × script/module): the issue's 9 + 5 + 3 matrix plus 27 further forms — mixed refined/unrefined in both orders, nested arrays and objects,for-in, plainfor-init,??=, sequence, chained assignment, parenthesized, and arrow-parameter controls. Written first and run against the unfixed parser: 40 failed, 48 passed, every failureAssert.Throws() Failure: No exception was thrown— the predicted mode.After: 88/88. Full suite
13098(net10.0) /13095(net9.0, net8.0, net462); test262 99090 passing, 0 failed; source generators 2/2; and the Debug configuration run too, since the newDebug.Assertonly exists there.Differential against V8 (node 24.19.0) over 684 generated forms — 36 shapes × 19 syntactic contexts:
Every one of the 165 behaviour changes is
accept → reject; nothing that already agreed with V8 moved.Two things deliberately left alone
The reported position stays Acornima's own — the
=token of the shorthand property, where V8 points at the key — and in the mixed case ([{a = 0}, {b = 0}.x] = []) it is the first recorded shorthand assign, which is not necessarily the one that stayed unrefined. V8 reports the first one there too. Making it exact would mean recording every position rather than the first, growingDestructuringErrors.Two pre-existing ordering divergences from V8 are preserved rather than "fixed":
[{a = 0}.x] += 1and({a = 0} += 1)report Invalid left-hand side in assignment where V8 reports the shorthand error, and({a = 0}.x) => 0reports Illegal property in declaration context. Changing them would contradict the existingShouldHandleVariableAssignmentEdgeCasespins. All three carry// V8 reports "…"comments in the new theory.Note the issue's finding that acorn 8.18.0 shares this defect, so this is not a port of an upstream fix — happy to adjust the approach if you would rather it track acorn.
Found while working on Jint; it is one of the two parser gaps blocking test262
staging/there (sebastienros/jint#3021). I hand-checkedstaging/sm/destructuring/bug1396261.js: all 10 must-parse statements parse and all 4 must-throw sources now throw. Acornima's ownTest262Harness.settings.jsondoes not includestaginginSubDirectories, which is why the suite stayed green on this — adding those ~2,800 cases felt like a separate change with its own triage rather than something to fold in here.