Generators: a yield* delegation both re-delegates and keeps its place (backport of #3506 and #3518)

[lahma]

Backports #3506 and #3518 to 4.x. Both are silent-wrong-result defects in generator
resumption; together they are 16 lines of production code across four files.

Neither is theoretical on this branch. The ten tests the two PRs added were ported onto unmodified 4.x first, and
every one of them fails there — counts below.

What is wrong on 4.x today

#3506 — the yield memo is never invalidated. Jint resumes a generator by replaying its body from the top and
skipping the work earlier passes already did, using a memo of what each yield node last returned. Nothing ever removed
an entry, so the second time a loop reached the same yield node the memo answered it with the first iteration's
value — without evaluating its operand at all. For yield (yield* g()) that abandons the new delegation before it
starts, and when the operand carries the loop's own decrement the loop never ends:

function* countdown(n) {
    while (n > 0) {
        yield (yield* countdown(--n));   // the operand carries the decrement
    }
    return 34;
}
collect(countdown(3));   // 4.x: never returns from the sixth next()

That is staging/sm/generators/delegating-yield-9.js, which this branch removes from the exclusion list. Move the
decrement into its own statement and the loop terminates but answers with six results instead of eight — the silent half
of the same defect, which is why it gets its own test.

#3518 — a yield* delegation publishes no suspension node. Each statement on the way back into a replayed body has
to recognise it is being re-entered rather than entered, or a while re-runs the test that already chose the iteration
it is inside. JintWhileStatement, JintForStatement, JintDoWhileStatement, JintIfStatement, JintSwitchStatement
and JintTryStatement all ask JintStatement.GetSuspensionNode, which reads exactly one property:
ISuspendable.LastSuspensionNode. A delegation parks its suspension point in _delegatingYieldNode — necessarily a
separate field, because JintYieldExpression selects its delegation branch on that one and its plain-yield branch on
_lastYieldNode — and nothing published it. Every resume taken with a delegation in flight therefore looked to all six
statements like a resume that had suspended nowhere, and each re-ran its test against state the delegated code had
already moved.

The synchronous half needs no recursion to show it:

function* leaf() { yield 1; }
function* outer() {
    var log = [];
    var n = 2;
    while (n > 0) {
        n = 0;
        yield* leaf();
        log.push('after');
    }
    return log.join(',');
}
// node: 1, then 'after'.  4.x: 1, then ''.

The fix

One Remove on the fresh-evaluation path of JintYieldExpression, and one ?? _delegatingYieldNode fallback on each of
the two instance types:

object? ISuspendable.LastSuspensionNode => _lastYieldNode ?? _delegatingYieldNode;

It cannot answer with a stale node: _lastYieldNode is cleared the moment the yield it names is resumed, so it is null
for as long as a delegation is in flight, and _delegatingYieldNode is cleared when the delegation ends.

Per 14.4.14 every evaluation
of yield * AssignmentExpression evaluates its operand and drives the resulting iterator, and
AsyncGeneratorYield suspends the generator at the yield; neither
re-evaluates the iteration statement the yield* sits inside.

All four production hunks are byte-identical to the ones merged on main. JintYieldExpression.cs,
GeneratorInstance.cs and ISuspendable.cs are in fact the same blob as on main after the pick;
AsyncGeneratorInstance.cs differs only in pre-existing 4.x/main drift elsewhere in the file (two spec anchors in
unrelated <summary> comments), never in the changed lines.

Evidence: the ten tests on unmodified 4.x

Ported onto e556d274 with no production change, in Jint.Tests/Runtime/GeneratorTests.cs:

	before	after
net472	Failed: 11, Passed: 40, Total: 51	Failed: 0, Passed: 50, Total: 50
net10.0	Failed: 12, Passed: 40, Total: 52	Failed: 0, Passed: 50, Total: 50

Ten distinct test methods fail on both; the runner emits 10–12 records for them from run to run, because the cases
that do not terminate at all are recorded a second time when their ten-second timeout fires. Repeated before runs gave
10, 11 and 12 — the distinct set of ten never varied.

(Jint.Tests targets net472 and net10.0 on this branch. net462 is the Jint library's floor, not a test leg, so
the interpreter change is compiled for net462 by dotnet build and exercised on net472.)

As the runner reported them on unmodified 4.x — which is what "silently wrong" means in practice: a shorter sequence,
or the other branch, never an exception:

Failed ADelegatingYieldInsideAYieldKeepsItsPlaceWhenTheDecrementIsElsewhere
  Expected engine.Evaluate(Script) to be
    "34:false 34:false 34:false 34:false 34:false 34:false 34:false 34:true (String)",
  but found
    "34:false 34:false 34:false 34:false 34:false 34:true (String)".      6 results, not 8

Failed AnAsyncDelegatingYieldInsideAYieldKeepsItsPlaceWhenTheDecrementIsElsewhere
  "…34:false 34:true"                                       (actual)      4 results, not 8
  "…34:false 34:false 34:false 34:false 34:false 34:false…"  (expected)

Failed ADelegatingYieldSuspensionKeepsTheEnclosingWhileLoopFromRestarting
Failed AnAsyncDelegatingYieldSuspensionKeepsTheEnclosingWhileLoopFromRestarting
  "1:false :true"                                           (actual)      'after' never ran
  "1:false after:true"                                      (expected)

Failed ADelegatingYieldSuspensionKeepsTheEnclosingIfFromTakingTheOtherBranch
Failed AnAsyncDelegatingYieldSuspensionKeepsTheEnclosingIfFromTakingTheOtherBranch
  "1:false else:true"                                       (actual)      wrong branch on resume
  "1:false then:true"                                       (expected)

Failed ADelegatingYieldSuspensionKeepsTheEnclosingForLoopFromRestarting
Failed AnAsyncDelegatingYieldSuspensionKeepsTheEnclosingForLoopFromRestarting
  "1:false 1:false 1:false 1:false 1:false 1:false 1:false…" (actual)     only the guard ends it
  "1:false body:true"                                       (expected)

Failed ADelegatingYieldInsideAYieldStartsOverOnEveryLoopIteration           does not terminate (10 s)
Failed AnAsyncDelegatingYieldInsideAYieldStartsOverOnEveryLoopIteration     does not terminate (10 s)

Note the last pair: on 4.x the async recursive shape does not return either, where on main (with #3506 already
in) it returned four results. #3518's write-up says as much — "on 4.16.1 the same script does not return at all" — and
that is confirmed here. The two fixes are needed together on this branch.

Each test asserts the exact sequence of value:done pairs rather than a count, and drains through a twenty-result guard
so a regression that never terminates fails with a wrong string instead of hanging the run.

Differences from the two PRs on main

Faithful except where 4.x has no equivalent:

• Jint.Tests is xUnit here, NUnit on main. [Test, CancelAfter(10000)] → [Fact(Timeout = 10000)],
  TestContext.CurrentContext.CancellationToken → TestContext.Current.CancellationToken, and
  Options.ObserveCancellation → Options.CancellationToken. The token is threaded through the Drain helper as a
  parameter rather than read inside it, because xUnit's xUnit1069 analyzer requires the test method itself to
  reference TestContext.Current.CancellationToken for a Timeout to be able to bite.
• No docs/v5-migration.md on this branch, so both migration-guide hunks are dropped.
• No Jint/Runtime/Interpreter/AGENTS.md on this branch — 4.x has a single root AGENTS.md and no co-located
  ones — so Generators: a yield* delegation keeps its place in the statement around it (#3509) #3518's gotcha entry is dropped with it.

Production code and the test bodies themselves are otherwise unchanged.

Test runs

dotnet build -c Release clean, dotnet test -c Release green:

project	net472	net10.0
Jint.Tests	6758 passed / 0 failed / 4 skipped	6843 passed / 0 failed / 4 skipped
Jint.Tests.PublicInterface	1493 / 0 / 9	1500 / 0 / 9
Jint.Tests.CommonScripts	28 / 0 / 0	28 / 0 / 0
Jint.Tests.SourceGenerators	—	52 / 0 / 0

test262

Both runs in isolation on net10.0, same machine, nothing else running:

	passed	failed	skipped	total
before (e556d274)	102,495	0	189	102,684
after	102,497	0	187	102,684

Exactly +2 passed / −2 skipped: the two modes of staging/sm/generators/delegating-yield-9.js, which this branch removes
from the exclusion list, and nothing else. No other exclusion is touched.

---

Original PRs: #3506 and #3518 (the latter fixes #3509).

🤖 Generated with Claude Code

https://claude.ai/code/session_014W5mbjGhyvgAS4pivXoc4S