Skip to content

Account memory across async continuations - #3036

Merged
lahma merged 1 commit into
mainfrom
sebros/improve-memory-accounting
Aug 21, 2026
Merged

lahma merged 1 commit into
mainfrom
sebros/improve-memory-accounting

Conversation

@sebastienros

Copy link
Copy Markdown
Owner

Summary

  • account managed allocations per engine operation across synchronous segments and thread-changing async continuations
  • carry memory state through promise reactions, event-loop jobs, and asynchronous module completion without charging unrelated process allocations
  • add explicit accuracy/usage diagnostics and Begin/End multi-entry scopes guarded by the engine ownership contract
  • document guarantees and residual risks in the README and threat model

Stack

This PR is stacked on #3035 (sebros/reject-concurrent-engine-use), which is stacked on #3030 (initial-threat-model). Review only the diff against sebros/reject-concurrent-engine-use.

Validation

  • dotnet build Jint/Jint.csproj -c Release across all target frameworks
  • dotnet test Jint.Tests/Jint.Tests.csproj -c Release excluding the known unrelated timezone-sensitive year-10000 test: 5,760 passed, 4 skipped
  • dotnet test Jint.Tests.PublicInterface/Jint.Tests.PublicInterface.csproj -c Release: 1,529 passed, 9 configuration skips
  • host-contract verification public suite: 1,533 passed, 5 inverse-configuration skips
  • combined memory/concurrency stress selection repeated 5 times: 41 passed per run
  • dotnet build Jint.Benchmark/Jint.Benchmark.csproj -c Release
  • git diff --check origin/sebros/reject-concurrent-engine-use...HEAD

@lahma lahma left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed as the incremental diff against sebros/reject-concurrent-engine-use. The segment/operation-state design is sound: per-thread counters sampled only inside synchronous segments, the accumulated total carried by the reaction/job/module-completion records, and the three entry lifecycles (host entry, event-loop job, implicit execution-context) each bracketing cleanly. The 25 HostMemoryLimitTests hit exactly the edge cases I went looking for — the final-callback allocation with no later statement check, exceeded-state not replayed by an idle Check(), nested entries sharing the outermost budget, per-engine isolation on shared Options, and the cross-thread rejection surface. Docs (README, threat model, AGENTS.md contract table and the constraints gotcha) were all kept in step, and LimitMemory's sentinel semantics are preserved while the now-public constructor validates its range. Approving.

Three non-blocking notes:

  1. Unconstrained-path cost. EnterExecutionContext/LeaveExecutionContext are per-function-call hot paths and now carry a _memoryLimitConstraint is not null / _implicitMemoryContextDepth != 0 branch even for engines with no memory limit, and ExecuteWithConstraints gained the _hostEntryDepth term. These should be branch-predictor noise, but since the suite already has the right instrument, it would be good to attach one JINT_BENCH_MODE=gate run of ConstrainedExecutionBenchmark's all-false row (base vs. this branch) when you re-roll the stack on top of the #3035 fix.
  2. isNested semantics tightened for everyone, not just memory-limited engines: engine.Invoke(hostFn) where the CLR function re-enters the engine used to re-arm budgets (no interpreter frame on the stack, so _executionContexts.Count == 1); with _hostEntryDepth > 0 in the disjunction it no longer does. I read that as a bug fix in the safe direction, and the AGENTS.md gotcha was updated to match — just calling it out so it's a deliberate decision on record, since HostCallLoopConstraintTests pins this area.
  3. Modules.StartImport now funnels through ExecuteWithConstraints, so it resets ordinary constraints as a top-level entry where it previously didn't. Consistent with treating it as an entry; worth a line in the PR body.

Merge is blocked beneath this by the #3035 admission-token leak; once that lands, this layer is good to go after the cascade rebase.

Base automatically changed from sebros/reject-concurrent-engine-use to main August 21, 2026 13:00
@lahma

lahma commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

#3035 is merged (8b980dc71), and this PR is now the bottom of the stack, auto-retargeted to main. The server-side cascade could not rewrite the branch, and after looking at it I don't think any mechanical rebase can — handing this back rather than guessing:

The twelve approvals stand for the designs. Once the stack is cascaded onto post-#3035 main I'll re-review each layer's refreshed diff and merge bottom-up as each goes green.

@lahma
lahma force-pushed the sebros/improve-memory-accounting branch from bce249f to 7719ee8 Compare August 21, 2026 17:20
@lahma
lahma merged commit fd50151 into main Aug 21, 2026
5 checks passed
@lahma
lahma deleted the sebros/improve-memory-accounting branch August 21, 2026 17:33
@sebastienros

Copy link
Copy Markdown
Owner Author

Thanks for stopping rather than guessing — the conflicts were semantic exactly as described.

I replayed the memory layer onto current main, including the merged #3035 ownership protocol, pump-driven Atomics/timers, the expanded web task sources, host-stream bridges, and the latest QuotaExceededError surface. During the refresh #3036 was merged server-side, so I preserved the post-merge fixes as follow-up #3216: #3216

Key resolutions in #3216:

  • finite continuations retain their originating allocation state (promises/modules/Atomics, fetch/streams, timers, idle callbacks and individual scheduler tasks);
  • persistent external event sources start a fresh per-delivery budget rather than accumulating forever against object creation;
  • transferred callbacks suspend/restore the original thread segment and account callback-thread allocations, including nested transfers;
  • host Task failures preserve explicitly marked Jint constraint provenance through aggregates without treating ordinary host TimeoutException as fatal;
  • a memory breach clears transient work, abandons stream bridges and advances the generation in finally, while retaining cleanup failures as InnerException;
  • host-stream I/O ends its Busy state on the completion thread before any generation/memory-gated delivery;
  • current-main QuotaExceededError, WPT and web API changes remain intact.

Validation: all-target Release build; 9,810 unit tests passed / 5 skipped (known local timezone test excluded); public interface 2,105 passed / 9 config skips; host-contract public interface 2,109 passed / 5 inverse skips; focused 112-test memory/concurrency/web selection passed three times; post-rebase focused suites passed 352 + 112. The benchmark restore is blocked only because the mandated proxy lacks YantraJS.Core 1.2.462, so I did not use another feed.

lahma added a commit to lahma/jint that referenced this pull request Aug 21, 2026
…r's budget

`docs/design/web-workers.md` predates the finalized plan in sebastienros#3167 and disagreed
with it in nine places that matter. Rewritten decision for decision:

- `close()` is not `terminate()`. The spec's *close a worker* discards the
  worker's own queued tasks and sets the closing flag; it pointedly does not
  abort the running script or empty the parent-side queue. The doc had them
  sharing one three-step teardown, which would kill `close(); flushMetrics();`
  and lose `postMessage(result); close()`.
- A load failure fires a plain `Event`, not an `ErrorEvent`, and ends the
  connection as `StartupFailed` rather than `Terminated`.
- The parent-side error relay is an internal per-connection hook gated on
  *notHandled*, sitting beside the `DiagnosticsSink` rather than being it — the
  sink is deliberately unsuppressible by script and cannot carry that gate.
- Nesting is OFF by default; `MaxWorkers` (16) is a per-engine backstop that
  bounds a branching factor, not a tree, which is what the old text claimed.
- Security posture: restrictions travel, grants never travel by implication,
  and the classification is pinned reflectively.
- `importScripts` is present and throwing, because the spec's own step 1 for a
  module worker prescribes the throw.
- The agent-cluster claim is corrected: HTML puts a dedicated worker in its
  creator's cluster, so refusing `SharedArrayBuffer` is Jint's isolation policy
  and not a fact about agents.
- "Close both ports is already the shipped rule" is withdrawn: the shipped rule
  is the opposite one-sided close, and the worker rule is argued on its own
  merits (cost, not delivery).
- The generation fence is per **port**, captured at port construction, which is
  what lets a transferred side rejoin the receiving engine's current cycle.
- The `AWorkerCannotBeSentAMessagePort` pin is deleted; it contradicted sebastienros#3197.

Line-number citations are replaced with file-and-member ones throughout. Most
of the old numbers were already wrong, and a wrong line number reads like a
fact. sebastienros#3215's transferable-streams paragraph is preserved, integrated into the
transfer section beside the port transfer it rides on.

The same correction lands in the XML docs, where it is load-bearing rather than
merely explanatory. `WorkerRequest.CreateDefaultOptions` claimed a replayed
`LimitMemory` never fires on a pumped worker. Since sebastienros#3036 that is false: every
event-loop job runs inside an allocation segment and is checked as it completes,
carrying the operation state captured at registration across continuations and
thread hops, so a replayed factory genuinely bounds each job chain. The worker
budget is a pair — `OperationDeadlineConstraint` for time, `MemoryLimitConstraint`
for allocations. The same change cites sebastienros#3036's other consequence: instance
copying is now loudly wrong, because `MemoryLimitConstraint.Attach` throws when
one instance meets a second engine.

`Options`'s wholesale exclusion of the `WebApi` group also stops glossing: its
value settings are per-feature caps living on the sub-groups, and each rides
with the feature it bounds rather than being a restriction that stayed behind
by accident.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lahma added a commit to lahma/jint that referenced this pull request Aug 21, 2026
…r's budget

`docs/design/web-workers.md` predates the finalized plan in sebastienros#3167 and disagreed
with it in nine places that matter. Rewritten decision for decision:

- `close()` is not `terminate()`. The spec's *close a worker* discards the
  worker's own queued tasks and sets the closing flag; it pointedly does not
  abort the running script or empty the parent-side queue. The doc had them
  sharing one three-step teardown, which would kill `close(); flushMetrics();`
  and lose `postMessage(result); close()`.
- A load failure fires a plain `Event`, not an `ErrorEvent`, and ends the
  connection as `StartupFailed` rather than `Terminated`.
- The parent-side error relay is an internal per-connection hook gated on
  *notHandled*, sitting beside the `DiagnosticsSink` rather than being it — the
  sink is deliberately unsuppressible by script and cannot carry that gate.
- Nesting is OFF by default; `MaxWorkers` (16) is a per-engine backstop that
  bounds a branching factor, not a tree, which is what the old text claimed.
- Security posture: restrictions travel, grants never travel by implication,
  and the classification is pinned reflectively.
- `importScripts` is present and throwing, because the spec's own step 1 for a
  module worker prescribes the throw.
- The agent-cluster claim is corrected: HTML puts a dedicated worker in its
  creator's cluster, so refusing `SharedArrayBuffer` is Jint's isolation policy
  and not a fact about agents.
- "Close both ports is already the shipped rule" is withdrawn: the shipped rule
  is the opposite one-sided close, and the worker rule is argued on its own
  merits (cost, not delivery).
- The generation fence is per **port**, captured at port construction, which is
  what lets a transferred side rejoin the receiving engine's current cycle.
- The `AWorkerCannotBeSentAMessagePort` pin is deleted; it contradicted sebastienros#3197.

Line-number citations are replaced with file-and-member ones throughout. Most
of the old numbers were already wrong, and a wrong line number reads like a
fact. sebastienros#3215's transferable-streams paragraph is preserved, integrated into the
transfer section beside the port transfer it rides on.

The same correction lands in the XML docs, where it is load-bearing rather than
merely explanatory. `WorkerRequest.CreateDefaultOptions` claimed a replayed
`LimitMemory` never fires on a pumped worker. Since sebastienros#3036 that is false: every
event-loop job runs inside an allocation segment and is checked as it completes,
carrying the operation state captured at registration across continuations and
thread hops, so a replayed factory genuinely bounds each job chain. The worker
budget is a pair — `OperationDeadlineConstraint` for time, `MemoryLimitConstraint`
for allocations. The same change cites sebastienros#3036's other consequence: instance
copying is now loudly wrong, because `MemoryLimitConstraint.Attach` throws when
one instance meets a second engine.

`Options`'s wholesale exclusion of the `WebApi` group also stops glossing: its
value settings are per-feature caps living on the sub-groups, and each rides
with the feature it bounds rather than being a restriction that stayed behind
by accident.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lahma added a commit that referenced this pull request Aug 21, 2026
#3219)

* Workers: the host-facing types, the options group and the posture rule

The foundation of the Worker feature (#3167), with no script surface at all:
`typeof Worker` stays `undefined` on every engine, nothing is installed, and
no engine runtime behaviour changes.

What lands:

- `Jint/WebApi/Workers/` — `WorkerProvider` (the host's answer to
  `new Worker(...)`, since Jint never starts a thread), `WorkerRequest`
  (what is being asked for, plus `CreateDefaultOptions()`), `WorkerConnection`
  (one live parent-worker pair) and the two enums. The connection is
  thread-safe now rather than later: `End()` is idempotent under *concurrent*
  callers, every property reads safely from any thread, and the end callback
  runs outside the lock so host code never runs under one.
- `Options.WebApi.Workers` — `Provider` (null by default, which is what keeps
  the global uninstalled), `MaxWorkers` (16, a per-engine backstop and not the
  policy) and `MaxQueuedMessages` (16384, a backstop against a never-pumped
  worker rather than flow control), plus `WebApiFeatures.Workers = 1 << 24`,
  never in `Default`, and `UseWorkers` which sets flag and provider together.
- `Options.CopySecurityPosture` — restrictions travel, grants never travel by
  implication. It copies the seven `Constraints` value settings,
  `Host.StringCompilationAllowed`, `AgentCanSuspend` and `Json.MaxParseDepth`,
  and lives beside the properties it names with three explicit
  classification lists: what is inherited, what is deliberately not (with the
  reason), and which option groups are excluded wholesale as grant-shaped.

`CreateDefaultOptions()` returns a fresh `Options` each call: the termination
token as a cancellation constraint, every parent constraint *factory* replayed
(never an instance — those are single-engine-only), the copied posture, the
feature mask minus network/storage/routing/nesting plus `Messaging` and
`GlobalEvents`, and `DiagnosticsSink.Null`.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Workers: pin the foundation, and the classification that outlives it

Mechanism in Jint.Tests, third-party reachability in Jint.Tests.PublicInterface,
each pin written so that removing the mechanism it names makes it fail.

`OptionsSecurityPostureTests` is the one worth keeping past this feature: it
reflects over the value-typed settings of `Options` and its `Constraints`,
`Host` and `Json` groups and fails unless each is either copied by
`CopySecurityPosture` or named as deliberately not inherited, and it does the
same for the option groups themselves. It found one on its first run —
`Options.Profiling`, which is now classified (host diagnostics, script cannot
reach it, and its gate already defaults to refusing).

`TheDefaultOptionsCopyTheParentsSecurityPosture` is a theory with one case per
copied setting, so dropping a single line of the copy fails exactly the case
that names it rather than one arbitrary assertion.

`TypeofWorkerIsUndefinedEvenWithTheFlagAndProvider` pins that this change is
not also the one that moves the engine: with the flag on, a provider
registered and the web APIs enabled, a script still cannot tell any of this
exists.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Workers: sync the design doc, and tell the truth about a pumped worker's budget

`docs/design/web-workers.md` predates the finalized plan in #3167 and disagreed
with it in nine places that matter. Rewritten decision for decision:

- `close()` is not `terminate()`. The spec's *close a worker* discards the
  worker's own queued tasks and sets the closing flag; it pointedly does not
  abort the running script or empty the parent-side queue. The doc had them
  sharing one three-step teardown, which would kill `close(); flushMetrics();`
  and lose `postMessage(result); close()`.
- A load failure fires a plain `Event`, not an `ErrorEvent`, and ends the
  connection as `StartupFailed` rather than `Terminated`.
- The parent-side error relay is an internal per-connection hook gated on
  *notHandled*, sitting beside the `DiagnosticsSink` rather than being it — the
  sink is deliberately unsuppressible by script and cannot carry that gate.
- Nesting is OFF by default; `MaxWorkers` (16) is a per-engine backstop that
  bounds a branching factor, not a tree, which is what the old text claimed.
- Security posture: restrictions travel, grants never travel by implication,
  and the classification is pinned reflectively.
- `importScripts` is present and throwing, because the spec's own step 1 for a
  module worker prescribes the throw.
- The agent-cluster claim is corrected: HTML puts a dedicated worker in its
  creator's cluster, so refusing `SharedArrayBuffer` is Jint's isolation policy
  and not a fact about agents.
- "Close both ports is already the shipped rule" is withdrawn: the shipped rule
  is the opposite one-sided close, and the worker rule is argued on its own
  merits (cost, not delivery).
- The generation fence is per **port**, captured at port construction, which is
  what lets a transferred side rejoin the receiving engine's current cycle.
- The `AWorkerCannotBeSentAMessagePort` pin is deleted; it contradicted #3197.

Line-number citations are replaced with file-and-member ones throughout. Most
of the old numbers were already wrong, and a wrong line number reads like a
fact. #3215's transferable-streams paragraph is preserved, integrated into the
transfer section beside the port transfer it rides on.

The same correction lands in the XML docs, where it is load-bearing rather than
merely explanatory. `WorkerRequest.CreateDefaultOptions` claimed a replayed
`LimitMemory` never fires on a pumped worker. Since #3036 that is false: every
event-loop job runs inside an allocation segment and is checked as it completes,
carrying the operation state captured at registration across continuations and
thread hops, so a replayed factory genuinely bounds each job chain. The worker
budget is a pair — `OperationDeadlineConstraint` for time, `MemoryLimitConstraint`
for allocations. The same change cites #3036's other consequence: instance
copying is now loudly wrong, because `MemoryLimitConstraint.Attach` throws when
one instance meets a second engine.

`Options`'s wholesale exclusion of the `WebApi` group also stops glossing: its
value settings are per-feature caps living on the sub-groups, and each rides
with the feature it bounds rather than being a restriction that stayed behind
by accident.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Workers: make the End()-idempotency pin actually race

The pin claimed to prove that `WorkerConnection.End()` is idempotent under
*concurrent* callers rather than merely repeated ones — which is the entire
reason `TryEnd` takes a lock instead of testing and setting a `volatile bool`.
It did not prove it. Replacing the lock with a plain check-then-set left the
test passing five times out of five: `Parallel.For` ramps its workers up, so by
the time the second one starts the first has long since set the flag and there
is no race left to lose. The pin named a mechanism it could not see.

A `Barrier` releases every racer into the same instant instead, the threads are
reused across 64 rounds so that many attempts cost a handful of threads rather
than thousands, and each round gets its own connection and its own callback
counter — so the failure message names the round rather than an aggregate. The
unlocked version now loses within the first couple of rounds, verified five
times out of five, and the assertion that fails is the callback count, which is
exactly the mechanism removed.

Nothing here is timed: there is no wall-clock assertion, so a slow machine makes
this test slower and never redder, and the joins carry a 60-second ceiling only
so that an `End()` that deadlocked fails the test instead of hanging the run.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* Workers: classify the security settings the stack landed under us

Four security-stack layers (#3037, #3045, #3046, #3051) merged between this
branch's base and its landing, and each added exactly the shape of setting the
posture rule exists to classify. The parser bounds, the four module-graph
limits and the ResultLimits bundle are restrictions and travel; detailed-error
exposure and the require shim are grants and stay behind. Modules stops being
excludable wholesale — the group now mixes its loader (a grant) with numeric
limits (restrictions), so it joins the scanned set with each member classified.
ResultLimits is class-typed and outside the reflective pin's value-typed scan,
so it is classified in CopySecurityPosture's body instead, and the copy is by
reference because the bundle is sealed and immutable.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PnghUC3EKrYa12xQ2LaYyd

* Workers: inherit the untrusted-code profile's expansion, never its marker

#3060 landed the hardened profile while this branch was in review, and it
resolved a question the design had left open in the profile's favour — almost.
An untrusted parent's Engine.Options is the profile's expanded clone, so the
posture copy already takes every value the expansion set and the factory replay
already carries its budget constraints; propagating the marker itself would be
strictly worse, because a marked options object re-expands at engine
construction and that expansion clears the constraint registrations, including
the cancellation constraint a worker's terminate() depends on. The unmarked
worker loses only the diagnostics label, and the test pins both halves.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PnghUC3EKrYa12xQ2LaYyd

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
lahma added a commit that referenced this pull request Aug 22, 2026
The rebase across the eighteen commits since #3036 leaves five things that had
to change, plus two defects the refresh introduced.

Reconciliation. `ConstraintFailure.MustPropagate` composes over
`Throw.MustPropagateHostException` on main, and that list already carries a
marker — `Throw.TimeoutException` and `Throw.OperationCanceledException` register
the exception they mint in `EngineAbortRegistry`, so Jint's own timeout and
cancellation are already distinguishable from a host's. The refresh brought a
second marker of its own (an `Exception.Data` key, `Throw.Constraint*Exception`
duplicates of three throw helpers, a `MemoryLimitPlatformNotSupportedException`
subclass) and a standalone list to read it with. All of that is gone; what it was
for is kept, in the one place it was actually needed: the faulted-Task scan in
`JsValue.ConvertTaskToPromise` reads `Throw.MustPropagateHostException` rather
than `ConstraintFailure.MustPropagate`, because for a queued turn a
`TimeoutException` is Jint's own and nothing else, while for a host Task it is an
`HttpClient` timeout and must stay a promise rejection.

`Constraint.BeforeFailure` is gone too: the same refresh added
`Engine.CheckConstraint`, which does the same job at the call site and covers
user-derived constraints the hook could never reach. It is now a `try` around
each of the three constraint loops rather than a wrapper per constraint, so the
non-throwing path is unchanged instruction for instruction, and the filter
excludes `JavaScriptException` — script can catch that one, and abandoning the
operation under it would hand the script a fresh budget.

The five `AddToEventLoop` overloads collapse to three: `EventLoopRegistration`
now carries every deferred registration, so `(Action, int, OperationState?)` and
`EnqueueModuleLoadCompletion`'s triple go away, and `ModuleLoadCompletion` and
the `Atomics.waitAsync` waiter each lose a field. `HostStreamBridge` hands the
registration back from `TryBeginOperation` instead of parking it in a field for
the next line to read, and its now-unread `Generation` goes with it.
`GetHostCallbackOwner` no longer returns an owner, so it is
`GetHostCallbackAuthorization`.

Defects. A `setInterval` inherited the budget of the operation that scheduled it
and charged every tick to it, so an engine whose heartbeat allocates a few bytes
a second died after enough ticks — the refresh's own rule says repeating work
starts fresh, and its docs said timers were finite continuations. A repeating
`TimerEntry` now keeps the cycle stamp and drops the budget, which is what every
persistent event source already does; `AnIntervalStartsAFreshBudgetOnEveryTick`
fails without it. And `MemoryLimitConstraint`'s teardown runs host code — a
worker host told its connection ended, a host stream disposed — which may
re-enter the engine and find a fresh operation over the same budget, so it is
guarded to run once.

Two tests join it: an engine is usable again after a teardown, and a teardown
discards the timers the failed operation scheduled.

Co-authored-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VecfHbij4EMQFzL7NCpdWW
lahma added a commit that referenced this pull request Aug 22, 2026
* Refresh memory accounting for current main

Carry allocation state through the pump-driven web continuations, preserve persistent-event task boundaries, coordinate transferred callback threads, and abort transient work safely when a memory operation fails.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 0d07275e-227c-4672-becd-4fe336386da4

* Reconcile the refresh with what main grew under it

The rebase across the eighteen commits since #3036 leaves five things that had
to change, plus two defects the refresh introduced.

Reconciliation. `ConstraintFailure.MustPropagate` composes over
`Throw.MustPropagateHostException` on main, and that list already carries a
marker — `Throw.TimeoutException` and `Throw.OperationCanceledException` register
the exception they mint in `EngineAbortRegistry`, so Jint's own timeout and
cancellation are already distinguishable from a host's. The refresh brought a
second marker of its own (an `Exception.Data` key, `Throw.Constraint*Exception`
duplicates of three throw helpers, a `MemoryLimitPlatformNotSupportedException`
subclass) and a standalone list to read it with. All of that is gone; what it was
for is kept, in the one place it was actually needed: the faulted-Task scan in
`JsValue.ConvertTaskToPromise` reads `Throw.MustPropagateHostException` rather
than `ConstraintFailure.MustPropagate`, because for a queued turn a
`TimeoutException` is Jint's own and nothing else, while for a host Task it is an
`HttpClient` timeout and must stay a promise rejection.

`Constraint.BeforeFailure` is gone too: the same refresh added
`Engine.CheckConstraint`, which does the same job at the call site and covers
user-derived constraints the hook could never reach. It is now a `try` around
each of the three constraint loops rather than a wrapper per constraint, so the
non-throwing path is unchanged instruction for instruction, and the filter
excludes `JavaScriptException` — script can catch that one, and abandoning the
operation under it would hand the script a fresh budget.

The five `AddToEventLoop` overloads collapse to three: `EventLoopRegistration`
now carries every deferred registration, so `(Action, int, OperationState?)` and
`EnqueueModuleLoadCompletion`'s triple go away, and `ModuleLoadCompletion` and
the `Atomics.waitAsync` waiter each lose a field. `HostStreamBridge` hands the
registration back from `TryBeginOperation` instead of parking it in a field for
the next line to read, and its now-unread `Generation` goes with it.
`GetHostCallbackOwner` no longer returns an owner, so it is
`GetHostCallbackAuthorization`.

Defects. A `setInterval` inherited the budget of the operation that scheduled it
and charged every tick to it, so an engine whose heartbeat allocates a few bytes
a second died after enough ticks — the refresh's own rule says repeating work
starts fresh, and its docs said timers were finite continuations. A repeating
`TimerEntry` now keeps the cycle stamp and drops the budget, which is what every
persistent event source already does; `AnIntervalStartsAFreshBudgetOnEveryTick`
fails without it. And `MemoryLimitConstraint`'s teardown runs host code — a
worker host told its connection ended, a host stream disposed — which may
re-enter the engine and find a fresh operation over the same budget, so it is
guarded to run once.

Two tests join it: an engine is usable again after a teardown, and a teardown
discards the timers the failed operation scheduled.

Co-authored-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VecfHbij4EMQFzL7NCpdWW

---------

Co-authored-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Co-authored-by: Marko Lahma <marko.lahma@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Copilot-Session: 0d07275e-227c-4672-becd-4fe336386da4
lahma added a commit to lahma/jint that referenced this pull request Aug 23, 2026
…lly has

Main has moved 181 commits past v4.16.0 and nothing records what an embedder
has to react to. Most of it is invisible to a compiler: every entry below still
compiles exactly as it did in 4.16 and behaves differently at run time, and six
of them flip a default.

`docs/v5-migration.md` is the artefact those entries go in. It is deliberately
not a second README: a table per change, before/after code where it helps, and
the rationale left in the pull request it cites.

Seeded from git history, every claim checked against the tree rather than
against the pull request title:

  * sebastienros#3054 `Interop.AllowWrite` true -> false. A projected CLR write is now
    silently ignored in sloppy mode and a TypeError in strict mode.
  * sebastienros#3056 `Interop.ArrayConversion` LiveView -> Copy. `Array.isArray` flips to
    true, `push`/`length` stop throwing, and CLR-side mutations after the
    crossing stop being visible.
  * sebastienros#3057 `Constraints.StackOverflowGuard` false -> true. RangeError instead of
    a process that ends with no exception in the log.
  * sebastienros#3058 `AgentCanSuspend` true -> false. `Atomics.wait` is a TypeError before
    a waiter is registered; `waitAsync` is untouched.
  * sebastienros#3052 namespace type discovery loses its implicit fallback to
    `Assembly.GetCallingAssembly()` / `GetExecutingAssembly()` /
    `Type.GetType(name)` and becomes the `AllowedAssemblies` allow-list.
  * sebastienros#3051 host exception, module-load and CLR-resolution messages are redacted
    from script; `ExposeDetailedErrors()` restores all three.
  * sebastienros#3035 concurrent `Engine` use fails fast, and an engine stays reserved for
    the lifetime of a returned async Task.
  * sebastienros#3036 `LimitMemory` charges allocations across async continuations, so a
    budget can now trip where one synchronous segment never reached it.
  * sebastienros#3252 every `*Async` entry reports the operation's failures through the
    task and only a usage error out of the call.
  * sebastienros#3248 an array-like `length` above 2^32-1 stops answering differently per
    target framework.
  * sebastienros#3037, sebastienros#3045, sebastienros#3046 add parser, module-graph and result bounds that all
    default to unlimited, so they change nothing until configured; sebastienros#3059 and
    sebastienros#3060 add the diagnostics and the hardened profile on top.

Two entries the prompt for this work had slightly differently, both checked and
written as the tree has them: the stack-overflow guard raises a catchable
`RangeError`, not a `RecursionDepthOverflowException`, and `ArrayOperations` is
internal, so sebastienros#3248 removes no public member — its break is what a script sees.

Sections 2, 3 and 6 (removed API, renamed API, AOT) are explicit empty
placeholders. Nothing public has been removed or renamed since v4.16.0, and a
parallel task is measuring the AOT state.

The target-framework section is written and marked pending: `Jint.csproj` still
lists net462, and the net472 change is not on main yet.

`Jint/AGENTS.md` gains the rule that makes the guide stay current — a change to
anything in its public-contract table is a row in the guide, in the same pull
request, including a change that breaks nothing at compile time. The root
`AGENTS.md` is untouched; it is at 23 KB against a 24 KiB budget.

README's "Branches and releases" described `main` alone and did not mention the
`3.x` branch at all. It now has a row per live branch, in the same wording
sebastienros#3291 gives the 4.x branch's own copy, plus a pointer to the guide.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014W5mbjGhyvgAS4pivXoc4S
lahma added a commit that referenced this pull request Aug 23, 2026
…lly has (#3294)

Main has moved 181 commits past v4.16.0 and nothing records what an embedder
has to react to. Most of it is invisible to a compiler: every entry below still
compiles exactly as it did in 4.16 and behaves differently at run time, and six
of them flip a default.

`docs/v5-migration.md` is the artefact those entries go in. It is deliberately
not a second README: a table per change, before/after code where it helps, and
the rationale left in the pull request it cites.

Seeded from git history, every claim checked against the tree rather than
against the pull request title:

  * #3054 `Interop.AllowWrite` true -> false. A projected CLR write is now
    silently ignored in sloppy mode and a TypeError in strict mode.
  * #3056 `Interop.ArrayConversion` LiveView -> Copy. `Array.isArray` flips to
    true, `push`/`length` stop throwing, and CLR-side mutations after the
    crossing stop being visible.
  * #3057 `Constraints.StackOverflowGuard` false -> true. RangeError instead of
    a process that ends with no exception in the log.
  * #3058 `AgentCanSuspend` true -> false. `Atomics.wait` is a TypeError before
    a waiter is registered; `waitAsync` is untouched.
  * #3052 namespace type discovery loses its implicit fallback to
    `Assembly.GetCallingAssembly()` / `GetExecutingAssembly()` /
    `Type.GetType(name)` and becomes the `AllowedAssemblies` allow-list.
  * #3051 host exception, module-load and CLR-resolution messages are redacted
    from script; `ExposeDetailedErrors()` restores all three.
  * #3035 concurrent `Engine` use fails fast, and an engine stays reserved for
    the lifetime of a returned async Task.
  * #3036 `LimitMemory` charges allocations across async continuations, so a
    budget can now trip where one synchronous segment never reached it.
  * #3252 every `*Async` entry reports the operation's failures through the
    task and only a usage error out of the call.
  * #3248 an array-like `length` above 2^32-1 stops answering differently per
    target framework.
  * #3037, #3045, #3046 add parser, module-graph and result bounds that all
    default to unlimited, so they change nothing until configured; #3059 and
    #3060 add the diagnostics and the hardened profile on top.

Two entries the prompt for this work had slightly differently, both checked and
written as the tree has them: the stack-overflow guard raises a catchable
`RangeError`, not a `RecursionDepthOverflowException`, and `ArrayOperations` is
internal, so #3248 removes no public member — its break is what a script sees.

Sections 2, 3 and 6 (removed API, renamed API, AOT) are explicit empty
placeholders. Nothing public has been removed or renamed since v4.16.0, and a
parallel task is measuring the AOT state.

The target-framework section is written and marked pending: `Jint.csproj` still
lists net462, and the net472 change is not on main yet.

`Jint/AGENTS.md` gains the rule that makes the guide stay current — a change to
anything in its public-contract table is a row in the guide, in the same pull
request, including a change that breaks nothing at compile time. The root
`AGENTS.md` is untouched; it is at 23 KB against a 24 KiB budget.

README's "Branches and releases" described `main` alone and did not mention the
`3.x` branch at all. It now has a row per live branch, in the same wording
#3291 gives the 4.x branch's own copy, plus a pointer to the guide.


Claude-Session: https://claude.ai/code/session_014W5mbjGhyvgAS4pivXoc4S

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
lahma added a commit to lahma/jint that referenced this pull request Oct 5, 2026
…y, so `s = s + x` is linear like `s += x`

Backport of PR sebastienros#3386 (commit 9d80990) from main.

`s += x` and `s = s + x` mean the same thing and did not cost the same
thing. The compound form builds into `JsString.ConcatenatedString`, which is
`StringBuilder`-backed and amortised linear; a plain `+` coerced both operands
to `string` and returned `JsString.Create(string.Concat(left, right))`, so
every iteration of an accumulator loop copied the whole left operand, and
prepending (`s = x + s`) had no fast path at all.

`JsString.RopeString` is an immutable deferred form: two operands and a total
length, flattened once on the first read that needs characters, memoized, and
with both operand references released at that point. `JsString.Concat` builds
one when the result reaches 512 characters and concatenates flat below that.
`Length` - and so truthiness and the length comparison string equality
performs first - is answered from the node; everything else flattens,
`this[int]` included, since descending per character would be a new
quadratic. The flatten walk is iterative over a heap array and descends
right-first, so depth is not capped and cannot overflow the stack. The
flattened `a + b + c` chain gets the same decision, so `s = s + a + b` is
linear too, and the MaxLength guard still runs on the summed lengths before
anything is built. `+=` is untouched. Also adds the four Assign* lanes to
StringConcatLargeBenchmark.

Adapted for 4.x:

- JsString.cs, class remarks (conflict): main rewrote the subclassing
  paragraphs around LazyJsString, which is v5's sebastienros#3340 and absent here. Kept
  4.x's two paragraphs and their contract; only the list of Jint's own
  representations gains the deferred one.
- JsValueExtensions.cs (conflict; the file is Jint/JsValueExtensions.cs on
  this branch): the "InternalTypes.String is set only by" list gains
  RopeString, without main's LazyJsString; 4.x's `IsString` cref is kept.
- LazyJsString.cs (absent): main's hunk there is a comment. It adds RopeString
  to the list of Jint's own lazy strings that are built on JsString directly
  rather than on LazyJsString, which is what lets LazyJsString's declared-
  length verification run with no out-of-assembly gate. The invariant behind
  it is one a node depends on: it answers its own Length as the sum of its
  operands' Lengths, sizes the flatten buffer from that sum and writes each
  operand's text at an offset computed from it, so every operand's Length has
  to be the length of its text. On main a host string is a LazyJsString whose
  declared length host-contract verification checks against what it produces.
  On 4.x the same role is played by a host's direct JsString subclass - the
  documented subclassing contract, pinned by LazyHostStringTests and by
  RavenApiUsageTests' CustomString - whose Length is its own unverified claim.
  So main's `Immutable` (snapshot a ConcatenatedString, hold everything else)
  becomes `IsRetainable`: a node holds only Jint's own immutable
  representations - an exact JsString, a SlicedString or a RopeString - and a
  ConcatenatedString or a host subclass is snapshotted at the `+`, with the
  node's length summed from what it actually holds. That keeps a host's
  ToString() at the `+`, where 4.x has always called it, and keeps a host
  whose Length disagrees with its text producing that text. The two tests
  added to Jint.Tests.PublicInterface's LazyHostStringTests pin both; run
  against main's rule verbatim, all three cases fail - the host is not
  materialized at the `+` (count 0, expected 1), a host reporting 5 for "ab"
  flattens to "\0\0\0ab...", and one reporting 3 for "abcd" throws
  ArgumentOutOfRangeException out of RopeString.CopyInto. The accumulator a
  loop builds is always Jint's own after its first deferred `+`, so the
  snapshot does not reintroduce a copy per iteration.
- README.md (conflict): main's hunk edits the "Lazy strings" section, which
  this branch's README does not have, and this README never described
  `s = s + x` as quadratic or recommended `+=` over `+`; dropped.
- docs/v5-migration.md: absent on this branch; dropped.
- StringConcatenationTests: this branch's GCPolyfills has no
  AllocatedBytesForCurrentThreadIsSupported (main's sebastienros#3036); the guard asks
  TryGetAllocatedBytesForCurrentThread instead. The file was already xUnit on
  main at sebastienros#3386.

Evidence - the ported tests against unfixed 4.x (engine files at upstream/4.x
plus a never-committed compile stub declaring RopeString and the cutoff
constant), net10.0 and net472 alike: in StringConcatenationTests,
StringRepresentationKeyTests and StringLengthLimitTests 32 fail and 55 pass.
Three of the failures are the asymptotic claim itself -
AccumulatingWithPlusIsNoLongerQuadratic allocates 640,758,656 B
(`s = s + chunk`), 640,744,128 B (`s = chunk + s`) and 1,281,226,960 B
(`s = s + chunk + chunk`) for 8,000 iterations against a 16 MB bound; the
other 29 stop at the "is a RopeString" premise assertion. The 55 passes pin
what must not change (characters, order, keys, the existing rows of both
older classes). AVeryDeepTreeFlattensWithoutRecursing (4x10^10 characters
copied at 200,000 iterations without the fix) and
ConcatenationPastTheLimitIsACatchableRangeError (about 1.3 GB of flat
strings without the fix) were not run against unfixed code on this shared
machine. With the package all of them pass on both frameworks.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanPJbBD7pQC9fRiTpHxvs
lahma added a commit to lahma/jint that referenced this pull request Oct 5, 2026
…y, so `s = s + x` is linear like `s += x`

Backport of PR sebastienros#3386 (commit 9d80990) from main.

`s += x` and `s = s + x` mean the same thing and did not cost the same
thing. The compound form builds into `JsString.ConcatenatedString`, which is
`StringBuilder`-backed and amortised linear; a plain `+` coerced both operands
to `string` and returned `JsString.Create(string.Concat(left, right))`, so
every iteration of an accumulator loop copied the whole left operand, and
prepending (`s = x + s`) had no fast path at all.

`JsString.RopeString` is an immutable deferred form: two operands and a total
length, flattened once on the first read that needs characters, memoized, and
with both operand references released at that point. `JsString.Concat` builds
one when the result reaches 512 characters and concatenates flat below that.
`Length` - and so truthiness and the length comparison string equality
performs first - is answered from the node; everything else flattens,
`this[int]` included, since descending per character would be a new
quadratic. The flatten walk is iterative over a heap array and descends
right-first, so depth is not capped and cannot overflow the stack. The
flattened `a + b + c` chain gets the same decision, so `s = s + a + b` is
linear too, and the MaxLength guard still runs on the summed lengths before
anything is built. `+=` is untouched. Also adds the four Assign* lanes to
StringConcatLargeBenchmark.

Adapted for 4.x:

- JsString.cs, class remarks (conflict): main rewrote the subclassing
  paragraphs around LazyJsString, which is v5's sebastienros#3340 and absent here. Kept
  4.x's two paragraphs and their contract; only the list of Jint's own
  representations gains the deferred one.
- JsValueExtensions.cs (conflict; the file is Jint/JsValueExtensions.cs on
  this branch): the "InternalTypes.String is set only by" list gains
  RopeString, without main's LazyJsString; 4.x's `IsString` cref is kept.
- LazyJsString.cs (absent): main's hunk there is a comment. It adds RopeString
  to the list of Jint's own lazy strings that are built on JsString directly
  rather than on LazyJsString, which is what lets LazyJsString's declared-
  length verification run with no out-of-assembly gate. The invariant behind
  it is one a node depends on: it answers its own Length as the sum of its
  operands' Lengths, sizes the flatten buffer from that sum and writes each
  operand's text at an offset computed from it, so every operand's Length has
  to be the length of its text. On main a host string is a LazyJsString whose
  declared length host-contract verification checks against what it produces.
  On 4.x the same role is played by a host's direct JsString subclass - the
  documented subclassing contract, pinned by LazyHostStringTests and by
  RavenApiUsageTests' CustomString - whose Length is its own unverified claim.
  So main's `Immutable` (snapshot a ConcatenatedString, hold everything else)
  becomes `IsRetainable`: a node holds only Jint's own immutable
  representations - an exact JsString, a SlicedString or a RopeString - and a
  ConcatenatedString or a host subclass is snapshotted at the `+`, with the
  node's length summed from what it actually holds. That keeps a host's
  ToString() at the `+`, where 4.x has always called it, and keeps a host
  whose Length disagrees with its text producing that text. The two tests
  added to Jint.Tests.PublicInterface's LazyHostStringTests pin both; run
  against main's rule verbatim, all three cases fail - the host is not
  materialized at the `+` (count 0, expected 1), a host reporting 5 for "ab"
  flattens to "\0\0\0ab...", and one reporting 3 for "abcd" throws
  ArgumentOutOfRangeException out of RopeString.CopyInto. The accumulator a
  loop builds is always Jint's own after its first deferred `+`, so the
  snapshot does not reintroduce a copy per iteration.
- README.md (conflict): main's hunk edits the "Lazy strings" section, which
  this branch's README does not have, and this README never described
  `s = s + x` as quadratic or recommended `+=` over `+`; dropped.
- docs/v5-migration.md: absent on this branch; dropped.
- StringConcatenationTests: this branch's GCPolyfills has no
  AllocatedBytesForCurrentThreadIsSupported (main's sebastienros#3036); the guard asks
  TryGetAllocatedBytesForCurrentThread instead. The file was already xUnit on
  main at sebastienros#3386.

Evidence - the ported tests against unfixed 4.x (engine files at upstream/4.x
plus a never-committed compile stub declaring RopeString and the cutoff
constant), net10.0 and net472 alike: in StringConcatenationTests,
StringRepresentationKeyTests and StringLengthLimitTests 32 fail and 55 pass.
Three of the failures are the asymptotic claim itself -
AccumulatingWithPlusIsNoLongerQuadratic allocates 640,758,656 B
(`s = s + chunk`), 640,744,128 B (`s = chunk + s`) and 1,281,226,960 B
(`s = s + chunk + chunk`) for 8,000 iterations against a 16 MB bound; the
other 29 stop at the "is a RopeString" premise assertion. The 55 passes pin
what must not change (characters, order, keys, the existing rows of both
older classes). AVeryDeepTreeFlattensWithoutRecursing (4x10^10 characters
copied at 200,000 iterations without the fix) and
ConcatenationPastTheLimitIsACatchableRangeError (about 1.3 GB of flat
strings without the fix) were not run against unfixed code on this shared
machine. With the package all of them pass on both frameworks.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanPJbBD7pQC9fRiTpHxvs
lahma added a commit that referenced this pull request Oct 5, 2026
…uilding, safe across engines and charged to LimitMemory (#4160)

* Backport #3386 to 4.x: Strings: a long `+` defers its copy, so `s = s + x` is linear like `s += x`

Backport of PR #3386 (commit 9d80990) from main.

`s += x` and `s = s + x` mean the same thing and did not cost the same
thing. The compound form builds into `JsString.ConcatenatedString`, which is
`StringBuilder`-backed and amortised linear; a plain `+` coerced both operands
to `string` and returned `JsString.Create(string.Concat(left, right))`, so
every iteration of an accumulator loop copied the whole left operand, and
prepending (`s = x + s`) had no fast path at all.

`JsString.RopeString` is an immutable deferred form: two operands and a total
length, flattened once on the first read that needs characters, memoized, and
with both operand references released at that point. `JsString.Concat` builds
one when the result reaches 512 characters and concatenates flat below that.
`Length` - and so truthiness and the length comparison string equality
performs first - is answered from the node; everything else flattens,
`this[int]` included, since descending per character would be a new
quadratic. The flatten walk is iterative over a heap array and descends
right-first, so depth is not capped and cannot overflow the stack. The
flattened `a + b + c` chain gets the same decision, so `s = s + a + b` is
linear too, and the MaxLength guard still runs on the summed lengths before
anything is built. `+=` is untouched. Also adds the four Assign* lanes to
StringConcatLargeBenchmark.

Adapted for 4.x:

- JsString.cs, class remarks (conflict): main rewrote the subclassing
  paragraphs around LazyJsString, which is v5's #3340 and absent here. Kept
  4.x's two paragraphs and their contract; only the list of Jint's own
  representations gains the deferred one.
- JsValueExtensions.cs (conflict; the file is Jint/JsValueExtensions.cs on
  this branch): the "InternalTypes.String is set only by" list gains
  RopeString, without main's LazyJsString; 4.x's `IsString` cref is kept.
- LazyJsString.cs (absent): main's hunk there is a comment. It adds RopeString
  to the list of Jint's own lazy strings that are built on JsString directly
  rather than on LazyJsString, which is what lets LazyJsString's declared-
  length verification run with no out-of-assembly gate. The invariant behind
  it is one a node depends on: it answers its own Length as the sum of its
  operands' Lengths, sizes the flatten buffer from that sum and writes each
  operand's text at an offset computed from it, so every operand's Length has
  to be the length of its text. On main a host string is a LazyJsString whose
  declared length host-contract verification checks against what it produces.
  On 4.x the same role is played by a host's direct JsString subclass - the
  documented subclassing contract, pinned by LazyHostStringTests and by
  RavenApiUsageTests' CustomString - whose Length is its own unverified claim.
  So main's `Immutable` (snapshot a ConcatenatedString, hold everything else)
  becomes `IsRetainable`: a node holds only Jint's own immutable
  representations - an exact JsString, a SlicedString or a RopeString - and a
  ConcatenatedString or a host subclass is snapshotted at the `+`, with the
  node's length summed from what it actually holds. That keeps a host's
  ToString() at the `+`, where 4.x has always called it, and keeps a host
  whose Length disagrees with its text producing that text. The two tests
  added to Jint.Tests.PublicInterface's LazyHostStringTests pin both; run
  against main's rule verbatim, all three cases fail - the host is not
  materialized at the `+` (count 0, expected 1), a host reporting 5 for "ab"
  flattens to "\0\0\0ab...", and one reporting 3 for "abcd" throws
  ArgumentOutOfRangeException out of RopeString.CopyInto. The accumulator a
  loop builds is always Jint's own after its first deferred `+`, so the
  snapshot does not reintroduce a copy per iteration.
- README.md (conflict): main's hunk edits the "Lazy strings" section, which
  this branch's README does not have, and this README never described
  `s = s + x` as quadratic or recommended `+=` over `+`; dropped.
- docs/v5-migration.md: absent on this branch; dropped.
- StringConcatenationTests: this branch's GCPolyfills has no
  AllocatedBytesForCurrentThreadIsSupported (main's #3036); the guard asks
  TryGetAllocatedBytesForCurrentThread instead. The file was already xUnit on
  main at #3386.

Evidence - the ported tests against unfixed 4.x (engine files at upstream/4.x
plus a never-committed compile stub declaring RopeString and the cutoff
constant), net10.0 and net472 alike: in StringConcatenationTests,
StringRepresentationKeyTests and StringLengthLimitTests 32 fail and 55 pass.
Three of the failures are the asymptotic claim itself -
AccumulatingWithPlusIsNoLongerQuadratic allocates 640,758,656 B
(`s = s + chunk`), 640,744,128 B (`s = chunk + s`) and 1,281,226,960 B
(`s = s + chunk + chunk`) for 8,000 iterations against a 16 MB bound; the
other 29 stop at the "is a RopeString" premise assertion. The 55 passes pin
what must not change (characters, order, keys, the existing rows of both
older classes). AVeryDeepTreeFlattensWithoutRecursing (4x10^10 characters
copied at 200,000 iterations without the fix) and
ConcatenationPastTheLimitIsACatchableRangeError (about 1.3 GB of flat
strings without the fix) were not run against unfixed code on this shared
machine. With the package all of them pass on both frameworks.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanPJbBD7pQC9fRiTpHxvs

* Backport #3571 to 4.x: Strings: a short `+` chain no longer pays for a deferred representation it never takes

Backport of PR #3571 (commit ef706d7) from main.

#3386 cost SunSpider's date-format-xparb 3.9% on main (#3527), and not
because of the deferral: xparb's chains are ten to thirty-five characters,
far below the 512-character cutoff. What moved onto the eager path was its
shape. The chain lane coerced every operand with TypeConverter.ToJsString
before the cutoff was read, allocating a JsString wrapper per non-string
operand, and ConcatMany built a string[] on top of the JsString[] the
operands already lived in.

An operand is now carried in whichever form its coercion produced - the
JsString it already was, or the plain text a non-string primitive coerced
to - both of which answer their length without materializing anything, and
below the cutoff the result is joined through a ValueStringBuilder over a
cutoff-sized stack buffer: no second array and no wrapper. Above the cutoff
the fold through JsString.Concat is unchanged. The pairwise `+` gets the same
treatment: two string operands take a branch that coerces nothing, and the
mixed pair (`n + "x"`) builds the wrapper only if it goes on to defer.
TypeConverter.ToStringNonString becomes internal for that. `+=` and
String.prototype.concat are untouched.

Adapted for 4.x:

- Engine hunks (JsString.cs, JintBinaryExpression.cs, TypeConverter.cs) apply
  verbatim; ValueStringBuilder and the assembly's SkipLocalsInit are already
  on this branch.
- StringConcatenationTests: transcribed from NUnit to xUnit v3 ([Test] to
  [Fact], [TestCase] to [Theory] with [InlineData]); the allocation guard
  asks TryGetAllocatedBytesForCurrentThread, as in the previous commit.

Evidence: bytes per evaluation of the eleven-operand short chain
(y + '-' + m + '-' + d + ' ' + y + ':' + m + ':' + d), measured as a delta of
the thread-local allocation counter over 20,000 evaluations - unfixed 4.x
272 on net10.0 and 408 on net472; with this package 272 and 296, the same
after-figures main measured. So a short chain now costs no more than it did
on 4.x before #3386, and 27% less on net472. The ported tests against
unfixed 4.x: the eleven new cases fail 3 on net10.0 and 4 on net472 -
AccumulatingWithPlusIsNoLongerQuadratic's two five-operand rows (2,562,225,160
and 2,562,134,112 B for 8,000 iterations against a 16 MB bound),
APairWithOneCoercedOperandTakesBothSidesOfTheCutoff at its "is a RopeString"
premise, and on net472 AShortChainDoesNotPayForTheDeferredRepresentation at
408 B against its 400 B ceiling. The seven
AChainMixingCoercedAndStringOperandsProducesTheSameCharacters rows pass,
pinning characters that must not change. With the package all pass on both
frameworks.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanPJbBD7pQC9fRiTpHxvs

* Backport #4130 to 4.x: Build a bound function's name as a rope so a chain of binds stays linear

Backport of PR #4130 (commit 5eac242) from main.

SetFunctionName composed step 4's "prefix name" with a CLR string
concatenation, which flattens whatever it is handed. Binding an already-bound
function therefore materialized "bound bound ... f" afresh at every level and
left that copy in the level's own name descriptor, so N binds retained about
3*N^2 characters (#4129). The prefixed name is now a JsString.Concat node over
the level below - O(1) to build, flattened once if something ever reads the
text - and every observable name is byte-identical.

Adapted for 4.x:

- On this branch BindFunction derives from ObjectInstance, not Function (main
  made it a Function in #3658, which is not here), so Function.prototype.bind
  names the function it creates through ObjectInstance.SetFunctionName, which
  main's hunk does not touch and which still did `prefix + " " + name`. With
  main's Function.cs hunk alone the bind chain stayed exactly as quadratic
  (measured: 9.5, 36.9, 145.3, 576.7 MB allocated for 1,250, 2,500, 5,000 and
  10,000 levels). Function.PrefixName becomes internal rather than private and
  ObjectInstance.SetFunctionName calls it, so both overloads defer. Main's
  Function.cs hunk applies verbatim otherwise; it still covers the get/set
  accessor names and ShadowRealm's wrapped functions, which are Functions.
- BoundFunctionNameTests: transcribed from NUnit to xUnit v3. The wedge
  ceiling is 256 MB instead of main's 768 MB: with the fix the whole script
  allocates 53.0 MB on net10.0 and 55.3 MB on net472, and against the defect
  the per-level copies are all retained, so main's ceiling would let a run
  against unfixed code hold about 770 MB of live text before it tripped.
  256 MB trips at level 6,649 (net10.0) / 6,648 (net472) with a peak working
  set of 291 / 284 MB. Added ABoundNameLongEnoughToDeferIsANodeOverTheLevelBelow,
  which asserts the premise directly - a 100-level bound name is a RopeString -
  because on this branch the route bind takes is the one main's hunk misses.

Evidence - the bind chain, measured as allocated and retained bytes around
`for (i < N) f = f.bind(null)` on net10.0:

  depth     unfixed 4.x allocated / retained    with this package
  1,250       9.5 MB /   9.4 MB                   0.6 MB / 0.5 MB
  2,500      36.9 MB /  36.7 MB                   1.1 MB / 1.0 MB
  5,000     145.3 MB / 145.0 MB                   2.2 MB / 1.9 MB
  100,000   (not run: about 60 GB)               45.5 MB / 36.7 MB, 98 ms

Unfixed quadruples per doubling; fixed doubles. net472 unfixed: 9.7, 37.1,
145.6 MB for the same three depths. The ported tests against unfixed 4.x fail
2 of 8 on both frameworks - ADeepChainOfBoundFunctionsDoesNotCopyTheNameAtEveryLevel
with MemoryLimitExceededException at 256 MB, and the new premise test at "is a
RopeString" - and the six APrefixedNameIsTheSameTextItAlwaysWas rows pass,
pinning the names that must not change. With the package all 8 pass on both.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PanPJbBD7pQC9fRiTpHxvs

* Backport #4164 to 4.x: Fall back to a rope's published text when a concurrent flatten has released its operands

RopeString.Flatten memoized its text and then released both operands with
plain writes, while CopyInto tested the memo and then dereferenced the
operands. A second thread finishing its own flatten of the same node between
those two reads left the walk holding a null operand, and the next
ToString() threw NullReferenceException. On 4.x, too, preparation folds a
literal-plus-literal into one constant on the shared Prepared<Script>, so
every engine running a preparation reads the same RopeString.

The engine change applies as is. The tests are transcribed from NUnit to
xUnit v3: the two tests are async and wait on a local two-minute wedge
ceiling through Task.WhenAny (4.x has no TestBudgets, and xUnit1031 rejects
a blocking Task.WaitAll).

(cherry picked from commit 78e9bb0)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Backport #4173 to 4.x: Charge a deferred string concatenation to LimitMemory when it is built

A long `a + b` returns a RopeString, whose characters are allocated by whoever
flattens it, and that can be a host's ToString() after every constraint is
disarmed. JsString.Concat now charges the memory limit on its deferring
branches: the shorter operand, which keeps `s = s + x` linear, or the whole
length, refused before the node exists, when the flat form alone exceeds the
budget. A constant fold evaluates with an engine-less context and is not
charged.

Adapted for 4.x's MemoryLimitConstraint, which predates main's operation
state and async segments: it measures the thread's allocation counter
against a per-entry baseline. The charge is kept in a _deferredBytes total
that Reset() clears with the baseline and Check() adds to the measured
allocation. A result whose flat form alone exceeds the budget calls Check()
and then throws regardless, because Check() measures nothing on a thread
other than the one the entry started on. The engine finds its constraint
once at construction (Engine._memoryLimitConstraint). 4.x's ConcatSnapshotting
branch, which main does not have, is charged too, for the lengths the node
actually holds. Function.PrefixName is called from ObjectInstance.SetFunctionName
on 4.x as well, and both pass the engine's evaluation context.

Not ported: main's AllocatedBytes doc line and the
TheCharactersADeferredConcatenationAppendsAreReportedAsAllocated test
(MemoryLimitConstraint.AllocatedBytes is not public API on 4.x), the
docs/guide and migration-guide hunks (absent on 4.x; the README's
constraints section gets the paragraph instead) and the Jint/Constraints/AGENTS.md
hunk (absent on 4.x). Tests transcribed from NUnit to xUnit v3.

Adapted from 7d9cdcc

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants