Repository navigation
Web streams: transfer a ReadableStream, a WritableStream or a TransformStream - #3215
Merged
Merged
Conversation
…rmStream The Streams Standard makes all three transferable, and Jint refused them outright — the last part of that standard it did not implement, on the grounds that there was nothing to transfer them to. There is now: transferring a stream turns out to need no new transport at all, because the standard's own mechanism is a MessagePort pair plus a pipe, and both of those already span engines. The transfer steps are the standard's. Transferring a ReadableStream creates an entangled pair in the sending realm, sets a cross-realm transform writable up on one port, pipes the original stream into it, and puts the other port in the data holder; the transfer-receiving steps build a cross-realm transform readable over the port that arrived. A WritableStream is the mirror image, and a TransformStream is both, so it costs two channels. The port rides the transport the previous change shipped: the standard's [[port]] is the result of a nested StructuredSerializeWithTransfer of one port, and SerializedMessagePort *is* that result, so the stream's record carries one directly and the nesting collapses to a field. Those holders are marked Nested — they share the record's port list, so stranding and a discarded port message queue reach them, and event.ports does not, because a stream's private channel is not one of the caller's ports. Backpressure is the whole protocol. The receiving side's high water mark is 0 and the sending side starts with an unresolved backpressure promise, so the first chunk is not posted until the receiving script reads and one `pull` releases exactly one chunk. Chunks cross as ordinary structured-serialized messages, which is why an uncloneable chunk fails its write with a DataCloneError and reports it to the far side, exactly as the standard writes it. The refusals are the standard's too: a locked stream is a DataCloneError, a stream that is not in the transfer list is a DataCloneError because a stream is transferable and not serializable, and after a transfer the original is locked and disturbed because the pipe holds a reader on it. A [[Detached]] slot is what still refuses a second transfer once that pipe has finished and released the lock. One thing goes beyond the standard, and it is what keeps a transferred stream from leaving a pipe running against nothing. HTML drops a message posted into a disentangled channel silently, so PackAndPostMessage cannot fail that way and the standard's writable side would go on draining its source into a channel whose far end has been ended — by a message that was serialized and never delivered, by a close(), or by a RestoreGlobalSnapshot on the receiving engine. The write and pull algorithms therefore consult JsMessagePort.IsChannelExhausted and end their stream instead. That predicate asks about the queue as well as the far side, and the queue half is not defensive: a sender's last act is to post `close` and then disentangle, so asking only "has the far side gone" would throw away the very message that closes the stream cleanly. The one .any.js file of wpt's streams/transferable/ is vendored at the pin and all four of its assertions pass; the rest of that directory is a browsing context and stays out, by name rather than by silence. Closes sebastienros#3199 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014n5uCpi2vvHWBSiewUwBiY
This was referenced Aug 21, 2026
lahma
added a commit
to lahma/jint
that referenced
this pull request
Aug 21, 2026
…r's budget `docs/design/web-workers.md` predates the finalized plan in sebastienros#3167 and disagreed with it in nine places that matter. Rewritten decision for decision: - `close()` is not `terminate()`. The spec's *close a worker* discards the worker's own queued tasks and sets the closing flag; it pointedly does not abort the running script or empty the parent-side queue. The doc had them sharing one three-step teardown, which would kill `close(); flushMetrics();` and lose `postMessage(result); close()`. - A load failure fires a plain `Event`, not an `ErrorEvent`, and ends the connection as `StartupFailed` rather than `Terminated`. - The parent-side error relay is an internal per-connection hook gated on *notHandled*, sitting beside the `DiagnosticsSink` rather than being it — the sink is deliberately unsuppressible by script and cannot carry that gate. - Nesting is OFF by default; `MaxWorkers` (16) is a per-engine backstop that bounds a branching factor, not a tree, which is what the old text claimed. - Security posture: restrictions travel, grants never travel by implication, and the classification is pinned reflectively. - `importScripts` is present and throwing, because the spec's own step 1 for a module worker prescribes the throw. - The agent-cluster claim is corrected: HTML puts a dedicated worker in its creator's cluster, so refusing `SharedArrayBuffer` is Jint's isolation policy and not a fact about agents. - "Close both ports is already the shipped rule" is withdrawn: the shipped rule is the opposite one-sided close, and the worker rule is argued on its own merits (cost, not delivery). - The generation fence is per **port**, captured at port construction, which is what lets a transferred side rejoin the receiving engine's current cycle. - The `AWorkerCannotBeSentAMessagePort` pin is deleted; it contradicted sebastienros#3197. Line-number citations are replaced with file-and-member ones throughout. Most of the old numbers were already wrong, and a wrong line number reads like a fact. sebastienros#3215's transferable-streams paragraph is preserved, integrated into the transfer section beside the port transfer it rides on. The same correction lands in the XML docs, where it is load-bearing rather than merely explanatory. `WorkerRequest.CreateDefaultOptions` claimed a replayed `LimitMemory` never fires on a pumped worker. Since sebastienros#3036 that is false: every event-loop job runs inside an allocation segment and is checked as it completes, carrying the operation state captured at registration across continuations and thread hops, so a replayed factory genuinely bounds each job chain. The worker budget is a pair — `OperationDeadlineConstraint` for time, `MemoryLimitConstraint` for allocations. The same change cites sebastienros#3036's other consequence: instance copying is now loudly wrong, because `MemoryLimitConstraint.Attach` throws when one instance meets a second engine. `Options`'s wholesale exclusion of the `WebApi` group also stops glossing: its value settings are per-feature caps living on the sub-groups, and each rides with the feature it bounds rather than being a restriction that stayed behind by accident. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lahma
added a commit
to lahma/jint
that referenced
this pull request
Aug 21, 2026
…r's budget `docs/design/web-workers.md` predates the finalized plan in sebastienros#3167 and disagreed with it in nine places that matter. Rewritten decision for decision: - `close()` is not `terminate()`. The spec's *close a worker* discards the worker's own queued tasks and sets the closing flag; it pointedly does not abort the running script or empty the parent-side queue. The doc had them sharing one three-step teardown, which would kill `close(); flushMetrics();` and lose `postMessage(result); close()`. - A load failure fires a plain `Event`, not an `ErrorEvent`, and ends the connection as `StartupFailed` rather than `Terminated`. - The parent-side error relay is an internal per-connection hook gated on *notHandled*, sitting beside the `DiagnosticsSink` rather than being it — the sink is deliberately unsuppressible by script and cannot carry that gate. - Nesting is OFF by default; `MaxWorkers` (16) is a per-engine backstop that bounds a branching factor, not a tree, which is what the old text claimed. - Security posture: restrictions travel, grants never travel by implication, and the classification is pinned reflectively. - `importScripts` is present and throwing, because the spec's own step 1 for a module worker prescribes the throw. - The agent-cluster claim is corrected: HTML puts a dedicated worker in its creator's cluster, so refusing `SharedArrayBuffer` is Jint's isolation policy and not a fact about agents. - "Close both ports is already the shipped rule" is withdrawn: the shipped rule is the opposite one-sided close, and the worker rule is argued on its own merits (cost, not delivery). - The generation fence is per **port**, captured at port construction, which is what lets a transferred side rejoin the receiving engine's current cycle. - The `AWorkerCannotBeSentAMessagePort` pin is deleted; it contradicted sebastienros#3197. Line-number citations are replaced with file-and-member ones throughout. Most of the old numbers were already wrong, and a wrong line number reads like a fact. sebastienros#3215's transferable-streams paragraph is preserved, integrated into the transfer section beside the port transfer it rides on. The same correction lands in the XML docs, where it is load-bearing rather than merely explanatory. `WorkerRequest.CreateDefaultOptions` claimed a replayed `LimitMemory` never fires on a pumped worker. Since sebastienros#3036 that is false: every event-loop job runs inside an allocation segment and is checked as it completes, carrying the operation state captured at registration across continuations and thread hops, so a replayed factory genuinely bounds each job chain. The worker budget is a pair — `OperationDeadlineConstraint` for time, `MemoryLimitConstraint` for allocations. The same change cites sebastienros#3036's other consequence: instance copying is now loudly wrong, because `MemoryLimitConstraint.Attach` throws when one instance meets a second engine. `Options`'s wholesale exclusion of the `WebApi` group also stops glossing: its value settings are per-feature caps living on the sub-groups, and each rides with the feature it bounds rather than being a restriction that stayed behind by accident. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
lahma
added a commit
that referenced
this pull request
Aug 21, 2026
#3219) * Workers: the host-facing types, the options group and the posture rule The foundation of the Worker feature (#3167), with no script surface at all: `typeof Worker` stays `undefined` on every engine, nothing is installed, and no engine runtime behaviour changes. What lands: - `Jint/WebApi/Workers/` — `WorkerProvider` (the host's answer to `new Worker(...)`, since Jint never starts a thread), `WorkerRequest` (what is being asked for, plus `CreateDefaultOptions()`), `WorkerConnection` (one live parent-worker pair) and the two enums. The connection is thread-safe now rather than later: `End()` is idempotent under *concurrent* callers, every property reads safely from any thread, and the end callback runs outside the lock so host code never runs under one. - `Options.WebApi.Workers` — `Provider` (null by default, which is what keeps the global uninstalled), `MaxWorkers` (16, a per-engine backstop and not the policy) and `MaxQueuedMessages` (16384, a backstop against a never-pumped worker rather than flow control), plus `WebApiFeatures.Workers = 1 << 24`, never in `Default`, and `UseWorkers` which sets flag and provider together. - `Options.CopySecurityPosture` — restrictions travel, grants never travel by implication. It copies the seven `Constraints` value settings, `Host.StringCompilationAllowed`, `AgentCanSuspend` and `Json.MaxParseDepth`, and lives beside the properties it names with three explicit classification lists: what is inherited, what is deliberately not (with the reason), and which option groups are excluded wholesale as grant-shaped. `CreateDefaultOptions()` returns a fresh `Options` each call: the termination token as a cancellation constraint, every parent constraint *factory* replayed (never an instance — those are single-engine-only), the copied posture, the feature mask minus network/storage/routing/nesting plus `Messaging` and `GlobalEvents`, and `DiagnosticsSink.Null`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Workers: pin the foundation, and the classification that outlives it Mechanism in Jint.Tests, third-party reachability in Jint.Tests.PublicInterface, each pin written so that removing the mechanism it names makes it fail. `OptionsSecurityPostureTests` is the one worth keeping past this feature: it reflects over the value-typed settings of `Options` and its `Constraints`, `Host` and `Json` groups and fails unless each is either copied by `CopySecurityPosture` or named as deliberately not inherited, and it does the same for the option groups themselves. It found one on its first run — `Options.Profiling`, which is now classified (host diagnostics, script cannot reach it, and its gate already defaults to refusing). `TheDefaultOptionsCopyTheParentsSecurityPosture` is a theory with one case per copied setting, so dropping a single line of the copy fails exactly the case that names it rather than one arbitrary assertion. `TypeofWorkerIsUndefinedEvenWithTheFlagAndProvider` pins that this change is not also the one that moves the engine: with the flag on, a provider registered and the web APIs enabled, a script still cannot tell any of this exists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Workers: sync the design doc, and tell the truth about a pumped worker's budget `docs/design/web-workers.md` predates the finalized plan in #3167 and disagreed with it in nine places that matter. Rewritten decision for decision: - `close()` is not `terminate()`. The spec's *close a worker* discards the worker's own queued tasks and sets the closing flag; it pointedly does not abort the running script or empty the parent-side queue. The doc had them sharing one three-step teardown, which would kill `close(); flushMetrics();` and lose `postMessage(result); close()`. - A load failure fires a plain `Event`, not an `ErrorEvent`, and ends the connection as `StartupFailed` rather than `Terminated`. - The parent-side error relay is an internal per-connection hook gated on *notHandled*, sitting beside the `DiagnosticsSink` rather than being it — the sink is deliberately unsuppressible by script and cannot carry that gate. - Nesting is OFF by default; `MaxWorkers` (16) is a per-engine backstop that bounds a branching factor, not a tree, which is what the old text claimed. - Security posture: restrictions travel, grants never travel by implication, and the classification is pinned reflectively. - `importScripts` is present and throwing, because the spec's own step 1 for a module worker prescribes the throw. - The agent-cluster claim is corrected: HTML puts a dedicated worker in its creator's cluster, so refusing `SharedArrayBuffer` is Jint's isolation policy and not a fact about agents. - "Close both ports is already the shipped rule" is withdrawn: the shipped rule is the opposite one-sided close, and the worker rule is argued on its own merits (cost, not delivery). - The generation fence is per **port**, captured at port construction, which is what lets a transferred side rejoin the receiving engine's current cycle. - The `AWorkerCannotBeSentAMessagePort` pin is deleted; it contradicted #3197. Line-number citations are replaced with file-and-member ones throughout. Most of the old numbers were already wrong, and a wrong line number reads like a fact. #3215's transferable-streams paragraph is preserved, integrated into the transfer section beside the port transfer it rides on. The same correction lands in the XML docs, where it is load-bearing rather than merely explanatory. `WorkerRequest.CreateDefaultOptions` claimed a replayed `LimitMemory` never fires on a pumped worker. Since #3036 that is false: every event-loop job runs inside an allocation segment and is checked as it completes, carrying the operation state captured at registration across continuations and thread hops, so a replayed factory genuinely bounds each job chain. The worker budget is a pair — `OperationDeadlineConstraint` for time, `MemoryLimitConstraint` for allocations. The same change cites #3036's other consequence: instance copying is now loudly wrong, because `MemoryLimitConstraint.Attach` throws when one instance meets a second engine. `Options`'s wholesale exclusion of the `WebApi` group also stops glossing: its value settings are per-feature caps living on the sub-groups, and each rides with the feature it bounds rather than being a restriction that stayed behind by accident. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Workers: make the End()-idempotency pin actually race The pin claimed to prove that `WorkerConnection.End()` is idempotent under *concurrent* callers rather than merely repeated ones — which is the entire reason `TryEnd` takes a lock instead of testing and setting a `volatile bool`. It did not prove it. Replacing the lock with a plain check-then-set left the test passing five times out of five: `Parallel.For` ramps its workers up, so by the time the second one starts the first has long since set the flag and there is no race left to lose. The pin named a mechanism it could not see. A `Barrier` releases every racer into the same instant instead, the threads are reused across 64 rounds so that many attempts cost a handful of threads rather than thousands, and each round gets its own connection and its own callback counter — so the failure message names the round rather than an aggregate. The unlocked version now loses within the first couple of rounds, verified five times out of five, and the assertion that fails is the callback count, which is exactly the mechanism removed. Nothing here is timed: there is no wall-clock assertion, so a slow machine makes this test slower and never redder, and the joins carry a 60-second ceiling only so that an `End()` that deadlocked fails the test instead of hanging the run. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * Workers: classify the security settings the stack landed under us Four security-stack layers (#3037, #3045, #3046, #3051) merged between this branch's base and its landing, and each added exactly the shape of setting the posture rule exists to classify. The parser bounds, the four module-graph limits and the ResultLimits bundle are restrictions and travel; detailed-error exposure and the require shim are grants and stay behind. Modules stops being excludable wholesale — the group now mixes its loader (a grant) with numeric limits (restrictions), so it joins the scanned set with each member classified. ResultLimits is class-typed and outside the reflective pin's value-typed scan, so it is classified in CopySecurityPosture's body instead, and the copy is by reference because the bundle is sealed and immutable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PnghUC3EKrYa12xQ2LaYyd * Workers: inherit the untrusted-code profile's expansion, never its marker #3060 landed the hardened profile while this branch was in review, and it resolved a question the design had left open in the profile's favour — almost. An untrusted parent's Engine.Options is the profile's expanded clone, so the posture copy already takes every value the expansion set and the factory replay already carries its budget constraints; propagating the marker itself would be strictly worse, because a marked options object re-expands at engine construction and that expansion clears the constraint registrations, including the cancellation constraint a worker's terminate() depends on. The unmarked worker loses only the diagnostics label, and the test pins both halves. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PnghUC3EKrYa12xQ2LaYyd --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 21, 2026
lahma
added a commit
that referenced
this pull request
Aug 22, 2026
The workers/ corpus is the capstone of #3167: it is reachable at all only because Jint now has a Worker. Eleven files, 15 assertions, run as the body of a real module worker built by a real `new Worker(…, { type: 'module' })` — not in the driver's top-level engine, where every one of them would have passed and proved nothing. Two lanes. WptWorkerProvider is a WorkerProvider — the shipped host extension point — and the driver pumps the parent and every live worker in turn on its own thread; nothing here starts one, so a suite's outcome cannot depend on how two schedulers interleaved. Results are read off the worker engine between turns rather than posted back, so a serializer defect would present as a failing assertion rather than as every worker suite reporting nothing. WptHarness.RunsInAWorker picks the lane from the directory *and* the `// META: global=` key, because neither half does the job alone, and EveryWorkerLaneFileIsAWorkersFile pins both directions — no previously vendored file moves. Eight of the 15 pass, and what passes is the worker global doing its job: dispatching a custom event at itself, taking `onmessage` eight times over, replacing `self.MessageEvent`, exposing atob/btoa, and — the best of them — refusing `postMessage(formData)` with a DataCloneError, so the worker global's postMessage really is the port's, running the real serializer. Six of the seven failures were decided in writing before a line of this ran, which is the point of running it: five are the three globals the design declines by name (WorkerGlobalScope, WorkerLocation, WorkerNavigator — divergences 5, 6 and 7) and one is nesting being off by default. The seventh is NeedsTriage and is not in the worker code: `self` is installed once for every global as a writable data property, against Window's [Replaceable] definition, where WorkerGlobalScope's is read-only. Recorded rather than fixed — the install predates Worker and is shared with the top-level lane. The shim stops assuming `self` is assignable so that fix will not break the harness. Twenty upstream .any.js files cannot be reached and are named with reasons. One row is a finding rather than a decision: workers/modules/ holds the flagship module-worker suite, and every one of its nine fixtures installs its onmessage handler inside `if ('DedicatedWorkerGlobalScope' in self && self instanceof DedicatedWorkerGlobalScope)`. Jint deliberately exposes no such interface object, so no handler is installed, the worker never answers, and the file hangs — the canonical "am I in a worker" sniff, failing in the fixture where no per-test exclusion can name it. The design named that wrinkle; this is what it costs. Also fixes leftover merge-conflict markers in Vendor/README.md, left by #3215. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements transferable streams:
ReadableStream,WritableStreamandTransformStreamcan now ride apostMessagetransfer list — orstructuredClone(stream, {transfer: [stream]})— landing as live streams on the receiving side, per the Streams Standard's cross-realm transform.Closes #3199.
The design is the standard's own, on transport that already existed. A transferred stream is a
MessagePortpair plus a pipe: transferring a readable sets a cross-realm-transform writable over one port and pipes the original into it; a writable is the mirror image; a transform is both sides, two channels. Every chunk crosses as an ordinary port message ({type, value}structured-serialized on the sender, deserialized on the receiver), which is what makes an uncloneable chunk fail at the write with theDataCloneErrorreported to the far side, exactly as the spec writes it. Backpressure is the spec's too: the receiving side's HWM is 0 and onepullmessage releases one chunk, so nothing is posted until the receiving script actually reads. The standard's nestedStructuredSerializeWithTransferof the port collapses to a field:SerializedMessagePortis that result, markedNestedso it shares the record's port list — stranding and the discard cascade reach it — whileevent.portsskips it.One deliberate divergence, documented on
JsMessagePort.IsChannelExhausted: HTML drops a message posted into a disentangled channel silently, so the standard's sender would drain its source into nothing forever once the far end died (a receiver-sideRestoreGlobalSnapshot, a stranded record). The write and pull algorithms consult the predicate and end their stream with aTypeErrorinstead. It can only fire where the standard leaves the outcome undefined — every orderly shutdown postsclose/errorbefore disentangling, and the queue half of the predicate is what stops that final message being thrown away. Known limit, also documented: it fires at the sender's next write, so a pipe already parked on a backpressure promise whose resolver died stays idly parked; closing that would need theMessagePortcloseevent Jint deliberately does not have.WPT:
streams/transferable/transform-stream-members.any.jsvendored at the standing pin (byte-verified), a seventh streams suite — 4/4, no exclusion entry of any kind; the.window.js/resources/browsing-context files stay out by name. 46 new tests in-repo (30 same-engine, 14 cross-engine over the public surface, 2 record-shape). Mutation: 16 mutants, 14 killed; the two survivors are the standard's own "Disentangle port" on shutdown paths whose stream the immediately-preceding message already finished — nothing script-visible changes, and observing the channel-object release would need a live-port diagnostic the change has no other reason to add.Full matrix green both TFMs including HCV legs; whole Wpt filter 346/346 post-rebase on #3208.
🤖 Generated with Claude Code
https://claude.ai/code/session_014n5uCpi2vvHWBSiewUwBiY