Web Streams cleanup #56819

wyattjoh · 2023-10-13T23:58:12Z

This updates some code related to web streams and encoding.

Removes some unused code related to base64 encoding/decoding (Edge runtime currently supports it natively via Buffer)
Prefer readable stream pull versus .on("data", (chunk) => { ... }) event handlers (simplifies execution)
Utilize pipeTo and pipeThrough on web streams to remove custom code related to stream pumping
Updates pipe readable function to utilize web streams first class rather than relying on manual pumping + stream management
- This also takes advantage of the AbortController when piping so that the response can use it to cancel the stream

wyattjoh · 2023-10-13T23:58:23Z

Current dependencies on/for this PR:

canary
- PR Web Streams cleanup #56819 👈

This comment was auto-generated by Graphite.

ijjk · 2023-10-14T00:00:23Z

Tests Passed

ijjk · 2023-10-14T00:04:21Z

Stats from current PR

Default Build (Increase detected ⚠️)

General Overall increase ⚠️

	vercel/next.js canary	vercel/next.js refactor/web-streams-cleanup	Change
buildDuration	10.3s	10.4s	N/A
buildDurationCached	6.2s	6.3s	⚠️ +119ms
nodeModulesSize	174 MB	175 MB	⚠️ +994 kB
nextStartRea..uration (ms)	510ms	515ms	N/A

Client Bundles (main, webpack)

	vercel/next.js canary	vercel/next.js refactor/web-streams-cleanup	Change
199-HASH.js gzip	27.5 kB	27.5 kB	N/A
3f784ff6-HASH.js gzip	53.3 kB	53.3 kB	✓
99.HASH.js gzip	182 B	182 B	✓
framework-HASH.js gzip	45.3 kB	45.3 kB	✓
main-app-HASH.js gzip	254 B	252 B	N/A
main-HASH.js gzip	32.9 kB	32.9 kB	N/A
webpack-HASH.js gzip	1.75 kB	1.75 kB	N/A
Overall change	98.8 kB	98.8 kB	✓

Legacy Client Bundles (polyfills)

	vercel/next.js canary	vercel/next.js refactor/web-streams-cleanup	Change
polyfills-HASH.js gzip	31 kB	31 kB	✓
Overall change	31 kB	31 kB	✓

Client Pages

	vercel/next.js canary	vercel/next.js refactor/web-streams-cleanup	Change
_app-HASH.js gzip	206 B	205 B	N/A
_error-HASH.js gzip	182 B	180 B	N/A
amp-HASH.js gzip	506 B	505 B	N/A
css-HASH.js gzip	322 B	323 B	N/A
dynamic-HASH.js gzip	2.57 kB	2.57 kB	N/A
edge-ssr-HASH.js gzip	260 B	259 B	N/A
head-HASH.js gzip	350 B	350 B	✓
hooks-HASH.js gzip	369 B	369 B	✓
image-HASH.js gzip	4.35 kB	4.35 kB	N/A
index-HASH.js gzip	256 B	256 B	✓
link-HASH.js gzip	2.64 kB	2.63 kB	N/A
routerDirect..HASH.js gzip	312 B	311 B	N/A
script-HASH.js gzip	385 B	384 B	N/A
withRouter-HASH.js gzip	307 B	308 B	N/A
1afbb74e6ecf..834.css gzip	106 B	106 B	✓
Overall change	1.08 kB	1.08 kB	✓

Client Build Manifests

	vercel/next.js canary	vercel/next.js refactor/web-streams-cleanup	Change
_buildManifest.js gzip	485 B	482 B	N/A
Overall change	0 B	0 B	✓

Rendered Page Sizes

	vercel/next.js canary	vercel/next.js refactor/web-streams-cleanup	Change
index.html gzip	527 B	529 B	N/A
link.html gzip	541 B	542 B	N/A
withRouter.html gzip	523 B	524 B	N/A
Overall change	0 B	0 B	✓

Edge SSR bundle Size Overall increase ⚠️

	vercel/next.js canary	vercel/next.js refactor/web-streams-cleanup	Change
edge-ssr.js gzip	93.6 kB	93.7 kB	N/A
page.js gzip	154 kB	155 kB	⚠️ +674 B
Overall change	154 kB	155 kB	⚠️ +674 B

Middleware size

	vercel/next.js canary	vercel/next.js refactor/web-streams-cleanup	Change
middleware-b..fest.js gzip	624 B	624 B	✓
middleware-r..fest.js gzip	150 B	151 B	N/A
middleware.js gzip	22.5 kB	22.5 kB	N/A
edge-runtime..pack.js gzip	1.92 kB	1.92 kB	✓
Overall change	2.55 kB	2.55 kB	✓

Diff details

Diff for page.js

Diff too large to display

Diff for edge-ssr.js

Diff too large to display

Commit: 6f3f023

wyattjoh · 2023-10-13T23:59:12Z

packages/next/src/server/lib/patch-fetch.ts

-              if (process.env.NEXT_RUNTIME === 'edge') {
-                const { decode } =
-                  require('../../shared/lib/base64-arraybuffer') as typeof import('../../shared/lib/base64-arraybuffer')
-                decodedBody = decode(resData.body)
-              } else {
-                decodedBody = Buffer.from(resData.body, 'base64').subarray()
-              }


This was added before the edge runtime supported Buffer, it now does, so we can remove!

wyattjoh · 2023-10-14T00:00:48Z

packages/next/src/server/stream-utils/encode-decode.ts

+export function createDecodeTransformStream(decoder = new TextDecoder()) {
+  return new TransformStream<Uint8Array, string>({
+    transform(chunk, controller) {
+      return controller.enqueue(decoder.decode(chunk, { stream: true }))
+    },
+  })
 }

-export function decodeText(
-  input: Uint8Array | undefined,
-  textDecoder: TextDecoder
-) {
-  return textDecoder.decode(input, { stream: true })
+export function createEncodeTransformStream(encoder = new TextEncoder()) {
+  return new TransformStream<string, Uint8Array>({
+    transform(chunk, controller) {
+      return controller.enqueue(encoder.encode(chunk))
+    },
+  })


Ideally we can swap this out for TextEncoderStream and TextDecoderStream but this hasn't been polyfilled in the edge runtime yet 👀

wyattjoh · 2023-10-14T00:05:39Z

packages/next/src/server/web-server.ts

-      // No, this cannot be replaced with `finally`, because early cancelling
-      // the stream will create a rejected promise, and finally will create an
-      // unhandled rejection.
-      writer.closed.then(onClose, onClose)


We've leveraging the native reader.pipeTo(writer) now which correctly handles cancellation.

wyattjoh · 2023-10-14T00:06:20Z

packages/next/src/server/stream-utils/node-web-streams-helper.ts

        return controller.enqueue(chunk)
      }

-      const content = decodeText(chunk, textDecoder)


The single layer of abstraction here made it difficult to understand, these are web primatives, we should use them as-is wherever possible.

wyattjoh · 2023-10-14T03:00:22Z

packages/next/src/server/node-polyfill-web-streams.ts

-    global.ReadableStream = ReadableStream
+// NOTE: any changes to this file should be mirrored in: test/__mocks__/node-polyfill-web-streams.js
+
+if (process.env.NEXT_RUNTIME !== 'edge') {


This extra wrap ensures that this polyfill doesn't run on edge.

wyattjoh · 2023-10-14T03:01:35Z

packages/next/src/server/lib/patch-fetch.ts

@@ -457,7 +460,7 @@ export function patchFetch({
              cacheKey &&
              isCacheableRevalidate
            ) {
-              const bodyBuffer = Buffer.from(await res.arrayBuffer())
+              const bodyBuffer = Buffer.from(await res.clone().arrayBuffer())


Reusing a consumed response is prohibited, so we have to clone it first.

That's unneeded in this case, because we return a new Response from the body buffer we create here.

wyattjoh · 2023-10-14T05:22:08Z

packages/next/src/server/font-utils.ts

+async function getFontForUA(url: string, UA: string): Promise<string> {
+  const res = await fetch(url, { headers: { 'user-agent': UA } })
+  return await res.text()


This was too simple to pass up updating 😅

shuding · 2023-10-16T15:26:14Z

packages/next/src/server/stream-utils/node-web-streams-helper.ts

+  const encoder = new TextEncoder()
  return new TransformStream({
-    async transform(chunk, controller) {
-      const insertedHTMLChunk = encodeText(await getServerInsertedHTML())
-      controller.enqueue(insertedHTMLChunk)
-      controller.enqueue(chunk)
+    start: async (controller) => {
+      const html = await getServerInsertedHTML()
+      controller.enqueue(encoder.encode(html))


This looks like a behavior change. The old code ensures that if there is new inserted chunk during the stream (after the start phase), it will still be enqueued. The new code changes that.

The default behevior for the TransformStream without a transform option provided for the sink is to pass it through without transforming, thereby ensuring that every future chunk will be just passed through.

https://developer.mozilla.org/en-US/docs/Web/API/TransformStream/TransformStream#transformchunk_controller

Never mind! I see, updating :)

packages/next/src/server/app-render/action-handler.ts

jridgewell · 2023-10-16T17:33:47Z

packages/next/src/server/body-streams.ts

-    start(controller) {
-      stream.on('data', (chunk) =>
+    start: async (controller) => {
+      for await (const chunk of stream) {
        controller.enqueue(new KUint8Array([...new Uint8Array(chunk)]))


Do we need to construct a Uint8Array just to clone it into a KUint8Array?

This was matching the existing beheviour seen in:

https://github.com/vercel/edge-runtime/blob/b53a2809bf9cb3f47e3f6e658b7a83f2069813b5/packages/runtime/src/server/body-streams.ts#L58

jridgewell · 2023-10-16T17:36:56Z

packages/next/src/server/base-http/web.ts

+
+  public async toResponse() {
+    // If we haven't called `send` yet, wait for it to be called.
+    if (!this.sent) await this.sendPromise.promise


Nit: Why not just return a promise, and resolve that promise with the response?

toResponse is (as expected) called every request. As soon as you call await it will pause execution to get the value of the promise (even if it's already resolved). To avoid unnecissary pauses, this check bypasses that.

But this is an async function, so this will always have at least 1 tick delay. If you just return a promise resolved to the Response, we keep the same number of overall ticks and (slightly) simplify this method.

packages/next/src/server/lib/incremental-cache/index.ts

jridgewell · 2023-10-16T17:41:14Z

packages/next/src/server/lib/incremental-cache/index.ts

          }
-          reader.read().then(processValue)
+
+          ;(init as any)._ogBody = arrayBuffer


Nit: Can this be done lazily? We need bodyChunks to generate the cache string, but do we always need this?

The body is used by the patchFetch as it's used to provide the original response:

next.js/packages/next/src/server/lib/patch-fetch.ts

Line 431 in 29c31d4

body: (init as any)._ogBody || init.body,

packages/next/src/server/web-server.ts

packages/next/src/server/pipe-readable.ts

packages/next/src/server/app-render/action-handler.ts

packages/next/src/server/stream-utils/node-web-streams-helper.ts

packages/next/src/server/lib/incremental-cache/index.ts

packages/next/src/server/pipe-readable.ts

jridgewell · 2023-10-17T03:15:57Z

packages/next/src/server/stream-utils/node-web-streams-helper.ts

+        controller.enqueue(encoder.encode(before))
+
+        // Flush the remaining part of the buffer.
+        controller.enqueue(encoder.encode(decoder.decode()))


Should this be pushing into buf?

buf isn't consumed after foundSuffix is set to true

ijjk added created-by: Next.js team PRs by the Next.js team. type: next labels Oct 13, 2023

wyattjoh force-pushed the refactor/web-streams-cleanup branch 2 times, most recently from 29a5605 to 6b1b79a Compare October 14, 2023 01:49

wyattjoh commented Oct 14, 2023

View reviewed changes

wyattjoh marked this pull request as ready for review October 14, 2023 05:24

wyattjoh requested review from timneutkens, ijjk, shuding, huozhi, feedthejim and ztanner as code owners October 14, 2023 05:24

wyattjoh force-pushed the refactor/web-streams-cleanup branch from cb33108 to beb5264 Compare October 16, 2023 05:38

shuding reviewed Oct 16, 2023

View reviewed changes

wyattjoh force-pushed the refactor/web-streams-cleanup branch from beb5264 to 9dba803 Compare October 16, 2023 15:43

shuding previously approved these changes Oct 16, 2023

View reviewed changes

jridgewell suggested changes Oct 16, 2023

View reviewed changes

wyattjoh dismissed shuding’s stale review via a55ad50 October 16, 2023 23:25

wyattjoh force-pushed the refactor/web-streams-cleanup branch from a55ad50 to 7f41b92 Compare October 16, 2023 23:42

jridgewell reviewed Oct 17, 2023

View reviewed changes

jridgewell previously approved these changes Oct 17, 2023

View reviewed changes

wyattjoh dismissed jridgewell’s stale review via 14bee29 October 17, 2023 20:05

wyattjoh force-pushed the refactor/web-streams-cleanup branch 4 times, most recently from 081c68f to 6928ffb Compare October 18, 2023 03:08

wyattjoh added 16 commits October 18, 2023 14:29

refactor: cleanup of web streams usage and render result streaming

5117ed1

fix: polyfill WritableStream

b383545

fix: ensure we use res.end() to flush buffers

1648a1e

fix: fixed import issue for edge

7253f6c

fix: ensure tests have WritableStream mocked

d930873

fix: send before pipe is finished to trigger send to return

5b04279

fix: re-introduce flushing due to compression middleware

c8bf05b

fix: actually wait on send promise..

d0b0bdc

fix: replace https with fetch

675d20d

fix: prefer async iteration vs callbacks

b325d8f

fix: ensure server inserted html is added as it's injected

e5e27f6

review: review comments

34f6286

review: update based on review

0ebb92f

fix: fix hanging on broken responses

c33cda8

fix: fix logic around suffix move stream

9b754da

fix: ensure we add an abort reason

c242a5f

wyattjoh force-pushed the refactor/web-streams-cleanup branch from 6928ffb to c242a5f Compare October 18, 2023 20:33

ijjk added Font (next/font) Related to Next.js Font Optimization. area: tests labels Oct 18, 2023

fix: always add the suffix, even if we don't find it

6f3f023

ztanner approved these changes Oct 18, 2023

View reviewed changes

kodiakhq bot merged commit 07c434d into canary Oct 18, 2023
58 of 60 checks passed

kodiakhq bot deleted the refactor/web-streams-cleanup branch October 18, 2023 21:38

github-actions bot added the locked label Nov 2, 2023

github-actions bot locked as resolved and limited conversation to collaborators Nov 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Web Streams cleanup #56819

Web Streams cleanup #56819

wyattjoh commented Oct 13, 2023 •

edited

Loading

wyattjoh commented Oct 13, 2023

ijjk commented Oct 14, 2023 •

edited

Loading

ijjk commented Oct 14, 2023 •

edited

Loading

wyattjoh Oct 13, 2023

wyattjoh Oct 14, 2023

wyattjoh Oct 14, 2023

wyattjoh Oct 14, 2023

wyattjoh Oct 14, 2023

wyattjoh Oct 14, 2023

jridgewell Oct 16, 2023

wyattjoh Oct 14, 2023

shuding Oct 16, 2023

wyattjoh Oct 16, 2023

wyattjoh Oct 16, 2023

jridgewell Oct 16, 2023

wyattjoh Oct 16, 2023

jridgewell Oct 16, 2023

wyattjoh Oct 16, 2023

jridgewell Oct 16, 2023

jridgewell Oct 16, 2023

wyattjoh Oct 16, 2023

jridgewell Oct 17, 2023

wyattjoh Oct 17, 2023

Web Streams cleanup #56819

Web Streams cleanup #56819

Conversation

wyattjoh commented Oct 13, 2023 • edited Loading

wyattjoh commented Oct 13, 2023

ijjk commented Oct 14, 2023 • edited Loading

Tests Passed

ijjk commented Oct 14, 2023 • edited Loading

Stats from current PR

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wyattjoh commented Oct 13, 2023 •

edited

Loading

ijjk commented Oct 14, 2023 •

edited

Loading

ijjk commented Oct 14, 2023 •

edited

Loading