Skip to content

Align process.nextTick execution order with Node - #4409

Merged
Jarred-Sumner merged 7 commits into
mainfrom
jarred/redo-nextitck
Sep 6, 2023
Merged

Jarred-Sumner merged 7 commits into
mainfrom
jarred/redo-nextitck

Conversation

@Jarred-Sumner

@Jarred-Sumner Jarred-Sumner commented Aug 30, 2023 •

Copy link
Copy Markdown
Collaborator

What does this PR do?

In Node, process.nextTick runs before microtasks are drained. Previously, Bun was treating process.nextTick as the same queue as the microtask queue. This led to subtle bugs.

Fixes #4252, probably others

The code is loosely based on https://github.com/nodejs/node/blob/main/lib/internal/process/task_queues.js. The main difference is that our implementation is lazier and we don't emit on unhandledexception callbacks currently.

How did you verify your code works?

A couple tests. Have to avoid referencing the process object inside outer function to avoid causing it to infinite loop. Using the $createFIFO builtin creates too large of an array in stress tests, causing an infinite loop. That's why it uses this fixed buffer abstraction from node.

I think we will need to make one small tweak to support AsyncLocalStorage, but @paperdave will know more about that than me

@github-actions

github-actions Bot commented Aug 30, 2023 •

Copy link
Copy Markdown
Contributor

❌ @paperdave 7 files with test failures on bun-darwin-aarch64:

  • test/cli/install/bunx.test.ts
  • test/js/bun/resolve/resolve.test.ts
  • test/js/bun/spawn/spawn.test.ts
  • test/js/bun/stream/direct-readable-stream.test.tsx
  • test/js/bun/test/test-test.test.ts
  • test/js/node/fs/fs.test.ts
  • test/js/node/watch/fs.watch.test.ts

View test output

#1ea08176d387f4c96fed69aca03ca8becae1cded

@github-actions

github-actions Bot commented Aug 30, 2023 •

Copy link
Copy Markdown
Contributor

❌ @paperdave 2 files with test failures on linux-x64:

  • test/bundler/esbuild/splitting.test.ts
  • test/js/web/fetch/fetch-gzip.test.ts

View test output

#1ea08176d387f4c96fed69aca03ca8becae1cded

@github-actions

github-actions Bot commented Aug 30, 2023 •

Copy link
Copy Markdown
Contributor

❌ @paperdave 2 files with test failures on linux-x64-baseline:

  • test/bundler/esbuild/splitting.test.ts
  • test/js/bun/util/filesink.test.ts

View test output

#1ea08176d387f4c96fed69aca03ca8becae1cded

@github-actions

github-actions Bot commented Aug 30, 2023 •

Copy link
Copy Markdown
Contributor

❌ @paperdave 8 files with test failures on bun-darwin-x64-baseline:

  • test/js/bun/spawn/spawn-streaming-stdin.test.ts
  • test/js/bun/spawn/spawn.test.ts
  • test/js/bun/sqlite/sqlite.test.js
  • test/js/bun/util/filesink.test.ts
  • test/js/node/fs/fs.test.ts
  • test/js/third_party/postgres/postgres.test.ts
  • test/js/third_party/webpack/webpack.test.ts
  • test/js/web/timers/setTimeout.test.js

View test output

#1ea08176d387f4c96fed69aca03ca8becae1cded

@paperclover paperclover left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been checking with node and this implementation is incorrect, nextTick runs after queueMicrotask as of node 20. Hopefully switching this around isn't too hard, but I don't think onEachMicrotaskTick will do the trick.

let resolve;
let promise = new Promise(res => {
  resolve = res;
});

const order = [];
let nextTickI = 0;
let microtaskI = 0;
let remaining = 20;
let runs = [];
for (let i = 0; i < 10; i++) {
  queueMicrotask(() => {
    runs.push(queueMicrotask);
    order.push("queueMicrotask " + microtaskI++);
    if (--remaining === 0) resolve(order);
  });
  process.nextTick(() => {
    runs.push(process.nextTick);
    order.push("process.nextTick " + nextTickI++);
    if (--remaining === 0) resolve(order);
  });
}

await promise;

console.log(order);

yields:

image

I'm going to update the tests to reflect this, as well as use jest as a baseline to ensure that they pass on node too.

Comment thread src/bun.js/bindings/ZigGlobalObject.cpp Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is the handler cleared here? it doesn't seem to ever be restored.

also we now need to coordinate this with the async_hooks setOnEachMicrotaskTick overwriting this handler temporarily.

I will push some tests that hit these edge cases

@paperclover

paperclover commented Aug 30, 2023 •

Copy link
Copy Markdown
Contributor

Actually, I'm incorrect. Having this not be at the top level in node causes the order to be as you implemented it. going off my prev script,

 let resolve;
 let promise = new Promise(res => {
   resolve = res;
 });

 const order = [];
 let nextTickI = 0;
 let microtaskI = 0;
 let remaining = 20;
 let runs = [];
 for (let i = 0; i < 10; i++) {
+  process.nextTick(() => {
     queueMicrotask(() => {
       runs.push(queueMicrotask);
       order.push("queueMicrotask " + microtaskI++);
       if (--remaining === 0) resolve(order);
     });
     process.nextTick(() => {
       runs.push(process.nextTick);
       order.push("process.nextTick " + nextTickI++);
       if (--remaining === 0) resolve(order);
     });
+  });
 }

 await promise;

console.log(order);

image

I'm sure you checked this page: https://nodejs.org/en/docs/guides/event-loop-timers-and-nexttick#phases-overview

@paperclover paperclover left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

merge if tests pass

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator Author

Tests do not pass, we need to address the exit code issue and possibly this readline promises test failure

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator Author

Let’s merge after tests run (you’ll need to fix the conflicts)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unpredictable behavior using fastify server encapsulation

2 participants