[RFC] Next.js as a test runner

[dvoytenko]

Goals

1. Allow unit and light integration tests to be executed within Next
2. Enable tests to mock browser APIs, such as fetch
3. Compile test code together with the user code to enable mocking of application APIs, such as classes, functions, and methods.

Non-Goals

1. Implementing comprehensive test framework, such as Jest/Karma/Jasmine/Sinon
2. Implementing custom testing tools, such as Fetch and other API mocking.

Background

There are many ways to test Next applications, such as using Jest or other test runners. However, these tests have to mock away Next APIs and workflows. This approach works well for many tests, but it's missing deeper integration testing with Next. This is especially relevant for App Router.

Proposal

Consider next test command that would enable:

1. Running a part of Next server with a broad set of Next APIs enabled for testing, such as next/navigation, next/headers, and so on.
2. Enable tests to use tools, such as MSW and nock, that can intercept Node and Browser APIs in the same VM to enable mocking, including the Fetch API.
3. Compile test and user code together to allow tests to mock user APIs, such as classes and functions.

A lot of nuances for the test runner are still TBD:

• Do tests run on the server, in the browser, or both?
• Is it possible to compile tests to mock code in both server and browser?

Context:

• See the Concept: test mode for Playwright and similar integration tools #52520 concept that explores e2e and deep integration testing using Playwright and similar tools. Tests, browser, and Next server are all running in separate VMs. This concept allows tests to perform limited mocking of Node/Next/Browser APIs, such as Fetch API. However, it doesn't allow a test to mock the application code, and can only mock a limited set of APIs directly supported by Next.
• See Support Next.js 13 (App Router) mswjs/msw#1644 on struggles of mocking fetch in Next for MSW tests.

[dvoytenko]

From @kettanaito from the POV of MSW:

On whether the Next's instrumentation hook helps:

Regarding the instrumentation hook, you can read mswjs/msw#1644 (comment) to see why the instrumentation hook in its current shape doesn't give us the right means to integrate API mocking into Next.

On fetch patching in DEV:

I'd like to see more confirmation that fetch patching is a dev-only thing. Next doesn't provide any surface to implement side-effects, like patching fetch before its internal store/restore logic of fetch. If it did, that could've been one way to approach it.

On overall needs for Next and MSW integration:

1. Next shouldn't meddle with fetch. If it does, it should give third-parties the means to meddle with fetch and don't override their functionality, like it does with MSW by overriding global fetch with whichever state of that function Next stored when it was bootstrapped.
2. Next should give the means to apply process-wide server-side side-effects. A very simple example of this is the means to introduce a global shared between server/client processes (Next has a few Node.js processes running concurrently, and they don't share the context, naturally). This is not possible in Next as of now, and this is perhaps the biggest blocker.

[kettanaito]

Happy to see this proposal being brought up.

One important aspect that I also shared in #52520, is that API mocking transcends the use cases for testing, as people are also using it for development and debugging. This is why I find the next test proposal to be rather limited and inefficient to cover all those use cases.

Instead of focusing inward on Next, as in adding support for testing capabilities within Next, I highly encourage the team to think outward, and instead provide versatile implementation from Next to the third-party consumers.

I believe that the instrumentation.js hook is a good starting point. I like that hook. My only concern is about when it executes (see mswjs/msw#1644 (comment)). Shortly, it executes after Next does its internal fetch patching, which is unfortunate. Can somebody assess the impact of the instrumentation hook executing before Next patches fetch?

The expected order to support third-party API mocking libraries that exclusively rely on patching request-issuing modules would be the following:

• Next server starts.
• Next executes instrumentation.js, which allows third-party modules to patch fetch.
• Next stores the state of fetch internally, just as it does now, with the exception that this is a potentially modified state by the instrumentation hook. Upon consequent re-renders, Next restores the fetch from the same state, meaning that whichever modifications were made to it by third-party libraries in the instrumentation hook persist.
• Next keeps serving the app.

If that would be the case, then third-party libraries would be able to do their job and enable API mocking for any use case with Next: development, testing, debugging, etc. It is simply a more versatile approach and, potentially, even more concise one.

[kettanaito]

The above also works on the assumption that the instrumentation hook executes for all internal Node.js processes of Next (Next has two), since both those processes are responsible for rendering the app, and both should have the request-issuing modules' patches applied to enable API mocking across the entire app.

[philwolstenholme]

+1 for this approach, as a Next user interested in mocking and using MSW, this is the primary functionality that I have been missing in my day job, not using Next in a special test mode (which sounds interesting but not something my team have run into needing yet, whereas we'd love to be able to use MSW on both the server and client with Next!)

[nickserv]

I'm working on a renderServer API for React Testing Library. Can we discuss API architecture? Would be great to understand how this works internally so React Testing Library can consume it and we can deduplicate work.

[ganicus]

Reliable, documented testing practices for API mocking with next app router would be wonderful. It's hard to justify adopting the framework for larger projects without this.

[kaikun213]

Bumping this, any updates?