Improving Apollo's testing experience

[klaaspieter]

Introduction

A new Apollo link solely for testing purposes to allow for more finegrained control over what is mocked and when. And a way to automatically generate mock data for any operation.

Inspired by Relay: https://relay.dev/docs/en/testing-relay-components

Motivation

MockedProvider works for simple cases, but providing mocks upfront can get complicated fast. Components at the leaves of the component tree tend to do a single query, but when testing components at the root one has to provide mocks for each component in the tree. It can be very hard to figure out which queries will be done. Even when that work is done the test remains fragile because a single query being added or changed can change the result of the test.

Then there is the problem of large nested queries. Generating data for these can be error prone and it can be hard to debug when fields are missing (#115). I think the Relay documentation actually describes it best. Replace Relay with Apollo:

One of the patterns you may see in the tests for Relay components: 95% of the test code, is the test preparation: the gigantic mock object with dummy data, manually created, or just a copy of a sample server response that needs to be passed as the network response. And rest 5% is actual test. As a result, people don't test much. It's hard to create and manage all these dummy payloads for different cases. Hence, writing tests are time-consuming and painful to maintain.
— https://relay.dev/docs/en/testing-relay-components#testing-with-relay

Testing components with polling reliable is even near impossible. I have done some tests where extra mocks are provided. The results in the final mocks are then used to ensure that polling was stopped for example. Very roughly something like this:

<MockedProvider mocks={[
  { data: { status: "Loading" } },
  { data: { status: "Ready" } },
  { error: "Should never happen"} 
]}>
  <Component pollInterval={1} />
</MockedProvider>

/* 
  Wait an arbitrary amount of time to ensure that we get to the ready state.
  There is no 100% guarantee that the error state is ever hit when `stopPolling` isn't called
  because it all depends on these timeouts.

  In other words it's possible this test succeeds even when `stopPolling` is never called. 
  One can add more waits, at the cost of slower tests.
*/
await wait()
await wait()
await wait()

expect(queryByText("Should never happen")).not.toBeInTheDocument()

Proposed solution

A new Link: apollo-mock-link. The link queues all operations and only resolves or rejects them when asked to do so. It can be combined with a mock data generator that automatically generates valid responses for each operation.

Let's look at an example of nested component that runs 3 queries. One for the current user me, one for all todos which have a title field and finally a query for all unresolved todos.

const link = new MockLink()

const { getByText } = render(<MockedProvider link={link}><Component /></MockedProvider>)

expect(getByText("Loading…"))

// Return a response manually
link.resolveMostRecentOperation({ me: { name: "Hello" })

expect(getByText("Hello")).toBeInTheDocument()

// Return a generated response with default values for each field
link.resolveMostRecentOperation((operation) => fakeQL({ operation }))

expect(getByText(`mock-value-for-field-"title"`)).toBeInTheDocument()

// Return a generated response with the a specific `Todo.title`:
link.resolveMostRecentOperation((operation) => fakeQL({ 
  operation,
  resolvers: {
    Todo: { return { title: "Unresolved Todo" }} 
  }
}))

expect(getByText("Unresolved Todo")).toBeInTheDocument()

The above also works for polling which needs to stop after some sentinal value. Let's say we're polling for a status field to become READY:

link.resolveMostRecentOperation({ status: "LOADING" })

expect(getByText("Loading")).toBeInTheDocument()

link.resolveMostRecentOperation({ status: "LOADING" })

expect(getByText("Loading")).toBeInTheDocument()

link.resolveMostRecentOperation({ status: "READY })

expect(getByText("Ready")).toBeInTheDocument()
expect(link.pendingOperations.count).toBe(0)

Note: Astute observers may realize that the above example need waits because each query will be run on the next tick. These examples show ideal usage. I hope to figure out and propose a solution for the (un)necessary waits in tests in the future.

Alternatives

Shallow rendering

One alternative for component trees with many queries is to shallow render components at the root. This works if the queries are in nested components. If the root itself however has all the queries the same problems arise. Granted this is probably not great API design but not something discouraged by Apollo's APIs.

Manual mocks

The approach described here is mentioned often (also on this issue tracker). It however still requires making manual mocks for individual types in your schema. Large nested types are still hard to generate data for

Keep everything as is

The current MockLink already has the undocumented addMockedResponse method. An alternative could be to document that method. However it still requires mocking responses before they happen which has some flaws (#206, #173, #115)

[insidewhy]

Released this package which supports wildcards in mocks and a bunch of other nice functionality to help writing tests: https://github.com/insidewhy/wildcard-mock-link

I find that specifying requests ahead of time is acceptable, as long as you can make wildcard matches on the variables as this package supports. When it comes to subscriptions, it's very useful to send out responses using methods, so this package also supports that. It is documented and tested, and very happy to get feedback and add more functionality.

[klaaspieter]

@insidewhy That's awesome 🙏. In general I'm not a fan of the mock the world upfront it's definitely an improvement over what's there right now. I'm happy there are more people working on the same problem. For a while I was afraid it was just me experiencing these problems 😬

[insidewhy]

@klaaspieter I think I'll add a "match all requests" wildcard option, and then have methods for pushing out responses as needed. That should pretty much match your proposal while also supporting scenarios where it's useful to mock up front.

[martdavidson]

Just piling on that testing needs some love - especially around firing subscription events manually with MockedProvider and the combination of mocking useQuery and useSubscription in the same MockedProvider.

@insidewhy I'll be checking out your package shortly to see if I can just use it as a drop in replacement.

[insidewhy]

@martdavidson It's fully compatible and lets you fire subscription events manually, other features too!