@defer timeout in clients

[smyrick]

Overview

Today you can configure a timeout for a http request in apollo clients (http-link in React, ect), however @defer now allows us to split our operations into many smaller chunks. Operating at the HTTP level we currently can only sent a timeout for the entire request and there is no understanding that responses may be coming back in many parts

Suggested Changes

Allow clients to add a timeout value or a hook for calculating one based on the @defer label. This would allow us to specify a timeout per chunked response

[jerelmiller]

Hey @smyrick 👋

I'm curious, where are you seeing the timeout option for HttpLink? Perhaps I'm missing something here, otherwise I had no idea this was possible 🤣.

I'd be curious what you'd expect the behavior to be if that @defer timeout was reached for a deferred chunk. Seeing as @defer chunks can also be nested, how would you expect a potential "waterfall" of @defer directives behave? Would it throw a timeout error and abort the request, or skip that chunk of data?

Would love to get some more thoughts from you on how you see this working! Appreciate the feature request 🙂

[smyrick]

I'm curious, where are you seeing the timeout option for HttpLink? Perhaps I'm missing something here, otherwise I had no idea this was possible 🤣.

It is not an option directly, but something you can build with wrappers, see: https://github.com/drcallaway/apollo-link-timeout

I would expect if any defer timeout was hit that it would error out the remaining response. I know that means we might get partial data back, and not null values instead, but that first response should still be valid data.

Maybe to use defer timeouts it requires use of the defer.label arg?

query ProductQuery {
   products {
     id
     name
     ... @defer(label: "reviews") {
       reviews {
         title
         text
         ... @defer(label: "reviews.user") {
           user {
             name
           }
         }
       }
     }
   }
}

Then on query you pass in additional config

useQuery(PRODUCT_QUERY, {
  deferTimeout: {
    "reviews": 1000,
    "reviews.user": 3000
  }
})

Let me ask others who more directly want to use the feature for feedback

[smyrick]

The customer helped with some clarifications so let me add those.

I think the ideal scenario we want to support is to give a little more control to the client teams on how data is resolved and when a timeout is used. In a Federated architecture this shifts the logic from just client side to the Router + subgraphs and we need some way of communicating that from client to subgraphs as they are the ones actually resolving data.

Lets use 1 client, 1 Router, 1 subgraph for simplicity. This is the operation today:

query MyOperation {
  products {
    name
    reviews {
      text
    }
  }
}

In the subgraph code we do call two different databases to resolve this, but that is hidden in the schema, but the same concept works even if this is across two subgraphs.

For the Product.name field the server has a configured timeout of 5s and Review.text is 10s. We rarely ever hit these but they are there just to not keep connections open for long. However that means, that we may be still waiting around for 10s for this operation if the name field was fast under it's timeout and the reviews field was slow and did timeout. Instead we want the client to be able to indicate what the server-side timeout should be

query MyOperation {
  products {
    name @defer(timeout: 3000)
    reviews {
      text @defer(timeout: 5000)
    }
  }
}

This should let subgraphs know they should wrap their resolver with a timeout block and error if that is not met.

This would require a lot of coordination across client, Router, and subgraph.