Support defer

[wincent]

In our public talks we've discussed Relay's ability to divide your queries into required and deferred parts, wherein your component won't get rendered until all of the required data that it requested is available, and it will get rendered again subsequently once the deferred data has arrived. There are a bunch of use cases for this, but the typical pattern is dividing your UI into a cheap, critical section that must be displayed fast before you can declare TTI, and a potentially more expensive but less crucial section which you're happy to display later on (example: a post [required] and its attached comments section [deferred]).

So we've talked about it, it works internally at FB, and you can see traces of support for in the code (such as this), but it's not currently exposed externally. This issue is for tracking progress towards getting this ported and released in some form.

(Extracted per comment here.)

[devknoll]

@wincent Any info you can provide for how this will work & what it'll entail to set up on an existing GraphQL server? 👍

[wincent]

@devknoll: Internally, the way defer works is that you can denote certain fragments as "deferred", distinguishing them from the "required" part of the data. Canonical use-case example: the data for showing a post is "required" and rendering should be blocked until it arrives, but the data for the comments on the post can be "deferred" and get shown later, on arrival.

You can see code in splitDeferredRelayQueries that traverses the query splitting off these deferred fragments and producing a tree of new queries: the required part, and any deferred parts; note that the deferred parts are themselves recursively broken down into "required" and deferred parts. Note that you can't necessarily fetch all these parts in parallel, because there may be a data dependency here where you won't know for which object to fetch some deferred data until you get the ID of the object in the response to the required part of the query.

Internally, we have a "batch" endpoint for the incumbent version of GraphQL that knows how to understand this kind of graph of queries with possible interdependencies. It also understands that one of these dependent queries may need to reference the result of the query that is its dependency, and it knows how to orchestrate all this and flush it all back to the client in chunks.

When we open-sourced GraphQL, we did not include an equivalent notion to this batch endpoint because we wanted to keep the spec minimal and we view the batch endpoint as a separate layer above and not part of GraphQL itself. Additionally, we wanted to take the opportunity to reset and revisit the assumptions that we'd accumulated over years of internal GraphQL use. We didn't want to bake too much in, in a way that would limit our options for implementing batch-like semantics, or others like streaming, prioritization and subscriptions and so on. Many of these ideas are being discussed on the GraphQL Slack channel and on GitHub if you want to learn more.

Anyway, in terms of what all this means for defer, there is a short-term plan and a longer-term one.

The short-term plan is to enable use of defer by doing query splitting and managing the orchestration of the split queries entirely on the client side. In other words, we can send batches of queries and do dependency management to hold back dependent queries until they have the data they need in order to be fetchable. This approach will incur some overhead, because it may require multiple round trips to the server, but it at least enables defer to be used.

The longer-term plan is to continue to work with GraphQL to flesh out the semantics and directives that would need to be in place to get a more integrated approach to deferring data, one which wouldn't depend on client-side management of multiple round trips. We're being deliberately non-committal about this because we want to involve the community and don't want to commit prematurely to a course of action that could limit our options later on.

In the meantime, there is a mid-term possibility, which is that it's all just JavaScript: if you want to take the client side batching logic and put a server-side layer in front of your GraphQL endpoint that gets rid of the roundtrips and does all the management server-side, then that should be totally possible. But for now, the immediate step is to get the purely client-side prototype out the door, which is what I am working on.

[anytimecoder]

Any reason why getFragment().defer() is not yet officially out? Relay seems to be supporting it - there's lots of isDeferred() checks in the code and looks like class GraphQLQueryRunner ignores deferred queries when updating the readyState.
Furthermore - I've just removed the supports('defer') check in GraphQLQueryRunner and at first glance everything seems to work as expected, at least for my simple needs:

• the query is split into deferred and non deferred queries,
• component is rendered on first response to arrive,
• then re-rendered when the next (deferred) response arrives.

That's of course the short-term plan described in the post above, but I'm perfectly happy to accept the overhead of extra queries for the deferred functionality to be available

[wincent]

@anytimecoder: I'm guessing it only seems to work because the RelayDefaultNetworkLayer is going to send the queries in parallel and in your case, your split deferred queries don't happen to have any dependencies on one another and they also happen to be arriving back in the desired order.

In the real batch implementation, we have the notion of dependent queries, such that you might have an initial query foo{bar{baz{id}}} and then something else that is deferred underneath baz, and which we can't even begin to fetch until we know baz's id. We fetch baz first, and only once we know its id do we proceed to fetch the deferred stuff under baz: eg. node(id: $baz_id){stuff{under{baz}}}) (note: another place where this can easily come up is if you defer something that is inside a connection). If your queries are independent, or baz isn't independently refetchable, Relay will end up querying for foo{bar{baz{id}}} and foo{bar{baz{stuff{under{baz}}}}}, which is what may be making it look like it's working.

[anytimecoder]

@wincent you're right - I'm deferring independent queries and I don't care about the order they're coming back, but I'm happy to comply to these restrictions to have defer() working.

[hmaurer]

What is the progress on this issue? Can we expect to see defer in the OS version any time soon? Thanks!

[josephsavona]

@A3gis Thanks for asking.

We aren't actively working on this right now and probably won't be able to focus on soon (see the Roadmap and Meeting Notes for current priorities of the core team). However, we're happy to support the community in building support for this in an open-source network layer. This could be a good candidate for adding to https://github.com/nodkz/react-relay-network-layer, for example (cc @nodkz).

[nodkz]

Just released new version of [email protected]:

• add gqErrorsMiddleware to display graphql errors from server
• add experimental deferMiddleware.

Right now deferMiddleware just set defer as supported option for Relay. So @anytimecoder may use this middleware to avoid hacking GraphQLQueryRunner. Also this middleware allow to community use defer in cases, which was described above by @wincent.

So I'll try to play deeper with defer fragments in near future.

[Globegitter]

I have started to work on this for a pure client side solution as @wincent suggested in one of the posts above. For some use-cases simply activating defer does indeed work. For other use cases we quickly ran into: Uncaught Invariant Violation: printRelayOSSQuery(): Deferred queries are not supported..

I managed to hack together a NetworkLayer that gets around this invariant, by accessing request._query and other bits I probably shouldn't touch. But I could not find another way to just in the NetworkLayer. Does that mean to support all use-cases code also needs to change in Relay and not just in the NetworkLayer?

The other thing I don't entirely understand is the splitDeferredRelayQueries function. Why does it split certain queries into ref queries?
E.g. we have

fragments: {
  customer: () => Relay.QL`
    fragment on Customer {
        holding {
          critical
          ${Component.getFragment('holding').defer()} // which then has a fragment on Holding
        }
        ${Component.getFragment('customer').defer()}
       some
       required
       fields
    }`
}

As long as I just have ${Component.getFragment('customer').defer()} it all works fine and the split queries look as I would have expected. But as soon as I add the holding fragment it splits the deffered fragment off into a nodes query as seen in this test: https://github.com/facebook/relay/blob/master/src/traversal/__tests__/splitDeferredRelayQueries-test.js#L290

What I would have expected is simply these 3 queries:
1:

{
  customer {
        id
        holding {
          critical
        }
       some
       required
       fields
  }
}

2:

{
  customer {
        id
        holding {
          deferred
          fields
        }
  }
}

3:

{
  customer {
        id
       other
      customer
      deferred
      fields
  }
}

Then at least conceptually all queries should work independently. Relay only needs to know that 2 and 3 are deferred, so it can render as soon as 1 has arrived. How come there is this distinction of ref queries? Is this normally handled by the server? How exactly can one use this batchCall information that is stored on the query?

Would my suggested approach make sense for handling everything on the client side? Or given some other directions I would be happy to take a stab at this.

@nodkz have you had any chance to look into this more?

[josephsavona]

Great questions. Deferred queries use "ref" queries to handle cases where a fragment is deferred at a point without a known ID. In the example you gave, it's possible for the value of customer.holding to change between execution of the original and deferred fragments, but we want the results of the fragments to be internally consistent. Ref queries allow a deferred query's input to depend upon the results of a previous query in the same "batch" (a set of queries that are requested at the same time).

To handle ref queries, you'd have to delay fetching them until the query they reference is fetched, extract the value from the parent, and use that as the input to the dependent query. Feel free to share a gist with what you have so far, and I'd be happy to comment with suggestions for handling the ref query case.

However, we have some good news on this issue: we've implemented a variant of our batch API in pure JavaScript as a layer over the standard GraphQL resolve function. The actual async function to resolve a query is passed as an argument, so it can be used by Relay on the client (with multiple round trips) or in the server to fetch multiple queries in one http request (if you're using node & GraphQL-JS). We'll be open sourcing this soon as part of the new relay core.

[Globegitter]

@josephsavona Yes, at the very least for understanding's sake it would be great if you could comment on my little hack: https://gist.github.com/Globegitter/553f7dffd1f7ceaead1aba0dd56c5554

Even though we are not using js on the backend this is great news! Either if we can use it on the client-side or just port it ourselves to python. And this batch API would just as-is take care of deferred queries?

[wincent]

I'm going through and cleaning out old issues that were filed against Relay 1, so I'll close this one. As @josephsavona mentioned, we have the primitives in place to straightforwardly implement a @defer-like directive in Relay 2. Thanks to everybody who commented on this issue!

[bchenSyd]

@wincent @josephsavona @devknoll does Relay 2 officially support 'short term' deferred queries?

I have a similar user case as your "post and comments" example.
let's say I came to the post page via "search by keyword", and I have a section that will display either 'loading comments' or actual comments depending on whether comments data is ready.

At the moment we are using relay 1 and I'm handling the 2 queries manually. Like you said, the comments query depends on post query because we don't know the post id until the post query is resolved by server (we only know keywords initially). This is my demo code

import React from 'react'
import Relay, { createContainer } from 'react-relay'

class MyPost extends Component {


    componentWillMount() {
        //relay guarantee that post query has returned by now
        const {viewer: {post_by_keyword: {id}}, relay} = this.props
        relay.setVariables({
            post_id:id,
            should_fetch_comments: true //variables set here only availble in the next render
        })
    }

    isCommentsReady() {
        const {  relay: {
            variables,
            variables: {
                should_fetch_comments
            },
            pendingVariables
        }
        } = this.props

        if (!should_fetch_comments) {
            //this is the initial render where we haven't even started fetching comments
            return false
        }
        if(pendingVariables && 'should_fetch_comments' in pendingVariables){
            //we have started fetching, but the data hasn't arrived yet
            //this is called multiple times "NETWORK_QUERY_START"..etc
            return false
        }
        //I've finally got NETWORK_QUERY_RECEIVED_ALL and data is ready now
        return true

    }

    render() {
        return <div>
            <div>
                {/*display post content here*/}
            </div>
            <div>
                {this.isCommentsReady() ?
                    <div>loading comments...</div>
                    : <div>{/*display comment list here*/}</div>
                }
            </div>
        </div>
    }
}

export default createContainer(MyPost, {
    initialVariables: {
        post_keyword: '',  //keywords is passed from Parent component via 'Overridding Fragment Variables'
        post_id: null,
        should_fetch_comments: false
    },
    fragment: {
        viewr: () => Relay.QL`
             fragment on viewer{
                    post_by_keyword(keyword:$post_keyword){
                     id,
                     title,
                     create_time,
                     content
                    },
                    comments_by_post_id(post_id:$post_id) @include(if:$should_fetch_comments){
                           user,
                           create_time,
                           content
                     }
             }
        `}
})

as you can see, the componentWillMount and isCommentsReady are boilerplate code and should be handled somewhere else. I would assume I could

• remove boilerplate code. remove should_fetch_comments
• wrap my comnets_by_post_id in a component
• call ${comments_by_postid.getFrament('comments',variables}.defer()?

My questions are:
(---try to anwser my own question)

• how do I pass additional parameters to my deferred query?
  ----no you can't. dependent node 's id is the only parameter that Relay passes to your deferred query. Not more other parameters can be passed
• in my MyPost component, how do I know whencomments fragment gets resolved?
  ---you can only check whether the field in props is null. There is no pendingDeferredQueries available in relay as this feature is not supported yet
  I hope it all makes sense. thanks for reading my long post
  (also can you also point out whether I'm following the right relay pattern?
  --yes it is as conformed by Greg int the follow ups)

[bchenSyd]

(i know it's awkward to explicitly pass $post_id to comments_by_post_id.
If I use deferred query, should I define comments inside post so that once post is resolved, Relay will automatically passes post's id to comments's deferred query by magic?

[wincent]

The initial release of the new core and API will not include @defer support. We've found in porting existing large apps at Facebook that the manual equivalent (I should really say near-equivalent because there is a difference between running multiple round trips from the client and doing a server-side phased request, but in practical terms in real apps it is close) has been sufficient, so we don't want to delay the release for a feature which is closer to a "nice to have" than a "launch blocker".

Having said that, the core primitives are in place to make it possible to add this, so it's something that we look forward to doing after the release. Clearly there is an interest in making a convenient abstraction over this usage pattern and other similar ones around streaming data, optional data etc.

[bchenSyd]

thanks @wincent . is it a way for client to know when deferred queries are resolved (like pendingDeferredQueries, just like pendingVariables)? or do I have to check the deferred fields are null in code.
@josephsavona. I'm not quite familiar with nodes query. How is it different from node query and connections query? is the query type created only for deferred query?

 expect(deferred[0].required).toEqualQueryRoot(
      filterGeneratedRootFields(getRefNode(
        Relay.QL`
          query {
            nodes(ids:$ref_q1) { // nodes query?
              ${fragment}
            }
          }
        `,
        {path: '$.*.hometown.id'} //path to extract  result from dependent query
      ))
    );

https://github.com/facebook/relay/blob/master/src/traversal/__tests__/splitDeferredRelayQueries-test.js#L323

[wincent]

is it a way for client to know when deferred queries are resolved

Given that there is no support, no. When the feature is actually built, we'll have some mechanism in place for this. For now the way to fake it is to either render a new root container after the initial fetch and render (triggered by calling setState in your componentDidMount callback) or fetch additional data by toggling a variable (setVariables) that switches an @include/@skip directive on a fragment (again, from componentDidMount).

nodes is just a shorthand that skips over the edges field in the connection; ie. the following are equivalent:

someConnection(first: 10) {
  edges {
    node {
      name
    }
  }
}

someConnection(first: 10) {
  nodes {
    name
  }
}

In practice, we don't recommend implementing nodes: explicit is generally better than implicit, and having two ways to access connection nodes can lead to confusion about which one is best used, and it can shield people from learning how connections are actually structured in reality, something which will likely cost them later on.

[bchenSyd]

thanks @wincent that helps me a lot!
"to either render a new root container after the initial fetch and render (triggered by calling setState in your componentDidMount callback) or .." ((I'm familar with the latter one and use it a lot in my work)
can you please elaborate on the former pattern? e.g. what is the user case for it. thanks