nested @defer ordering

[yaacovCR]

EDITED BELOW 2023-12-31 and 2024-01-03

Continuation of #43, as updated by the new response format.

TLDR:

In ExampleB below, we have assumed that Inner must always follow Outer. The implementation currently makes Inner and Outer siblings for below ExampleA, but not for similar ExampleC. Why?

Nested Defers at the Same Level

Example operation, ExampleA:

      query ExampleA {
        me{
          ... @defer(label: "Outer") {
            fieldForOuter
            ... @defer(label: "Inner") {
              fieldForInner
            }
          }
        }
      }

Should Inner be sent once after Outer or should Outer and Inner be considered sibling defers?

Benefits of Always Preserving Defer Dependencies

Full conformance to "actionable payloads only" principle

The client has indicated that the component for Outer has an Inner portion that is "deferrable"; it also has non-deferrable portions. Those non-deferrable portions are not sent prior to all of the fields for Outer being ready. Why should the deferrable portions of Outer (i.e., Inner) be sent when ready before non-deferrable portions are ready? Just as individual fields for Outer such as fieldForOuter ((or nested fragment spreads!) are not sent until all of the fields for Outer are ready, neither should Inner.

More consistent handling of errors.

If Outer and Inner are made siblings:

• If Inner completes first but with an error causing the entire fragment to bubble up, Inner would send me as null with errors, and Outer would be suppressed as non-mergable, even though Inner is deferrable => Outer can be interactive without it => Inner could be useful as an error boundary. Actually, Inner would complete with errors as non-mergable into the final response, and then Outer would complete normally. This seems fine, as long as the client knows that a non-mergable fragment as a particular path does not preclude later mergable data at that same path, which seems fine, as that's the basic definition of non-mergable.

• If Outer completes second but with an error causing the entire fragment to bubble up, without this change, Inner would be successfully sent first with non-null me and a notification for Outer is sent as non-mergable. In this situation, Inner is actually useless to the client.

If the defer dependency is preserved:

• If Inner completes first but with an error causing the entire fragment to bubble up, Outer is sent first without errors, Outer is interactive, after which a notification for Inner is sent as non-mergable.
• If Outer completes second but with an error causing the entire fragment to bubble up, me is sent as null, useless Inner is not sent.

Benefits of Allowing Sibling Defers

What does "actionable" mean?

Inner is in a sense actionable as soon as it completes, because the graphql client building the "final reconcilable object" can do something with it, i.e. can insert the data under the me key within the initial result. Although the client application cannot yet use this data, the graphql client has a place to store it. This is as opposed to the following counter-example ExampleB:

      query ExampleB {
        ... @defer(label: "Outer") {
          me{
            fieldForOuter
            ... @defer(label: "Inner") {
              fieldForInner
            }
          }
        }
      }

In this case, until Outer completes, the graphql client cannot even store Inner in the final reconcilable object, it would instead have to save Inner in some sort of queue and then when Outer is sent with its "pending" notification for Inner, merge both. We have already considered that to be undesirable secondary to the "no action-less payloads" principle, but we allowed Inner to be sent prior to Outer in ExampleA, because even though it's not actionable by the client application, it's actionable by the graphql client.

To buttress support for the actionable claim for ExampleA, we could argue that it is useful for the client application if not immediately actionable, because it allows a form of pre-loading; Inner will be instantly available within the final reconcilable object. A distinction must then be drawn between preloading in ExampleA and Example B.

Avoiding Defer Creep

@leebyron has raised the concern of fragments sprawled across a code base leading to unintended unnecessary defers. This would seem to be a broader concern not entirely mitigated by specifically changing the behavior for nested defers at the same level, but it would certainly cut down on the number of unintentional defers (at the expense of ignoring intentional defers!).

Allowing More Siblings Based on Deduplication Effects

Let us assume that in Example A we have decided to make Outer and Inner siblings.

Consider the following operation, Example C:

      query ExampleC {
        me { fieldForInitial }
        ... @defer(label: "Outer") {
          me{
            fieldForOuter
            ... @defer(label: "Inner") {
              fieldForInner
            }
          }
        }
      }

Because of deduplication, me is sent within the initial result rather than within Outer, and so the operation is equivalent to:

      query ExampleCPrime {
        me { 
          fieldForInitial
          ... @defer(label: "Outer") {
            fieldForOuter
            ... @defer(label: "Inner") {
              fieldForInner
            }
          }
        }
      }

Similar to ExampleA, we might then expect Outer and Inner to be treated as siblings:

• The actionable claim for Inner in ExampleC and ExampleA are essentially equivalent. Offline, @robrichard noted to me that this was his expectation.
• The calculus in terms of unintentional defers vs. ignored defers raised by @leebyron above may be different with a potentially different expectation.
• One could also argue that because the actionability of Inner is not immediately obvious (and in fact may be unknowable until runtime depending in the presence on type dependencies/variables), only Outer and Inner should only be considered siblings in ExampleA and not in ExampleC. In fact, this is the state of the current implementation in graphql-js

Conclusion

In ExampleB, we have assumed that Inner must always follow Outer.

The status quo on main within graphql-js makes Inner and Outer siblings for ExampleA, but not for ExampleC, where Inner still follow Outer, i.e. Inner always follows Outer, except when they are at the same level.

I have raised separate PRs to explore the options of:

(A) making Inner always follow Outer, including where they are at the same level like in ExampleA, and
(B) sending Inner as soon as the graphql client can merge it into the final reconcilable object (i.e. making Inner and Outer siblings for ExampleC just like in ExampleA, but still not for ExampleB).

I have not raised a PR for the more aggressive option (C) of always sending Inner as soon as it completes, i.e. even in ExampleB, although I consider this not so much less reasonable than (B).

I lean at this point toward Option A, followed by Option B, with the current status quo my third choice, and lastly option C, but eager for input from others!

[yaacovCR]

I have modified the spec PR I have been working on for deduplicated incremental delivery to reflect this discussion.

It turns out that what I had been working on actually most closely matches Option B which is significantly easier to specify.

I have raised a different PR to implement option A. The first commit in that PR retains defer ordering across multiple levels, but not the same level, corresponding to the status quo of the implementation, and the last commit retains defer ordering even at the same level, i.e. Option A above. Most of the "work" within the PR is done by that first commit, with Option A is only slightly more complicated to specify than the implementation status quo.

We shouldn't necessarily do what's easiest to specify! It would be even easier to specify Option (C) where all defers are siblings, i.e. sent as soon as they complete, even if they cannot yet be merged into the final reconcilable object. That's still not the best option, because the simplicity of the specification created somewhat of a processing headache for the graphql client!

On the other hand, ease of specification is certainly something to keep in mind.

[robrichard]

My opinion is that we should move forward with Option B (graphql/graphql-js#4003 (i.e. making Inner and Outer siblings for ExampleC just like in ExampleA, but still not for ExampleB).

I have two reasons for this:

1. It makes the spec and implementation is simpler. In option B deferred groups can be created as the execution traverses into tree of fields, where in option A there is extra logic required to additionally track a separate tree of how the deferred fragments are nested. Option B therefore makes the algorithm simpler to understand and implement.
2. If we decide this is the wrong approach and this causes difficulty for many clients, we can change course in the future and update the spec to option A. At that point all clients that were implemented to follow the option B spec will still continue to work, as option A only requires additional restrictions on top of option B. The reverse is not true. If we start out with Option A, it would be a breaking change to introduce Option B in the future as clients would be dependent on the ordering specified in Option A.

[yaacovCR]

Above, I listed the benefits of option A and option B, but I left out an important point, related to early execution. @benjie in the linked discussion did a great job of explaining why early execution might lead to delayed delivery of the initial result. The example he used had only a single defer:

  {
    ... @defer { superExpensive }
    cheap
  }

My general conclusions from that discussion were that:
(A) we should try to write the specification in such a way that the "diff" between the early and deferred execution paths is minimal => this allows us to include both within the spec, or just specify one or the other and add a note.
(B) we should encourage server implementations to disable early execution by default and/or provide management tools for mitigating any deleterious effects.

Let's apply the above to the following operation:

  {
    ... @defer {
      ... @defer { superExpensive }
      cheap
    }
    reallyCheap
  }

According to Option B, this is the same as:

  {
    ... @defer { cheap }
    ... @defer { superExpensive }
    reallyCheap
  }

And so "technically" in some sense, cheap is not executed early with respect to superExpensive because we have promoted superExpensive to be a sibling.

But that very decision causes early execution with respect to developer intent (as well as early delivery)!

So I think we have to preserve (B) in this context => allow disabling this by default/provide mechanisms for management.

But what about (A), minimizing the difference in terms of the spec?

My initial experience was that delayed execution was easier to specify, but with some work I was able to pare that down under Option B. Now Option B (with its own early execution problems) is actually simpler to specify than Option A!

Bottom line, I don't think we have a choice though, because of the early execution issues, I think we need to specify Option A, with a normative note that this might be unnecessary if Option B is pursued. Although I will continue iterating! Continued feedback welcome! Looking forward to discussing at the next meeting!

[benjie]

Example of why the nested Inner must not be delievered until Outer is delivered:

fragment PostDetails on Post {
  body
  ... @defer {
    bodyHighlights # Uses an LLM
    PostDetailsIsReady: __typename
  }
}

query ($id: ID!) {
  currentUser { name }
  post(id: $id) {
    id
    title
    icon
    ... @defer { ...PostDetails }
  }
}

Here the PostDetails fragment's component knows it can render bodyHighlights against body once PostDetailsIsReady is present. If it tried to render bodyHighlights but body wasn't there yet, it would be very unexpected to the programmer when they have a fragment-centric understanding of the data.

[yaacovCR]

based on feedback from meeting today:

1. closed emit all deferred fragments as soon as possible graphql-js#4003 in favor of allow nested defers at the same level graphql-js#4002
2. pulled incremental: maintain defer fragment ordering graphql-spec#1072 into RFC: deduplicated incremental delivery graphql-spec#1052

[robrichard]

In today's incremental delivery wg, we decided to proceed with Option A. The example in #80 (comment) demonstrates that is likely what is most expected by the user, or require more complexity in clients to work around it.

Marking this discussion as resolved and opened #83 to discuss how the "pending/complete" objects in the response format should behave.