Can a client know when individual streams are complete?

[robrichard]

Context

Originally posted by loganfsmyth graphql/graphql-spec#830.

It is not possible to know the status of individual streams if there are multiple fields with @stream directives in a single GraphQL operation, only a single hasNext field is provided in the payloads.

Decision

Not in scope for current spec proposal. Additional exploration needed (potentially using extensions field) to be done for a future enhancement.

[yaacovCR]

I think it makes sense to add this into the payload rather than the extension, although I understand it’s no emergency, makes sense to start speculating how we might do that to make sure current implementation doesn’t conflict.

i might naively imagine we would just have two different possible payloads, 1) stream payloads and 2) defer payloads (whether from original operation or deferred from stream payloads) where the former have an additional field streamHasNext or streamIsFinal.

But sometimes you don’t know the status of whether there is a subsequent stream until after the prior is delivered, and so I imagine that former approach would require a separate payload to signify that the stream is done, otherwise you would have to hold back a payload, which is a no no.

So I don’t know if an extra field or extensions field on a payload would work, we would need an entire separate payload (an extensions payload?) which unless built into the spec now, would presumably require breaking changes later. What would the path of that payload be? You can’t usually have repeat paths for stream fields, so that is another aspect that if not standardized now, might break later. Perhaps should be explicitly allowed to have repeat paths?

[yaacovCR]

if this potential feature whenever introduced was breaking, it wouldn’t be a tragedy, but we should avoid that if possible

[yaacovCR]

See https://github.com/graphql/graphql-spec/pull/742/files#r756814376

[yaacovCR]

I think we need to explicitly state within the spec that data and errors could both be empty for subsequent payloads, with path and extensions populated in that case

[robrichard]

I updated the spec to explicitly allow payloads with only hasNext and extensions, which should allow enough flexibility for any kind of additional metadata or experimentation. We can't also allow path because then we'd need to specify exactly what it means when data is not present, but that would be dependent on what's in the extensions.

[yaacovCR]

Agreed, even better! Thanks for the quick response!

[robrichard]

Discussed with @mjmahone and others at GraphQL Conf.

There was general agreement that this should not be part of the initial defer/stream spec and would benefit from going through its own RFC process.

One potential area of discovery is if this could be supported via the schema, i.e. metadata fields on Connections and/or Connection Edges.

Another option is a new boolean response field. There was strong preference that the hasNext field in the current proposal should not have overloaded meaning, and that nothing in the current spec proposal would block a new response field from being added in the future.

In current proposal, clients should consider streams completed when receiving hasNext: false. If a new field is introduced, clients can use this field to consider streams completed before receiving the final hasNext: false.

[robrichard]

Just to clarify, there was no opposition against this feature and we don't intend to discourage exploration. We agreed that it would be better to limit scope of the initial defer/stream proposal and this could be introduced in a future proposal.