Repository navigation
[Fusion] Write single JSON results straight into the response pipe - #10624
Merged
Merged
Conversation
The composite result is written straight into the response buffer when the writer neither indents nor omits nulls. Property names come from a token cached on the selection that already holds the quotes and the colon, so no name is checked for escaping, and every token is placed through one cached span.
The headers of a single result written as JSON are committed before the body is formatted, so Kestrel writes the body straight into its output pipe instead of buffering it until the first flush and copying it behind the headers.
The minimized result emitter requests spans of at least 2 KB, which the response buffer writer now hands out from its underlying pipe, so the composite result is no longer copied from the 64 KB staging buffer into the response pipe. Token by token writers keep using the staging buffer.
tobias-tengler
changed the base branch from
main
to
tte/fusion-minimized-result-writer
October 19, 2026 06:44
…ipe-handoff # Conflicts: # src/HotChocolate/Fusion/src/Fusion.Execution/Execution/Nodes/Selection.cs # src/HotChocolate/Fusion/src/Fusion.Execution/Text/Json/MinimizedJsonEmitter.cs # src/HotChocolate/Fusion/src/Fusion.Execution/Text/Utf8StringCache.cs # src/HotChocolate/Fusion/test/Fusion.AspNetCore.Tests/ResponseFormattingTests.cs
michaelstaib
marked this pull request as ready for review
October 22, 2026 06:07
Contributor
Patch coverage100.0% of changed lines covered (6/6)
Project coverage: 59.2% (314248/530638 lines) |
This was referenced Oct 25, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A single JSON result body is copied twice after it is formatted: once from the 64 KB staging buffer of
ChunkedPipeBufferWriterinto Kestrel's body writer, and again by Kestrel, because the response has not started yet and Kestrel has to move the buffered bytes behind the headers at the first flush.This PR removes both copies with two separate commits. The first starts single results written as
application/graphql-response+jsonorapplication/jsonright before the body is formatted (after all headers are set), so Kestrel writes the body straight into its output pipe. The second letsMinimizedJsonEmitterrequest 2 KB spans and lowers theChunkedPipeBufferWriterdirect-write threshold to 2 KB, so the minimized result is written into the response pipe memory instead of the staging buffer.Stacked on #10617 (the base branch is
tte/fusion-minimized-result-writer, so the diff shows only this change).ResponseSerializationBenchmark(87 KB result), base is a tree with #10617 applied.FormatAsync_BlockPipeuses a pipe with 4 KB blocks like Kestrel's output pipe.k6 end-to-end screen (
constant, A B B A, 3 x 30 s per invocation), base includes #10617:In the stalled Apollo Federation pair (one base and one PR invocation) the gateway sat at about 22% CPU and half the throughput, an environmental stall that hit both builds alike.
The handoff cost on top of plain formatting drops by about 60% in the micro. End to end, composite gateway CPU per request drops 2.4%, beyond main's A/A noise (CV 0.5 to 0.8%), and both PR invocations are below both base invocations; the composite RPS delta is within main's A/A noise (sigma about 65 RPS) and p99 is flat. Apollo Federation points the same way in both pairs, but with one clean pair it is supporting evidence only.
Behavior change:
OnStartingcallbacks now run before the body is serialized, and an exception thrown while serializing a single JSON result after the response started aborts the connection instead of producing a 500.