[Fusion] Support request batching for legacy servers

[tobias-tengler]

Adds a new setting to enable request batching to communicate with legacy servers to the schema-settings.json:

{
  "name": "products",
  "transports": {
    "http": {
      "url": "...",
      "batchingMode": "REQUEST_BATCHING"
    }
  }
}

Batching is disabled by default since Hot Chocolate v13, so you'll have to enable this in your subgraphs:

builder.MapGraphQL()
      .WithOptions(new GraphQLServerOptions { EnableBatching = true }));

[Copilot]

Pull request overview

This pull request adds support for Apollo request batching (legacy servers) to the HotChocolate Fusion GraphQL gateway. The main objective is to enable compatibility with older GraphQL servers that use the traditional array-based batching format instead of the newer JSONL/SSE streaming formats.

Changes:

• Added SourceSchemaHttpClientBatchingMode enum to support both variable batching and Apollo request batching modes
• Implemented JsonResultEnumerable to parse both single JSON objects and JSON arrays in responses
• Modified the JSON parser to support early termination when parsing individual objects from arrays
• Updated test infrastructure to support batching configuration and validation

Reviewed changes

Copilot reviewed 77 out of 80 changed files in this pull request and generated 14 comments.

Show a summary per file

File	Description
SourceSchemaHttpClientConfiguration.cs	Added batching mode configuration and Accept header management for different request types
SourceSchemaHttpClientBatchingMode.cs	New enum defining batching modes (VariableBatching, ApolloRequestBatching)
SourceSchemaHttpClient.cs	Implemented Apollo request batching support using OperationBatchRequest
JsonResultEnumerable.cs	New reader for parsing JSON array responses from legacy servers
SourceResultDocument.Parse.cs	Added early exit support for parsing individual documents from arrays
SourceSchemaResult.cs	Optimized error property access by caching during construction
SseEventParser.cs	Fixed line ending calculation (contains bugs)
SseReader.cs	Fixed buffer cleanup order (contains bugs)
LookupTests.cs	Added comprehensive tests for all batching modes
Migration documentation	Added TODO for documenting JSONL response format change

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

🚀 Fusion Gateway Performance Results

Simple Composite Query

Constant Load (50 VUs)

Requests/sec	Error Rate
5822.31 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
0.81ms	7.09ms	162.30ms	8.43ms	13.15ms	18.57ms

Ramping Load (0→50→500→50 VUs)

Requests/sec	Error Rate
4560.13 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
0.84ms	39.24ms	267.05ms	48.76ms	108.97ms	125.83ms

Executed Query

fragment User on User {
  id
  username
  name
}

fragment Review on Review {
  id
  body
}

fragment Product on Product {
  inStock
  name
  price
  shippingEstimate
  upc
  weight
}

query TestQuery {
  topProducts(first: 5) {
    ...Product
    reviews {
      ...Review
      author {
        ...User
      }
    }
  }
}

Deep Recursion Query

Constant Load (50 VUs)

Requests/sec	Error Rate
269.24 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
13.17ms	174.88ms	741.11ms	179.97ms	226.21ms	253.53ms

Ramping Load (0→50→500→50 VUs)

Requests/sec	Error Rate
307.00 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
2.99ms	656.83ms	1698.40ms	687.68ms	1357.97ms	1429.44ms

Executed Query

fragment User on User {
  id
  username
  name
}

fragment Review on Review {
  id
  body
}

fragment Product on Product {
  inStock
  name
  price
  shippingEstimate
  upc
  weight
}

query TestQuery {
  users {
    ...User
    reviews {
      ...Review
      product {
        ...Product
        reviews {
          ...Review
          author {
            ...User
            reviews {
              ...Review
              product {
                ...Product
              }
            }
          }
        }
      }
    }
  }
  topProducts(first: 5) {
    ...Product
    reviews {
      ...Review
      author {
        ...User
        reviews {
          ...Review
          product {
            ...Product
          }
        }
      }
    }
  }
}

Variable Batching Throughput

Constant Load (50 VUs)

Requests/sec	Error Rate
23551.17 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
0.09ms	1.71ms	43.95ms	2.07ms	3.93ms	4.83ms

Ramping Load (0→50→500→50 VUs)

Requests/sec	Error Rate
18771.00 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
0.09ms	9.20ms	104.19ms	11.26ms	23.33ms	28.24ms

Executed Query

query TestQuery_8f7a46ce_2(
  $__fusion_1_upc: ID!
  $__fusion_2_price: Long!
  $__fusion_2_weight: Long!
) {
  productByUpc(upc: $__fusion_1_upc) {
    inStock
    shippingEstimate(weight: $__fusion_2_weight, price: $__fusion_2_price)
  }
}

Variables (5 sets batched in single request)

[
  { "__fusion_1_upc": "1", "__fusion_2_price": 899, "__fusion_2_weight": 100 },
  { "__fusion_1_upc": "2", "__fusion_2_price": 1299, "__fusion_2_weight": 1000 },
  { "__fusion_1_upc": "3", "__fusion_2_price": 15, "__fusion_2_weight": 20 },
  { "__fusion_1_upc": "4", "__fusion_2_price": 499, "__fusion_2_weight": 100 },
  { "__fusion_1_upc": "5", "__fusion_2_price": 1299, "__fusion_2_weight": 1000 }
]

No baseline data available for comparison.

---

Run 21728317158 • Commit 5d71413 • Thu, 05 Feb 2026 21:20:06 GMT

[github-actions]

🚀 Fusion Gateway Performance Results

Simple Composite Query

Constant Load (50 VUs)

Requests/sec	Error Rate
5864.67 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
0.81ms	7.07ms	162.23ms	8.37ms	13.44ms	18.76ms

Ramping Load (0→50→500→50 VUs)

Requests/sec	Error Rate
4547.75 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
0.83ms	39.75ms	229.39ms	48.84ms	110.50ms	127.04ms

Executed Query

fragment User on User {
  id
  username
  name
}

fragment Review on Review {
  id
  body
}

fragment Product on Product {
  inStock
  name
  price
  shippingEstimate
  upc
  weight
}

query TestQuery {
  topProducts(first: 5) {
    ...Product
    reviews {
      ...Review
      author {
        ...User
      }
    }
  }
}

Deep Recursion Query

Constant Load (50 VUs)

Requests/sec	Error Rate
266.35 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
12.87ms	175.80ms	691.97ms	182.05ms	226.77ms	251.80ms

Ramping Load (0→50→500→50 VUs)

Requests/sec	Error Rate
307.06 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
2.95ms	662.63ms	1718.51ms	686.22ms	1360.79ms	1481.07ms

Executed Query

fragment User on User {
  id
  username
  name
}

fragment Review on Review {
  id
  body
}

fragment Product on Product {
  inStock
  name
  price
  shippingEstimate
  upc
  weight
}

query TestQuery {
  users {
    ...User
    reviews {
      ...Review
      product {
        ...Product
        reviews {
          ...Review
          author {
            ...User
            reviews {
              ...Review
              product {
                ...Product
              }
            }
          }
        }
      }
    }
  }
  topProducts(first: 5) {
    ...Product
    reviews {
      ...Review
      author {
        ...User
        reviews {
          ...Review
          product {
            ...Product
          }
        }
      }
    }
  }
}

Variable Batching Throughput

Constant Load (50 VUs)

Requests/sec	Error Rate
24067.73 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
0.09ms	1.67ms	41.89ms	2.03ms	3.87ms	4.75ms

Ramping Load (0→50→500→50 VUs)

Requests/sec	Error Rate
18890.80 req/s	0.00%

📊 Response Time Metrics

Min	Med	Max	Avg	P90	P95
0.09ms	9.21ms	107.56ms	11.21ms	23.20ms	27.97ms

Executed Query

query TestQuery_8f7a46ce_2(
  $__fusion_1_upc: ID!
  $__fusion_2_price: Long!
  $__fusion_2_weight: Long!
) {
  productByUpc(upc: $__fusion_1_upc) {
    inStock
    shippingEstimate(weight: $__fusion_2_weight, price: $__fusion_2_price)
  }
}

Variables (5 sets batched in single request)

[
  { "__fusion_1_upc": "1", "__fusion_2_price": 899, "__fusion_2_weight": 100 },
  { "__fusion_1_upc": "2", "__fusion_2_price": 1299, "__fusion_2_weight": 1000 },
  { "__fusion_1_upc": "3", "__fusion_2_price": 15, "__fusion_2_weight": 20 },
  { "__fusion_1_upc": "4", "__fusion_2_price": 499, "__fusion_2_weight": 100 },
  { "__fusion_1_upc": "5", "__fusion_2_price": 1299, "__fusion_2_weight": 1000 }
]

No baseline data available for comparison.

---

Run 21742617996 • Commit 42cb921 • Fri, 06 Feb 2026 07:53:49 GMT