Skip to content

feat(grpc_datasource): add ConnectRPC transport for gRPC datasource#1480

Draft
fengyuwusong wants to merge 7 commits intowundergraph:masterfrom
fengyuwusong:feat/connectrpc-grpctest-buf
Draft

feat(grpc_datasource): add ConnectRPC transport for gRPC datasource#1480
fengyuwusong wants to merge 7 commits intowundergraph:masterfrom
fengyuwusong:feat/connectrpc-grpctest-buf

Conversation

@fengyuwusong
Copy link
Copy Markdown
Contributor

@fengyuwusong fengyuwusong commented Apr 30, 2026

@coderabbitai summary

Summary

This PR introduces an RPCTransport abstraction inside the gRPC data source so that subgraph calls can be made over either native gRPC or ConnectRPC. It then adds the Connect implementation, regenerates the grpctest fixture with buf so the same backing MockService can be served over both transports, and locks in end-to-end coverage through the data source layer.

ConnectRPC is the right transport whenever end-to-end HTTP/2 to the subgraph is unavailable. The feature is consumed by the matching cosmo router work (see "Related").

Related

What's changed

Data source transport abstraction

  • New RPCTransport interface (transport.go) wraps the previous direct grpc.ClientConnInterface use site so the data source can dispatch through any concrete transport.
  • NewDataSource(transport, config) accepts the abstraction directly; NewDataSourceGRPC(conn, config) is a thin convenience wrapper that preserves the existing call shape for gRPC consumers.

ConnectRPC transport

  • transport_connect.go implements the Connect protocol over HTTP/1.1 with both Protobuf and JSON encodings.
  • Follow-up fixes from earlier reviews are folded in:
    • base64-encode binary metadata headers,
    • nil-guard the response body and cap it at 10 MB,
    • detect oversized responses and truncate the surfaced error body.
  • transport_connect_test.go covers Protobuf, JSON, Connect-protocol errors, non-JSON errors, and ASCII / binary header forwarding.

grpctest fixture migration to buf

  • New buf.yaml and buf.gen.yaml drive protoc-gen-go, protoc-gen-go-grpc, and protoc-gen-connect-go from a single generation step. The Mproduct.proto= plugin option remaps the proto's go_package onto the actual repository path so the generated Connect package can import productv1 cleanly.
  • Makefile generate-proto target now invokes buf generate.
  • productv1/productv1connect/product.connect.go is checked in.

Connect-compatible MockService

  • pkg/grpctest/mockservice_connect.go: mechanically generated MockServiceConnect adapter that forwards every Connect call to the existing MockService (one method per RPC, 94 in total). The same backing implementation can now serve Connect, gRPC, and gRPC-Web from a single HTTP handler.

Data source end-to-end coverage

  • pkg/engine/datasource/grpc_datasource/grpc_datasource_connect_test.go: drives the data source through the new Connect transport against the Connect-wrapped MockService for both Protobuf and JSON encodings.
  • A small fixup commit migrates two test sites that landed on master between rebase points (calling NewDataSource(conn, …)) onto NewDataSourceGRPC(conn, …).

Testing

  • pkg/engine/datasource/grpc_datasource: 456 tests pass (Connect cases added; existing gRPC / federation / cost / lookup tests unchanged).
  • pkg/grpctest: package builds with both protoc and buf-generated outputs in sync.

Backward compatibility

  • The RPCTransport introduction keeps NewDataSourceGRPC as the drop-in replacement for the old NewDataSource(conn, …) call shape; existing gRPC consumers do not need to touch anything else.
  • The Connect handler/client live in a new productv1/productv1connect sub-package so they don't affect existing imports of productv1.

Checklist

  • I have discussed my proposed changes in an issue and have received approval to proceed.
  • I have followed the coding standards of the project.
  • Tests or benchmarks have been added or updated.
  • Documentation has been updated on https://github.com/wundergraph/docs-website — N/A: router-side documentation for the consuming feature lives in the matching cosmo PR (docs-website/router/...).
  • I have read the Contributors Guide.

Open Source AI Manifesto

This project follows the principles of the Open Source AI Manifesto. Please ensure your contribution aligns with its principles.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 30, 2026

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 81b5cb89-91c3-4593-acf0-56dfb927f7cd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@fengyuwusong
Copy link
Copy Markdown
Contributor Author

CC @biubiubiuboomboomboom

fengyuwusong and others added 7 commits April 30, 2026 19:55
Introduce RPCTransport interface to abstract the transport protocol
in grpc_datasource, enabling both gRPC and Connect protocol support.

Key changes:
- New RPCTransport interface with grpcTransport and connectTransport impls
- connectTransport implements Connect protocol over HTTP (Protobuf/JSON)
- DataSource.cc refactored to DataSource.transport (RPCTransport)
- NewDataSourceGRPC convenience function for backward compatibility
- NewFactoryConnect in graphql_datasource for Connect protocol entry
- ConnectConfiguration for Connect-specific settings
- 6 unit tests for connectTransport (protobuf, json, errors, headers)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…t transport

Keys ending in "-bin" carry binary values per gRPC convention. The Connect
protocol requires these to be base64-encoded when placed in HTTP headers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…orts

Address code review feedback:
- Add nil check in grpcTransport.Invoke to prevent panics
- Limit Connect response body to 10MB to prevent memory exhaustion

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…odies

- Read maxSize+1 bytes to detect truncation and return a clear
  "response body exceeds N bytes" error instead of cryptic unmarshal failures
- Truncate non-JSON error bodies to 256 chars in parseConnectError
  to prevent log bloat and sensitive data exposure

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the hand-written protoc invocation with `buf generate`, driven by
new buf.yaml and buf.gen.yaml. The same configuration runs three plugins
side by side:

  - protoc-gen-go         (messages, unchanged from before)
  - protoc-gen-go-grpc    (gRPC server/client, unchanged)
  - protoc-gen-connect-go (new: ConnectRPC handler/client)

The Connect handler accepts gRPC, gRPC-Web, and Connect over the same
HTTP endpoint, which lets the mock service serve both transports from a
single registration. The generated code lives at
productv1/productv1connect/product.connect.go.

Both plugins use Mproduct.proto to remap the proto's go_package
(cosmo/pkg/proto/productv1) onto the actual repository path so the
generated Connect package can import productv1 cleanly.
…se base

Two test sites added on master between v2.0.0-rc.265 (the original base
of feat/connectrpc-datasource) and v2.0.0 still call NewDataSource(conn,
...). After the RPCTransport abstraction landed, NewDataSource expects
an RPCTransport; gRPC connections must go through NewDataSourceGRPC.
Update both call sites so the package builds against v2.0.0.
Add comprehensive end-to-end tests that drive the gRPC data source
pipeline through the new Connect transport against the existing
MockService.

  - v2/pkg/grpctest/mockservice_connect.go:
    Mechanically-generated MockServiceConnect adapter (one method per
    RPC, 94 in total) that wraps the gRPC MockService onto the
    productv1connect.ProductServiceHandler interface. The same backing
    implementation can therefore serve Connect, gRPC, and gRPC-Web from
    a single HTTP handler.

  - v2/pkg/engine/datasource/grpc_datasource/grpc_datasource_connect_test.go:
    setupTestConnectServer spins the adapter up on httptest.NewUnstartedServer
    + h2c.NewHandler. Two new tests drive a representative query through
    the data source via the Connect transport, once with Protobuf encoding
    and once with JSON, to lock in wire-format parity for both encodings
    end-to-end.
@fengyuwusong fengyuwusong force-pushed the feat/connectrpc-grpctest-buf branch from 2d0024b to 3a0d79e Compare April 30, 2026 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant