Pulsar: built-in Avro schema support (typed ISchema<T>) (#3213)#3214
Closed
jeremydmiller wants to merge 1 commit into
Closed
Pulsar: built-in Avro schema support (typed ISchema<T>) (#3213)#3214jeremydmiller wants to merge 1 commit into
jeremydmiller wants to merge 1 commit into
Conversation
Follow-up to the JSON schema work (#3210). JSON used a pass-through schema where Wolverine owns the body bytes; Avro can't — DotPulsar's Schema.AvroISpecificRecord<T>() owns the encode/decode, so the body must be genuine Avro on the wire. Adds a codec seam that keeps the existing byte-oriented sender/listener (and all the ack/retry/DLQ logic) intact: opts.PublishMessage<OrderPlaced>().ToPulsarTopic(topic).UseAvroSchema<OrderPlaced>(); opts.ListenToPulsarTopic(topic).UseAvroSchema<OrderPlaced>(); - IPulsarMessageCodec / PulsarAvroCodec<T> wraps Schema.AvroISpecificRecord<T>() (T must be an Apache.Avro ISpecificRecord at runtime; no compile-time Avro dependency in the transport — DotPulsar resolves it reflectively). - UseAvroSchema<T>() sets endpoint.MessageCodec + a pass-through PulsarSchema carrying the Avro SchemaInfo so the broker registers the Avro schema. - PulsarSender encodes envelope.Message through the codec (falling back to Wolverine's serialized body for non-codec messages such as a ping); PulsarListener decodes to the message object and sets envelope.Message directly, so the pipeline skips its own body deserialization (HandlerPipeline.TryDeserializeEnvelope returns early when Message is set). Tests: Avro codec round-trip (unit) + end-to-end publish/consume under an Avro schema with broker-side AVRO registration verified via the admin REST API; JSON schema path unchanged (no regression). Docs: Avro subsection. Apache.Avro added to the test project only. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Member
Author
|
Superseded by the rebased branch (resolves conflicts with the merged dedup PR #3212). See the replacement PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3213. Follow-up to the JSON schema work (#3210), completing the Pulsar schema story from #3183.
JSON used a pass-through schema where Wolverine owns the body bytes. Avro can't work that way — DotPulsar's
Schema.AvroISpecificRecord<T>()owns the encode/decode, so the body must be genuine Avro on the wire. This adds a small codec seam that keeps the existing byte-oriented sender/listener (and all the ack/retry/DLQ logic) fully intact.How it works
IPulsarMessageCodec/PulsarAvroCodec<T>wraps DotPulsar's built-inSchema.AvroISpecificRecord<T>().Tmust be an Apache.AvroISpecificRecordat runtime — there is no compile-time Avro dependency in the transport (DotPulsar resolves the schema reflectively from the type's static_SCHEMA).UseAvroSchema<T>()setsendpoint.MessageCodecand a pass-throughPulsarSchemacarrying the AvroSchemaInfo, so the broker registers the Avro schema for the topic.PulsarSenderencodesenvelope.Messagethrough the codec (falling back to Wolverine's serialized body for non-codec messages such as a ping);PulsarListenerdecodes to the message object and setsenvelope.Messagedirectly — the pipeline then skips its own body deserialization (HandlerPipeline.TryDeserializeEnvelopereturns early whenMessageis already set).This reuses the
UsePulsarSchema(...)seam direction noted in #3210; Protobuf/other schema types can plug in the same way via a custom codec.Acceptance criteria (from #3183 / #3213)
type: AVRO.Tests / build
pulsar_avro_schema: codec round-trip (unit) + end-to-end broker round-trip with AVRO registration — green against a Testcontainers broker.Apache.Avroadded to the test project only.wolverine.slnxbuilds clean in Release.🤖 Generated with Claude Code