Add Avro DataEncoding to the stack (1/3, supersedes part of #7) - #10
Merged
Merged
Conversation
This was referenced Jul 10, 2026
marcschier
force-pushed
the
marcschier/encoding-avro
branch
2 times, most recently
from
July 12, 2026 06:43
eb66b17 to
788aa83
Compare
Ports the reference Avro encoder/decoder, SchemaId and Avro schema-exchange types into Stack/Opc.Ua.Types/Encoders (where Binary/Json/Xml live), and the Avro PubSub NetworkMessage adapter plus shared schema-exchange/cache infra into Libraries/Opc.Ua.PubSub/Encoding (where Json/Uadp live). This removes the need for the Opc.Ua.Core.Experimental / Opc.Ua.PubSub.Experimental packages for Avro. - The encoders implement the standard IEncoder/IDecoder and already use namespace Opc.Ua / Opc.Ua.PubSub.Encoding, so this is a relocation, not a rewrite. - They require net5+ BCL APIs, so they are compile-excluded on the legacy target frameworks (net472/net48/netstandard2.0/netstandard2.1) via IsTargetFrameworkCompatible(net8.0) conditions; modern net8/9/10 builds include them. - AvroDecoder.DecodeMessage<T> uses an idiomatic UnconditionalSuppressMessage for the AOT trimming warning (all IEncodeable message types have a public parameterless constructor). - Renamed the internal shared helper ExperimentalMessageEncoding -> PubSubMessageEncoding. - Arrow-specific members in the shared schema-exchange files are added by the stacked Arrow PR. - Tests: AvroRoundTripTests + SchemaIdTests -> Opc.Ua.Core.Encoders.Tests; AvroNetworkMessageTests -> Opc.Ua.PubSub.Tests. net10: core 8/8 + pubsub 1/1 pass; Opc.Ua.Types/PubSub build clean on netstandard2.0/2.1.
…imits) Addresses two review findings on the ported Avro codec: 1. (High) AvroDecoder had no recursion-depth guard. ReadDiagnosticInfo (InnerDiagnosticInfo), ReadDataValue<->ReadVariant, ReadExtensionObject/ReadEncodeable body decode are mutually/self recursive on attacker-controlled PubSub input, so a small deeply-nested frame could exhaust the stack (uncatchable StackOverflowException -> process crash). Added CheckAndIncrementNestingLevel() enforcing Context.MaxEncodingNestingLevels around ReadVariantBody, ReadDataValue, ReadDiagnosticInfo, ReadExtensionObject and both ReadEncodeable<T> overloads, mirroring BinaryDecoder/XmlDecoder/JsonDecoder. 2. (High) Decoded length/count fields sized allocations with no limit. AvroBinaryReader.ReadBytes/ ReadString now take a maxLength and throw BadEncodingLimitsExceeded when exceeded; AvroDecoder passes Context.MaxByteStringLength/MaxStringLength. AvroNetworkMessageDecoder validates the DataSetMessage count and field count against Context.MaxArrayLength before pre-sizing the lists. Because the guards raise ServiceResultException(BadEncodingLimitsExceeded), the PubSub AvroNetworkMessageDecoder catch filter now also catches ServiceResultException so limit violations mark the message invalid (ReceivedInvalidNetworkMessages) instead of escaping TryDecode. Tests: added deep-nesting, oversized-string and oversized-bytestring decode regressions (assert BadEncodingLimitsExceeded). Core Avro/SchemaId: 11 passed; PubSub Avro: 1 passed.
avro3 (Low): AvroSchemaAnnouncement.Encode/Decode and AvroSchemaRequest.Encode/Decode construct an AvroBinaryWriter/AvroBinaryReader (each rents an 8 KB ArrayPool buffer) but only called Flush(), never Release(), so the rented buffers were dropped instead of returned to the pool on this hot handshake path. Wrapped usage in try/finally and call Release(). No behavior change; Core Avro/SchemaId: 11 passed.
marcschier
force-pushed
the
marcschier/encoding-avro
branch
from
July 12, 2026 12:45
01278aa to
7796c37
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part 1 of 3 splitting #7 (Experimental Avro/Protobuf/Arrow encoders) into one PR per encoding, porting each codec into the production stack so the
*.Experimentalpackages are no longer needed.This PR — Avro
Ports the reference Avro encoder/decoder + schema-exchange into the stack, next to the existing codecs:
Stack/Opc.Ua.Types/Encoders/Avro/(AvroEncoder/Decoder + AvroBinaryReader/Writer), plus sharedSchemaIdandEncoders/SchemaExchange/Avro*— alongside Binary/Json/Xml.Libraries/Opc.Ua.PubSub/Encoding/Avro/(NetworkMessage adapter) + sharedEncoding/SchemaExchange/*and the renamed internal helperPubSubMessageEncoding— alongside Json/Uadp.Notes
IEncoder/IDecoderand already usenamespace Opc.Ua/Opc.Ua.PubSub.Encoding, so this is a relocation, not a rewrite.IsTargetFrameworkCompatible('net8.0')conditions; net8/9/10 include them.AvroDecoder.DecodeMessage<T>uses an idiomaticUnconditionalSuppressMessagefor the AOT trim warning (allIEncodeablemessages have a public parameterless ctor).Validation
Opc.Ua.Core.Encoders.TestsAvro/SchemaId 8/8,Opc.Ua.PubSub.TestsAvro 1/1 pass.Opc.Ua.TypesandOpc.Ua.PubSubbuild clean (0/0) on netstandard2.0 / netstandard2.1 (exclusion verified) and net10.Split plan