Skip to content

Support pluggable binary event serialization (IEventBinarySerializer), at parity with Marten #4515 #388

Description

@jeremydmiller

What

Polecat has no equivalent of Marten's IEventBinarySerializer (Marten #4515, shipped in 9.20.2). Verified against Polecat 5.7.1 — the assembly contains no IEventBinarySerializer / BinaryEventAttribute / UseBinarySerializer symbols.

Why it matters

CritterWatch ships two flavors from one shared source tree: Marten/PostgreSQL and Polecat/SQL Server. CritterWatch #896 has just opted its highest-volume internal event (ShardStatesUpdated) into binary serialization on the Marten flavor. Measured at field scale (15 tenants × 10 async projections × 2 databases = 330 snapshots per event, appended every 15s per monitored service):

JSON MessagePack + LZ4
one event on the wire 225.9 KB 4.7 KB −97.9%
mt_events on disk (incl. TOAST) 424.0 KB 168.0 KB −60.4%
append p50 3.92 ms 1.04 ms −73.4%
steady state 904 KB/min 19 KB/min

The SqlServer flavor keeps writing these events as JSON. That is correct and lossless, but it means the two flavors have materially different write-amplification characteristics on the same workload, and the Polecat one carries the cost the Marten one just shed.

The API, for reference

Marten's surface is small and worth mirroring as-is so a store-agnostic consumer can wire either:

public interface IEventBinarySerializer
{
    byte[] Serialize(Type type, object data);
    object Deserialize(Type type, byte[] data);
}

Registration, per event type rather than store-wide:

opts.Events.UseBinarySerializer<TEvent>(serializer);   // explicit
[BinaryEvent] + opts.Events.DefaultBinarySerializer;   // attribute-driven

The part most worth copying is the coexistence design: an additive nullable bdata column alongside data, with the per-row discriminator being bdata IS NULL. JSON and binary rows live in the same table, so the feature can be switched on for an existing store with no migration of existing event data, and switched back off just as safely. That property is what made this a low-risk change to adopt on the Marten side.

Note for whoever picks this up

Marten shipped it in two passes — #4578 was Rich-append-mode only, and Quick mode came in a follow-up. Quick mode is the one that mattered for CritterWatch (its store is configured EventAppendMode.Quick), so it is worth treating Quick as in-scope from the start rather than as a phase 2.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions