Master: test harness standardization → cross-store event sourcing compliance library

[jeremydmiller]

Master tracking issue for the whole test-harness standardization program (started 2026-08-01). The five-step goal: (1) rewrite non-standard-harness tests onto standard context base types, (2) retire the old tests, (3) compare Marten↔Polecat functional tests, (4) lift out shared compliance test libraries of behavioral expectations — event sourcing first, document-db later, (5) move Marten and Polecat onto them and retire the duplicates. A future Sqlite-based minimal document db + event store (JasperFx.Events + Weasel.Storage + Weasel.Sqlite) becomes the third consumer.

Phase 1 — multi-store parameterization harness ✅ COMPLETE

• Collapse EventSourcingTests/QuickAppend forks via new MultiStoreFixture harness #5098 — MultiStoreFixture (profile-registry collection fixture) + collapse of the four EventSourcingTests/QuickAppend/ fork pairs (−1,258 lines; Rich append mode gained coverage it never had)

Phase 2 — migrate non-standard harnesses onto the standard bases ✅ COMPLETE

• Migrate raw-DocumentStore.For event sourcing tests onto standard harnesses #5099 — ES raw-DocumentStore.For tests → BugIntegrationContext / OneOff / SeparateStore
• Absorb the hand-rolled IAsyncLifetime template in TenantPartitionedEventsTests #5100 — TenantPartitionedEventsTests hand-rolled IAsyncLifetime template → PartitionedStoreContext (18 files, −449)
• Delete the vestigial TestingSupport project and dangling TestSetup.cs solution item #5101 — delete vestigial TestingSupport project + dangling TestSetup.cs
• Add HostedStoreContext harness for IHost + AddMarten tests #5102 — new HostedStoreContext harness for IHost + AddMarten tests, first 6 migrations
• Absorb the sharded scrub template in TenantPartitionedEventsTests/Sharded #5103 — sharded TPE scrub template → ShardedPartitionedContext (14 files, −351)
• Migrate DaemonTests inline-host tests onto HostedStoreContext #5104 — DaemonTests inline hosts → HostedStoreContext (9 files, −205; also fixed 4 ManualOnly blue/green tests silently broken on master)
• Migrate ValueTypeTests hand-rolled stores onto OneOffConfigurationsContext #5105 — ValueTypeTests 16 hand-rolled-store classes → OneOffConfigurationsContext (−444; retired strong_typed1..24 schemas + 3 inconsistent disposal variants)
• Migrate CoreTests hand-rolled stores and hosts onto the standard harnesses #5106 — CoreTests → HostedStoreContext/OneOff (−168; ~20 schema literals retired)
• Share tenant-database provisioning and dedupe the multi-tenancy collection #5107 — MultiTenancyTests: shared TenantDatabases provisioning helper (10 copies deleted) + "multi-tenancy" collection dedup (9 definitions → 1, classes properly enrolled)

Documented deliberate non-migrations (all with recorded rationale): TPE multi-node files and CoreTests Partitioning/* (disciplined ProcessId-suffixed schema convention), MultiHostConfigurationContext (primary/standby NpgsqlMultiHostDataSource harness), AspNetCore.Testing IntegrationContext (is the doc sample), disposal-semantics tests, DI-registration-count tests, doc-snippet files, pure options unit tests.

Phase 3 — event sourcing compliance library ✅ COMPLETE (2026-08-02)

Tracked in detail by epic #5110. Children:

• Compliance seam: EventStoreComplianceFixture<TOperations, TQuerySession> in Marten.Testing (P3.1a) #5111 — P3.1a: EventStoreComplianceFixture<TOperations, TQuerySession> seam in Marten.Testing
• Seed the compliance suites: self-aggregating evolve, DCB tag query/consistency, AssignTagWhere (P3.1b) #5112 — P3.1b: seed suites (self-aggregating evolve, DCB tag query/consistency, AssignTagWhere) + strongest-assertion drift fixes
• Async-daemon compliance suite (P3.1c) #5113 — P3.1c: async-daemon compliance suite
• [jasperfx] Additive IEventRegistry generic DIMs: AddEventType<T>() and EventMappingFor<T>() (P3.2a) #5114 (closed, not planned) — P3.2a: [jasperfx] IEventRegistry generic DIMs
• [jasperfx] Shared batch-DCB query interface + Marten/Polecat implementations (P3.2b) #5115 (closed, not planned) — P3.2b: [jasperfx] shared batch-DCB query interface
• Extract the compliance library as source-only NuGet JasperFx.Events.ComplianceTests (P3.3) #5116 — P3.3: extract as source-only NuGet JasperFx.Events.ComplianceTests
• [polecat] Adopt JasperFx.Events.ComplianceTests and retire the mirrored test files (P3.4) #5117 — P3.4: [polecat] adopt the library, retire mirrored test files
• Compliance wave 2: grow suite coverage from the remaining ported pairs (P3.5) #5118 — P3.5: wave 2 suite growth, which became waves 2–8 (remaining ~25–35 ported pairs, ProjectionScenario lift)

Phase 3 wave growth — ✅ COMPLETE, backlog empty (2026-08-08)

The library is at 28 suites / 230 tests, both stores enrolled, zero capability gates, and every per-suite child of #5118 is closed. Waves 2–8 shipped through JasperFx 2.37.2 → 2.45.0; the final wave (#5148 conjoined event tenancy, #5151 subscriptions) landed in 2.45.0 via #642, adopted by Marten in #5205.

Two of the twenty children closed without a suite, for recorded reasons rather than by attrition: #5152 (IEventStoreInstrumentation) NOT_PLANNED — Marten only implements it on a build-time DI shim, never on the store, so casting EventGraph to it compiles and throws; #5156 (Sqlite third consumer) closed because it is happening as "Fisher". Two needed an upstream lift first and got one: #5153 CompactStreamAsync<T> and #5154 IEventDataMasking, both now on shared JasperFx surfaces.

The adoption found 11 real product defects across the program — the point of the exercise, not a side effect. The sharpest was EventGraph.TryMask composing maskers with || instead of |= in both products (marten#5199 / polecat#422): the first matching rule short-circuited every later one, so on a GDPR right-to-erasure path protected data silently survived while the operation reported success. No existing test caught it because every masking test in both repos registered rules over disjoint event types.

#5155 CLOSED 2026-08-08 (not planned) — what remained was a standing convention with no finish line, owner or trigger, and the measurement it existed to support had already moved to the child-suite checklist. The insight it recorded is kept there: a marker grep is not an exit criterion, because provenance citations grow as parity work succeeds.

#5177 CLOSED 2026-08-05 (not planned) — Jeremy's call: the high-water identity grammar does not become a compliance suite, the mechanics are too different between the stores. Consistent with what #5178 found: each product composes the grammar through its own constants. The related live bug #5171 (unescaped LIKE in the per-tenant progression filter) was fixed and closed 2026-08-04.

Later phases

• LINQ / query-provider compliance — OUT OF SCOPE, permanently (decided 2026-08-08 on Adopt distinct ported:/parity: provenance markers in both repos #5155). The store query languages diverge structurally enough that a shared suite would pin coincidence rather than contract. This restores the day-one non-goal from the Phase 3 design rather than setting a new one; it had drifted into "blocked pending convergence" framing, which wrongly reclassified a permanent exclusion as pending work. Consequence: Polecat.Tests/Linq/additional_linq_operator_tests.cs (19 tests) is not an unabsorbed port — it is a Polecat-owned test file and stays one.
• Document-db compliance suite — still open as a question, narrowed by the above: no shared IDocumentStore contract exists yet (Polecat re-declares its own session interfaces) and that convergence settles in Weasel.Storage first. Whatever it eventually covers, it does not cover LINQ.
• Sqlite minimal store as the third compliance consumer #5156 — Sqlite minimal store as the third compliance-fixture consumer; the first true proof the seam carries no PostgreSQL/SQL Server residue. Closed: it is happening as "Fisher".

Marten is the tracking repository for the whole program; issues targeting jasperfx/polecat are marked in their bodies.

🤖 Generated with Claude Code

[jeremydmiller]

Phase 3 is complete end to end and shipped. Summary for the program record.

What landed

Repo	PR	Contents
jasperfx	#607 merged + published	New source-only package JasperFx.Events.ComplianceTests, shipped as part of the JasperFx 2.37.1 line
marten	#5122	Seam, four suites, Marten enrollment, retirements — closes #5111, #5112, #5113, #5116
polecat	polecat#393	Adoption, mirror retirement, and a product fix — closes #5117

Both stores now run the same 42 compliance tests: Marten EventSourcingTests 1613 passed / 0 failed, Polecat Polecat.Tests 1629 passed / 0 failed / 3 skipped, both clean on net9.0 and net10.0.

The program worked

Decision D5 (strongest assertion wins) paid off on day one, in both directions:

• Marten gained two tests it never had, both from the Polecat side, both green immediately: can_query_events_across_distinct_tag_types_with_or and fetch_for_writing_appends_to_existing_tag_derived_stream_without_collision.
• Polecat failed 5 of 42 on its first run, and two of those were a real product bug (polecat#392): QueryByTagsAsync selected stream_id and never mapped it onto the envelope, so every DCB tag query returned StreamId == Guid.Empty. It had been invisible because Polecat's mirror of assign_tag_where_by_stream_id was the weakened copy this epic's "Why" section called out. The shared suite kept Marten's assertion and the bug fell out in the first minute of adoption.

Corrections to the plan

Three things the design got wrong, all recorded on their own issues:

• [jasperfx] Additive IEventRegistry generic DIMs: AddEventType<T>() and EventMappingFor<T>() (P3.2a) #5114 (registry DIMs) — recommend won't fix. Both proposed DIMs collide with existing EventGraph members that have incompatible return types (CS0738). Making room means a source-breaking change to AddEventType<T>()'s public return type. The seam needs neither and requires zero InternalsVisibleTo as designed.
• [jasperfx] Shared batch-DCB query interface + Marten/Polecat implementations (P3.2b) #5115 (shared batch-DCB interface) — recommend defer. It would not remove IComplianceBatch, because Execute(ct) belongs to the batch rather than to the event queries. That was the exact open question this issue asked P3.1b to settle, and real usage settled it.
• A ninth drift item the design's list of eight missed: Polecat runs its unit of work in parallel and surfaces DcbConcurrencyException inside an AggregateException where Marten throws it directly. The suites assert the semantics through ShouldFailWithAsync<T> rather than forcing either product to change.

Also worth recording for the Sqlite consumer: doc snippets stay repo-owned. Both products keep a dcb_documentation_samples.cs holding the sample_*_dcb_* blocks their docs pull from, because after extraction the suite sources no longer live in either repo and mdsnippets cannot reach them.

Remaining: #5118 (wave 2 suite growth) is now unblocked — both stores are on the library.

[jeremydmiller]

Correction to my summary above, on the ninth drift item.

I described Polecat as "runs its unit of work in parallel and aggregates." That is wrong about the mechanism. DocumentSessionBase.SaveChangesAsync batches the DCB assertions into a single command, postprocesses them in a loop collecting failures into a list, and then throws AggregateException unconditionally when the list is non-empty — including when it holds exactly one exception. Nothing runs in parallel.

The consequence is the same and the seam handling is unchanged (ShouldFailWithAsync<T> accepts the exception directly or flattened), but the distinction matters for whether it is worth fixing: unwrapping the single-exception case is a small, local change rather than a concurrency-model question. Tracked as polecat#394.

Also filed from the adoption, for the Polecat side to pick up independently: polecat#395 — EventStoreOptions exposes no AddEventType, so event-type pre-registration is only reachable via StoreOptions.EventGraph. The compliance fixture had to take that detour.

Note for whoever bumps JasperFx next: the same inaccurate "in parallel" wording is in the doc comment on EventStoreComplianceSuite.ShouldFailWithAsync as shipped in 2.37.1. Comment only, no behavior — worth correcting on the next release rather than respinning for it.