Marten 8.23 - DCB support, natural keys, EF Core projections

Directory.Packages.props

@@ -13,9 +13,9 @@
     <PackageVersion Include="FluentAssertions" Version="6.12.0" />
     <PackageVersion Include="FSharp.Core" Version="9.0.100" />
     <PackageVersion Include="FSharp.SystemTextJson" Version="1.3.13" />
-    <PackageVersion Include="JasperFx" Version="1.20.0" />
-    <PackageVersion Include="JasperFx.Events" Version="1.22.0" />
-    <PackageVersion Include="JasperFx.Events.SourceGenerator" Version="1.0.0" />
+    <PackageVersion Include="JasperFx" Version="1.21.0" />
+    <PackageVersion Include="JasperFx.Events" Version="1.23.1" />
+    <PackageVersion Include="JasperFx.Events.SourceGenerator" Version="1.2.0" />
     <PackageVersion Include="JasperFx.RuntimeCompiler" Version="4.4.0" />
     <PackageVersion Include="Jil" Version="3.0.0-alpha2" />
     <PackageVersion Include="Lamar" Version="7.1.1" />
@@ -59,11 +59,23 @@
     <PackageVersion Include="StronglyTypedId" Version="1.0.0-beta08" />
     <PackageVersion Include="Swashbuckle.AspNetCore" Version="6.5.0" />
     <PackageVersion Include="Vogen" Version="7.0.0" />
-    <PackageVersion Include="Weasel.Postgresql" Version="8.8.1" />
+    <PackageVersion Include="Weasel.EntityFrameworkCore" Version="8.9.0" />
+    <PackageVersion Include="Weasel.Postgresql" Version="8.9.0" />
     <PackageVersion Include="WolverineFx.Marten" Version="4.2.0" />
     <PackageVersion Include="xunit" Version="2.9.3" />
     <PackageVersion Include="xunit.runner.visualstudio" Version="3.1.5" />
   </ItemGroup>
+  <!-- EF Core packages - requires net9.0+ due to Npgsql 9.x compatibility -->
+  <ItemGroup Condition="'$(TargetFramework)' == 'net9.0'">
+    <PackageVersion Include="Microsoft.EntityFrameworkCore" Version="9.0.2" />
+    <PackageVersion Include="Microsoft.EntityFrameworkCore.Relational" Version="9.0.2" />
+    <PackageVersion Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="9.0.3" />
+  </ItemGroup>
+  <ItemGroup Condition="'$(TargetFramework)' == 'net10.0'">
+    <PackageVersion Include="Microsoft.EntityFrameworkCore" Version="10.0.0" />
+    <PackageVersion Include="Microsoft.EntityFrameworkCore.Relational" Version="10.0.0" />
+    <PackageVersion Include="Npgsql.EntityFrameworkCore.PostgreSQL" Version="10.0.0" />
+  </ItemGroup>
   <!-- Framework-specific package versions -->
   <ItemGroup Condition="'$(TargetFramework)' == 'net8.0'">
     <PackageVersion Include="Microsoft.Extensions.Diagnostics.HealthChecks" Version="8.0.0" />

centralization-for-marten-9.md

@@ -0,0 +1,114 @@
+# Centralization Plan for Marten 9: Shared Infrastructure with Polecat via Weasel.Core
+
+## Context
+
+Marten (PostgreSQL) and Polecat (SQL Server 2025) share significant infrastructure patterns. Common base types and interfaces should move to Weasel.Core so both projects can share them without database-specific coupling.
+
+---
+
+## Category 1: Metadata Interfaces — Identical, move to Weasel.Core
+
+These are exact duplicates (modulo namespace and nullability) with zero database-specific logic. They're pure document metadata contracts.
+
+| Interface | Marten (`Marten.Metadata`) | Polecat (`Polecat.Metadata`) | Status |
+|-----------|--------|---------|--------|
+| `IVersioned` | `Guid Version { get; set; }` | `Guid Version { get; set; }` | **Identical** |
+| `ISoftDeleted` | `bool Deleted; DateTimeOffset? DeletedAt` | `bool Deleted; DateTimeOffset? DeletedAt` | **Identical** |
+| `IRevisioned` | `int Version { get; set; }` | Does not exist yet | Marten-only; worth putting in Weasel.Core for Polecat to adopt later |
+| `ITracked` | Non-nullable `string` members | Nullable `string?` members | **Needs alignment** — Polecat's nullable approach is more correct |
+
+**Action:** Move all four to `Weasel.Core.Metadata` namespace. Align `ITracked` on nullable strings. Both Marten and Polecat type-forward or alias from their own namespaces.
+
+---
+
+## Category 2: Serialization Enums — Already partially shared
+
+| Type | Weasel.Core | Marten | Polecat |
+|------|-------------|--------|---------|
+| `EnumStorage` | Yes (canonical) | Uses Weasel.Core's | Own copy (duplicate) |
+| `Casing` | **No** | Defined in `ISerializer.cs` | Own copy (duplicate) |
+| `CollectionStorage` | **No** | Defined in `ISerializer.cs` | Own copy (duplicate) |
+| `NonPublicMembersStorage` | **No** | Defined in `ISerializer.cs` | Own copy (duplicate) |
+
+**Action:** Move `Casing`, `CollectionStorage`, and `NonPublicMembersStorage` to Weasel.Core alongside the existing `EnumStorage`. Polecat already duplicates all three and can switch to the Weasel.Core versions.
+
+---
+
+## Category 3: ISerializer Interface — Strong overlap, worth unifying
+
+### Shared surface (present in both Marten and Polecat)
+
+```csharp
+EnumStorage EnumStorage { get; }
+Casing Casing { get; }
+string ToJson(object document);
+T FromJson<T>(Stream stream);
+T FromJson<T>(DbDataReader reader, int index);
+object FromJson(Type type, Stream stream);
+object FromJson(Type type, DbDataReader reader, int index);
+ValueTask<T> FromJsonAsync<T>(Stream stream, CancellationToken cancellationToken = default);
+ValueTask<object> FromJsonAsync(Type type, Stream stream, CancellationToken cancellationToken = default);
+```
+
+### Marten-only additions
+
+- `ValueCasting ValueCasting { get; }` — controls LINQ Select() casting behavior
+- `string ToCleanJson(object? document)` — serialize without type metadata
+- `string ToJsonWithTypes(object document)` — serialize with embedded type info
+- `ValueTask<T> FromJsonAsync<T>(DbDataReader reader, int index, CancellationToken)` — async reader deserialization
+- `ValueTask<object> FromJsonAsync(Type type, DbDataReader reader, int index, CancellationToken)` — async reader deserialization (non-generic)
+
+### Polecat-only additions
+
+- `T FromJson<T>(string json)` — string-based deserialization
+- `object FromJson(Type type, string json)` — string-based deserialization (non-generic)
+
+**Action:** Define a common `ISerializer` interface in Weasel.Core with the shared members. Both Marten and Polecat extend it with project-specific additions via their own derived interfaces.
+
+---
+
+## Category 4: IStorageOperation — Similar but divergent patterns
+
+Both have a storage operation concept with `DocumentType`, `Role`, and `PostprocessAsync(DbDataReader)`, but the interfaces diverge:
+
+| Aspect | Marten | Polecat |
+|--------|--------|---------|
+| Inheritance | Inherits `IQueryHandler` (LINQ) | Standalone |
+| Role | `OperationRole Role()` (method) | `OperationRole Role { get; }` (property) |
+| Postprocess | `IList<Exception>` parameter | No exceptions parameter |
+| Command setup | Via `IQueryHandler` | `ConfigureCommand(ICommandBuilder)` |
+| Document ID | Not on interface | `object? DocumentId` default method |
+| Enum values | `Upsert, Insert, Update, Deletion, Patch, Events, Other` | `Upsert, Insert, Update, Delete, Patch` |
+
+**Action:** Extract a minimal common `OperationRole` enum and a slim `IStorageOperation` base to Weasel.Core. Both projects extend with their specific needs. Lower priority since the divergence is larger.
+
+---
+
+## Category 5: Session Interfaces — Too divergent, do not share
+
+`IQuerySession`, `IDocumentSession`, `IDocumentOperations`, `IDocumentStore` are conceptually similar but:
+
+- **Marten** is much larger: `NpgsqlConnection Connection`, full-text search, bulk insert via PostgreSQL COPY, dirty tracking sessions, serializable isolation variants, 65+ members on `IDocumentStore`
+- **Polecat** is intentionally minimal: ~10 members on `IDocumentStore`, no dirty tracking, no full-text search
+
+Forcing a common interface would either bloat Polecat or gut Marten.
+
+**Action:** Keep project-specific. No shared base.
+
+---
+
+## Recommended Priority Order
+
+1. **Metadata interfaces** (`IVersioned`, `ISoftDeleted`, `ITracked`, `IRevisioned`) → Weasel.Core
+   - Easiest win, zero risk, zero database coupling
+
+2. **Serialization enums** (`Casing`, `CollectionStorage`, `NonPublicMembersStorage`) → Weasel.Core
+   - Alongside existing `EnumStorage`. Eliminates Polecat duplicates
+
+3. **ISerializer base interface** → Weasel.Core with shared members
+   - Both projects extend for their extras
+
+4. **OperationRole enum + minimal IStorageOperation** → Weasel.Core
+   - Lower priority, interfaces diverge more
+
+5. **Session interfaces** → Do not share

dcb-concepts.md

@@ -0,0 +1,191 @@
+# DCB (Dynamic Consistency Boundary) Implementation in Marten
+
+## What is DCB?
+
+DCB is a technique for enforcing consistency in event-driven systems without rigid aggregate-based transactional boundaries. It allows events to be assigned to **multiple domain concepts** via tags, and enforces consistency across them using conditional appends.
+
+**Core spec (https://dcb.events/):**
+- **Read**: Filter events by event type (OR) and/or tags (AND within a query item, OR across items)
+- **Write**: Atomically persist events, failing if any event matching a query exists after a given global sequence position
+
+## Design Decisions
+
+| Decision | Choice | Rationale |
+|----------|--------|-----------|
+| Coexistence | DCB coexists with traditional stream-based event sourcing | Purely additive — users tag some events, use streams for others |
+| Stream requirement | DCB events always belong to a stream | Keeps existing stream infrastructure intact |
+| Condition model | First-class "condition" pattern | Encapsulates query + state + consistency check as a formal concept |
+| Append atomicity | Condition check within `SaveChangesAsync()` transaction | True consistency, no check-then-act race |
+| Append modes | Both Rich and Quick | Full compatibility |
+| Tenancy | Scoped to current tenant by default | Opt-in for cross-tenant queries |
+| Portability | Abstractions in JasperFx.Events, PostgreSQL impl in Marten | Maximizes portability |
+| Tag extraction | Write-time with backfill tool | Pre-computed for query performance; backfill for migration |
+| Schema | Opt-in `tags TEXT[]` column + GIN index on `mt_events` | No impact to existing users |
+| Performance target | ~1ms per event at 1M events | Aligned with DCB FAQ benchmarks |
+
+## Storage Approach
+
+### Recommended: `TEXT[]` column with GIN index
+
+When DCB is enabled, add to `mt_events`:
+
+```sql
+ALTER TABLE mt_events ADD COLUMN tags TEXT[];
+CREATE INDEX ix_mt_events_tags ON mt_events USING GIN (tags);
+```
+
+**Why this over alternatives:**
+- **vs normalized junction table**: No JOINs, single-table queries, simpler writes
+- **vs JSONB headers**: Dedicated column with native array operators, proper GIN indexing, no mixing concerns
+- **vs query-time computation**: Pre-computed tags enable indexed conditional append checks — the core DCB performance requirement
+
+### Conditional Append SQL
+
+The append condition check runs within the same transaction as event insertion:
+
+```sql
+SELECT EXISTS (
+  SELECT 1 FROM mt_events
+  WHERE seq_id > @after
+  AND (
+    -- Query Item 1 (types OR'd, tags AND'd within item)
+    (type = ANY(@types1) AND tags @> @tags1)
+    OR
+    -- Query Item 2
+    (type = ANY(@types2) AND tags @> @tags2)
+  )
+  -- Tenant scoping (default)
+  AND tenant_id = @tenantId
+)
+```
+
+If this returns `true`, the append fails with a concurrency exception.
+
+## API Syntax
+
+### Tag Registration
+
+**Fluent configuration in StoreOptions:**
+
+```csharp
+opts.Events.EnableDcb(); // opt-in, triggers schema addition
+
+opts.Events.TagEvent<StudentSubscribedToCourse>(e => new[]
+{
+    $"student:{e.StudentId}",
+    $"course:{e.CourseId}"
+});
+
+opts.Events.TagEvent<CourseCapacityChanged>(e => new[]
+{
+    $"course:{e.CourseId}"
+});
+```
+
+**Interface on the event (alternative):**
+
+```csharp
+public record StudentSubscribedToCourse(Guid StudentId, Guid CourseId) : ITaggedEvent
+{
+    public IEnumerable<string> GetTags() => [$"student:{StudentId}", $"course:{CourseId}"];
+}
+```
+
+### Condition Pattern
+
+A "condition" is a first-class concept that encapsulates:
+1. **Query**: What event types and tags to look for
+2. **State**: Projecting matching events into a decision model
+3. **Check**: Whether the append should proceed
+4. **Enforcement**: Atomic validation during `SaveChangesAsync()`
+
+```csharp
+public class CourseSubscriptionCondition : AppendCondition
+{
+    public bool CourseExists { get; set; }
+    public int Capacity { get; set; }
+    public int Subscriptions { get; set; }
+
+    // Define which events this condition queries
+    public override void ConfigureQuery(DcbQueryBuilder query, Guid courseId, Guid studentId)
+    {
+        query
+            .Match<CourseDefined>([$"course:{courseId}"])
+            .Match<CourseCapacityChanged>([$"course:{courseId}"])
+            .Match<StudentSubscribedToCourse>([$"course:{courseId}"])
+            .Match<StudentSubscribedToCourse>([$"student:{studentId}"]);
+    }
+
+    // Build state from matching events
+    public void Apply(CourseDefined e) => CourseExists = true;
+    public void Apply(CourseCapacityChanged e) => Capacity = e.NewCapacity;
+    public void Apply(StudentSubscribedToCourse e) => Subscriptions++;
+
+    // Evaluate whether append is allowed
+    public override bool CanAppend() =>
+        CourseExists && Subscriptions < Capacity;
+}
+```
+
+**Usage:**
+
+```csharp
+var condition = await session.Events
+    .BuildCondition<CourseSubscriptionCondition>(courseId, studentId);
+
+if (condition.CanAppend())
+{
+    session.Events.AppendWithCondition(
+        streamId,
+        condition,
+        new StudentSubscribedToCourse(studentId, courseId)
+    );
+    await session.SaveChangesAsync();
+    // ^ condition check + event insert in same transaction
+    // throws concurrency exception if condition violated
+}
+```
+
+### Backfill Tool
+
+For existing events when enabling DCB or adding new tag definitions:
+
+```csharp
+await store.Events.BackfillTagsAsync(CancellationToken.None);
+```
+
+This reads events in batches, applies registered tag extractors, and updates the `tags` column.
+
+## Architecture Split
+
+### JasperFx.Events (portable)
+
+- `ITaggedEvent` interface
+- `AppendCondition` base class / condition model
+- `DcbQueryBuilder` — query construction
+- `IDcbQuery` / `DcbQueryItem` — query representation
+- Tag extraction registration abstractions
+- Apply method conventions for condition state
+
+### Marten (PostgreSQL-specific)
+
+- `EnableDcb()` opt-in configuration
+- `tags TEXT[]` column addition to `EventsTable`
+- GIN index creation
+- SQL generation for conditional append check
+- Integration with Rich and Quick append paths
+- `BackfillTagsAsync()` migration tool
+- Tenant scoping in condition queries
+
+## DCB Spec Mapping
+
+| DCB Spec Concept | Marten Implementation |
+|---|---|
+| Sequence Position | `seq_id` (existing global sequence) |
+| Event Type | `type` column (existing) |
+| Tags | `tags TEXT[]` column (new, opt-in) |
+| Query | `DcbQueryBuilder` → SQL with `type = ANY(...)` and `tags @> ARRAY[...]` |
+| Query Item (types OR, tags AND) | Single `WHERE` clause per item, items combined with `OR` |
+| Append Condition `failIfEventsMatch` | `SELECT EXISTS(...)` check in `SaveChangesAsync()` transaction |
+| Append Condition `after` position | `WHERE seq_id > @after` in the condition query |
+| Atomic append | PostgreSQL transaction wrapping condition check + event INSERT |