Skip to content

Compliance suite: dead letters and the projection error path #5149

Description

@jeremydmiller

Part of #5118 (compliance wave growth), epic #5110.

AsyncDaemonCompliance is deliberately a smoke test: one async snapshot, one rebuild. The error path — a projection that throws, the event being dead-lettered, the shard continuing — is the behavior that actually costs users when it drifts, and it is declared on shared interfaces:

  • JasperFx.Events.Daemon.DeadLetterEvent, DeadLetterShardCount
  • IEventDatabase.CountDeadLetterEventsAsync(ShardName), QueryDeadLetterEventsAsync(ShardName, tenantId), FetchDeadLetterCountsAsync() and its per-tenant overload
  • IEventStore.AllDatabases()IReadOnlyList<IEventDatabase>
  • IEventFailureContext

Seam additions required

Small. AllDatabases() is already on the shared IEventStore the fixture exposes, so the dead-letter queries are reachable today. What is missing is a way for a suite to register a projection that fails deterministically and to configure the error policy:

  1. ComplianceStoreConfig needs an error-handling knob — at minimum "skip poison events after N attempts and record a dead letter", which both products spell differently (Marten OnException(...).SkipEvent() policies on DaemonSettings; Polecat's equivalent). Follows the MaxConcurrentRebuildsPerDatabase precedent: fixture-owned property, not registrar-routed.
  2. Nothing else — the failing projection itself is shared source.

What each store tests today

Marten Polecat
DaemonTests error-handling and skip tests Daemon/dead_letter_count_tests.cs
marking_events_as_skipped_as_Guid_identified.cs Daemon/skipped_events_count_augmenter_tests.cs
marking_events_as_skipped_as_string_identified.cs Daemon/shard_failure_progression_columns_tests.cs
Events/event_failure_context_tests.cs

Scope

New DeadLetterCompliance suite:

  1. A projection that throws on one event type dead-letters that event and the shard keeps advancing
  2. CountDeadLetterEventsAsync / FetchDeadLetterCountsAsync report the shard and the count
  3. QueryDeadLetterEventsAsync returns the event body, the exception type and the shard name
  4. IEventFailureContext correlates the failure with the dead-letter row
  5. A rebuild clears prior dead letters for the shard
  6. Marking events as skipped, for Guid and string stream identity

Explicitly out of scope: pause/resume policies, circuit breakers, multi-node failure handling — daemon-topology behavior that stays product-specific.

Acceptance

  • One config knob added; suite enrolled in both stores
  • Originals retired for the absorbed behavior in both repos

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions