Add a "from the end" (last-N) query Offset to Akka.Persistence.Query by Aaronontheweb · Pull Request #8245 · akkadotnet/akka.net

Aaronontheweb · 2026-06-11T02:53:11Z

Closes #8244 (phase 1).

Summary

Adds a new Offset.FromEnd(int count) query offset to Akka.Persistence.Query so callers can retrieve the last N events (per-tag or across all events) without replaying the whole journal or maintaining a custom projection.

FromEnd is a query input only — it is never emitted in an EventEnvelope. A read journal resolves it at materialization into a concrete Sequence start offset (the Nth-from-last matching Ordering) and then reuses its existing forward-streaming pipeline. The change is fully additive: no interface signature changes and no wire-format changes. Backends that don't recognize it throw on the unknown offset exactly as they do today.

Current queries (CurrentEventsByTag/CurrentAllEvents): resolve start → stream forward → complete = exactly the last N.
Live queries (EventsByTag/AllEvents): resolve start once, then continue forward = last N, then everything new.

Release plan

This ships in an upcoming 1.5 beta, and the beta is the validation vehicle for the offset contract across the wider Akka.NET ecosystem. The public Offset.FromEnd surface and its API approvals are included in this PR so that:

plugin authors (Akka.Persistence.Sql, Akka.Persistence.MongoDb, community backends) can implement and validate against the offset + the TCK during the beta, and
we surface any contract/ergonomics issues before the API stabilizes in v1.6.0.

Only InMemory implements FromEnd in this PR; every other backend throws on the unknown offset until it opts in (tracked in #8244). That is expected for the beta — the TCK below is the spec those backends build against.

What's in this PR (phase 1)

Core: FromEnd : Offset + Offset.FromEnd(int) factory (input-only; CompareTo throws because the offset is relative, not a stream position). Constructor rejects count <= 0.
InMemory reference implementation: publishers resolve FromEnd(N) via a new MemoryJournal.SelectEventCount/EventCount round-trip (shared FromEndResolvingPublisher base), reusing the existing replay machinery; a faulted count query fails the stream (EventCountFailure) rather than hanging it.
TCK FromEndOffsetSpec — the cross-backend contract. Opt-in spec covering the full {by-tag, all-events} × {current, live} matrix against an interleaved, multi-persistence-id fixture that mixes tagged and untagged events:
- last-N, N > total (returns all), empty / no-matching-tag (completes, doesn't hang)
- live continuation for both by-tag and all-events
- a discriminator test pinning that by-tag FromEnd resolves against the per-tag count while all-events resolves against the total count — a distinction a single-persistence-id fixture cannot express, so a backend resolving both against a global ordinal would pass a naive spec but fail this one
- a deep-history test exercising the start = count - N arithmetic at depth
Unit tests: OffsetSpec/OffsetCompareSpecs cover the FromEnd factory, the count <= 0 guard, ToString, Equals/GetHashCode, and CompareTo throwing.
API approval updates for the additive surface (required for the beta — included).

Review status

Opened as a design-review draft; has since had a high-effort review pass with all findings addressed. InMemory is green against the hardened TCK (10 TCK tests + 15 Offset unit tests), -warnaserror clean, CI passing.

Design notes

FromEnd participates in Offset's IComparable<Offset> but CompareTo throws — generic comparison-based code paths (e.g. sorting a mixed offset set) will throw on a FromEnd. Intentional: it is a relative, input-only offset with no stream position, and is never emitted in an envelope. Calling it out so beta consumers know not to put FromEnd in a comparison/sort path.

Scope / follow-ups

Deferred (tracked in Add a "from the end" (last-N) query Offset to Akka.Persistence.Query #8244): Akka.Persistence.Sql (Linq2Db) and Akka.Persistence.MongoDb implementations + their TCK opt-ins. Azure is awkward (Table Storage has no server-side DESC/LIMIT); Redis is out of scope until its tag/all-events queries exist.
Backward pagination (older pages) is an explicit phase-2 — forward continuation from a captured offset already works today.
Per-persistence-id "last N" via a HighestSequenceNr - N + 1 helper (listed in Add a "from the end" (last-N) query Offset to Akka.Persistence.Query #8244) is not in this PR; defer or fold into a follow-up.
Docs for the Persistence.Query offset types — follow-up.

Caveat

The from-the-end position is resolved at materialization by reading the current matching count; for live queries it is best-effort — events written between resolving the count and reading the first batch may widen the initial window beyond N (documented on the type).

…Query Introduce Offset.FromEnd(int count), a query-input-only offset that begins a query at the Nth event from the end of history rather than the beginning. Read journals resolve it at materialization into a concrete Sequence start offset and then reuse their existing forward-streaming pipeline, so the change is fully additive: no interface signature changes and no wire-format changes. - Core: FromEnd : Offset type + Offset.FromEnd(int) factory - InMemory: resolve FromEnd in the by-tag and all-events publishers via a new MemoryJournal.SelectEventCount/EventCount round-trip - TCK: opt-in FromEndOffsetSpec (last-N, N > total, live continuation) and InMemoryFromEndOffsetSpec - Update API approval files for the additive surface SQL and MongoDB plugin implementations to follow. See akkadotnet#8244

- Fail the FromEnd resolution stream instead of hanging it: MemoryJournal now pipes a failure (EventCountFailure) for SelectEventCount, and the resolving state surfaces it via OnErrorThenStop, matching the sibling replay handlers. - Extract the duplicated FromEnd resolution handshake from both InMemory query publishers into a shared FromEndResolvingPublisher base. - Deduplicate the event-count predicate behind MemoryJournal.CountEvents and reuse it from the replay paths. - FromEnd no longer implements IComparable: it is a relative, input-only offset with no stream position, so CompareTo now always throws. - Document that the from-the-end window is resolved at materialization and is best-effort under concurrent writes. See akkadotnet#8244

Harden the FromEnd ("last N events") contract before it backs SQL/Mongo: - FromEndOffsetSpec: grow from 4 to 9 tests covering the full {by-tag, all-events} x {current, live} matrix against an interleaved, multi-persistence-id fixture mixing tagged and untagged events. Adds N>total and empty/zero-match cases for all-events, live AllEvents continuation, and a discriminating test pinning that by-tag FromEnd resolves against the per-tag count while all-events resolves against the total count (a single-persistence-id fixture cannot tell these apart, so a backend resolving both against a global ordinal would pass today). Converted to async TestKit style (ExpectMsgAsync/ExpectNextAsync/ AwaitConditionAsync) to match the sibling AllEventsSpec. - OffsetSpec/OffsetCompareSpecs: add FromEnd unit tests for the factory, the count<=0 guard, ToString, Equals/GetHashCode, and CompareTo throwing. See akkadotnet#8244

Fixes from a high-effort review of the FromEnd spec expansion: - FromEndOffsetSpec: the cross-query discriminator test now also waits for the green tag index to settle (not just all-events) before resolving its by-tag from-end window. Previously it could resolve against a partial tag set and flake on backends whose tag projection lags the global ordering (hidden on InMemory, where writes are immediately visible). - FromEndOffsetSpec: the live tests now require the current-query counterpart (RequireQuery) instead of silently skipping stabilization via an "is ICurrent...Query" check. The from-end window is resolved once at materialization, so a live-only backend with read-side lag would otherwise resolve it against an unsettled journal. Fail loud rather than flake. - FromEndOffsetSpec: derive AllEvents/GreenEvents from a single Fixture table (tagging by event text) instead of hand-maintaining two parallel arrays that could drift from the writes. - FromEndOffsetSpec: collapse the two near-identical WaitFor* helpers into one WaitForVisibleAsync(query, count); project the discriminator's result tuples once instead of recomputing them per assertion. - FromEndOffsetSpec: add a deep-history by-tag test (25 events, FromEnd(4)) to exercise the "start = count - N" arithmetic at depth and a long forward replay, restoring coverage the single-PID fixture had lost. - OffsetSpec/OffsetCompareSpecs: add the missing #nullable enable directive (CLAUDE.md: enable nullable in new/modified files). All green: 10 InMemory FromEnd TCK tests + 15 Offset unit tests; -warnaserror clean. See akkadotnet#8244

Supporting a "from the end" start is an internal detail of how the InMemory read journal interprets an Offset, not something every publisher constructor should know about. The previous design threaded two parallel ints — fromOffset and fromEndCount — through Props and three constructor layers, where fromEndCount > 0 was an implicit "resolve from the end, ignore fromOffset" sentinel (primitive-obsession + boolean-blindness). Replace that pair with a single internal readonly struct, ReplayStart, with named factories (At(offset) / LastN(count)) and an IsFromEnd discriminator. The read journal's Offset resolvers now return a ReplayStart; publishers take one argument instead of two and no longer re-derive the mode from a >0 check. Also drops the now-dead FromOffset property from both publishers (only ever assigned; CurrentOffset drives the replay). Purely internal: every affected type is internal, so there is no public API or wire change and no API-approval delta. Behavior is unchanged — full InMemory query suite (56 tests incl. the 10 FromEnd TCK tests) green, -warnaserror clean. See akkadotnet#8244

Aaronontheweb

LGTM - good to go for a prototype I think

Aaronontheweb · 2026-06-23T12:59:09Z

+    /// detail of how the read journal interprets an <see cref="Query.Offset"/>, instead of an extra argument threaded
+    /// through every publisher constructor.
+    /// </summary>
+    internal readonly struct ReplayStart


Aaronontheweb · 2026-06-23T13:41:09Z

+            var probe = queries.CurrentEventsByTag("green", FromEnd(2))
+                .RunWith(this.SinkProbe<EventEnvelope>(), Materializer);
+            probe.Request(10);
+            // the last two green events, across persistence ids, in ascending order


Aaronontheweb · 2026-06-23T13:41:40Z

+        }
+
+        [Fact]
+        public virtual async Task ReadJournal_query_CurrentEventsByTag_with_FromEnd_larger_than_total_should_return_all_events()


…8245) (#8308) * feat: add FromEnd ("last N events") query offset to Akka.Persistence.Query Introduce Offset.FromEnd(int count), a query-input-only offset that begins a query at the Nth event from the end of history rather than the beginning. Read journals resolve it at materialization into a concrete Sequence start offset and then reuse their existing forward-streaming pipeline, so the change is fully additive: no interface signature changes and no wire-format changes. - Core: FromEnd : Offset type + Offset.FromEnd(int) factory - InMemory: resolve FromEnd in the by-tag and all-events publishers via a new MemoryJournal.SelectEventCount/EventCount round-trip - TCK: opt-in FromEndOffsetSpec (last-N, N > total, live continuation) and InMemoryFromEndOffsetSpec - Update API approval files for the additive surface SQL and MongoDB plugin implementations to follow. See #8244 * refactor: address code review for FromEnd query offset - Fail the FromEnd resolution stream instead of hanging it: MemoryJournal now pipes a failure (EventCountFailure) for SelectEventCount, and the resolving state surfaces it via OnErrorThenStop, matching the sibling replay handlers. - Extract the duplicated FromEnd resolution handshake from both InMemory query publishers into a shared FromEndResolvingPublisher base. - Deduplicate the event-count predicate behind MemoryJournal.CountEvents and reuse it from the replay paths. - FromEnd no longer implements IComparable: it is a relative, input-only offset with no stream position, so CompareTo now always throws. - Document that the from-the-end window is resolved at materialization and is best-effort under concurrent writes. See #8244 * test: expand FromEnd TCK to full query matrix + add Offset unit tests Harden the FromEnd ("last N events") contract before it backs SQL/Mongo: - FromEndOffsetSpec: grow from 4 to 9 tests covering the full {by-tag, all-events} x {current, live} matrix against an interleaved, multi-persistence-id fixture mixing tagged and untagged events. Adds N>total and empty/zero-match cases for all-events, live AllEvents continuation, and a discriminating test pinning that by-tag FromEnd resolves against the per-tag count while all-events resolves against the total count (a single-persistence-id fixture cannot tell these apart, so a backend resolving both against a global ordinal would pass today). Converted to async TestKit style (ExpectMsgAsync/ExpectNextAsync/ AwaitConditionAsync) to match the sibling AllEventsSpec. - OffsetSpec/OffsetCompareSpecs: add FromEnd unit tests for the factory, the count<=0 guard, ToString, Equals/GetHashCode, and CompareTo throwing. See #8244 * test: address code-review findings on FromEnd TCK + Offset unit tests Fixes from a high-effort review of the FromEnd spec expansion: - FromEndOffsetSpec: the cross-query discriminator test now also waits for the green tag index to settle (not just all-events) before resolving its by-tag from-end window. Previously it could resolve against a partial tag set and flake on backends whose tag projection lags the global ordering (hidden on InMemory, where writes are immediately visible). - FromEndOffsetSpec: the live tests now require the current-query counterpart (RequireQuery) instead of silently skipping stabilization via an "is ICurrent...Query" check. The from-end window is resolved once at materialization, so a live-only backend with read-side lag would otherwise resolve it against an unsettled journal. Fail loud rather than flake. - FromEndOffsetSpec: derive AllEvents/GreenEvents from a single Fixture table (tagging by event text) instead of hand-maintaining two parallel arrays that could drift from the writes. - FromEndOffsetSpec: collapse the two near-identical WaitFor* helpers into one WaitForVisibleAsync(query, count); project the discriminator's result tuples once instead of recomputing them per assertion. - FromEndOffsetSpec: add a deep-history by-tag test (25 events, FromEnd(4)) to exercise the "start = count - N" arithmetic at depth and a long forward replay, restoring coverage the single-PID fixture had lost. - OffsetSpec/OffsetCompareSpecs: add the missing #nullable enable directive (CLAUDE.md: enable nullable in new/modified files). All green: 10 InMemory FromEnd TCK tests + 15 Offset unit tests; -warnaserror clean. See #8244 * refactor: collapse FromEnd plumbing into an internal ReplayStart value Supporting a "from the end" start is an internal detail of how the InMemory read journal interprets an Offset, not something every publisher constructor should know about. The previous design threaded two parallel ints — fromOffset and fromEndCount — through Props and three constructor layers, where fromEndCount > 0 was an implicit "resolve from the end, ignore fromOffset" sentinel (primitive-obsession + boolean-blindness). Replace that pair with a single internal readonly struct, ReplayStart, with named factories (At(offset) / LastN(count)) and an IsFromEnd discriminator. The read journal's Offset resolvers now return a ReplayStart; publishers take one argument instead of two and no longer re-derive the mode from a >0 check. Also drops the now-dead FromOffset property from both publishers (only ever assigned; CurrentOffset drives the replay). Purely internal: every affected type is internal, so there is no public API or wire change and no API-approval delta. Behavior is unchanged — full InMemory query suite (56 tests incl. the 10 FromEnd TCK tests) green, -warnaserror clean. See #8244 (cherry picked from commit 938a629)

Aaronontheweb added 2 commits June 10, 2026 19:57

Aaronontheweb added the akka-persistence-query label Jun 11, 2026

Aaronontheweb added this to the 1.5.70 milestone Jun 15, 2026

Aaronontheweb and others added 3 commits June 22, 2026 15:16

Merge branch 'v1.5' into feature/persistence-query-fromend-offset

1d28573

Aaronontheweb marked this pull request as ready for review June 23, 2026 00:55

Aaronontheweb and others added 2 commits June 22, 2026 20:47

Merge branch 'v1.5' into feature/persistence-query-fromend-offset

2091b67

Aaronontheweb commented Jun 23, 2026

View reviewed changes

Aaronontheweb merged commit 938a629 into akkadotnet:v1.5 Jun 23, 2026
11 checks passed

Aaronontheweb deleted the feature/persistence-query-fromend-offset branch June 23, 2026 13:42

Aaronontheweb mentioned this pull request Jun 23, 2026

Add a "from the end" (last-N) query Offset to Akka.Persistence.Query #8244

Open

6 tasks

This was referenced Jul 2, 2026

Release Akka.NET v1.5.70 #8306

Merged

Forward-port #8245 to dev: Offset.FromEnd (last-N events) query offset #8308

Merged

This was referenced Jul 27, 2026

Bump Akka from 1.5.68 to 1.5.70 Aaronontheweb/akka.net-aspire-plugin#50

Merged

Bump the akka group with 10 updates sbdevman/Thunderbolt#49

Open

dependabot Bot mentioned this pull request Sep 6, 2026

Bump Akka.Persistence and Akka.Persistence.Query Kuestenlogik/Akka.Surgewave#30

Merged

Navigation Menu

Add a "from the end" (last-N) query Offset to Akka.Persistence.Query - #8245

Add a "from the end" (last-N) query Offset to Akka.Persistence.Query#8245
Aaronontheweb merged 7 commits into
akkadotnet:v1.5from
Aaronontheweb:feature/persistence-query-fromend-offset

Aaronontheweb commented Jun 11, 2026 •

edited

Loading

Uh oh!

Aaronontheweb left a comment

Uh oh!

Aaronontheweb Jun 23, 2026

Uh oh!

Aaronontheweb Jun 23, 2026

Uh oh!

Aaronontheweb Jun 23, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

Aaronontheweb commented Jun 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Release plan

What's in this PR (phase 1)

Review status

Design notes

Scope / follow-ups

Caveat

Uh oh!

Aaronontheweb left a comment

Choose a reason for hiding this comment

Uh oh!

Aaronontheweb Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

Aaronontheweb Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

Aaronontheweb Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Aaronontheweb commented Jun 11, 2026 •

edited

Loading