mailbox+serverconn: durable response store and unified connector boundary - #116
Conversation
In this commit, we introduce a Store abstraction that the mailbox RPC client uses to persist response payloads and the pull cursor across restarts. The interface defines five operations: LoadCursor, SaveCursor, PutResponse, GetResponse, and DeleteResponse, with idempotency and monotonicity contracts that match the cursor-based AckUpTo protocol. A MemoryStore implementation is included for tests and short-lived processes that don't need crash safety. The Config struct gains a Store field that defaults to MemoryStore when unset, preserving backward compatibility with existing callers.
With the Store interface in place, we now integrate it into the client's main run loop to achieve crash-safe response handling under the cursor-based AckUpTo protocol. On startup the client loads the persisted cursor so it resumes pulling from where it left off. Each inbound response envelope is written to the Store before any in-memory waiter is notified, ensuring the payload survives a crash between dispatch and consumption. The cursor is persisted to the Store before acking with the remote mailbox, so a restart replays only unacked envelopes rather than the entire stream. AwaitRPC now peeks through the in-memory pending map into the Store, and only deletes the response from both layers after successful proto unmarshal. A best-effort retry loop handles transient delete failures without blocking the caller. This two-tier lookup means responses are available even if the process restarted between PutResponse and the caller's AwaitRPC. The malformed response body error is extracted to a package-level format string so it can be matched in tests.
Summary of ChangesHello @Roasbeef, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request significantly enhances the reliability and functionality of the client-side mailbox communication. It introduces a durable storage mechanism for RPC responses, ensuring that client operations can gracefully recover from crashes without losing critical data. Concurrently, it refactors and expands the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces significant changes to the mailbox client and server connection components, primarily focusing on crash-safe operation and improved RPC handling. Key changes include the addition of a Store interface and MemoryStore implementation for persisting pull cursors and response payloads, ensuring at-least-once delivery even across restarts. The Client struct in mailbox/client/client.go was refactored to utilize this new store, incorporating logic for loading/saving cursors and persisting/retrieving RPC responses. A new ackTo field and associated methods were added to manage the acknowledgment watermark. The AwaitRPC method was updated to use peekResponse and deleteResponseBestEffort for more robust response handling and cleanup. Concurrently, a new serverconn package was introduced, defining a ServerConnectionActor that acts as a unified connector for client-server mailbox traffic. This actor includes an ingress loop for pulling, dispatching, and acknowledging envelopes, and an egress path for sending FSM events and RPC requests. It uses a DeliveryStore for checkpointing its AckState to ensure durability. The ServerConnectionActor also manages an in-memory responseRegistry for delivering unary RPC responses. A UnaryFacade was added to implement the mailboxrpc.RPCClient interface, leveraging the ServerConnectionActor for sending and awaiting RPCs. The review comment highlights an inconsistency in logging, noting that serverconn/log.go initializes a btclog.Logger while other files in the package use log/slog calls, suggesting a conversion to log/slog for consistency and to resolve compilation errors.
| package serverconn | ||
|
|
||
| import "github.com/btcsuite/btclog/v2" | ||
|
|
||
| // Subsystem defines the logging code for this subsystem. | ||
| const Subsystem = "SVRC" | ||
|
|
||
| // log is a logger that is initialized with no output filters. This means the | ||
| // package will not perform any logging by default until the caller requests | ||
| // it. | ||
| var log = btclog.Disabled | ||
|
|
||
| // DisableLog disables all library log output. Logging output is disabled by | ||
| // default until UseLogger is called. | ||
| func DisableLog() { | ||
| UseLogger(btclog.Disabled) | ||
| } | ||
|
|
||
| // UseLogger uses a specified Logger to output package logging info. This | ||
| // should be used in preference to SetLogWriter if the caller is also using | ||
| // btclog. | ||
| func UseLogger(logger btclog.Logger) { | ||
| log = logger | ||
| } |
There was a problem hiding this comment.
The serverconn package appears to have inconsistent logging setups. This file initializes a btclog.Logger, but other files in the package (ingress.go, actor.go, unary_facade.go) make calls assuming a log/slog logger (e.g., log.InfoS). This will lead to compilation errors because btclog.Logger does not have an InfoS method.
To ensure consistency and fix the compilation issue, this logging setup should be converted to use log/slog, which seems to be the intended logging library for new code in this repository.
package serverconn
import (
"io"
"log/slog"
)
// Subsystem defines the logging code for this subsystem.
const Subsystem = "SVRC"
// log is a logger that is initialized with no output filters. This means the
// package will not perform any logging by default until the caller requests
// it.
var log = slog.New(slog.NewTextHandler(io.Discard, nil))
// DisableLog disables all library log output. Logging output is disabled by
// default until UseLogger is called.
func DisableLog() {
UseLogger(slog.New(slog.NewTextHandler(io.Discard, nil)))
}
// UseLogger uses a specified Logger to output package logging info.
func UseLogger(logger *slog.Logger) {
log = logger
}![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 34b21f7bc9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if committedCursor > state.PullCursor { | ||
| state.AdvanceDispatch(committedCursor) | ||
| state.PullCursor = committedCursor |
There was a problem hiding this comment.
dispatchBatch returns committedCursor as the last successfully processed event_seq, but on partial failure the loop stores that value directly in state.PullCursor. Since the pull cursor is the next expected sequence, resuming from committedCursor re-pulls the last already-dispatched envelope and can dispatch it twice before ack catches up, which violates the intended “don’t re-dispatch already committed envelopes” behavior and can duplicate downstream side effects.
Useful? React with 👍 / 👎.
| if !delivered { | ||
| log.DebugS(ctx, | ||
| "No waiter for response, dropping", |
There was a problem hiding this comment.
When deliverResponse cannot find a waiter, the response is explicitly dropped; the loop still advances/acks the cursor later, so that response is permanently lost. In the normal generated client flow (SendRPC then AwaitRPC), a fast response can be pulled before AwaitRPC registers its waiter, which leads to intermittent timeouts even though the server already replied successfully.
Useful? React with 👍 / 👎.
| return err | ||
| } | ||
|
|
||
| if _, err := stream.DecodeWithParsedTypes(r); err != nil { |
There was a problem hiding this comment.
Could just store pure protos here and bypass the TLV round trip since we send protos in the end.
There was a problem hiding this comment.
Ah we require TLV serialization rn with the package. So let's do the composite generic type thing mentioned below.
|
@claude review this |
|
Claude finished @Roasbeef's task in 5m 22s —— View job PR Review: mailbox+serverconn: durable response store and unified connector boundaryTasks
This is a well-structured PR with good test coverage. The ack state machine design is sound and the 1. Pull cursor off-by-one on partial dispatch failure (correctness bug)
if committedCursor > state.PullCursor {
state.AdvanceDispatch(committedCursor)
state.PullCursor = committedCursor // ← sets to event_seq, not event_seq+1On the success path, The comment on line 120 says "so we don't re-dispatch already committed envelopes" — but the implementation does exactly that. At-most-once dispatch is rescued by Fix: use if committedCursor > 0 {
nextCursor := committedCursor + 1
if nextCursor > state.PullCursor {
state.AdvanceDispatch(nextCursor)
state.PullCursor = nextCursor
...
}
}2. Misleading "No waiter for response, dropping" log message
delivered := a.deliverResponse(corrID, env)
if !delivered {
log.DebugS(ctx, "No waiter for response, dropping", ...)
}
3. Stale-waiter prune can permanently strand a blocked
|
e7c3e85 to
2b40b23
Compare
|
@claude review this |
|
Claude finished @Roasbeef's task in 7m 2s —— View job PR Review: mailbox+serverconn: durable response store and unified connector boundaryTasks
This PR significantly restructures the mailbox client layer: Previous review issues — status
1.
|
155a0c6 to
744bf65
Compare
|
@claude review this |
|
Claude finished @Roasbeef's task in 4m 38s —— View job PR Review: mailbox+serverconn: durable response store and unified connector boundaryTasks
This is the third review pass. Most of the previously reported issues have been fixed. Below is the current state of the open issues, followed by new findings. Previous review issues — status
1.
|
744bf65 to
44e0502
Compare
|
@claude review this |
|
Claude finished @Roasbeef's task in 4m 38s —— View job PR Review: mailbox+serverconn: durable response store and unified connector boundaryTasks
This is the fourth review pass. Most of the previously-flagged issues have been addressed. Below is the current status of open items, followed by new findings. Previous review issues — status
1. Data race on
|
| } | ||
| } | ||
|
|
||
| return waiter.Ch |
There was a problem hiding this comment.
Should this be an actor.Future[mailboxpb.Envelope] instead?
|
|
||
| if waiter, ok := r.waiters[id]; ok { | ||
| select { | ||
| case waiter.Ch <- env: |
There was a problem hiding this comment.
This would then complete the promise.
| // later. | ||
| type Runtime struct { | ||
| connector *ServerConnectionActor | ||
| durable *actor.DurableActor[ServerConnMsg, ServerConnResp] |
There was a problem hiding this comment.
Embed instead? Then an actual actor.
There was a problem hiding this comment.
So it can be registered directly.
| } | ||
| idempotencyBytes := []byte(idempotencyKey) | ||
|
|
||
| records := []tlv.Record{ |
There was a problem hiding this comment.
We can make a composite type wrapper here, which handles this boiler plate for us (wrap proto in TLV for now).
| return err | ||
| } | ||
|
|
||
| if _, err := stream.DecodeWithParsedTypes(r); err != nil { |
There was a problem hiding this comment.
Ah we require TLV serialization rn with the package. So let's do the composite generic type thing mentioned below.
| // | ||
| // Cursor never advances past non-durable local work. Repeated acks are safe | ||
| // and idempotent. | ||
| type AckState struct { |
There was a problem hiding this comment.
Moved this here so it can be used across other instances. As part of the durable actor, you store this state for server/client conn and use that to remember what was sent vs not.
44e0502 to
d8b972e
Compare
|
@claude review this |
|
Claude finished @Roasbeef's task in 3m 25s —— View job PR Review: mailbox+serverconn: durable response store and unified connector boundaryTasks
This is the fifth review pass. The codebase is in good shape — the previous round of fixes was thorough. Two genuine issues remain, along with a minor documentation inconsistency. Previous review issues — status
1. Data race on
|
This commit introduces the foundational types for the serverconn connector, which will serve as the unified boundary for all mailbox traffic between the client and the remote server. The AckState struct models the four-cursor state machine that governs the pull-dispatch-ack cycle: PullCursor tracks where to resume pulling, DispatchCommittedTo records the last durably dispatched position, AckTarget holds the cursor to send in the next AckUpTo call, and AckCommittedTo tracks what the remote has actually acknowledged. The state machine enforces the critical invariant that AckUpTo only advances after durable local dispatch, preventing message loss on crash. TLV encode/decode methods allow the state to be persisted via the durable actor checkpoint mechanism. Also included are the ConnectorConfig struct with all dependency injection points (Edge client, dispatch table, delivery store, pull tuning knobs), named types for CorrelationID and IdempotencyKey, the EnvelopeDispatcher function type for ServiceKey-based routing, and the ResponseWaiter struct for in-memory unary RPC correlation.
d8b972e to
c573fc3
Compare
|
@claude review this |
|
Claude finished @Roasbeef's task in 5m 29s —— View job PR Review (Pass 6): mailbox+serverconn: durable response store and unified connector boundaryTasks
This is the sixth review pass. The PR is in excellent shape — all substantive issues from prior passes have been addressed. Two minor structural notes below. Previous review issues — status
1.
|
The ServerConnectionActor is transformed from a stub relay into the unified connector boundary described in the architecture plan. Message types gain full TLV serialization so they can be persisted in the durable actor mailbox. SendClientEventRequest wraps the outbound proto in an anypb.Any to preserve type information across the encode/decode boundary, using the global protobuf type registry for reconstruction. SendRPCRequest serializes the entire mailbox Envelope via proto.Marshal for the unary facade path. Both message types carry stable TLV type identifiers (2000, 2001) for the durable actor codec. The actor struct gains a response registry (correlation ID to waiter channel map) for unary RPC response routing, protected by a dedicated mutex separate from the actor's mailbox processing. Egress handlers now build proper mailbox envelopes with protocol version, sender/ recipient addressing, and RPC metadata, sending them through the configured Edge client. A MessageCodec factory registers both message types for the durable actor runtime. The ingress loop lifecycle is managed via StartIngress/StopIngress, running as a background goroutine independent of the actor mailbox.
The ingress loop is the heart of the connector's inbound message processing. It runs as a self-driven background goroutine that continuously long-polls the remote mailbox via Edge.Pull and routes received envelopes to local consumers. The loop follows a strict three-phase cycle per iteration: First, if a prior dispatch committed successfully, it acks the remote mailbox up to the committed cursor via Edge.AckUpTo. This is done before pulling new work so the remote can garbage-collect delivered envelopes. The ack only advances after the checkpoint has been persisted, maintaining the invariant that AckUpTo never races ahead of durable local state. Second, it pulls a batch of envelopes starting from the current cursor. Empty long-poll returns reset the backoff counter and loop immediately since the long-poll timeout already provides the delay. Third, each envelope is dispatched based on its RPC metadata kind. KIND_RESPONSE envelopes are delivered to the in-memory response registry for unary RPC waiters. KIND_REQUEST and KIND_EVENT envelopes are routed through the configured dispatch table to local durable actors via their ServiceKey. If any dispatch fails mid-batch, the loop advances state only through the last successfully committed position, avoiding re-dispatch of already-committed envelopes on retry. All checkpoint state is persisted through the DeliveryStore's checkpoint mechanism using a stable actor ID derived from the local mailbox ID. Transient failures trigger exponential backoff with jitter to prevent busy-spinning, capped at the configured maximum delay.
The UnaryFacade provides the mailboxrpc.RPCClient interface on top of the ServerConnectionActor, giving generated service stubs a synchronous send-and-await API for unary RPCs. SendRPC builds a complete mailbox envelope with correlation ID, idempotency key, service/method addressing, and protocol version, then sends it directly via Edge.Send rather than going through the durable actor mailbox. This direct path keeps unary RPC latency low since the caller already handles retries on failure and doesn't need durable egress guarantees. AwaitRPC registers a waiter in the actor's response registry, then blocks on the waiter channel until the ingress loop delivers a matching KIND_RESPONSE envelope or the caller's context is cancelled. On receipt, it unmarshals the proto body from the envelope's Any value into the caller-provided response message. The waiter is always cleaned up on exit to prevent registry leaks. Package documentation is included describing the connector's role as the single ingress/egress boundary, the ack watermark invariants, and the relationship between the dispatch table and the unary facade.
This commit adds comprehensive tests covering the three main layers of the serverconn connector. The ack state machine tests verify the cursor advancement invariants: AdvanceDispatch sets the ack target monotonically, AdvanceAck updates the committed cursor and pull position, NeedsAck correctly reflects pending acks, and TLV round-trip encoding preserves all four cursor fields including the zero-value edge case. The connector integration tests exercise the full ingress loop against an in-memory mailbox edge. They verify that dispatched envelopes are acked, responses are delivered to registered waiters, failed dispatches don't advance the ack cursor, shutdown drains cleanly without goroutine leaks, and checkpoint state survives a simulated restart cycle. The exponential backoff helper is tested for range bounds and zero-value defaults. The unary facade tests cover the SendRPC envelope construction path (verifying correlation ID, idempotency key, and service/method metadata in the sent envelope), the AwaitRPC round-trip through the response registry including context cancellation, concurrent inflight request isolation, and nil body handling. Test infrastructure includes an in-memory mailbox with long-poll support, a fake MailboxServiceClient adapter, and a minimal checkpoint store that implements the full DeliveryStore interface with panicking stubs for unused methods.
In this commit, we introduce the mailbox/conn package to house reusable building blocks shared by client-side and server-side connector runtimes. The AckState struct and its TLV encode/decode are moved here wholesale from serverconn/types.go, along with the CorrelationID and IdempotencyKey named types. A ResponseRegistry provides in-memory correlation waiter tracking with early-response buffering (a response that arrives before its waiter is registered gets cloned and held until the waiter appears) and configurable TTL-based stale cleanup. Envelope identity helpers (StableEventMsgID, StableEventIdempotencyKey) derive deterministic SHA-256-based identifiers from a serialized payload so durable retries of the same semantic event produce the same message and idempotency keys for remote dedupe. The test suite includes both table-driven unit tests and property-based randomized checks (via pgregory.net/rapid) for the ack state invariants and response registry interleaving behavior.
…dentity In this commit, we wire the serverconn package to use the shared mailbox/conn primitives and fix two correctness gaps identified during review. The type aliases for CorrelationID, IdempotencyKey, AckState, and ResponseWaiter now point to mailbox/conn, and the actor's hand-rolled response registry is replaced by conn.ResponseRegistry. The registry's early-response buffering closes the race where a fast response from the remote mailbox could arrive (via the ingress loop) before AwaitRPC had a chance to register its waiter -- previously that response would be silently dropped. SendRPC now registers the waiter before calling Edge.Send, and on any send failure the waiter is cleaned up immediately. AwaitRPC uses a defer to remove its waiter on all exit paths. SendClientEventRequest gains MsgID and IdempotencyKey fields that survive TLV round-trips. When unset, Encode derives deterministic defaults from the serialized proto payload via conn.StableEventMsgID and conn.StableEventIdempotencyKey, ensuring that durable replay of the same semantic event produces the same identifiers for remote dedupe. The egress handler stamps these identifiers onto the outbound envelope. The standalone ack_state_test.go is removed since those tests now live in mailbox/conn.
In this commit, we add a Runtime struct that wires the ServerConnectionActor, its DurableActor wrapper, and the UnaryFacade into a single composition unit that higher-level daemon bootstrap can use. NewRuntime validates required config (Store, LocalMailboxID), fills a default codec when one isn't supplied, and returns a fully assembled but inert runtime. Start launches both durable egress processing and the ingress pull loop; Stop tears them down in reverse order. Accessors expose the Ref/TellRef (for round actor egress) and Unary (for typed RPC stubs). The DurableActorID helper extracts the "serverconn-" + mailboxID convention into a single function shared by both the runtime and the ingress checkpoint load/save paths. Package docs are updated to describe the runtime composition stage and note the shared mailbox/conn primitives.
In this commit, we remove the entire mailbox/client package. With the serverconn package now serving as the unified connector boundary for all mailbox traffic, the legacy client runtime is redundant. It maintained its own pull loop, cursor persistence, response store, and ack logic -- all of which duplicated what serverconn and the durable actor runtime already provide. Keeping a parallel connector path would mean two competing durability stores, two cursor management schemes, and two sets of retry semantics. Removing it now prevents that divergence from accumulating.
In this commit, we add tests covering the new functionality introduced in the prior commits. The TLV round-trip tests verify that SendClientEventRequest produces stable deterministic MsgID and IdempotencyKey for equal payloads, that explicitly set identifiers survive encode/decode unchanged, and that SendRPCRequest round-trips its full envelope. The Receive path is tested for unknown message rejection and the SendRPCRequest dispatch handler. The runtime tests verify config validation rejects missing Store and LocalMailboxID, and that NewRuntime fills a default codec and wires all accessors (Ref, TellRef, Unary, Connector) correctly. The connector test gains TestEgress_EventRetriesPreserveIdempotencyKey, which verifies that two egress sends of the same semantic event produce identical MsgId and IdempotencyKey on the wire. The unary facade test gains TestUnaryFacade_ResponseBeforeAwait, which exercises the early- response buffering path where the ingress loop delivers a response before the caller starts AwaitRPC.
In this commit, we replace the panicking stubs in memCheckpointStore with working in-memory implementations of the full actor.DeliveryStore interface. The store now supports EnqueueMessage with ON CONFLICT DO NOTHING semantics for receiver-side dedup, LeaseNextMessage with FIFO ordering and lease expiry, AckMessage, NackMessage with retry-after delays, ExtendLease, MoveToDeadLetter, IsProcessed/MarkProcessed, and the outbox claim/complete/fail cycle. This upgrade is needed by upcoming tests that exercise the durable actor runtime integration path where the runtime calls through to the full DeliveryStore contract rather than just the checkpoint subset.
In this commit, we add tests covering the remaining gaps identified during review: ingress error paths, crash/restart replay semantics, property-based ack invariant checks, high-concurrency unary response delivery, codec round-trips, and runtime lifecycle. The ingress error tests use a configurable MailboxServiceClient stub to verify behavior under Pull failures, AckUpTo failures, partial dispatch failures, and responses with missing or nil RPC metadata. The restart/replay test exercises egress idempotency by verifying that a failed Send followed by a retry produces identical MsgId and IdempotencyKey on the wire. The property-based test uses pgregory.net/rapid to randomize ack and partial-dispatch progressions, asserting that AckCommittedTo never exceeds DispatchCommittedTo under arbitrary interleaving. The high-concurrency unary test fires 200 requests, injects responses in shuffled order before AwaitRPC begins, and verifies each caller receives its correct response through the early-response buffering path. The runtime lifecycle test verifies Start/Stop complete promptly on context cancellation without goroutine leaks.
c573fc3 to
2b19f47
Compare
|
@claude review this |
|
Claude finished @Roasbeef's task in 3m 37s —— View job PR Review (Pass 7): mailbox+serverconn: durable response store and unified connector boundaryTasks
This is the seventh review pass. All issues from the previous six passes have been addressed. The codebase is clean and ready for merge. Previous review issues — status
Verification of the two Pass 6 open items
ingressCtx, cancel := context.WithCancel(ctx)
a.wg.Add(1) // ← counter incremented first
a.cancelCh <- cancel // ← then delivered to StopIngress
go a.ingressLoop(ingressCtx, state)
No new issues foundAfter reading all key files (
LGTM. The PR has had a thorough review cycle and all identified issues have been resolved. No further action required. |
1 participant
In this PR, we lay the groundwork for making the client's mailbox RPC
layer crash-safe by introducing a
Storeabstraction for responsepersistence and expanding
serverconninto the unified connectorboundary for all client↔server mailbox traffic.
mailbox/client: durable response store
We add a
Storeinterface tomailbox/clientthat persists the pullcursor and response payloads keyed by correlation ID. The client's main
runloop is then wired to use this store: on startup we load thepersisted cursor so pulls resume from where we left off, each inbound
response envelope is written to the store before notifying in-memory
waiters, and the cursor is persisted before acking with the remote
mailbox.
AwaitRPCnow does a two-tier lookup — first the in-memorypending map, then the store — so responses survive a restart between
PutResponseand the caller'sAwaitRPC.A
MemoryStoreships for tests and short-lived processes that don'tneed crash safety. The
Config.Storefield defaults to it when unset,so existing callers are unaffected.
serverconn: unified connector boundary
The
serverconnpackage is expanded from its previous stub into thesingle ingress/egress boundary for mailbox traffic. The core pieces:
Ack state machine (
types.go): Four cursors —PullCursor,DispatchCommittedTo,AckTarget,AckCommittedTo— enforce theinvariant that
AckUpToonly advances after durable local dispatch.TLV encode/decode lets the state persist via the durable actor
checkpoint mechanism.
TLV message types (
actor.go):SendClientEventRequestwrapsoutbound FSM events in
anypb.Anyfor type-preserving TLVserialization.
SendRPCRequestmarshals the full mailbox envelope forthe unary path. Both carry stable TLV type identifiers for the durable
actor codec. The actor struct gains a response registry, egress
handlers that build proper mailbox envelopes, and ingress lifecycle
management.
Ingress loop (
ingress.go): A self-driven background goroutinethat long-polls the remote mailbox via
Edge.Pull, routesKIND_RESPONSEenvelopes to the response registry for unary waiters,routes
KIND_REQUEST/KIND_EVENTenvelopes through the dispatch tableto local durable actors, and advances the ack watermark only after
committed dispatch. Transient failures get exponential backoff w/
jitter.
Unary facade (
unary_facade.go): Implementsmailboxrpc.RPCClienton top of the connector.
SendRPCgoes directly throughEdge.Send(no durable mailbox — caller retries on failure), while
AwaitRPCblocks on the response registry channel until the ingress loop delivers
a matching response.
Tests
21 tests across three files cover the ack state machine invariants,
full ingress loop integration against an in-memory mailbox edge
(dispatch, ack, checkpoint restart, shutdown), and the unary facade
round-trip including concurrent inflight isolation and context
cancellation. See each commit message for a detailed description w.r.t
the incremental changes.