Skip to content

GH-4188: see past the pass-through receiver wrappers before branching on the receiver - #4190

Merged
jeremydmiller merged 1 commit into
mainfrom
gh-4188-unwrap-receiver-for-enqueue-directly
Aug 30, 2026
Merged

GH-4188: see past the pass-through receiver wrappers before branching on the receiver#4190
jeremydmiller merged 1 commit into
mainfrom
gh-4188-unwrap-receiver-for-enqueue-directly

Conversation

@jeremydmiller

@jeremydmiller jeremydmiller commented Aug 30, 2026

Copy link
Copy Markdown
Member

Closes #4188.

The defect

ListeningAgent.EnqueueDirectlyAsync type-switches over receiver implementations to decide how a replayed envelope re-enters the pipeline. It reads the raw _receiver field — but that field is routinely a pass-through wrapper, not the receiver the switch is looking for.

Endpoint.MaybeWrapReceiver installs ReceiverWithRules whenever RulesForIncoming() yields anything: an IncomingRules entry, an endpoint-level MessageType, or an endpoint-level TenantId. ReceiverWithRules is unconditionally an ILocalQueue, so a wrapped NativeAckReceiver or InlineReceiver matched the ILocalQueue branch ahead of its own and then threw from inside ReceiverWithRules.EnqueueAsync, whose Inner is not a local queue:

System.InvalidOperationException: There is no active, local queue for this listening endpoint at <uri>

Same exception, same call sites — DLQ replay per GH-1942 and scheduled-message firing — as the bug #4011 was supposed to have closed. #4011 added the missing branch; it did not fix the branch never being reached.

GlobalPartitionedInterceptor is the second wrapper on the same path. It is not an ILocalQueue at all, so everything behind it fell through to the throwing else.

A wrapped BufferedReceiver is the quieter third case: it matched the generic ILocalQueue branch instead of its own, so the replay was enqueued rather than dispatched through a RetryOnInlineChannelCallback — and that callback is the only thing that marks the inbox row handled on completion (GH-1942). The message ran; the row was left behind.

Noticed while working #4187 (GH-4186) and deliberately left out of scope there.

The fix

Both wrappers now implement a small internal IReceiverWrapper { IReceiver Inner { get; } }, and Unwrap() peels them off — they nest, GlobalPartitionedInterceptor(ReceiverWithRules(inner)) — so branch selection sees the receiver that actually executes messages.

Dispatch stays on the outer receiver everywhere it can, so the wrappers keep doing their jobs: incoming envelope rules still get applied, globally partitioned messages still get re-routed. Only the ILocalQueue branch enqueues into the unwrapped receiver, which is exactly what ReceiverWithRules.EnqueueAsync delegated to anyway.

Branch order is unchanged — BufferedReceiver is still tested ahead of the generic ILocalQueue branch, so #4011 does not regress — and a null _receiver still reaches the same throw it always did.

One behavior change beyond the throw

The wrapped-BufferedReceiver case now dispatches through ReceivedAsync instead of EnqueueAsync, so incoming envelope rules are applied to a replayed envelope where before they were skipped. That is a deliberate consequence, not a side effect of fixing the inbox row: a replay should look like a delivery, and every other path into that endpoint already stamps the endpoint's TenantId / MessageType. For the two rules RulesForIncoming() synthesizes on its own — TenantIdRule and MessageTypeRule, both unconditional assignments — re-running them on a DLQ replay is a no-op, because the envelope was stamped by the same rules when it first arrived. A user-supplied rule in IncomingRules carries no such guarantee, but re-running one on a replay is exactly what a broker redelivery already does. For a scheduled message the result is what a live delivery to that endpoint would have produced. The buffered test asserts it (envelope.TenantId).

It is limited to that one case. Wrapped NativeAck and Inline threw before, so they have no prior behavior to change, and an unwrapped BufferedReceiver has no rules to apply.

LatchReceiver hand-rolled a one-level ReceiverWithRules unwrap for the same reason (GH-3709) and so never latched anything behind the interceptor. It uses the shared helper now, which matters: an unlatched receiver's DrainAsync returns immediately instead of waiting for in-flight handlers.

Tests

CoreTests/Runtime/WorkerQueues/wrapped_receiver_enqueue_directly_4188.cs — five tests, all red before the change and each verified to fail for the right reason:

test pre-fix failure
NativeAck + incoming rule InvalidOperationException from ReceiverWithRules.EnqueueAsync
Inline + incoming rule same
NativeAck behind a GlobalPartitionedInterceptor the throwing else
Buffered + incoming rule takes the Buffered branch envelope.Listener was null — no RetryOnInlineChannelCallback
LatchReceiver through the interceptor the envelope was executed instead of deferred

Each asserts the receiver's real shape first, so none of them can pass vacuously against an unwrapped receiver. The rule-carrying tests also assert the rule was applied (envelope.TenantId), which would fail if dispatch bypassed the wrapper.

Verification

Follow-up, not in this PR

Filed as #4191: ReceiverHasFaulted and the receiver-rebuild path in StartAsync type-test IFaultTrackingReceiver against the raw _receiver too, so for any endpoint with an incoming rule a terminally faulted receiver (#506, CritterWatch#942) is never rebuilt. Same family, same Unwrap() helper — which is why #4191 is blocked on this PR.

🤖 Generated with Claude Code

… on the receiver

ListeningAgent.EnqueueDirectlyAsync type-switched on the raw _receiver field, but
that field is routinely a wrapper. ReceiverWithRules -- installed for any incoming
envelope rule, which includes an endpoint-level MessageType or TenantId -- is
unconditionally an ILocalQueue, so a wrapped NativeAck or Inline receiver matched
the ILocalQueue branch ahead of its own and threw from inside
ReceiverWithRules.EnqueueAsync, re-creating the exact GH-4011 failure on the
durability agent's re-entry path (DLQ replay per GH-1942, scheduled-message
firing). GlobalPartitionedInterceptor is not an ILocalQueue at all, so everything
behind it fell through to the throwing else.

A wrapped BufferedReceiver was the quieter case: it matched the generic
ILocalQueue branch instead of its own, so the replay was enqueued rather than
dispatched through a RetryOnInlineChannelCallback -- and that callback is what
marks the inbox row handled on completion. The message ran; the row was left
behind.

Both wrappers now implement IReceiverWrapper, and Unwrap() peels them off
(they nest) so branch SELECTION sees the receiver that actually executes
messages. Dispatch stays on the outer receiver everywhere it can, so the
wrappers keep applying incoming rules and re-routing globally partitioned
messages; only the ILocalQueue branch enqueues into the unwrapped receiver,
which is what ReceiverWithRules.EnqueueAsync delegated to anyway. Branch order
is unchanged, and a null receiver still reaches the same throw.

LatchReceiver hand-rolled a one-level ReceiverWithRules unwrap for the same
reason (GH-3709) and so never latched anything behind the interceptor; it uses
the shared helper now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Envelope rules or global partitioning hide the receiver from ListeningAgent.EnqueueDirectlyAsync, re-breaking GH-4011

1 participant