Allow Wolverine to accept the same message to multiple durable listeners in the same application

[jeremydmiller]

Here's the deal -- and it's come up twice recently -- as people start pursuing the "modular monolith" approach, they're effectively trying to run multiple applications in the same process. We've accounted for this through the local routing w/ the "Separated" approach, but this scenario doesn't work with Wolverine yet:

Say you're using the transactional inbox on listening endpoints, which includes a check for idempotency that today only uses the message id

1. An upstream process using Wolverine (or anything else I guess) publishes a message to a Rabbit MQ fanout exchange
2. The receiving Wolverine application has multiple bindings from that exchange to different Rabbit MQ queues that each have a separate "sticky" message handler. In effect, the receiving Wolverine application gets the exact same message twice, just in different listening endpoints
3. Because the fanout is happening in Rabbit MQ, Wolverine thinks that it's getting a duplicate message by the message id when the 2nd listener gets the message and rejects it

Possible approaches:

1. Document that Wolverine doesn't support this and recommend some kind of "splitter" approach that delegates the incoming messages to different "sticky" local queues
2. Try to do something systematic where Wolverine can read the incoming message id, and systematically transform the message id Guid to some other Guid using the endpoint Uri or name or something.
3. Have an opt in mode -- and must be opt in -- where the incoming storage keys off a combination of the message id and the receiver Uri

The first is probably just a bad "solution" and might not really solve anything. The 2nd solution would be the lightest touch and require much less change to the Wolverine internals, but I'm not sure how viable it is yet. The 3rd solution sounds like the most robust approach, provides traceability, but of course, requires the most changes to Wolverine internals and would cause quite a bit of permutation hell. At least we have the storage compliance now that might help.

[jay-zahiri]

I'm working on a modular monolith (Wolverine + Marten + GCP Pubsub), and the 3rd solution would be great

[jeremydmiller]

This is going to suck to do and test, but let's go down the "id + destination" path that's probably the most robust.

Actually, maybe not that bad. Don't think it hurts much to just always use the Destination/REceivedAt in most operations even if it isn't the PK

Tasks

• WolverineOptions.MessageIdentity = IdOnly/IdAndDestination
• Doc that in the new modular monolith guide
• Doc in persistence index page

MessageDatabase

• MarkIncomingEnvelopeAsHandledAsync needs to account for received_at if the flag is true
• WriteLoadScheduledEnvelopeSql too
• _markEnvelopeAsHandledById
• _incrementIncomingEnvelopeAttempts

PostgreSQL

• Add DatabaseConstants.ReceivedAt to the PK for incoming envelopes table
• PostgresqlMessageStore, change _deleteIncomingEnvelopesSql to add extra parameter
• PostgresqlMessageStore, change _reassignIncomingSql to add extra parameter

Sql Server

• Add DatabaseConstants.ReceivedAt to the PK for incoming envelopes table
• SqlServerlMessageStore, change _deleteIncomingEnvelopesSql to add extra parameter
• SqlServerMessageStore, change _reassignIncomingSql to add extra parameter

RavenDb

• Use Envelope.Id + / + Envelope.Destination for the RavenDb PK across the board

[JohanMulticase]

This would be really helpful for us as well, and we would rather wait for a robust solution than having a "hacky" solution faster!