Bump WolverineFx and WolverineFx.RuntimeCompilation - #56
Open
dependabot[bot] wants to merge 1 commit into
Open
Conversation
Bumps WolverineFx from 6.23.1 to 6.25.5 Bumps WolverineFx.RuntimeCompilation from 6.23.1 to 6.25.5 --- updated-dependencies: - dependency-name: WolverineFx dependency-version: 6.25.5 dependency-type: direct:production update-type: version-update:semver-minor - dependency-name: WolverineFx.RuntimeCompilation dependency-version: 6.25.5 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Updated WolverineFx from 6.23.1 to 6.25.5.
Release notes
Sourced from WolverineFx's releases.
6.25.5
6.25.5 supersedes the never-published 6.25.4 (its tag and release were retired), so the first two fixes below make their first NuGet appearance here.
Fixes
PostgreSQL dead-letter and outgoing counts are exact for small tables (GH-3885)
The PostgreSQL message-store counts for the dead-letter and outgoing tables now report exact numbers for small tables instead of the estimate that could read as zero right after activity. First staged for 6.25.4; this is its first published release.
The durable inbox routes by endpoint for sticky handlers (GH-3886)
Durable inbox recovery now routes each envelope by its owning endpoint, so sticky-handler (
[StickyHandler]/ endpoint-scoped) messages recovered from the inbox execute on the endpoint they were received on rather than falling back to the default route. Also first staged for 6.25.4.Never export empty metrics snapshots + idle-tenant eviction (#3891)
Wolverine no longer exports metrics snapshots that contain no data, and per-tenant metric state for tenants that have gone idle is evicted after a configurable number of cycles via
WolverineOptions.Metrics.TenantIdleEvictionCycles. This is the upstream half of CritterWatch#963 — at very high tenant counts, idle tenants no longer pin memory or pad every export.Agents that exhaust node-local auto-restarts are released to a capable peer (GH-3888, #3896)
When a stalled agent uses up its node-local auto-restart budget, the node now releases the agent so a capable peer can pick it up, instead of retrying forever on the same node. A capability embargo prevents the agent from bouncing straight back to the node that just failed it.
Short-circuiting Before + Finally middleware no longer NREs (GH-3892, #3895)
Middleware that combines a short-circuiting
Beforemethod with aFinallymethod no longer produces aNullReferenceExceptionat codegen time — andFinallynow runs on the short-circuit path, as the middleware contract promises.Saga diagnostics tolerate an unprovisioned saga table (GH-3887, #3894)
DatabaseSagaStoreDiagnostics.ReadSagaAsync/ListSagaInstancesAsynctreat a missing saga table (Postgres 42P01 / SQL Server 208) as null / empty rather than surfacing a raw undefined-table error. A declared-but-never-persisted saga is a legitimate state, sinceAddSagaTypeis optional.Polecat
TransportSchemaNameis honored (GH-3884, #3897)PolecatIntegration.TransportSchemaNameis now actually applied — previously the setting was inert and the transport tables always landed in the default schema.Improvements
The stalled-agent auto-restart path is testable (#3890)
The auto-restart path now runs on
TimeProvider, making it deterministic under test — with coverage added. Thanks @erdtsieck!Message types can be exempted from partitioned processing (GH-3899, #3902)
MessagePartitioning.ExemptFromPartitionedProcessing<T>()exempts a message type from partitioned (GroupId-keyed) processing — exempt types ride the endpoint's normal parallelism while partitioned types keep strict per-group ordering.Batched members'
DeliverByexpiry is enforced again (GH-3898, #3903)Expired members are shed at batch assembly with the normal discard observability, and a whole-batch backstop expires batches whose every member has lapsed.
The sharded execution block deserializes in parallel with ordered emission (GH-3900, #3904)
The sharded execution block's decompress/deserialize stage now runs N-wide while preserving per-group FIFO byte-for-byte.
... (truncated)
6.25.3
A silent data-plane bug for anyone combining Marten with a database-backed transport. Found from a user's minimal reproduction against CritterWatch.
What's Changed
IntegrateWithWolverine()registersMartenIntegrationas anIWolverineExtension, so itsConfigure()runs at host build — after an inlineUsePostgresqlPersistenceAndTransport(..., transportSchema: ...)in the same options lambda. It then stamped its own schema names onto the shared PostgreSQL transport unconditionally, so the integration's defaults silently overwrote whatever the caller asked for.The failure lands on the data plane rather than at startup, which is what makes it expensive to diagnose. A host without Marten honours the configured schema and publishes to
{configured}.wolverine_queue_x; a Marten-backed consumer listens onwolverine_queues.wolverine_queue_x. Auto-provision creates both tables happily, nothing is logged on either side, and no message is ever delivered — the publisher's rows just accumulate in a table nobody polls:TransportSchemaNamenow records whether it was explicitly assigned and is stamped onto the transport only then;MessageStorageSchemaNameis stamped only when non-empty. Both currently-working cases are unchanged — an explicitly-set Marten knob still wins, and a host that configures neither still lands onwolverine_queues.If you have been running Marten alongside
UsePostgresqlPersistenceAndTransportwith a customtransportSchema, check for a duplicatewolverine_queue_*table underwolverine_queues— that is undelivered mail, and it becomes reachable once you upgrade.Known related gap
PolecatIntegration.TransportSchemaNameis declared and documented but never applied — the mirror-image problem (inert rather than over-eager), so an explicit value there is silently ignored. Its siblingMessageStorageSchemaNameis wired correctly. Tracked as #3884, not addressed in this release.Full Changelog: JasperFx/wolverine@V6.25.2...V6.25.3
6.25.2
All related to CritterWatch
What's Changed
GlobalPartitionedMessageTopology.SetExternalTopologyforce-setEndpointMode.Durableon every external slot and companion local queue after the user's configure callback ran, with no way to opt out. For lossy, re-reported traffic — telemetry being the motivating case — that store-and-forwards every envelope through the application's own message store.The mode applies to the external slots and their companion local queues, and is order-independent (it may be set before or after the transport-specific
UseSharded*Queuescall).EndpointMode.Inlineis rejected — partitioned slots depend on the external-listener-to-companion-queue bridge that inline endpoints bypass.Full Changelog: JasperFx/wolverine@V6.25.1...V6.25.2
6.25.1
All related to CritterWatch
What's Changed
Full Changelog: JasperFx/wolverine@V6.25.0...V6.25.1
6.25.0
Couple bugs, one new API meant for CritterWatch
What's Changed
Full Changelog: JasperFx/wolverine@V6.24.10...V6.25.0
6.24.10
Small bug fix release: queue endpoints addressed only by Uri on the database-backed transports (SQL Server, PostgreSQL, SQLite, MySQL) now sanitize the queue name the same way the fluent API does, so a name like
sqlserver://my-service-controlno longer produces invalidwolverine_queue_*table DDL from the dash. This was uncovered by CritterWatch'ssystemControlUriusage in the field.What's Changed
Full Changelog: JasperFx/wolverine@V6.24.9...V6.24.10
6.24.9
This is mostly about CritterWatch uncovered issues with very high volumes of messaging via SQS and making the back pressure detection a bit more sophisticated
What's Changed
Full Changelog: JasperFx/wolverine@V6.24.8...V6.24.9
6.24.8
Bug fix release. Four durability and multi-tenancy fixes, all with regression coverage.
Fixes
#3856 — Dormant inbox rows for a durable local queue were never recovered (#3857)
PublishToPartitionedLocalMessaging()marks every slotListenerScope.Exclusive, and the GH-3590 carve-out then handed inbox recovery to a loop that is never constructed for a local queue — a local queue never gets aListeningAgentat all. Envelopes sat atstatus='Incoming',owner_id=0indefinitely, surviving rolling deploys. Both guards implementing that hand-off now ask a singleEndpoint.IsSingleNodeListenerpredicate, whichLocalQueueanswersfalse. Reported by @erdtsieck.#3815 —
forEveryDatabasevisited the main database twice (#3858)MultiTenantedMessageStore.ActiveDatabases()yieldsMainfirst, so on any multi-tenanted configuration the Oracle, PostgreSQL and MySQL queues counted the main database twice —GetAttributesAsync()reported a queue depth of 2 for a single row. Schema checks and purges also ran twice. SqlServer and Sqlite were already correct.#3859 — MySQL multi-tenanted queues shared one physical table (#3861)
A MySQL schema is a database, so the single
TransportSchemaNameresolved every tenant to the same queue table: no isolation, and counts that multiplied by the tenant count instead of summing. Queue tables now resolve inside each tenant's own database. Single-database hosts are unaffected.#3860 — MySQL database-per-tenant storage had no isolation (#3862)
The same root cause in the message stores: every tenant store received the one configured schema name, so inbox, outbox, dead letter, node and saga tables were shared across all tenants. Each tenant's database is now its own schema.
Upgrading
MySQL database-per-tenant users only. Before this release your tenant envelope rows all lived in the single configured schema. After upgrading, each tenant reads from its own database instead — drain or copy across any in-flight envelopes still sitting in the old shared tables before you upgrade. No other provider or configuration is affected.
Full changelog: JasperFx/wolverine@V6.24.7...V6.24.8
6.24.7
This is a fix release. Its centre of gravity is agent assignment: a leader that re-decided the same placements every cycle, and — hidden underneath that churn — a serial stop path that made every rebalance far slower than it needed to be.
Agent assignment converges much faster
#3852 — the leader re-decided placements it had already made. The GH-3698 pending-assignment ledger armed on a
ReassignAgentbut could never apply one: an agent being moved is still listed in its source node's persistedActiveAgents, so the guard that skips agents with a known original node skipped every reassignment. GH-3698 closed this hole for first-time placement and left it open for moves.On a 512-database / 5-node / ~8,700-agent cluster that reproduced as 3,468 decisions every cycle against a frozen snapshot, indefinitely — matching the ~45,000 decisions over six minutes reported from production. It converged in spite of itself, because the batched command carries set-based value equality and the dispatcher collapses an identical re-emitted batch while its lane is busy, so it read as benign. The telemetry was not deduplicated at all:
AssignmentsChangedfires before batching, so every one of those decisions wrote anAssignmentChangednode record.The churn was concealing a second defect.
StartAgentsgot bounded parallelism back in GH-3604 — a 50-agent chunk started one at a time was seconds of dead wall-clock that blew the reply window. The stop side is the same shape and never got it: a plainforeach, so atAgentStartBatchSize = 50an entire chunk's stop cost ran in series before a single start could cascade. It survived only because the per-cycle churn was trickling agents onto the destination alongside the batch. Fixing the churn exposed it.Measured against the 512-database reproduction:
Net 5.7x faster to converge than 6.24.6, not merely quieter.
#3850 — the cached node-number release is now bounded by a high-water mark, so a newcomer's messages cannot be released by a stale cache. Follow-up to GH-3846.
Node-number lookups happen once per node instead of once per database (#3847, thanks @erdtsieck) — a real saving on multi-database deployments, where the old shape scaled with the shard count.
Durability/projection affinity now reports whether it engaged
#3785 shipped in 6.24.5: a shard database's durability agent follows that database's event-subscription agents, so the database attracts one node's connection pool instead of two.
That join is deliberately fail-silent — a miss falls back to the even spread, because a miss is never wrong, only not-better. The problem is diagnostic: a join that never fires because the two descriptor pipelines spell the same database differently looks exactly like the feature working, minus the benefit. Verifying it meant joining
pg_stat_activityagainst the assignment table on a live cluster.It now says so directly, once, when the numbers change:
and escalates to a warning in the one unambiguous case — projection agents present, database-bearing durability agents present, zero matched. On a multi-database store that is a spelling divergence, not a coincidence. An application with no projections has nothing to follow and stays quiet.
Transport and listener fixes
#3832 — a deliberately paused listener now reports the distinct
ListeningStatus.Pausedinstead of being indistinguishable from back-pressureTooBusy. The contract now matches what the code actually does.#3842 —
RabbitMqListener.CreateAsyncno longer dereferences a nullChannelwhen the agent is disposed mid-startup.Testing and build
:latest, which used to hang silently when Docker ran out of memory.Exception, so dead-lettering by exception type can actually be tested;ErrorCausingMessagenever round-tripped through System.Text.Json.Pack, and the Polecat incident-service sample (whose tests had not compiled since April, with nothing noticing).... (truncated)
6.24.6
A bug-fix release. The headline is a message ordering regression affecting every transport built on
BatchedSender— if you rely on FIFO ordering anywhere, this release matters to you.Highlights
Message ordering restored in
BatchedSender(#3825).BatchedSenderran its serializing stage atEnvironment.ProcessorCount, so envelopes reached the batching block in serialization-completion order rather than enqueue order.This was a silent regression from the switch off TPL Dataflow.
ActionBlockdefaultsMaxDegreeOfParallelismto 1 — ordered by default — and the Channels rewrite raised it without the ordering guarantee being restated anywhere. The block was ordered for years, then quietly wasn't. The practical effect: FIFO ordering was not honored under Azure Service Bus sessions, SQS FIFO message groups, or global partitioning, on every transport that usesBatchedSender. Nothing was lost; messages arrived out of order. Fixed by returning the stage to a degree of parallelism of 1 — everything downstream was already serial.A second, independent defect fell out of the same investigation:
TrackedSession.AllRecordsInOrder()sorted bySessionTime, which isElapsedMilliseconds— whole milliseconds. An entire receive batch ties, and the stable sort then fell back to enumerating a Guid-keyed cache with no relation to real order. Every ordering assertion in the test suite was at the mercy of this. Records now carry a monotonic sequence number.Back-pressure now works on the right number, and says what it is (#3831, jasperfx#632). A latched listener logged exactly one
too busyline and then nothing — forever. An operator watching a queue grow for 40 minutes could not distinguish "still draining" from "wedged". Underneath that, the count aPartitionProcessingByGroupIdendpoint latched and resumed against was wrong: the downstream block holding the backlog was invisible to it, soCountreported zero for work that was really there.BackPressureAgentlogs a periodic warning while a listener stays latched, carrying the queue count and the restart threshold the resume decision is made from.ValueTaskfault, and the listener silently never resumed.BufferedReceiver/DurableReceiverwire the receiving block'sOnErrortoILogger. A terminally-faulted block freezes the queue count and permanently latches the listener; that now logs at Critical instead of vanishing to stderr.A tenanted Azure Service Bus endpoint could not send at all (#3826) — tenanted or untenanted.
TenantedSenderdeliberately does not implementISenderRequiresCallback, but callback registration did not recurse, so aBatchedSenderunderneath it kept a null callback and threwInvalidOperationException: This sender has not been registered.on every batch. The tenanted path now uses inline senders, matching how Redis, MQTT, and Pub/Sub already worked around this.Oracle queue identity round-trip (#3820).
System.Urilowercases the authority component while Oracle uppercases its queue identifiers, soToOracleQueue()resolved a second endpoint over the same physical tables. Also fixes a dead final-attempt error handler: awhenclause that included the loop counter made the descriptive exception at the bottom of the retry loop unreachable.Behavior change worth reading
TrackedSessionnow completes only when all conditions are satisfied, not the first (#3824). This is a public testing API. A tracked session configured with several expectations previously returned as soon as any one of them was met, which means some existing tests were passing vacuously. After upgrading, such a test waits for every condition — and may now fail where it previously passed. That failure is generally revealing a real gap rather than introducing one.Other changes
Countfix above, this carries jasperfx#600/#601 — the application-assembly stack walk could adopt a test-runner assembly and then scan an assembly holding none of your types — and jasperfx#599, whereDatabaseId's escaping now survives aSystem.Uriround trip.EventSubscriptionAgentFamily.DatabaseKeyOfandTenantNeutralKeyOfare now public (#3819).Testing and CI
No runtime behavior changes here, but this is why the fixes above became findable. The
Category=Flakyexclusion list went from 12 tagged classes to zero (#3763) — and several of those tags turned out to have been added in the very commit that introduced the feature they test, hiding working code rather than broken code. Every CI readiness gate now fails loudly instead of warning and continuing; the Kafka gate in particular was a no-op that passed in 0.0s against a broker that would not serve metadata for another 3 seconds (#3814). The retry ledger records why a test flaked rather than only which one (#3787), andCIAzureServiceBuswas sharded three ways on measured per-class durations (#3790).What's Changed
docker compose upso a registry timeout does not redden main by @jeremydmiller in Retrydocker compose upso a registry timeout does not redden main JasperFx/wolverine#3807... (truncated)
6.24.5
Highlights
Multi-database projection & subscription assignment got a major reliability pass. For sharded event stores, Wolverine assigns the agents for projections and subscriptions in groups by database — so connection pools scale with the number of databases rather than nodes × databases:
Durable outbox to SNS/SQS FIFO destinations is fixed (#3793):
EnvelopeSerializernever round-trippedEnvelope.DeduplicationId, so any envelope recovered from durable storage after an outage was re-sent withoutMessageDeduplicationIdand rejected deterministically by a FIFO destination without content-based deduplication — retrying forever or dead-lettering. Also fixed alongside it: the circuit-resume ping could never reach a FIFO destination (a latched sender could never unlatch), and SNS sentMessageDeduplicationIdto standard topics, which AWS rejects. The same fix is merged to the 5.x maintenance branch and will ship in the next 5.40.x release for .NET 8 users.Balanced-mode host shutdown no longer hangs (#3781): stopping a node while agent commands were queued could pay a full agent-batch reply window per queued command — measured at 17+ minutes. Now ~2 minutes on the same reproduction.
Azure Service Bus conventional routing sanitizes entity names (#3786): a handler for an array message type (e.g.
Handle(Foo[])) produced an illegal ASB entity name that broke broker startup for the whole assembly, and the real reason was lost. Names are sanitized and failures now carry the offending name.What's Changed
Full Changelog: JasperFx/wolverine@V6.24.3...V6.24.5
6.24.3
What's Changed
New Contributors
Full Changelog: JasperFx/wolverine@V6.24.2...V6.24.3
6.24.2
What's Changed
Full Changelog: JasperFx/wolverine@V6.24.1...V6.24.2
6.24.1
Patch release. Four reported issues, all with reproductions from production clusters.
Fixes
#3701 —
wolverine_node_recordsgrows without bound (#3734)A reporting cluster reached 36,135,221 rows / 16 GB in five days on a diagnostic table nothing on the hot path reads. Three distinct defects:
INodeAgentPersistence.DeleteOldNodeRecordsAsyncwas implemented for every relational store and never invoked outside tests.NodeEventRecordExpirationTime, 5 days), which is no ceiling at all at high write rates — every one of those 36M rows was inside the window.CS0649suppression on it said so), so a full-table delete went out on every recovery cycle — every 5 seconds by default.New
Durability.NodeRecordRetention(default 10,000 rows) andDurability.NodeRecordPruningPeriod(default hourly).MultiTenantedMessageStorenow delegates the trim to the main store instead of inheriting a no-op default, and Sqlite, MySQL and Oracle gained implementations they had also been missing.#3697 — no supported force-catch-up under Wolverine-managed event subscription distribution (#3735)
Wolverine already implemented the coordinator-driven catch-up path, but only exposed it as a
TrackActivity()stage. Adds the standalone entry point onIHostandIServiceProvider, plus<T>ancillary-store variants:It never calls
IProjectionDaemon.CatchUpAsync— doing so under a live coordinator is what produces theProgressionProgressOutOfOrderExceptionandpk_mt_event_progressionduplicate-key errors suites have been retrying around. Resuming the agents that already own the shards means there is only ever one writer.#3733 — a comma in an agent Uri voided a whole batch confirmation (#3736)
AgentsStarted,StartAgents,AgentsStoppedandStopAgentsjoined theirUri[]on a comma, which RFC 3986 permits unescaped in a path segment. Agent URIs embed tenant ids and projection names, so one comma shattered an agent into fragments — and because the read side built the array in a single projection, the resulting throw took out the confirmation for the entire batch. Newline is the delimiter now, and entries are parsed individually so a bad one names itself.The comma remains the default on the wire for payloads that do not contain one, so rolling upgrades keep working in both directions.
#3706 — RabbitMQ acks were cumulative (#3737)
Every ack went out as
BasicAckAsync(tag, multiple: true), acknowledging every lower delivery tag on the channel. That is only correct when completions happen in delivery order, and they do not withConsumerDispatchConcurrency > 1— acking one message silently acknowledged deliveries whose handlers were still running, and a crash at that moment lost them.Acks are now per message. Two dead-letter paths that relied on the cumulative sweep settle themselves, most importantly the un-mappable-message branch in
WorkerQueueMessageConsumer, which dead-lettered and returned without touching the delivery at all. This unblocks the planned native-ack parallel endpoint mode.Also included
6.24.0
Two data-loss fixes — but for unusual usages
This release closes two bugs that silently destroyed data rather than failing loudly. Both are worth reading before you skip the rest of these notes.
Durable inbox rows were orphaned when a circuit breaker tripped (#3680).
DurableReceiverchecked its latched flag before callingMarkReceived. The latched path still persists each envelope to the inbox as a safety net — but on an envelope that never went throughMarkReceived,Statusis the enum default (Outgoing) andDestinationis null. Both are filter columns for inbox recovery, so the rows were written in a state no recovery sweep on any node could ever see. The nullListeneralso skipped the nack back to the broker, and the broker's redelivery after restart hitDuplicateIncomingEnvelopeException— which acks and drops. Net result: genuine message loss under a durable inbox any time a circuit breaker trip latched the receiver mid-flight. Measured on the circuit-breaker suite, 9 of 1,200 messages were lost per run.Dropping one tenant from a shared partition bucket destroyed its co-tenants' data (#3686). Found alongside #3683. Tenant bucketing — registering several small tenants against one partition suffix so they share a physical partition — is documented and exposed through
PartitionPerTenant(p => p.AllowPartitionSharing = true), and it did not work on either engine. It had no test coverage, because the doc sample demonstrating it is compile-only and never executed.Global partitioning
Part of the GlobalPartitioning epic (#3482).
Wolverine.ComplianceTests.Partitioning.ShardedProcessing, so a new transport costs one small test classThe new suites immediately found two real bugs:
EndpointMode.Durableon every slot, and aNatsEndpointonly supportsDurablewhen JetStream-backed — so everyUseShardedNatsSubjects()call threw at configuration time. The topology now enables JetStream on its own endpoints and declares a work-queue stream per shard, without which the listener died at startup onstream not foundglobal-persistent://public/default/orders1. They now use the topic's short name, matching every other transportMulti-tenancy and persistence
ITenantedentity — had no partition for any tenant registered before that table existed.IConjoinedTenantPartitions<T>.MigrateTenantPartitionsAsync()reconciles every partitioned table against the full registered tenant set, with per-tableTenantPartitionResultreportingTransports
NullReferenceExceptionFilterSubjectwas only assigned whenConsumerNamewas empty, so every durable consumer on a stream received every message. The fix needs aFilterSubjectsmulti-filter — a single filter cannot cover both{subject}and{subject}.scheduled, and a work-queue stream discards an uncovered control message"OAUTH2-JWT". Azure Event Grid's custom JWT authentication requiresCUSTOM-JWT, so those brokers could not be reached through Wolverine's authentication support at all. You could already set the method by hand throughMqttClientOptionsBuilder.WithAuthentication(), but that gave up Wolverine's token refresh loop — the whole reason to useMqttJwtAuthenticationOptions. You no longer have to chooseWolverineHttpTransportClientused the endpoint'sOutboundUripurely as anIHttpClientFactoryclient name, then posted to that client'sBaseAddress— so operator commands sent back over the HTTP transport failed withAn invalid request URI was providedPerformance
RabbitMQ consumer dispatch concurrency is now per-endpoint (#3492). The client default of 1 was the bottleneck. Simulated handler, 2,000 msg/s offered load, 30s measured window:
ConsumerDispatchConcurrencyThe 5.1x and 12.2x multiples understate it — at 1 and 5 the listener never catches up at all.
Amazon SQS batches message deletions and chunks outgoing batches on the 256KB request size limit (#3493)
Azure Service Bus session listeners are no longer quadratic — the n² session loops are now n.
MaxConcurrentCallsis surfaced, and a batched defer settles the original message (#3494)HTTP and gRPC
... (truncated)
Commits viewable in compare view.
Updated WolverineFx.RuntimeCompilation from 6.23.1 to 6.25.5.
Release notes
Sourced from WolverineFx.RuntimeCompilation's releases.
6.25.5
6.25.5 supersedes the never-published 6.25.4 (its tag and release were retired), so the first two fixes below make their first NuGet appearance here.
Fixes
PostgreSQL dead-letter and outgoing counts are exact for small tables (GH-3885)
The PostgreSQL message-store counts for the dead-letter and outgoing tables now report exact numbers for small tables instead of the estimate that could read as zero right after activity. First staged for 6.25.4; this is its first published release.
The durable inbox routes by endpoint for sticky handlers (GH-3886)
Durable inbox recovery now routes each envelope by its owning endpoint, so sticky-handler (
[StickyHandler]/ endpoint-scoped) messages recovered from the inbox execute on the endpoint they were received on rather than falling back to the default route. Also first staged for 6.25.4.Never export empty metrics snapshots + idle-tenant eviction (#3891)
Wolverine no longer exports metrics snapshots that contain no data, and per-tenant metric state for tenants that have gone idle is evicted after a configurable number of cycles via
WolverineOptions.Metrics.TenantIdleEvictionCycles. This is the upstream half of CritterWatch#963 — at very high tenant counts, idle tenants no longer pin memory or pad every export.Agents that exhaust node-local auto-restarts are released to a capable peer (GH-3888, #3896)
When a stalled agent uses up its node-local auto-restart budget, the node now releases the agent so a capable peer can pick it up, instead of retrying forever on the same node. A capability embargo prevents the agent from bouncing straight back to the node that just failed it.
Short-circuiting Before + Finally middleware no longer NREs (GH-3892, #3895)
Middleware that combines a short-circuiting
Beforemethod with aFinallymethod no longer produces aNullReferenceExceptionat codegen time — andFinallynow runs on the short-circuit path, as the middleware contract promises.Saga diagnostics tolerate an unprovisioned saga table (GH-3887, #3894)
DatabaseSagaStoreDiagnostics.ReadSagaAsync/ListSagaInstancesAsynctreat a missing saga table (Postgres 42P01 / SQL Server 208) as null / empty rather than surfacing a raw undefined-table error. A declared-but-never-persisted saga is a legitimate state, sinceAddSagaTypeis optional.Polecat
TransportSchemaNameis honored (GH-3884, #3897)PolecatIntegration.TransportSchemaNameis now actually applied — previously the setting was inert and the transport tables always landed in the default schema.Improvements
The stalled-agent auto-restart path is testable (#3890)
The auto-restart path now runs on
TimeProvider, making it deterministic under test — with coverage added. Thanks @erdtsieck!Message types can be exempted from partitioned processing (GH-3899, #3902)
MessagePartitioning.ExemptFromPartitionedProcessing<T>()exempts a message type from partitioned (GroupId-keyed) processing — exempt types ride the endpoint's normal parallelism while partitioned types keep strict per-group ordering.Batched members'
DeliverByexpiry is enforced again (GH-3898, #3903)Expired members are shed at batch assembly with the normal discard observability, and a whole-batch backstop expires batches whose every member has lapsed.
The sharded execution block deserializes in parallel with ordered emission (GH-3900, #3904)
The sharded execution block's decompress/deserialize stage now runs N-wide while preserving per-group FIFO byte-for-byte.
... (truncated)
6.25.3
A silent data-plane bug for anyone combining Marten with a database-backed transport. Found from a user's minimal reproduction against CritterWatch.
What's Changed
IntegrateWithWolverine()registersMartenIntegrationas anIWolverineExtension, so itsConfigure()runs at host build — after an inlineUsePostgresqlPersistenceAndTransport(..., transportSchema: ...)in the same options lambda. It then stamped its own schema names onto the shared PostgreSQL transport unconditionally, so the integration's defaults silently overwrote whatever the caller asked for.The failure lands on the data plane rather than at startup, which is what makes it expensive to diagnose. A host without Marten honours the configured schema and publishes to
{configured}.wolverine_queue_x; a Marten-backed consumer listens onwolverine_queues.wolverine_queue_x. Auto-provision creates both tables happily, nothing is logged on either side, and no message is ever delivered — the publisher's rows just accumulate in a table nobody polls:TransportSchemaNamenow records whether it was explicitly assigned and is stamped onto the transport only then;MessageStorageSchemaNameis stamped only when non-empty. Both currently-working cases are unchanged — an explicitly-set Marten knob still wins, and a host that configures neither still lands onwolverine_queues.If you have been running Marten alongside
UsePostgresqlPersistenceAndTransportwith a customtransportSchema, check for a duplicatewolverine_queue_*table underwolverine_queues— that is undelivered mail, and it becomes reachable once you upgrade.Known related gap
PolecatIntegration.TransportSchemaNameis declared and documented but never applied — the mirror-image problem (inert rather than over-eager), so an explicit value there is silently ignored. Its siblingMessageStorageSchemaNameis wired correctly. Tracked as #3884, not addressed in this release.Full Changelog: JasperFx/wolverine@V6.25.2...V6.25.3
6.25.2
All related to CritterWatch
What's Changed
GlobalPartitionedMessageTopology.SetExternalTopologyforce-setEndpointMode.Durableon every external slot and companion local queue after the user's configure callback ran, with no way to opt out. For lossy, re-reported traffic — telemetry being the motivating case — that store-and-forwards every envelope through the application's own message store.The mode applies to the external slots and their companion local queues, and is order-independent (it may be set before or after the transport-specific
UseSharded*Queuescall).EndpointMode.Inlineis rejected — partitioned slots depend on the external-listener-to-companion-queue bridge that inline endpoints bypass.Full Changelog: JasperFx/wolverine@V6.25.1...V6.25.2
6.25.1
All related to CritterWatch
What's Changed
Full Changelog: JasperFx/wolverine@V6.25.0...V6.25.1
6.25.0
Couple bugs, one new API meant for CritterWatch
What's Changed
Full Changelog: JasperFx/wolverine@V6.24.10...V6.25.0
6.24.10
Small bug fix release: queue endpoints addressed only by Uri on the database-backed transports (SQL Server, PostgreSQL, SQLite, MySQL) now sanitize the queue name the same way the fluent API does, so a name like
sqlserver://my-service-controlno longer produces invalidwolverine_queue_*table DDL from the dash. This was uncovered by CritterWatch'ssystemControlUriusage in the field.What's Changed
Full Changelog: JasperFx/wolverine@V6.24.9...V6.24.10
6.24.9
This is mostly about CritterWatch uncovered issues with very high volumes of messaging via SQS and making the back pressure detection a bit more sophisticated
What's Changed
Full Changelog: JasperFx/wolverine@V6.24.8...V6.24.9
6.24.8
Bug fix release. Four durability and multi-tenancy fixes, all with regression coverage.
Fixes
#3856 — Dormant inbox rows for a durable local queue were never recovered (#3857)
PublishToPartitionedLocalMessaging()marks every slotListenerScope.Exclusive, and the GH-3590 carve-out then handed inbox recovery to a loop that is never constructed for a local queue — a local queue never gets aListeningAgentat all. Envelopes sat atstatus='Incoming',owner_id=0indefinitely, surviving rolling deploys. Both guards implementing that hand-off now ask a singleEndpoint.IsSingleNodeListenerpredicate, whichLocalQueueanswersfalse. Reported by @erdtsieck.#3815 —
forEveryDatabasevisited the main database twice (#3858)MultiTenantedMessageStore.ActiveDatabases()yieldsMainfirst, so on any multi-tenanted configuration the Oracle, PostgreSQL and MySQL queues counted the main database twice —GetAttributesAsync()reported a queue depth of 2 for a single row. Schema checks and purges also ran twice. SqlServer and Sqlite were already correct.#3859 — MySQL multi-tenanted queues shared one physical table (#3861)
A MySQL schema is a database, so the single
TransportSchemaNameresolved every tenant to the same queue table: no isolation, and counts that multiplied by the tenant count instead of summing. Queue tables now resolve inside each tenant's own database. Single-database hosts are unaffected.#3860 — MySQL database-per-tenant storage had no isolation (#3862)
The same root cause in the message stores: every tenant store received the one configured schema name, so inbox, outbox, dead letter, node and saga tables were shared across all tenants. Each tenant's database is now its own schema.
Upgrading
MySQL database-per-tenant users only. Before this release your tenant envelope rows all lived in the single configured schema. After upgrading, each tenant reads from its own database instead — drain or copy across any in-flight envelopes still sitting in the old shared tables before you upgrade. No other provider or configuration is affected.
Full changelog: JasperFx/wolverine@V6.24.7...V6.24.8
6.24.7
This is a fix release. Its centre of gravity is agent assignment: a leader that re-decided the same placements every cycle, and — hidden underneath that churn — a serial stop path that made every rebalance far slower than it needed to be.
Agent assignment converges much faster
#3852 — the leader re-decided placements it had already made. The GH-3698 pending-assignment ledger armed on a
ReassignAgentbut could never apply one: an agent being moved is still listed in its source node's persistedActiveAgents, so the guard that skips agents with a known original node skipped every reassignment. GH-3698 closed this hole for first-time placement and left it open for moves.On a 512-database / 5-node / ~8,700-agent cluster that reproduced as 3,468 decisions every cycle against a frozen snapshot, indefinitely — matching the ~45,000 decisions over six minutes reported from production. It converged in spite of itself, because the batched command carries set-based value equality and the dispatcher collapses an identical re-emitted batch while its lane is busy, so it read as benign. The telemetry was not deduplicated at all:
AssignmentsChangedfires before batching, so every one of those decisions wrote anAssignmentChangednode record.The churn was concealing a second defect.
StartAgentsgot bounded parallelism back in GH-3604 — a 50-agent chunk started one at a time was seconds of dead wall-clock that blew the reply window. The stop side is the same shape and never got it: a plainforeach, so atAgentStartBatchSize = 50an entire chunk's stop cost ran in series before a single start could cascade. It survived only because the per-cycle churn was trickling agents onto the destination alongside the batch. Fixing the churn exposed it.Measured against the 512-database reproduction:
Net 5.7x faster to converge than 6.24.6, not merely quieter.
#3850 — the cached node-number release is now bounded by a high-water mark, so a newcomer's messages cannot be released by a stale cache. Follow-up to GH-3846.
Node-number lookups happen once per node instead of once per database (#3847, thanks @erdtsieck) — a real saving on multi-database deployments, where the old shape scaled with the shard count.
Durability/projection affinity now reports whether it engaged
#3785 shipped in 6.24.5: a shard database's durability agent follows that database's event-subscription agents, so the database attracts one node's connection pool instead of two.
That join is deliberately fail-silent — a miss falls back to the even spread, because a miss is never wrong, only not-better. The problem is diagnostic: a join that never fires because the two descriptor pipelines spell the same database differently looks exactly like the feature working, minus the benefit. Verifying it meant joining
pg_stat_activityagainst the assignment table on a live cluster.It now says so directly, once, when the numbers change:
and escalates to a warning in the one unambiguous case — projection agents present, database-bearing durability agents present, zero matched. On a multi-database store that is a spelling divergence, not a coincidence. An application with no projections has nothing to follow and stays quiet.
Transport and listener fixes
#3832 — a deliberately paused listener now reports the distinct
ListeningStatus.Pausedinstead of being indistinguishable from back-pressureTooBusy. The contract now matches what the code actually does.#3842 —
RabbitMqListener.CreateAsyncno longer dereferences a nullChannelwhen the agent is disposed mid-startup.Testing and build
:latest, which used to hang silently when Docker ran out of memory.Exception, so dead-lettering by exception type can actually be tested;ErrorCausingMessagenever round-tripped through System.Text.Json.Pack, and the Polecat incident-service sample (whose tests had not compiled since April, with nothing noticing).... (truncated)
6.24.6
A bug-fix release. The headline is a message ordering regression affecting every transport built on
BatchedSender— if you rely on FIFO ordering anywhere, this release matters to you.Highlights
Message ordering restored in
BatchedSender(#3825).BatchedSenderran its serializing stage atEnvironment.ProcessorCount, so envelopes reached the batching block in serialization-completion order rather than enqueue order.This was a silent regression from the switch off TPL Dataflow.
ActionBlockdefaultsMaxDegreeOfParallelismto 1 — ordered by default — and the Channels rewrite raised it without the ordering guarantee being restated anywhere. The block was ordered for years, then quietly wasn't. The practical effect: FIFO ordering was not honored under Azure Service Bus sessions, SQS FIFO message groups, or global partitioning, on every transport that usesBatchedSender. Nothing was lost; messages arrived out of order. Fixed by returning the stage to a degree of parallelism of 1 — everything downstream was already serial.A second, independent defect fell out of the same investigation:
TrackedSession.AllRecordsInOrder()sorted bySessionTime, which isElapsedMilliseconds— whole milliseconds. An entire receive batch ties, and the stable sort then fell back to enumerating a Guid-keyed cache with no relation to real order. Every ordering assertion in the test suite was at the mercy of this. Records now carry a monotonic sequence number.Back-pressure now works on the right number, and says what it is (#3831, jasperfx#632). A latched listener logged exactly one
too busyline and then nothing — forever. An operator watching a queue grow for 40 minutes could not distinguish "still draining" from "wedged". Underneath that, the count aPartitionProcessingByGroupIdendpoint latched and resumed against was wrong: the downstream block holding the backlog was invisible to it, soCountreported zero for work that was really there.BackPressureAgentlogs a periodic warning while a listener stays latched, carrying the queue count and the restart threshold the resume decision is made from.ValueTaskfault, and the listener silently never resumed.BufferedReceiver/DurableReceiverwire the receiving block'sOnErrortoILogger. A terminally-faulted block freezes the queue count and permanently latches the listener; that now logs at Critical instead of vanishing to stderr.A tenanted Azure Service Bus endpoint could not send at all (#3826) — tenanted or untenanted.
TenantedSenderdeliberately does not implementISenderRequiresCallback, but callback registration did not recurse, so aBatchedSenderunderneath it kept a null callback and threwInvalidOperationException: This sender has not been registered.on every batch. The tenanted path now uses inline senders, matching ho...Description has been truncated