-
Notifications
You must be signed in to change notification settings - Fork 102
Add slow consumer docs #1073
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Add slow consumer docs #1073
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,82 @@ | ||
| // ReSharper disable SuggestVarOrType_SimpleTypes | ||
| // ReSharper disable SuggestVarOrType_Elsewhere | ||
| #pragma warning disable SA1123 | ||
| #pragma warning disable SA1124 | ||
| #pragma warning disable SA1509 | ||
| #pragma warning disable IDE0007 | ||
| #pragma warning disable IDE0008 | ||
|
|
||
| using System.Threading.Channels; | ||
| using NATS.Client.Core; | ||
|
|
||
| namespace NATS.Net.DocsExamples.Advanced; | ||
|
|
||
| public class SlowConsumerPage | ||
| { | ||
| public async Task Run() | ||
| { | ||
| Console.WriteLine("____________________________________________________________"); | ||
| Console.WriteLine("NATS.Net.DocsExamples.Advanced.SlowConsumerPage"); | ||
|
|
||
| { | ||
| #region message-dropped | ||
| await using NatsConnection nc = new NatsConnection(); | ||
|
|
||
| nc.MessageDropped += (sender, args) => | ||
| { | ||
| Console.WriteLine($"Dropped message on '{args.Subject}' with {args.Pending} pending messages"); | ||
| return default; | ||
| }; | ||
| #endregion | ||
| } | ||
|
|
||
| { | ||
| #region slow-consumer-detected | ||
| await using NatsConnection nc = new NatsConnection(); | ||
|
|
||
| nc.SlowConsumerDetected += (sender, args) => | ||
| { | ||
| Console.WriteLine($"Slow consumer detected on '{args.Subscription.Subject}'"); | ||
| return default; | ||
| }; | ||
| #endregion | ||
| } | ||
|
|
||
| { | ||
| #region tuning | ||
| // Set global default capacity | ||
| NatsOpts opts = new NatsOpts { SubPendingChannelCapacity = 4096 }; | ||
| await using NatsConnection nc = new NatsConnection(opts); | ||
|
|
||
| // Override capacity for a specific subscription | ||
| NatsSubOpts subOpts = new NatsSubOpts | ||
| { | ||
| ChannelOpts = new NatsSubChannelOpts { Capacity = 8192 }, | ||
| }; | ||
|
|
||
| await foreach (NatsMsg<string> msg in nc.SubscribeAsync<string>("events.>", opts: subOpts)) | ||
| { | ||
| Console.WriteLine($"Received: {msg.Data}"); | ||
| } | ||
| #endregion | ||
| } | ||
|
|
||
| { | ||
| #region suppress-warnings | ||
| NatsOpts opts = new NatsOpts | ||
| { | ||
| SuppressSlowConsumerWarnings = true, | ||
| }; | ||
|
|
||
| await using NatsConnection nc = new NatsConnection(opts); | ||
|
|
||
| // Events still fire even when log warnings are suppressed | ||
| nc.SlowConsumerDetected += (sender, args) => | ||
| { | ||
| Console.WriteLine($"Slow consumer on '{args.Subscription.Subject}'"); | ||
| return default; | ||
| }; | ||
| #endregion | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| # Slow Consumers | ||
|
|
||
| When a subscriber can't keep up with the rate of incoming messages, its internal bounded channel fills up. | ||
| By default, new messages are dropped when this happens. This is aligned with NATS at-most-once delivery. | ||
|
|
||
| ## Detecting Dropped Messages | ||
|
|
||
| The `MessageDropped` event fires every time a message is dropped from a subscription's internal channel: | ||
|
|
||
| [!code-csharp[](../../../../tests/NATS.Net.DocsExamples/Advanced/SlowConsumerPage.cs#message-dropped)] | ||
|
|
||
| ## Detecting Slow Consumers | ||
|
|
||
| The `SlowConsumerDetected` event (added in v2.7.2) fires once per slow consumer episode. | ||
| It resets after the subscription catches up (channel drains), so it will fire again if the subscription falls behind a second time: | ||
|
|
||
| [!code-csharp[](../../../../tests/NATS.Net.DocsExamples/Advanced/SlowConsumerPage.cs#slow-consumer-detected)] | ||
|
|
||
| ## Tuning Channel Capacity | ||
|
|
||
| Each subscription has an internal bounded channel with a default capacity of 1024 messages. | ||
| You can tune this globally or per subscription: | ||
|
|
||
| [!code-csharp[](../../../../tests/NATS.Net.DocsExamples/Advanced/SlowConsumerPage.cs#tuning)] | ||
|
|
||
| ## Channel Full Mode | ||
|
|
||
| The `SubPendingChannelFullMode` option controls what happens when the channel is full: | ||
|
|
||
| - `BoundedChannelFullMode.DropNewest` (default for `NatsConnection`) - newest messages are dropped | ||
| - `BoundedChannelFullMode.Wait` (default for `NatsClient`) - backpressure is applied, but the server may disconnect you as a slow consumer | ||
|
|
||
| > [!WARNING] | ||
| > Using `BoundedChannelFullMode.Wait` prevents message loss at the client level, but if the subscriber | ||
| > can't keep up, the NATS server itself may disconnect the client as a slow consumer. | ||
|
|
||
| ## Suppressing Warnings | ||
|
|
||
| By default, a warning is logged once per slow consumer episode. You can suppress these logs while still | ||
| receiving the events: | ||
|
|
||
| [!code-csharp[](../../../../tests/NATS.Net.DocsExamples/Advanced/SlowConsumerPage.cs#suppress-warnings)] | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The documentation states that
BoundedChannelFullMode.DropNewestis the "default forNatsConnection" andBoundedChannelFullMode.Waitis the "default forNatsClient". While technically accurate, this could be misleading becauseNatsClientis a higher-level wrapper that modifies theNatsOpts.SubPendingChannelFullModefrom its base default.Consider clarifying that
BoundedChannelFullMode.DropNewestis the default for the underlyingNatsOpts, whichNatsConnectionuses directly, whileNatsClientexplicitly overrides this to useBoundedChannelFullMode.Wait. This would make it clearer that the difference is due toNatsClientbeing an opinionated wrapper rather than two separate default configurations.