Skip to content

[Event Hubs] Handle cases where desired worker count exceeds int.MaxInt #39468

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JoshLove-msft merged 18 commits into from
Oct 26, 2023
Merged

[Event Hubs] Handle cases where desired worker count exceeds int.MaxInt #39468

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
e5a8756
handle long desired worker count
Oct 24, 2023
a0869c4
adding additional unit test
Oct 24, 2023
6acb4cd
use checked approach
Oct 24, 2023
95f1243
remove unncessary cast
Oct 24, 2023
4c2faca
add unit test
Oct 24, 2023
8237d21
update comment
Oct 24, 2023
c7fb27a
add unit tests
Oct 24, 2023
20811f1
fix merge conflicts
Oct 25, 2023
3d28bbc
Merge branch 'Azure:main' into user/vchiang/handleLongDesiredWorkerCount
chiangvincent Oct 25, 2023
093a1c2
handle long desired worker count
Oct 24, 2023
249e9fa
adding additional unit test
Oct 24, 2023
89262fc
use checked approach
Oct 24, 2023
b1adcb3
remove unncessary cast
Oct 24, 2023
50dc031
add unit test
Oct 24, 2023
49500f4
update comment
Oct 24, 2023
424658d
fix merge conflict
Oct 25, 2023
c23a6d7
Merge branch 'user/vchiang/handleLongDesiredWorkerCount' of https://g…
Oct 25, 2023
e15c310
fix concurrency in unit tests
Oct 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion ...nthub/Microsoft.Azure.WebJobs.Extensions.EventHubs/src/Listeners/EventHubsTargetScaler.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,20 @@ internal static TargetScalerResult ThrottleScaleDownIfNecessaryInternal(TargetSc
internal TargetScalerResult GetScaleResultInternal(TargetScalerContext context, long eventCount, int partitionCount)
{
int desiredConcurrency = GetDesiredConcurrencyInternal(context);
int desiredWorkerCount = (int)Math.Ceiling(eventCount / (decimal)desiredConcurrency);

int desiredWorkerCount;
try
{
checked
{
desiredWorkerCount = (int)Math.Ceiling(eventCount / (decimal)desiredConcurrency);
}
}
catch (OverflowException)
{
desiredWorkerCount = int.MaxValue;
}

int[] sortedValidWorkerCounts = GetSortedValidWorkerCountsForPartitionCount(partitionCount);
int validatedTargetWorkerCount = GetValidWorkerCount(desiredWorkerCount, sortedValidWorkerCounts);

Expand Down
20 changes: 11 additions & 9 deletions ...eventhub/Microsoft.Azure.WebJobs.Extensions.EventHubs/tests/EventHubsTargetScalerTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,16 +76,18 @@ public void ThrottleScaleDownIfNecessaryInternal_ReturnsExpected(int currentTarg
}

[Test]
// Using concurrency of 10.
[TestCase(10, 10, 1)]
[TestCase(20, 10, 2)]
[TestCase(30, 10, 3)]
[TestCase(60, 10, 10)]
[TestCase(70, 10, 10)]
[TestCase(150, 10, 10)]
public void GetScaleResultInternal_ReturnsExpected(long eventCount, int partitionCount, int expectedTargetWorkerCount)
// Using default concurrency of 10.
[TestCase(10, 10, 10, 1)]
[TestCase(20, 10, 10, 2)]
[TestCase(30, 10, 10, 3)]
[TestCase(60, 10, 10, 10)]
[TestCase(70, 10, 10, 10)]
[TestCase(150, 10, 10, 10)]
[TestCase(2147483650, 1, 1, 1)] // Testing eventCount > int.MaxInt is 2147483647
[TestCase(21474836500, 1000, 1, 1000)] // Testing eventCount > int.MaxInt is 2147483647, with concurrency 1 to force overflow
public void GetScaleResultInternal_ReturnsExpected(long eventCount, int partitionCount, int? concurrency, int expectedTargetWorkerCount)
{
TargetScalerResult result = _targetScaler.GetScaleResultInternal(new TargetScalerContext { InstanceConcurrency = 10 }, eventCount, partitionCount);
TargetScalerResult result = _targetScaler.GetScaleResultInternal(new TargetScalerContext { InstanceConcurrency = concurrency }, eventCount, partitionCount);
Assert.AreEqual(expectedTargetWorkerCount, result.TargetWorkerCount);
}

Expand Down