Remerge "Using new windowTimeout operator with backpressure support" by conniey · Pull Request #30111 · Azure/azure-sdk-for-java

conniey · 2022-07-24T16:38:15Z

Description

Remerges #29806 into beta release branch.

All SDK Contribution checklist:

The pull request does not introduce [breaking changes]
CHANGELOG is updated for new features, bug fixes or other significant changes.
I have read the contribution guidelines.

General Guidelines and Best Practices

Title of the pull request is clear and informative.
There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

Pull request includes test coverage for the included changes.

…libs defined to use v4.3.19 reactor-core

azure-sdk · 2022-07-24T17:27:38Z

API change check

API changes are not detected in this pull request.

…while iterating array.

JonathanGiles

Looks great - love to see this!

JonathanGiles · 2022-07-24T21:13:57Z

...saging-eventhubs/src/main/java/com/azure/messaging/eventhubs/implementation/ReactorShim.java

+            return MethodHandles.publicLookup().findVirtual(Flux.class, WINDOW_TIMEOUT_OPERATOR,
+                MethodType.methodType(Flux.class, int.class, Duration.class, boolean.class));
+        } catch (IllegalAccessException | NoSuchMethodException err) {
+            LOGGER.verbose("Failed to retrieve MethodHandle for backpressure aware windowTimeout Reactor operator.", err);


Do you think it is worthwhile letting the user who might be reading this log know that it isn't a catastrophic outcome? I.e. by adding something like " - falling back to non-backpressure aware operator instead. It is recommended you consider upgrading your Reactor dependency if possible to at least version X.Y.Z"

I'll second this, there are a few warning messages like this in azure-core that have been flagged as issues

// @lmolkova (in case you missed this comment)

…30111) * Using new windowTimeout operator with backpressure support * Using Flux.class directly for windowTimeout lookup, using azure core libs defined to use v4.3.19 reactor-core * Improving reflection logic and moving it to ReactorShim * Replace inline reflection with ReactorShim * Adding ReactorShim test for backpressure aware windowTimeout. * Changelog update for ReactorShim * Using ConcurrentLinkedQueue to avoid ConcurrentModificationException while iterating array. Co-authored-by: Anu Thomas Chandy <anuamd@hotmail.com>

…zure#30111) * Using new windowTimeout operator with backpressure support * Using Flux.class directly for windowTimeout lookup, using azure core libs defined to use v4.3.19 reactor-core * Improving reflection logic and moving it to ReactorShim * Replace inline reflection with ReactorShim * Adding ReactorShim test for backpressure aware windowTimeout. * Changelog update for ReactorShim * Using ConcurrentLinkedQueue to avoid ConcurrentModificationException while iterating array. Co-authored-by: Anu Thomas Chandy <anuamd@hotmail.com>

* Remerge "Using new windowTimeout operator with backpressure support" (#30111) * Using new windowTimeout operator with backpressure support * Using Flux.class directly for windowTimeout lookup, using azure core libs defined to use v4.3.19 reactor-core * Improving reflection logic and moving it to ReactorShim * Replace inline reflection with ReactorShim * Adding ReactorShim test for backpressure aware windowTimeout. * Changelog update for ReactorShim * Using ConcurrentLinkedQueue to avoid ConcurrentModificationException while iterating array. Co-authored-by: Anu Thomas Chandy <anuamd@hotmail.com> * Merge buffered producer into release/azure-messaging-eventhubs (#30224) * Adding initial Buffered Producer APIs with documentation. (#29525) * Adding skeleton classes. * Adding documentation to buffered producer models. * Adding minor implementation for Event HubBufferedProducerClientBuilder. * Adding some documentation to Event Hub Buffered Producer Client Builder. * Adding suppression for PagedFlux * Adds hashing algorithm for buffered producer (#30012) * Adding implementation of PartitionResolver * Adding tests for PartitionResolver. * Using unsigned right shift operator. * Adding concurrent test cases. * Add CHANGELOG * Adding internal classes to support buffered producer. (#30078) * Updating property name to getMaxEventBufferLengthPerPartition * Adds an aggregator that publishes events based on full batches and time. * Adding implementation for publishing events for a partition. * Adding license info. * Adding inline mock maker for final classes. * Fixing recursive subscription. * Do not get additional batches if the current instance is completed. * Rename to EventHubBufferedPartitionProducer * Applying retryWhen policy to Aggregator. * Remove unused EventData field * Adding documentation to mock helper. * Hack around the blocking call. * Adding tests for EventHubBufferedPartitionProducerTest * Populating PartitionProcessors for each partition id. * Creating private static class to hold PublishResults from buffered producer. * Adding tests to ensure upstream requests are respected. * Add suppression for fallthrough * Use --no-cone in pipeline sparse checkout script (#29905) Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com> * Adding comments for cases meant to fallthrough * Wrap synchronous lock. * Adding documentation to EventDataAggregatora Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com> * Connecting EventHubBufferedProducerClient to internal classes. (#30194) * Adds flush to PartitionProcessor. * Adds implementation to methods for async client. * Fixes build breaks. * Remove idempotent retries for next release. * Connect asynchronous client. * Closing producer client after test case. * Make default retry options package-private. * Adding integration test for BufferedProducerAsyncClient. * Adding partition id to EventDataAggregator * Adding defaults to builder and documentation. * Remove localizedBy usages. * Fixing switchMap logic. * Have PartitionProducer throw an UncheckedInterruptedException when a batch creation occurs while the thread is interrrupted. * Update PublishResultSubscriber to clear remaining queue items when closed. * Delay computation of EventDataAggregator to prevent multiple instances being created. * Adding support for flush(). * Add emitResult constant. * Complete EventDataAggregator when it is cancelled. * Implement flush and clean up logging. * Add implementation for sync-client. * Making non-implemented methods package-private. * Add changelog entry. Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com> * Update CHANGELOG.md * Fix CHANGELOG. Co-authored-by: Anu Thomas Chandy <anuamd@hotmail.com> Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com> Co-authored-by: Ben Broderick Phillips <bebroder@microsoft.com>

conniey requested review from JimSuplizio, ZejiaJiang, alzimmermsft, anuchandy, anushkasingh16, g2vinay, haolingdong-msft, ki1729, liukun-msft, samvaity, srnagar and weidongxu-microsoft July 24, 2022 16:38

ghost added the Event Hubs label Jul 24, 2022

conniey force-pushed the release/azure-messaging-eventhubs_5.13.0-beta.1 branch from 73b7513 to b28366f Compare July 24, 2022 17:04

conniey requested review from JonathanGiles, benbp, hallipr, mssfang and weshaggard as code owners July 24, 2022 17:04

anuchandy added 6 commits July 24, 2022 10:04

Using new windowTimeout operator with backpressure support

c786d83

Using Flux.class directly for windowTimeout lookup, using azure core …

38e49f7

…libs defined to use v4.3.19 reactor-core

Improving reflection logic and moving it to ReactorShim

861a1c4

Replace inline reflection with ReactorShim

429a9c0

Adding ReactorShim test for backpressure aware windowTimeout.

0ee2671

Changelog update for ReactorShim

365ede4

conniey force-pushed the eh-win-timeout2 branch from 815264b to 365ede4 Compare July 24, 2022 17:19

Using ConcurrentLinkedQueue to avoid ConcurrentModificationException …

22ed383

…while iterating array.

conniey merged commit b4b982e into Azure:release/azure-messaging-eventhubs_5.13.0-beta.1 Jul 24, 2022

JonathanGiles reviewed Jul 24, 2022

View reviewed changes

This was referenced Jul 28, 2022

Better log when windowTimeout overload is not found #30174

Merged

Few logging fixes for issues that have fallback #30178

Merged

conniey deleted the eh-win-timeout2 branch November 17, 2022 19:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remerge "Using new windowTimeout operator with backpressure support"#30111

Remerge "Using new windowTimeout operator with backpressure support"#30111
conniey merged 7 commits intoAzure:release/azure-messaging-eventhubs_5.13.0-beta.1from
conniey:eh-win-timeout2

conniey commented Jul 24, 2022 •

edited

Loading

Uh oh!

azure-sdk commented Jul 24, 2022

Uh oh!

JonathanGiles left a comment

Uh oh!

JonathanGiles Jul 24, 2022

Uh oh!

alzimmermsft Jul 25, 2022

Uh oh!

JonathanGiles Jul 25, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

conniey commented Jul 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

All SDK Contribution checklist:

General Guidelines and Best Practices

Testing Guidelines

Uh oh!

azure-sdk commented Jul 24, 2022

Uh oh!

JonathanGiles left a comment

Choose a reason for hiding this comment

Uh oh!

JonathanGiles Jul 24, 2022

Choose a reason for hiding this comment

Uh oh!

alzimmermsft Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

JonathanGiles Jul 25, 2022

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

conniey commented Jul 24, 2022 •

edited

Loading