Skip to content

Update ContinuablePagedFluxCore Page Retrieving Logic#15929

Merged
alzimmermsft merged 5 commits intoAzure:masterfrom
alzimmermsft:AzCore_InvestigateMemoryLeaks
Oct 5, 2020
Merged

Update ContinuablePagedFluxCore Page Retrieving Logic#15929
alzimmermsft merged 5 commits intoAzure:masterfrom
alzimmermsft:AzCore_InvestigateMemoryLeaks

Conversation

@alzimmermsft
Copy link
Member

This PR updates the page retrieving logic of ContinuablePagedFluxCore.

Previously, it used Reactor's concatWith recursively, this functioned correctly but had the unforeseen side affect of which each page retrieval Reactor would instantiate an internal buffer with length + 1. When dealing with restrictive memory scenarios and a large page count this could result in an OutOfMemoryError.

Now, it uses Reactor's expand functionality, which is loosely akin to a do/while loop that doesn't terminate until the do function returns a completion signal (either an empty Mono/Flux or an exception). This doesn't have the same side affects as the recursive concatWith where it needs to continually increment an internal buffer for each page retrieval, leading to more consistent memory usage with less overall spike and smaller GCs.

@alzimmermsft alzimmermsft added Client This issue points to a problem in the data-plane of the library. Azure.Core azure-core labels Oct 5, 2020
@alzimmermsft alzimmermsft self-assigned this Oct 5, 2020
Copy link
Member

@srnagar srnagar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Azure.Core azure-core Client This issue points to a problem in the data-plane of the library.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants