Reduce number of gen 2 GC under high memory load

[PeterSolMS]

We observed that under high memory load, we would sometimes get a blocking gen 2 collection right after a background gen 2 collection.

The reason is that the background gen 2 collection finds free spaces which increase gen 2 fragmentation. The next collection sees this higher fragmentation and decides to compact gen 2 via a blocking collection. That in itself is not wrong, but it arguably happens too early.

This PR changes the logic so we only check fragmentation in generation_to_condemn if we are about to do a gen 2 collection anyway, or we have observed growth in gen 2, implying that the fragmentation is no longer useful to fit objects from gen 1.

There is a remaining issue that I have observed - sometimes generation_to_condemn decides to do a blocking collection in order to compact, but then decide_on_compacting decides not to compact because fragmentation isn't high enough. The end result is that we payed the cost of a blocking collection, but don't get the benefit. Arguably this needs fixing - perhaps we should just have generation_to_condemn set the settings.compaction flag and only call decide_on_compacting if the flag is not already set.

[ghost]

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

---

We observed that under high memory load, we would sometimes get a blocking gen 2 collection right after a background gen 2 collection.

The reason is that the background gen 2 collection finds free spaces which increase gen 2 fragmentation. The next collection sees this higher fragmentation and decides to compact gen 2 via a blocking collection. That in itself is not wrong, but it arguably happens too early.

This PR changes the logic so we only check fragmentation in generation_to_condemn if we are about to do a gen 2 collection anyway, or we have observed growth in gen 2, implying that the fragmentation is no longer useful to fit objects from gen 1.

There is a remaining issue that I have observed - sometimes generation_to_collect decides to do a blocking collection in order to compact, but then decide_on_compacting decides not to compact because fragmentation isn't high enough. The end result is that we payed the cost of a blocking collection, but don't get the benefit. Arguably this needs fixing - perhaps we should just have generation_to_condemn set the settings.compaction flag and only call decide_on_compacting if the flag is not already set.

Author:	PeterSolMS
Assignees:	PeterSolMS
Labels:	area-GC-coreclr
Milestone:	-

[Maoni0]

I agree - we should not be doing a blocking GC as sweeping - the cons outweigh the pros in this scenario.

instead of setting settings.compaction to TRUE maybe it's easier for now to add a field to track the compacting decision by generation_to_condemn for this specific situation? something like high_memory_compact_gtc_p? settings.compaction is already initialized to TRUE by default, and we also will be establishing a relationship between generation_to_condemn and decide_on_compacting which we never did before. I'm thinking we could do some refactoring with the 8.0 work since we will undoubtedly touch logic in GTC anyway. what do you think?

[PeterSolMS]

I pushed a change adding an additional flag high_memory_compact_gtc_p as you suggested.

[mangod9]

@Maoni0 is this still required or is it ok to close?

[Maoni0]

since we haven't had time to test this, we can close for now and reopen when we have tested it.

[mangod9]

Closing for now.