Add an API to reduce GC memory usage

[PeterSolMS]

Background and Motivation

In some cases, it's desirable for GC to reduce memory usage at the possible expense of increased GC pause times.

The best tradeoff between memory usage and GC pause time will depend on the scenario, so what is proposed below is a dial where a value of 0 specifies the default behavior, and higher values specify increasingly aggressive measures to keep GC heap size low.

Proposed API

    public static partial class GCOptimizationGoal
    {
        /// <summary>
        /// This property is a dial that tells GC to try to reduce memory usage.
        /// A value of 0 specifies the default behavior.
        /// Values 1 through 9 specify increasingly aggressive behavior to reduce
        /// memory usage, at the possible expense of increased GC pause time.
        /// </summary>
        public static int Memory { get; set; }
    }

Usage Examples

// Example 1:
// Keep heap size fairly low
GCOptimizationGoal.Memory = 7;

// Example 2:
// save setting
int savedMemorySetting = GCOptimizationGoal.Memory;
try
{
    // Relax memory goal for the following to reduce pause times
    GCOptimizationGoal.Memory = 0;
    // do work
}
finally
{
    // restore setting
    GCOptimizationGoal.Memory = savedMemorySetting;
}

-->

Alternative Designs

A plausible alternative would be to add a static property to the GCSettings class instead, for example:

    public static partial class GCSettings
    {
        // ... existing members ...

        /// <summary>
        /// This property is a dial that tells GC to try to reduce memory usage.
        /// A value of 0 specifies the default behavior.
        /// Values 1 through 9 specify increasingly aggressive behavior to reduce
        /// memory usage, at the possible expense of increased GC pause time.
        /// </summary>
        public static int OptimizeMemory { get; set; }
    }

Risks

The risks can be kept fairly low by making sure the default behavior stays the same. As mentioned above, at the higher settings GC pause times will increase.

[ghost]

Tagging subscribers to this area: @dotnet/gc
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Background and Motivation

In some cases, it's desirable for GC to reduce memory usage at the possible expense of increased GC pause times.

The best tradeoff between memory usage and GC pause time will depend on the scenario, so what is proposed below is a dial where a value of 0 specifies the default behavior, and higher values specify increasingly aggressive measures to keep GC heap size low.

Proposed API

    public static partial class GCOptimizationGoal
    {
        /// <summary>
        /// This property is a dial that tells GC to try to reduce memory usage.
        /// A value of 0 specifies the default behavior.
        /// Values 1 through 9 specify increasingly aggressive behavior to reduce
        /// memory usage, at the possible expense of increased GC pause time.
        /// </summary>
        public static int Memory { get; set; }
    }

Usage Examples

// Example 1:
// Keep heap size fairly low
GCOptimizationGoal.Memory = 7;

// Example 2:
// save setting
int savedMemorySetting = GCOptimizationGoal.Memory;
try
{
    // Relax memory goal for the following to reduce pause times
    GCOptimizationGoal.Memory = 0;
    // do work
}
finally
{
    // restore setting
    GCOptimizationGoal.Memory = savedMemorySetting;
}

-->

Alternative Designs

A plausible alternative would be to add a static property to the GCSettings class instead, for example:

    public static partial class GCSettings
    {
        // ... existing members ...

        /// <summary>
        /// This property is a dial that tells GC to try to reduce memory usage.
        /// A value of 0 specifies the default behavior.
        /// Values 1 through 9 specify increasingly aggressive behavior to reduce
        /// memory usage, at the possible expense of increased GC pause time.
        /// </summary>
        public static int OptimizeMemory { get; set; }
    }

Risks

The risks can be kept fairly low by making sure the default behavior stays the same. As mentioned above, at the higher settings GC pause times will increase.

Author:	PeterSolMS
Assignees:	-
Labels:	api-suggestion, area-GC-coreclr, untriaged
Milestone:	-

[YairHalberstadt]

GCOptimizationGoal.Memory isn't a very a clear name - I would in fact assume that higher amounts meant use more memory.

I think an enum would be clearer.

What if you want to go the opposite way? Use as much memory as you want to increase performance? I presume the server GC is tuned for that anyways so there's no need.

I think with 10 levels it's very difficult to decide which one to pick. Maybe 2 modes are all that's really necessary? Optimize space vs Optimize performance?

[Maoni0]

GCOptimizationGoal.Memory isn't a very a clear name - I would in fact assume that higher amounts meant use more memory.
Optimize space vs Optimize performance?

that's interesting, when we talk about memory performance, it generally has 3 aspects, space (or more commonly referred to as memory size) is one of them. the other 2 are throughput/time and pause duration. this is explained here.

when folks talk about optimizing for an aspect, the convention from most folks I've worked with seems to be -

memory -> lower is better
time -> lower is better (which means throughput -> higher is better)
pause -> shorter is better

so when someone talks about "I want to optimize for memory" they mean "I want to use less memory".

interested to hear what other people think of this.

[mangod9]

@richlander since this is related to dotnet/core#5488. Perhaps we can broaden the knobs.

[PeterSolMS]

I think that having a dial is preferable over an enum because it allows for more fine-grained adjustment. Of course, some experimentation and measurement will be required to arrive at a suitable setting for each scenario, but a setting in the middle of the range (say 4 to 6) will be a reasonable starting point in most cases.

I am open to imroving the naming to reduce potential misunderstanding.

[Maoni0]

since there haven't been many comments I think we can take this to API review folks to figure out the naming/convention/etc so I've set it to ready for review.

[richlander]

This is interesting. The problem with these sorts of APIs is ensuring that it fits within an actual functioning app. For example, what's the data that would inform an app that it should ++ the value? If that data rich enough? We should get feedback from teams who run big sites (1P and 3P). I'm happy to help get that feedback.

[BreyerW]

Would it be possible to actually reduce gc pauses at the expense of bigger memory consumption compared to current default? I mean lets say increase allowed level to 11-15 and default level would be 2-5 and that would mean current behaviour while values lower than default would make gc "lazier" than now. Or there wont be any meangiful gains because gc is already as "lazy" as possible?

[jkotas]

A plausible alternative would be to add a static property to the GCSettings class instead

This looks better to me as it is similar to the existing GCSettings.LatencyMode property. We try to avoid having types that have just a single or very few members.

[bartonjs]

Video

• We changed the int 0-9 to a double 0-1
• We moved it onto GCSettings and renamed it.
• Otherwise, looks good.

namespace System.Runtime
{
    partial static class GCSettings
    {
        // NaN represents "there is no goal"
        // Setter throws for things outside the range [0, 1] union NaN
        public static double MemoryOptimizationGoal { get; set; }
    }
}

[davidfowl]

Doesn't seem like this is implemented yet. @PeterSolMS is this still on track of .NET 6?

[madhub]

Any update ?, is it planned for .NET 7
How does it related to Workstation or Server GC setting ? Does it automatically select GC mode based on this setting ?

[huoyaoyuan]

@madhub It needs real implementation in GC. We don't want to expose such API in advance with no actual effect.

How does it related to Workstation or Server GC setting ? Does it automatically select GC mode based on this setting ?

I don't think they should be related. There are more behavior difference between workstation and server GC.

[deeprobin]

Video

* We changed the int 0-9 to a double 0-1

* We moved it onto GCSettings and renamed it.

* Otherwise, looks good.

namespace System.Runtime
{
    partial static class GCSettings
    {
        // NaN represents "there is no goal"
        // Setter throws for things outside the range [0, 1] union NaN
        public static double MemoryOptimizationGoal { get; set; }
    }
}

@bartonjs

Wouldn't a float suffice for this case?

[bartonjs]

Wouldn't a float suffice for this case?

Possibly; but double is the go-to type in .NET for floating point values (just like how we use int even for API only capable of expressing values like 0..10)). System.Single only makes it into API when there's a compelling reason.

I doubt that the GC's backing integer values will ever exceed 24 bits of granularity, so System.Single probably isn't limiting; but it's a bit unnatural, and this API shouldn't really be on any hot path that would warrant the squeeze.

[deeprobin]

Can any of you explain roughly what this change involves on the GC side?

If this is a simple topic for beginners, it would certainly be interesting for me, because I have not had much to do with the GC.