From 261698cb9c56bab0bc7b54b938fa097d729a144f Mon Sep 17 00:00:00 2001 From: Richard Lander Date: Thu, 13 Dec 2018 11:05:14 -0500 Subject: [PATCH 1/5] Add proposal for docker limits --- proposed/support-for-docker-limits.md | 63 +++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 proposed/support-for-docker-limits.md diff --git a/proposed/support-for-docker-limits.md b/proposed/support-for-docker-limits.md new file mode 100644 index 000000000..ac206e41c --- /dev/null +++ b/proposed/support-for-docker-limits.md @@ -0,0 +1,63 @@ +# Proposal for .NET Core GC Support for Docker Limits + +.NET Core has support for [control groups](https://en.wikipedia.org/wiki/Cgroups) (cgroups), which is the basis of [Docker limits](https://docs.docker.com/config/containers/resource_constraints/). We found that the algorithm we use to honor cgroups works well for larger memory size limits (for example, >500MB), but that it is not possible to configure a .NET Core application to run indefinitely at lower memory levels. This document proposes an approach to support low memory size limits, <100MB. + +Note: Windows has a concept similar to cgroups called [job objects](https://docs.microsoft.com/windows/desktop/ProcThread/job-objects). .NET Core should honor job objects in the same way as cgroups, as appropriate. This document will focus on cgroups throughout. + +It is critical to provide effective and well defined experiences when .NET Core applications are run within memory-limited cgroups. An application should run indefinitely given a sensible configuration for that application. We considered relying on orchestators to manage failing applications (that can no longer satisfy the configuration of a cgroup), but believe this to be antithetical to building reliable systems. We also expect that there are scenarios where orchestrators will be unavailable or primitive or hardware will be constrainted, and therefore not tolerant of frequently failing applications. As a result, we need a better tuned algorithm for cgroup support to the end of running reliable software within constrained environments. + +## Archetypal Experiences + +We identified two archetypal experiences (with arbitrary metrics provided) that we want to enable: + +* Maximum RPS that can be maintained indefinitely with 64MB of memory allocated +* Minimum memory size required to indefinitely maintain 100 requests per second (RPS) + +These experiences fix a single metric and expect the application to otherwise function indefinitely. They each offer a specific characteristic that we expect will align with a specific kind of workload. + +A cloud hoster might want to fix the memory allocated to an application to improve hosting density and profitability. They will look to our documentation to understand the maximum RPS that they can promise their customers in this configuration, as demonstrated by a sample application like Music Store. + +An IoT developer might want to determine the minimum amount of memory that can be used to maintain a given RPS metric, typically a very low level <100 RPS. + +## GC Configuration + +We will expose the following configuration knobs (final naming TBD) to enable developers to define their own policies, with the following default values (final values TBD). + +* **Native memory budget**: 20MB +* **Minimum GC heap size**: 20MB +* **Maximum GC heap size**: 90% of (**cgroup limit** - **native memory budget**) +* **Safe GC heap size**: 80% of (**cgroup limit** - **native memory budget**) + +Small cgroup example: + +* **cgroup limit**: 60MB +* Defaults for other values +* **Maximum GC heap size**: 36MB +* **Safe GC heap size**: 32MB + +Big cgroup example: + +* **cgroup limit**: 1000MB +* Defaults for other values +* **Maximum GC heap size**: 882MB +* **Safe GC heap size**: 784MB + +Error cases: + +* **cgroup limit** - **native memory budget** - **minimum GC heap size** < 0 +* **maximum GC heap size** < **minimum GC heap size** +* **maximum GC heap size** < **safe GC heap size** + +Note: This means that the minimum cgroup size by default is 40MB. + +Today, the **maximum GC heap size** matches the cgroup limit. We found that the maximum GC heap size needs to be lower than the cgroup limit in order to account for native component memory requirements and to enable the GC to successfully maintain the managed heap at a sustainable level for a process. + +The heap sizes can be specified as a percentage of cgroup limit or as an absolute value. We expect a given application to be run in cgroups of varying sizes, making a percentage value attractive. **native memory budget** represents the amount of native memory that is expected to be used by native components in the process and should be unavailable to the GC to use. Specifying `0` for any of the configuration knobs disables the associated policy from being enforced for that knob. + +The GC will throw an `OutOfMemoryException` when an allocation exceeds the specified **maximum GC heap size**. + +The GC will more aggressive perform GCs after the GC heap grows beyond the **safe GC heap limit** with the goal of returning the heap under that limit. The GC will avoid continuously performing full blocking GCs if they are not considered productive. + +## Conclusion + +We believe that running .NET Core applications in cgroups set to low memory limits is critically important, and that the two archetypal scenarios are descriptive of real-world needs. We would appreciate feedback to determine if we're on the right track, and if we have thought broadly enough about scenarios that should be supported. \ No newline at end of file From 7d5852c7efb1c9c49480fb92304300357442fdbd Mon Sep 17 00:00:00 2001 From: Rich Lander Date: Wed, 27 Feb 2019 14:16:37 -0800 Subject: [PATCH 2/5] Update per design discussions and feedback --- proposed/support-for-docker-limits.md | 71 ++++++++++++--------------- 1 file changed, 32 insertions(+), 39 deletions(-) diff --git a/proposed/support-for-docker-limits.md b/proposed/support-for-docker-limits.md index ac206e41c..bb9f0f5b3 100644 --- a/proposed/support-for-docker-limits.md +++ b/proposed/support-for-docker-limits.md @@ -4,60 +4,53 @@ Note: Windows has a concept similar to cgroups called [job objects](https://docs.microsoft.com/windows/desktop/ProcThread/job-objects). .NET Core should honor job objects in the same way as cgroups, as appropriate. This document will focus on cgroups throughout. -It is critical to provide effective and well defined experiences when .NET Core applications are run within memory-limited cgroups. An application should run indefinitely given a sensible configuration for that application. We considered relying on orchestators to manage failing applications (that can no longer satisfy the configuration of a cgroup), but believe this to be antithetical to building reliable systems. We also expect that there are scenarios where orchestrators will be unavailable or primitive or hardware will be constrainted, and therefore not tolerant of frequently failing applications. As a result, we need a better tuned algorithm for cgroup support to the end of running reliable software within constrained environments. +It is critical to provide effective and well defined experiences when .NET Core applications are run within memory-limited cgroups. An application should run indefinitely given a sensible configuration for that application. We considered relying on orchestators to manage failing applications (that can no longer satisfy the configuration of a cgroup), but believe this to be antithetical as a primary solution for building reliable systems. We also expect that there are scenarios where orchestrators will be unavailable or primitive or hardware will be constrainted, and therefore not tolerant of frequently failing applications. As a result, we need a better tuned algorithm for cgroup support to the end of running reliable software within constrained environments. -## Archetypal Experiences +See [implementing hard limit for GC heap dotnet/coreclr #22180](https://github.com/dotnet/coreclr/pull/22180). -We identified two archetypal experiences (with arbitrary metrics provided) that we want to enable: +## GC Heap Hard Limit -* Maximum RPS that can be maintained indefinitely with 64MB of memory allocated -* Minimum memory size required to indefinitely maintain 100 requests per second (RPS) +The following configuration knobs will be exposed to enable developers to configure their applications: -These experiences fix a single metric and expect the application to otherwise function indefinitely. They each offer a specific characteristic that we expect will align with a specific kind of workload. +* `GCHeapHardLimit` - specifies a hard limit for the GC heap as an absolute value +* `GCHeapHardLimitPercent` - specifies a hard limit for the GC heap as a percentage of physical memory that the process is allowed to use -A cloud hoster might want to fix the memory allocated to an application to improve hosting density and profitability. They will look to our documentation to understand the maximum RPS that they can promise their customers in this configuration, as demonstrated by a sample application like Music Store. +If both are specified, `GCHeapHardLimit` is used. -An IoT developer might want to determine the minimum amount of memory that can be used to maintain a given RPS metric, typically a very low level <100 RPS. +The `GCHeapHardLimit` will be calculated using the following formular if it is not specified and the process is running inside a container (or cgroup or job object) with a memory limit specified: -## GC Configuration +```console +max (20mb, 75% of the memory limit on the container) +``` -We will expose the following configuration knobs (final naming TBD) to enable developers to define their own policies, with the following default values (final values TBD). +The GC will more aggressive perform GCs as the GC heap grows closer to the `GCHeapHardLimit` with the goal of making more memory available so that the application can continue to safely function. The GC will avoid continuously performing full blocking GCs if they are not considered productive. -* **Native memory budget**: 20MB -* **Minimum GC heap size**: 20MB -* **Maximum GC heap size**: 90% of (**cgroup limit** - **native memory budget**) -* **Safe GC heap size**: 80% of (**cgroup limit** - **native memory budget**) +The GC will throw an `OutOfMemoryException` when the committed heap size exceeds the `GCHeapHardLimit` memory size after a full compacting GC. -Small cgroup example: +## GC Heap Heap Minimum Size -* **cgroup limit**: 60MB -* Defaults for other values -* **Maximum GC heap size**: 36MB -* **Safe GC heap size**: 32MB +Using Server GC, there are multiple GC heaps created, up to one per core. This model doesn't scale well when a small memory limit is set on a machine with many cores. -Big cgroup example: +The minimum _reserved_ segment size per heap: 16mb -* **cgroup limit**: 1000MB -* Defaults for other values -* **Maximum GC heap size**: 882MB -* **Safe GC heap size**: 784MB +Example: -Error cases: +* 48 core machine +* cgroup has a 200MB memory limit +* cgroup has no CPU/core limit +* 160MB `GCHeapHardLimit` +* Server GC will create 10 GC heaps +* All 48 cores can be used by the application -* **cgroup limit** - **native memory budget** - **minimum GC heap size** < 0 -* **maximum GC heap size** < **minimum GC heap size** -* **maximum GC heap size** < **safe GC heap size** +Example: -Note: This means that the minimum cgroup size by default is 40MB. +* 48 core machine +* cgroup has a 200MB memory limit +* cgroup has 4 CPU/core limit +* 160MB `GCHeapHardLimit` +* Server GC will create 4 GC heaps +* Only 4 cores can be used by the application -Today, the **maximum GC heap size** matches the cgroup limit. We found that the maximum GC heap size needs to be lower than the cgroup limit in order to account for native component memory requirements and to enable the GC to successfully maintain the managed heap at a sustainable level for a process. +## Previous behavior -The heap sizes can be specified as a percentage of cgroup limit or as an absolute value. We expect a given application to be run in cgroups of varying sizes, making a percentage value attractive. **native memory budget** represents the amount of native memory that is expected to be used by native components in the process and should be unavailable to the GC to use. Specifying `0` for any of the configuration knobs disables the associated policy from being enforced for that knob. - -The GC will throw an `OutOfMemoryException` when an allocation exceeds the specified **maximum GC heap size**. - -The GC will more aggressive perform GCs after the GC heap grows beyond the **safe GC heap limit** with the goal of returning the heap under that limit. The GC will avoid continuously performing full blocking GCs if they are not considered productive. - -## Conclusion - -We believe that running .NET Core applications in cgroups set to low memory limits is critically important, and that the two archetypal scenarios are descriptive of real-world needs. We would appreciate feedback to determine if we're on the right track, and if we have thought broadly enough about scenarios that should be supported. \ No newline at end of file +Previously, the **maximum GC heap size** matched the cgroup limit. We found that the maximum GC heap size needs to be lower than the cgroup limit in order to account for native component memory requirements and to enable the GC to successfully maintain the managed heap at a sustainable level for a process. From b5713efa9c13a839a39d3d61221d10e116397839 Mon Sep 17 00:00:00 2001 From: Rich Lander Date: Mon, 4 Mar 2019 16:42:56 -0800 Subject: [PATCH 3/5] Update support-for-docker-limits.md --- proposed/support-for-docker-limits.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposed/support-for-docker-limits.md b/proposed/support-for-docker-limits.md index bb9f0f5b3..4070a0d7b 100644 --- a/proposed/support-for-docker-limits.md +++ b/proposed/support-for-docker-limits.md @@ -25,7 +25,7 @@ max (20mb, 75% of the memory limit on the container) The GC will more aggressive perform GCs as the GC heap grows closer to the `GCHeapHardLimit` with the goal of making more memory available so that the application can continue to safely function. The GC will avoid continuously performing full blocking GCs if they are not considered productive. -The GC will throw an `OutOfMemoryException` when the committed heap size exceeds the `GCHeapHardLimit` memory size after a full compacting GC. +The GC will throw an `OutOfMemoryException` for allocations that would cause the committed heap size to exceed the `GCHeapHardLimit` memory size, even after a full compacting GC. ## GC Heap Heap Minimum Size From 3e6282e19e56d4865ae1d76d6ea0bf51e23662e5 Mon Sep 17 00:00:00 2001 From: Rich Lander Date: Mon, 4 Mar 2019 18:15:14 -0800 Subject: [PATCH 4/5] Update support-for-docker-limits.md --- proposed/support-for-docker-limits.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposed/support-for-docker-limits.md b/proposed/support-for-docker-limits.md index 4070a0d7b..d1a41731b 100644 --- a/proposed/support-for-docker-limits.md +++ b/proposed/support-for-docker-limits.md @@ -53,4 +53,4 @@ Example: ## Previous behavior -Previously, the **maximum GC heap size** matched the cgroup limit. We found that the maximum GC heap size needs to be lower than the cgroup limit in order to account for native component memory requirements and to enable the GC to successfully maintain the managed heap at a sustainable level for a process. +Previously, the **maximum GC heap size** matched the cgroup limit. From 226b97d2eadae60ba4543c19c6dad86205f2b0c6 Mon Sep 17 00:00:00 2001 From: Richard Lander Date: Wed, 6 Mar 2019 10:12:25 -0800 Subject: [PATCH 5/5] File rename --- .../support-for-memory-limits.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposed/support-for-docker-limits.md => accepted/support-for-memory-limits.md (100%) diff --git a/proposed/support-for-docker-limits.md b/accepted/support-for-memory-limits.md similarity index 100% rename from proposed/support-for-docker-limits.md rename to accepted/support-for-memory-limits.md