Allow cpu/memory scaler to target a specific container in the pod

[trondhindenes]

Use-Case

One limitation with Kubernetes' default hpa implementation, is that scaling happens based on the "sum" of all containers in the targeted pod. This means that sidecar metrics affect the scaling, and thus sidecar cpu/memory requests/limits have to be very carefully managed in order to achieve the desired scaling dynamics. Especially in cases where sidecar cpu/memory resources are tiny compared to those of the "main" app container, tuning this correctly is often very difficult. The kubernetes metrics-server actually presents per-container metrics, it's just that the default hpa doesnt have a way of selecting which individual container to track metrics for.

Since keda implements its own cpu/memory scaler, it would be so awesome if a new optional field container was added, which allowed using only the metrics of a specific container (ignoring other containers in the pod) for scaling.

[silenceper]

ref:kubernetes/kubernetes#90691
I see that HPA has added the ContainerResourceMetricSource type, which should solve the problem you mentioned.

Maybe we can add this support in subsequent versions, when this feature is released in kubernetes 1.20

[trondhindenes]

Oh - my impression was that Keda implements its own cpu/memory scaler and thus wouldn't be bound by the limitations in Kubernetes (which now seem to be mitigated).

[silenceper]

NO, cpu/memory scaler depend k8s HPA. keda is not implemented separately.

[tomkerkhove]

Jup, we just provide a user-friendly scaler on top of it so that you can use the same experience for CPU/Memory as if it would be another scaler.

[trondhindenes]

makes total sense. Thanks for explaining! I guess we can close this.

[prasobhen]

Getting error on EKS fargate cluster
one or more objects failed to apply, reason: HorizontalPodAutoscaler.autoscaling "xxxxxxxx" is invalid: [spec.metrics[0].containerResource: Required value: must populate information for the given metric source (only allowed when HPAContainerMetrics feature is enabled), spec.metrics[1].containerResource: Required value: must populate information for the given metric source (only allowed when HPAContainerMetrics feature is enabled)]

[prasobhen]

ref:kubernetes/kubernetes#90691 I see that HPA has added the ContainerResourceMetricSource type, which should solve the problem you mentioned.

Maybe we can add this support in subsequent versions, when this feature is released in kubernetes 1.20

Please share the fix for this issue