Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document the support for different metric types #657

Merged
merged 4 commits into from
Mar 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion content/docs/2.7/concepts/external-scalers.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ The `Scaler` interface defines 4 methods:
- `Close` is called to allow the scaler to clean up connections or other resources.
- `GetMetricSpec` returns the target value for the HPA definition for the scaler. For more details refer to [Implementing `GetMetricSpec`](#5-implementing-getmetricspec).
- `GetMetrics` returns the value of the metric referred to from `GetMetricSpec`. For more details refer to [Implementing `GetMetrics`](#6-implementing-getmetrics).
> Refer to the [HPA docs](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/) for how HPA calculates `replicaCount` based on metric value and target value. KEDA uses the metric target type `AverageValue` for external metrics. This will cause the metric value returned by the external scaler to be divided by the number of replicas.
> Refer to the [HPA docs](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/) for how HPA calculates `replicaCount` based on metric value and target value. KEDA supports both `AverageValue` and `Value` metric target types for external metrics. When `AverageValue` (the default metric type) is used, the metric value returned by the external scaler will be divided by the number of replicas.

The `PushScaler` interface adds a `Run` method. This method receives a push channel (`active`), on which the scaler can send `true` at any time. The purpose of this mechanism is to initiate a scaling operation independently from `pollingInterval`.

Expand Down
29 changes: 24 additions & 5 deletions content/docs/2.7/concepts/scaling-deployments.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,8 +66,6 @@ spec:
# {list of triggers to activate scaling of the target resource}
```

> 💡 **NOTE:** You can find all supported triggers [here](/scalers).

### Details
```yaml
scaleTargetRef:
Expand Down Expand Up @@ -116,7 +114,7 @@ The `cooldownPeriod` only applies after a trigger occurs; when you first create

> 💡 **NOTE:** Due to limitations in HPA controller the only supported value for this property is 0, it will not work correctly otherwise. See this [issue](https://github.com/kedacore/keda/issues/2314) for more details.

If this property is set, KEDA will scale the resource down to this number of replicas. If there's some activity on target triggers KEDA will scale the target resource immediately to `minReplicaCount` and then will be scaling handled by HPA. When there is no activity, the target resource is again scaled down to `idleReplicaCount`. This seting must be less than `minReplicaCount`.
If this property is set, KEDA will scale the resource down to this number of replicas. If there's some activity on target triggers KEDA will scale the target resource immediately to `minReplicaCount` and then will be scaling handled by HPA. When there is no activity, the target resource is again scaled down to `idleReplicaCount`. This setting must be less than `minReplicaCount`.

**Example:** If there's no activity on triggers the target resource is scaled down to `idleReplicaCount` (0), once there is an activity the target resource is immediately scaled to `minReplicaCount` (10) and then up to `maxReplicaCount` (100) as needed. If there's no activity on triggers the resource is again scaled down to `idleReplicaCount` (0).

Expand Down Expand Up @@ -155,10 +153,9 @@ Due to the HPA metric being of type `AverageValue` (see below), this will have t
**Example:** When my instance of prometheus is unavailable 3 consecutive times, KEDA will change the HPA metric such that the deployment will scale to 6 replicas.

There are a few limitations to using a fallback:
- It is **not** supported by the CPU & memory scalers and will assume that fallback is disabled in those cases. This is because it only supports scalers that present their target as an `AverageValue` metric.
- It only supports scalers whose target is an `AverageValue` metric. Thus, it is **not** supported by the CPU & memory scalers, or by scalers whose metric target type is `Value`. In these cases, it will assume that fallback is disabled.
- It is only supported by `ScaledObjects` **not** `ScaledJobs`.


---
#### advanced
```yaml
Expand Down Expand Up @@ -195,6 +192,28 @@ Starting from Kubernetes v1.18 the autoscaling API allows scaling behavior to be

**Assumptions:** KEDA must be running on Kubernetes cluster v1.18+, in order to be able to benefit from this setting.

---
#### triggers
amirschw marked this conversation as resolved.
Show resolved Hide resolved
```yaml
triggers:
# {list of triggers to activate scaling of the target resource}
amirschw marked this conversation as resolved.
Show resolved Hide resolved
```

> 💡 **NOTE:** You can find all supported triggers [here](/scalers).

Trigger fields:
- **type**: The type of trigger to use. (Mandatory)
- **metadata**: The configuration parameters that the trigger requires. (Mandatory)
- **authenticationRef**: A reference to the `TriggerAuthentication` or `ClusterTriggerAuthentication` object that is used to authenticate the scaler with the environment.
- More details can be found [here](./authentication). (Optional)
- **metricType**: The type of metric that should be used. (Values: `AverageValue`, `Value`, `Utilization`, Default: `AverageValue`, Optional)
- Learn more about how the [Horizontal Pod Autoscaler (HPA) calculates `replicaCount`](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/) based on metric type and value.
- To show the differences between the metric types, let's assume we want to scale a deployment with 3 running replicas based on a queue of messages:
- With `AverageValue` metric type, we can control how many messages, on average, each replica will handle. If our metric is the queue size, the threshold is 5 messages, and the current message count in the queue is 20, HPA will scale the deployment to 20 / 5 = 4 replicas, regardless of the current replica count.
- The `Value` metric type, on the other hand, can be used when we don't want to take the average of the given metric across all replicas. For example, with the `Value` type, we can control the average time of messages in the queue. If our metric is average time in the queue, the threshold is 5 milliseconds, and the current average time is 20 milliseconds, HPA will scale the deployment to 3 * 20 / 5 = 12.

> ⚠️ **NOTE:** All scalers, except CPU and Memory, support metric types `AverageValue` and `Value` while CPU and Memory scalers both support `AverageValue` and `Utilization`.

## Long-running executions

One important consideration to make is how this pattern can work with long running executions. Imagine a deployment triggers on a RabbitMQ queue message. Each message takes 3 hours to process. It's possible that if many queue messages arrive, KEDA will help drive scaling out to many replicas - let's say 4. Now the HPA makes a decision to scale down from 4 replicas to 2. There is no way to control which of the 2 replicas get terminated to scale down. That means the HPA may attempt to terminate a replica that is 2.9 hours into processing a 3 hour queue message.
Expand Down
40 changes: 20 additions & 20 deletions content/docs/2.7/operate/events.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,23 @@ weight = 100

KEDA emits the following [Kubernetes Events](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#event-v1-core):

| Event | Type | Description |
|-------------------------------------|-----------|-----------------------------------------------------------------------------------------------------------------------------|
| `ScaledObjectReady` | `Normal` | On the first time a ScaledObject is ready, or if the previous ready condition status of the object was `Unknown` or `False` |
| `ScaledJobReady` | `Normal` | On the first time a ScaledJob is ready, or if the previous ready condition status of the object was `Unknown` or `False` |
| `ScaledObjectCheckFailed` | `Warning` | If the check validation for a ScaledObject fails |
| `ScaledJobCheckFailed` | `Warning` | If the check validation for a ScaledJob fails |
| `ScaledObjectDeleted` | `Normal` | When a ScaledObject is deleted and removed from KEDA watch |
| `ScaledJobDeleted` | `Normal` | When a ScaledJob is deleted and removed from KEDA watch |
| `KEDAScalersStarted` | `Normal` | When Scalers watch loop have started for a ScaledObject or ScaledJob |
| `KEDAScalersStopped` | `Normal` | When Scalers watch loop have stopped for a ScaledObject or a ScaledJob |
| `KEDAScalerFailed` | `Warning` | When a Scaler fails to create or check its event source |
| `KEDAScaleTargetActivated` | `Normal` | When the scale target (Deployment, StatefulSet, etc) of a ScaledObject is scaled to 1 |
| `KEDAScaleTargetDeactivated` | `Normal` | When the scale target (Deployment, StatefulSet, etc) of a ScaledObject is scaled to 0 |
| `KEDAScaleTargetActivationFailed` | `Warning` | When KEDA fails to scale the scale target of a ScaledObject to 1 |
| `KEDAScaleTargetDeactivationFailed` | `Warning` | When KEDA fails to scale the scale target of a ScaledObject to 0 |
| `KEDAJobsCreated` | `Normal` | When KEDA creates jobs for a ScaledJob |
| `TriggerAuthenticationAdded` | `Normal` | When a new TriggerAuthentication is added |
| `TriggerAuthenticationDeleted` | `Normal` | When a TriggerAuthentication is deleted |
| `ClusterTriggerAuthenticationAdded` | `Normal` | When a new ClusterTriggerAuthentication is added |
| `ClusterTriggerAuthenticationDeleted`| `Normal` | When a ClusterTriggerAuthentication is deleted |
| Event | Type | Description |
| ------------------------------------- | --------- | --------------------------------------------------------------------------------------------------------------------------- |
| `ScaledObjectReady` | `Normal` | On the first time a ScaledObject is ready, or if the previous ready condition status of the object was `Unknown` or `False` |
| `ScaledJobReady` | `Normal` | On the first time a ScaledJob is ready, or if the previous ready condition status of the object was `Unknown` or `False` |
| `ScaledObjectCheckFailed` | `Warning` | If the check validation for a ScaledObject fails |
| `ScaledJobCheckFailed` | `Warning` | If the check validation for a ScaledJob fails |
| `ScaledObjectDeleted` | `Normal` | When a ScaledObject is deleted and removed from KEDA watch |
| `ScaledJobDeleted` | `Normal` | When a ScaledJob is deleted and removed from KEDA watch |
| `KEDAScalersStarted` | `Normal` | When Scalers watch loop have started for a ScaledObject or ScaledJob |
| `KEDAScalersStopped` | `Normal` | When Scalers watch loop have stopped for a ScaledObject or a ScaledJob |
| `KEDAScalerFailed` | `Warning` | When a Scaler fails to create or check its event source |
| `KEDAScaleTargetActivated` | `Normal` | When the scale target (Deployment, StatefulSet, etc) of a ScaledObject is scaled to 1 |
| `KEDAScaleTargetDeactivated` | `Normal` | When the scale target (Deployment, StatefulSet, etc) of a ScaledObject is scaled to 0 |
| `KEDAScaleTargetActivationFailed` | `Warning` | When KEDA fails to scale the scale target of a ScaledObject to 1 |
| `KEDAScaleTargetDeactivationFailed` | `Warning` | When KEDA fails to scale the scale target of a ScaledObject to 0 |
| `KEDAJobsCreated` | `Normal` | When KEDA creates jobs for a ScaledJob |
| `TriggerAuthenticationAdded` | `Normal` | When a new TriggerAuthentication is added |
| `TriggerAuthenticationDeleted` | `Normal` | When a TriggerAuthentication is deleted |
| `ClusterTriggerAuthenticationAdded` | `Normal` | When a new ClusterTriggerAuthentication is added |
| `ClusterTriggerAuthenticationDeleted` | `Normal` | When a ClusterTriggerAuthentication is deleted |
2 changes: 1 addition & 1 deletion content/docs/2.7/scalers/cassandra.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ triggers:
- `protocolVersion` - CQL Binary Protocol. (Default: `4`, Optional)
- `keyspace` - The name of the keyspace used in Cassandra.
- `query` - A Cassandra query that should return single numeric value.
- `targetQueryValue` - The threshold value that is provided by the user and used as `targetAverageValue` in the Horizontal Pod Autoscaler (HPA).
- `targetQueryValue` - The threshold value that is provided by the user and used as `targetValue` or `targetAverageValue` (depending on the trigger metric type) in the Horizontal Pod Autoscaler (HPA).
- `metricName` - Name to assign to the metric. (Default: `s<X>-cassandra-<KEYSPACE>`, Optional, In case of `metricName` is specified, it will be used to generate the `metricName` like this: `s<X>-cassandra-<METRICNAME>`, where `<X>` is the index of the trigger in a ScaledObject)

### Authentication Parameters
Expand Down
8 changes: 5 additions & 3 deletions content/docs/2.7/scalers/cpu.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ This specification describes the `cpu` trigger that scales based on cpu metrics.
```yaml
triggers:
- type: cpu
metricType: Utilization/ AverageValue
amirschw marked this conversation as resolved.
Show resolved Hide resolved
metadata:
# Required
type: Utilization/ AverageValue
type: Utilization/ AverageValue # Deprecated in favor of trigger.metricType
value: "60"
```

Expand All @@ -31,6 +31,8 @@ triggers:
- When using `Utilization`, the target value is the average of the resource metric across all relevant pods, represented as a percentage of the requested value of the resource for the pods.
- When using `AverageValue`, the target value is the target value of the average of the metric across all relevant pods (quantity).

> 💡 **NOTE:** The `type` parameter is deprecated in favor of the global `metricType` and will be removed in a future release. Users are advised to use `metricType` instead.

### Example

```yaml
Expand All @@ -44,7 +46,7 @@ spec:
name: my-deployment
triggers:
- type: cpu
metricType: Utilization
metadata:
type: Utilization
value: "50"
```
9 changes: 6 additions & 3 deletions content/docs/2.7/scalers/datadog.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,11 @@ This specification describes the `datadog` trigger that scales based on a Datado
```yaml
triggers:
- type: datadog
metricType: Value
metadata:
query: "sum:trace.redis.command.hits{env:none,service:redis}.as_count()"
queryValue: "7"
type: "global"
type: "global" # Deprecated in favor of trigger.metricType
age: "120"
metricUnavailableValue: "0"
```
Expand All @@ -36,6 +37,8 @@ triggers:
- `age`: The time window (in seconds) to retrieve metrics from Datadog. (Default: `90`, Optional)
- `metricUnavailableValue`: The value of the metric to return to the HPA if Datadog doesn't find a metric value for the specified time window. If not set, an error will be returned to the HPA, which will log a warning. (Optional)

> 💡 **NOTE:** The `type` parameter is deprecated in favor of the global `metricType` and will be removed in a future release. Users are advised to use `metricType` instead.

### Authentication

Datadog requires both an API key and an APP key to retrieve metrics from your account.
Expand Down Expand Up @@ -97,13 +100,13 @@ spec:
name: worker
triggers:
- type: datadog
# Optional: (Value or AverageValue). Whether the target value is global or average per pod. Default: AverageValue
metricType: "Value"
metadata:
# Required: datadog metric query
query: "sum:trace.redis.command.hits{env:none,service:redis}.as_count()"
# Required: according to the number of query result, to scale the TargetRef
queryValue: "7"
# Optional: (Global or Average). Whether the target value is global or average per pod. Default: Average
type: "Global"
# Optional: The time window (in seconds) to retrieve metrics from Datadog. Default: 90
age: "120"
# Optional: The metric value to return to the HPA if a metric value wasn't found for the specified time window
Expand Down
8 changes: 5 additions & 3 deletions content/docs/2.7/scalers/memory.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ This specification describes the `memory` trigger that scales based on memory me
```yaml
triggers:
- type: memory
metricType: Utilization/ AverageValue
metadata:
# Required
type: Utilization/ AverageValue
type: Utilization/ AverageValue # Deprecated in favor of trigger.metricType
value: "60"
```

Expand All @@ -31,6 +31,8 @@ triggers:
- When using `Utilization`, the target value is the average of the resource metric across all relevant pods, represented as a percentage of the requested value of the resource for the pods.
- When using `AverageValue`, the target value is the target value of the average of the metric across all relevant pods (quantity).

> 💡 **NOTE:** The `type` parameter is deprecated in favor of the global `metricType` and will be removed in a future release. Users are advised to use `metricType` instead.

### Example

```yaml
Expand All @@ -44,7 +46,7 @@ spec:
name: my-deployment
triggers:
- type: memory
metricType: Utilization
metadata:
type: Utilization
value: "50"
```
2 changes: 1 addition & 1 deletion content/docs/2.7/scalers/mssql.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ triggers:
The `mssql` trigger always requires the following information:

- `query` - A [T-SQL](https://docs.microsoft.com/sql/t-sql/language-reference) query that returns a single numeric value. This can be a regular query or the name of a stored procedure.
- `targetValue` - A threshold that is used as `targetAverageValue` in the Horizontal Pod Autoscaler (HPA).
- `targetValue` - A threshold that is used as `targetValue` or `targetAverageValue` (depending on the trigger metric type) in the Horizontal Pod Autoscaler (HPA).

To connect to the MSSQL instance, you can provide either:

Expand Down
2 changes: 1 addition & 1 deletion content/docs/2.7/scalers/mysql.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ This specification describes the `mysql` trigger that scales based on result of
The trigger always requires the following information:

- `query` - A MySQL query that should return single numeric value.
- `queryValue` - A threshold that is used as `targetAverageValue` in HPA.
- `queryValue` - A threshold that is used as `targetValue` or `targetAverageValue` (depending on the trigger metric type) in HPA.

To provide information about how to connect to MySQL you can provide:

Expand Down
Loading