Multiple Azure Log Analytics scalers doesn't work with the same workspace

[trojandg]

Report

When trying to have at least two Azure Log Analytics scalers working for the same workspace/namespace we have an error with unique externalMetricName validation, since name for that is actually made like this: azure-log-analytics- {{workspaceId}}

Expected Behavior

Having option to deploy more then one Azure Log Analytics scaler for specified workspace.

Actual Behavior

While defining more than one scaler for Azure Log Analytics, it throws error about uniqueness violation.

Steps to Reproduce the Problem

1. Create deployment file like that:

apiVersion: v1
kind: Secret
metadata:
  name: kedaloganalytics
  namespace: kedaloganalytics
  labels:
    app: kedaloganalytics
type: Opaque
data:
  tenantId: "QVpVUkVfQURfVEVOQU5UX0lE" #Base64 encoded Azure Active Directory tenant id
  clientId: "U0VSVklDRV9QUklOQ0lQQUxfQ0xJRU5UX0lE" #Base64 encoded Application id from your Azure AD Application/service principal
  clientSecret: "U0VSVklDRV9QUklOQ0lQQUxfUEFTU1dPUkQ=" #Base64 encoded Password from your Azure AD Application/service principal
  workspaceId: "TE9HX0FOQUxZVElDU19XT1JLU1BBQ0VfSUQ=" #Base64 encoded Log Analytics workspace id
---
apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: trigger-auth-kedaloganalytics
  namespace: kedaloganalytics
spec:
  secretTargetRef:
    - parameter: tenantId
      name: kedaloganalytics
      key: tenantId
    - parameter: clientId
      name: kedaloganalytics
      key: clientId
    - parameter: clientSecret
      name: kedaloganalytics
      key: clientSecret
    - parameter: workspaceId
      name: kedaloganalytics
      key: workspaceId
---
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: kedaloganalytics-consumer-scaled-object
  namespace: kedaloganalytics
  labels:
    deploymentName: kedaloganalytics-consumer
spec:
  scaleTargetRef:
    kind: #Optional: Default: Deployment, Available Options: ReplicaSet, Deployment, DaemonSet, StatefulSet
    name: kedaloganalytics-consumer
  pollingInterval: 30
  cooldownPeriod: 30
  minReplicaCount: 1
  maxReplicaCount: 10
  triggers:
  - type: azure-log-analytics
    metadata:
      query: |
        let AppName = "web";
        let ClusterName = "demo-cluster";
        let AvgDuration = ago(10m);
        let ThresholdCoefficient = 0.8;
        Perf
        | where InstanceName contains AppName
        | where InstanceName contains ClusterName
        | where CounterName == "cpuUsageNanoCores"
        | where TimeGenerated > AvgDuration
        | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1)
        | summarize MetricValue=round(avg(CounterValue)) by CounterName, AppName
        | join (Perf 
                | where InstanceName contains AppName
                | where InstanceName contains ClusterName
                | where CounterName == "cpuLimitNanoCores"
                | where TimeGenerated > AvgDuration
                | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1)
                | summarize arg_max(TimeGenerated, *) by AppName, CounterName
                | project Limit = CounterValue, TimeGenerated, CounterPath, AppName)
                on AppName
        | project MetricValue, Threshold = Limit * ThresholdCoefficient
      threshold: "1900000000"
    authenticationRef:
      name: trigger-auth-kedaloganalytics
  - type: azure-log-analytics
    metadata:
      query: |
        let AppName = "web";
        let ClusterName = "demo-cluster";
        let AvgDuration = ago(10m);
        let ThresholdCoefficient = 0.8;
        Perf
        | where InstanceName contains AppName
        | where InstanceName contains ClusterName
        | where CounterName == "memoryWorkingSetBytes"
        | where TimeGenerated > AvgDuration
        | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1)
        | summarize MetricValue=round(avg(CounterValue)) by CounterName, AppName
        | join (Perf 
                | where InstanceName contains AppName
                | where InstanceName contains ClusterName
                | where CounterName == "memoryLimitBytes"
                | where TimeGenerated > AvgDuration
                | extend AppName = substring(InstanceName, indexof((InstanceName), "/", 0, -1, 10) + 1)
                | summarize arg_max(TimeGenerated, *) by AppName, CounterName
                | project Limit = CounterValue, TimeGenerated, CounterPath, AppName)
                on AppName
        | project MetricValue, Threshold = Limit * ThresholdCoefficient
      threshold: "1900000000"
    authenticationRef:
      name: trigger-auth-kedaloganalytics

2. Error while creating ExternalMetric object occurs

Logs from KEDA operator

2021-09-17T11:53:27.574Z	ERROR	controller	Reconciler error	{"reconcilerGroup": "keda.sh", "reconcilerKind": "ScaledObject", "controller": "scaledobject", "name": "kedaloganalytics-consumer-scaled-object", "namespace": "kedaloganalytics", "error": "metricName azure-log-analytics-00000000-0000-0000-0000-000000000000 defined multiple times in ScaledObject kedaloganalytics-consumer-scaled-object, please refer the documentation how to define metricName manually"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:218
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker
	/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:197
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155
k8s.io/apimachinery/pkg/util/wait.BackoffUntil
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156
k8s.io/apimachinery/pkg/util/wait.JitterUntil
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133
k8s.io/apimachinery/pkg/util/wait.Until
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90

KEDA Version

2.3.0

Kubernetes Version

1.20

Platform

Microsoft Azure

Scaler Details

Azure Log Analytics

Anything else?

No response

[JorTurFer]

Hi
As you said, the problem is how we are generating the metric name

Metric: v2beta2.MetricIdentifier{
	Name: kedautil.NormalizeString(fmt.Sprintf("%s-%s", "azure-log-analytics", s.metadata.workspaceID)),
},

Honestly, I don't have any idea about how we could generate it automatically without collisions in this case.
Maybe we can do here like we are doing in other scalers adding the option to set the metric name manually...
WDYT @zroubalik @ahmelsayed @tomkerkhove ?

[zroubalik]

@JorTurFer agree, adding metricName field here seems like the best option how to solve this.

Side note: we should probably refactor the code and solve the metricName generation once and for all.

[JorTurFer]

I will work on this today and I will create the PR ASAP

[tomkerkhove]

We will use a similar approach for all scalers #2123, instead of just this one.