Improving pre-aggregated Standard Metrics in SDK

[cijothomas]

Background read: https://docs.microsoft.com/en-us/azure/azure-monitor/app/pre-aggregated-metrics-log-metrics

The UI screenshots in this post are from Metric Explorer.

The default configuration of SDK (when used with Web, AspNetCore or WorkerService nugets) adds a special telemetry processor called AutocollectedMetricsExtractor before sampling processors. This processor inspects every Request and Dependency telemetry items, and aggregates them with key dimension. This pre-agg metric is sent to Application Insights once every min, and are available to query under StandardMetrics->Server in metric explorer.

As this metric is calculated before sampling, these metrics are superior in terms of accuracy.
There are 5 standard metrics which are populated from the above.

They are:
Dependency Calls - Count of dep calls.
Dependency duration - Duration of dep calls
Server request - Count of incoming Server request
Server response time - Duration of Server requests
Server request rate - Derived metric from server request count showing requests/sec.

As explained in the docs, these counters are sent with certain key dimensions only. (instead of entire dimensions available in log-based metrics). In UI, these dimensions allow splitting the metric chart, and Alerting allows alerting based on these dimensions.

For Requests, currently the only dimension populated by SDK is 'successful request', even though the UI lists 6 dimensions.

For Dependencies, SDK populates dimensions Dependency type and Sucessful call, though UI lists 6 dimensions.

As SDK pre-aggregation did not add all the dimensions supported in UI, it was not possible to split the charts using these dimensions. (eg: it is not possible to split based on cloudRoleInstance)

We plan to improve the metric experience for the Standard Metrics, by adding more dimension to the SDK pre-aggregators.

This will be done in several steps:
1 Add dimensions which are low/fixed cardinality like CloudRoleName, CloudRoleInstance, Dependency/Request performance, real/synthetic. Users will be allowed to pick a max limit to the number of values for these. (similar to MaxDependencyTypesToDiscover)

2. Add dimensions like OperationName, DependencyName which can potentially be high cardinality. We'll provide option for user to configure this. For example, user could say configure a regex on the OperationName before applying it as a dimension. (these needs to designed)

3. Allow users to add any custom dimensions. (this also need to be designed)

#1 is planned to be addressed soon (in 2.12.0 version of SDK (to be released as stable by end of year, with betas before)
#2, #3 will follow later. (no ETAs)

[cijothomas]

@Dawgfan

[cijothomas]

Dependency.Target and Dependency.ResultCode are additional being enabled in backend, hence SDKs should also add them as dimensions.

[cijothomas]

2.12.0-beta2 shipped with the following pre-aggregated standard metrics:

"Server response time" with dimensions:
"Request.Success"
"cloud/roleInstance"
"cloud/roleName"
"request/resultCode"
"request/performanceBucket"
"operation/synthetic"

“Dependency duration” with dimensions:
"Dependency.Success"
"cloud/roleInstance"
"cloud/roleName"
"Dependency.Type"
"dependency/target"
"operation/synthetic"
"dependency/performanceBucket"

Additional metric are generated from the above by the back end, leading the following full list of metrics:

[cijothomas]

Closing this as the original requirements are met. We'll create new issue to track adding more user-defined dimensions.