Target Based Scaling for CosmosDB + Separate out CosmosDBScaleMonitor from Listener

[chiangvincent]

Overview

This PR refactors the CosmosDBTriggerListener into two components, CosmosDBScaleMonitor, and CosmosDBMetricsProvider, and introduces new class called the CosmosDBTargetScaler to support target based scaling. The refactor separates CosmosDBScaleMonitor for the redesigned scale controller, which calls into CosmosDBScaleMonitor methods for scaling decisions. This is an effort to unify scaling logic between the scale controller and extensions SDKs, which have diverged over time. The CosmosDBMetricsProvider is shared by the CosmosDBScaleMonitor and CosmosDBTargetScaler classes, which use the provider class to retrieve metrics from the monitored container.

Here is a sample PR for the Service Bus SDK for reference.

Note: this PR includes a fix for the StatusFilePath_ThrowsWhenPathDoesNotExist unit test for timer triggers after upgrading the test csproj to netcore3.1:
To address the following changes in Exception logging, see:

• Github PR
• Github PR Files

TODO

• create CosmosDBScaleMonitor
• create CosmosDBMetricsProvider
• remove duplicate logic between CosmosDBScaleMonitor, CosmosDBMetricsProvider, and CosmosDBTriggerListener
• CosmosDBTriggerListener implement IScaleMonitorProvider and ITargetScalerProvider
• fix unit tests
• add CosmosDBTargetScaler
• add unit tests for CosmosDBTargetScaler
• E2E tests with cosmosdb trigger
• test with scale controller prototype

Logs from E2E Verification With Local Function App

1. Generate a local nuget package with ComosDB changes from this PR
2. Create function app referencing local CosmosDB nuget packages
3. Publish and run the application locally with azure-functions-host repo
4. Enable runtime driven scaling, target based scaling, and add ScmHostingConfigurations.txt to enable CosmosDB TBS
5. Create timer app that sends CosmosDb events every second to a production CosmosDB container
6. Ping admin/host/scale/status API using Postman, observe target worker count and local azure-functions-host logs emitted

Screenshot showing the target scaler requesting a worker count of 2, capped by the partition count which is 1 in this case

[ealsur]

Looks good on the trigger changes

[chiangvincent]

@ealsur are the scale monitors used for Cosmos V3 the same ones here? If not, can you send a pointer to the existing listener + scale monitor we will have to cherry-pick to?