Adding external cosmos db scaler by divyagandhisethi · Pull Request #1 · kedacore/external-scaler-azure-cosmos-db

divyagandhisethi · 2021-03-05T18:17:12Z

Provide a description of what has been changed

Checklist

Commits are signed with Developer Certificate of Origin (DCO)
A PR is opened to update the documentation on our docs repo

Fixes #

divyagandhisethi · 2021-03-05T22:37:21Z

@pragnagopa

tomkerkhove

This is a good start but think we need some more work.

Main topics are:

Please add a CI on GitHub Actions
Removing the deprecated approach and use new instead
Use connection string parser from official SDK, if available
Create a file per class instead

Thanks for the PR!

pragnagopa

Thanks for the PR! Added few comments. Please add unit test project as well.

pragnagopa · 2021-03-08T15:34:45Z

+        public string AccountName { get; internal set; }
+    }
+
+    internal class CosmosDbTriggerMetadata


I think you need to expose these through config via Options - similar to https://github.com/Azure/azure-webjobs-sdk-extensions/blob/dev/src/WebJobs.Extensions.CosmosDB/Config/CosmosDBOptions.cs

ahmelsayed · 2021-03-09T19:55:22Z

regarding @tomkerkhove note

Please add a CI on GitHub Actions

@divyagandhisethi I can help set this up if you're not familiar with github actions.

divyagandhisethi · 2021-03-10T17:14:37Z

regarding @tomkerkhove note

Please add a CI on GitHub Actions

@divyagandhisethi I can help set this up if you're not familiar with github actions.

Thanks. As discussed we will open a tracking issue for adding CI

divyagandhisethi · 2021-03-11T03:10:46Z

@tomkerkhove Can you please take a look?

pragnagopa

Added few more comments. Thanks!

tomkerkhove · 2021-03-12T06:24:41Z

Created an issue for CI - #2

tomkerkhove

A few more things that I'd change to build a solid foundation, but almost there.

Also opened an issue for integration tests: #4

tomkerkhove · 2021-03-12T06:39:06Z

+            CosmosDBTrigger trigger)
+        {
+            CosmosDBConnectionString triggerConnection = new CosmosDBConnectionString(trigger.CosmosDBConnectionString);
+            DocumentCollectionInfo documentCollectionLocation = new DocumentCollectionInfo


We definately should do that

tomkerkhove · 2021-03-12T06:40:51Z

+            return builder;
+        }
+
+        private static string NormalizeString(string inputString)


Let's move it to Extensions/StringExtensions instead, otherwise Utilities will become a dump :D

pragnagopa

Posting partial feedback.

Main feedback in this iteration - move to https://github.com/Azure/azure-cosmos-dotnet-v3

pragnagopa

Added more feedback. Please ping once review is addressed

pragnagopa

Added few more comments. Please open an issue here to add E2E test

pragnagopa

Changes look good. Please open following issues:

Add E2E test in K4Apps
Add readme in this repo - brief description and getting started instructions
If you do not have already, please follow up setting up CI to produce the right artifacts needed - I recommend publishing artifacts with -beta or -preview in the version / name

divyagandhisethi · 2021-05-20T18:46:11Z

Changes look good. Please open following issues:

* Add E2E test in K4Apps

* Add readme in this repo - brief description and getting started instructions

* If you do not have already, please follow up setting up CI to produce the right artifacts needed - I recommend publishing artifacts with   -beta or -preview in the version / name

Issue for readme - #3
Issue for integration tests - #4
For CI artifacts - we can generate the preview image manually until we have the CI setup

divyagandhisethi · 2021-05-20T18:52:17Z

@tomkerkhove This PR is blocked on your review. Please do take a look when possible.

tomkerkhove · 2021-05-25T14:55:37Z

For CI artifacts - we can generate the preview image manually until we have the CI setup

Let's have a GitHub action to build & push an image to GHCR similar to our core. We use a different experimental tag AFAIK

tomkerkhove · 2021-05-25T15:10:57Z

@divyagandhisethi Would you mind fixing DCO please?

tomkerkhove · 2021-06-24T06:35:53Z

What is the status on this PR?

pragnagopa · 2021-06-29T15:31:43Z

@divyagandhisethi - Please merge the PR once you address DCO

pragnagopa · 2021-07-30T15:08:10Z

cc @JatinSanghvi and @SushmithaVReddy

eskaufel · 2021-08-12T06:03:20Z

You are so close... you can do this! 👍🏻

borqosky · 2021-08-19T06:46:10Z

@divyagandhisethi could you just address DCO?
THIS PR is really interesting and as soon you merge it, I'm going to try it 😺

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

divyagandhisethi · 2021-08-19T17:14:22Z

You are so close... you can do this! 👍🏻

Thanks @eskaufel .

divyagandhisethi · 2021-08-19T17:15:54Z

@divyagandhisethi could you just address DCO?
THIS PR is really interesting and as soon you merge it, I'm going to try it 😺

Thanks @borqosky . The commits are now signed. I don't have write permissions on the repo, so can't merge this PR in.

divyagandhisethi · 2021-08-19T17:17:34Z

For CI artifacts - we can generate the preview image manually until we have the CI setup

Let's have a GitHub action to build & push an image to GHCR similar to our core. We use a different experimental tag AFAIK

@pragnagopa @JatinSanghvi @SushmithaVReddy - please follow up on this and publish this image as an experimental scaler.

divyagandhisethi · 2021-08-19T17:20:45Z

@pragnagopa @JatinSanghvi @SushmithaVReddy A PR needs to be open against the docs repo

tomkerkhove · 2021-08-19T17:26:29Z

@pragnagopa @JatinSanghvi @SushmithaVReddy A PR needs to be open against the docs repo

This is only for built-in scalers but feel free to tackle #7 and use documentation in the README!

tomkerkhove · 2021-08-19T17:26:49Z

We are planning to surface external scalers on KEDA.sh based on Artifact Hub

tomkerkhove · 2021-08-19T17:28:54Z

@divyagandhisethi could you just address DCO?
THIS PR is really interesting and as soon you merge it, I'm going to try it 😺

Thanks @borqosky . The commits are now signed. I don't have write permissions on the repo, so can't merge this PR in.

Happy to merge if you want.

divyagandhisethi · 2021-08-19T18:31:27Z

@divyagandhisethi could you just address DCO?
THIS PR is really interesting and as soon you merge it, I'm going to try it 😺

Thanks @borqosky . The commits are now signed. I don't have write permissions on the repo, so can't merge this PR in.

Happy to merge if you want.

Thanks @tomkerkhove . Yes, please go ahead and merge it in.

divyagandhisethi requested review from ahmelsayed, tomkerkhove and zroubalik as code owners March 5, 2021 18:17

ahmelsayed reviewed Mar 5, 2021

View reviewed changes

Comment thread Keda.Cosmosdb.Scaler/src/Services/ExternalScalerService.cs Outdated

ahmelsayed reviewed Mar 5, 2021

View reviewed changes

Comment thread Keda.Cosmosdb.Scaler/src/Services/ExternalScalerService.cs Outdated

ahmelsayed reviewed Mar 5, 2021

View reviewed changes

Comment thread Keda.Cosmosdb.Scaler/src/Services/ExternalScalerService.cs Outdated

tomkerkhove requested changes Mar 8, 2021

View reviewed changes

pragnagopa suggested changes Mar 8, 2021

View reviewed changes

divyagandhisethi force-pushed the main branch from 528054c to ce86198 Compare March 10, 2021 17:21

pragnagopa suggested changes Mar 11, 2021

View reviewed changes

tomkerkhove reviewed Mar 12, 2021

View reviewed changes

tomkerkhove reviewed Apr 9, 2021

View reviewed changes

Comment thread Keda.Cosmosdb.Scaler/src/Startup.cs Outdated

pragnagopa suggested changes Apr 10, 2021

View reviewed changes

pragnagopa mentioned this pull request Apr 14, 2021

Cosmos DB: Migration to V3 SDK Azure/azure-webjobs-sdk-extensions#717

Merged

divyagandhisethi requested a review from tomkerkhove May 6, 2021 16:08

ealsur reviewed May 7, 2021

View reviewed changes

Comment thread Keda.Cosmosdb.Scaler/src/Keda.Cosmosdb.Scaler.csproj Outdated

Comment thread Keda.Cosmosdb.Scaler/src/Repository/ChangeFeedEstimatorFactory.cs Outdated

Comment thread Keda.Cosmosdb.Scaler/src/Repository/CosmosDBRepository.cs Outdated

pragnagopa suggested changes May 8, 2021

View reviewed changes

divyagandhisethi force-pushed the main branch from 003a980 to f70ead1 Compare May 10, 2021 19:25

pragnagopa reviewed May 10, 2021

View reviewed changes

Comment thread Keda.CosmosDB.Scaler/src/Repository/ChangeFeedEstimatorFactory.cs

pragnagopa reviewed May 11, 2021

View reviewed changes

Comment thread Keda.CosmosDB.Scaler/src/Services/CosmosDBLease.cs Outdated

pragnagopa reviewed May 11, 2021

View reviewed changes

Comment thread Keda.CosmosDB.Scaler/src/Services/CosmosDBLease.cs Outdated

pragnagopa reviewed May 11, 2021

View reviewed changes

Comment thread Keda.CosmosDB.Scaler/src/Services/CosmosDBTrigger.cs Outdated

pragnagopa approved these changes May 19, 2021

View reviewed changes

tomkerkhove approved these changes May 25, 2021

View reviewed changes

divyagandhisethi added 10 commits August 19, 2021 10:09

Adding cosmos db scaler

106cb57

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

Fixing metric names

bf3cbaf

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

update2

0c2dcd2

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

update3

eaac7dc

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

adding in memory cache for changefeedprocessorbuilder

baa1469

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

Move to CosmosDB v3 package

cf7ea1b

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

Sum estimatedlag for all leases

d757067

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

Addressing CR Feedback

2da0c4c

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

Creating CosmosDBEstimator

7815c43

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

Update 20

9dd6be8

Signed-off-by: Divya Gandhi <digandhi@microsoft.com>

divyagandhisethi force-pushed the main branch from f01c73b to 9dd6be8 Compare August 19, 2021 17:09

ahmelsayed merged commit bcade15 into kedacore:main Aug 19, 2021

luna8418 mentioned this pull request Sep 11, 2025

Support java SDK #85

Open

1 task

Conversation

divyagandhisethi commented Mar 5, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

Uh oh!

Uh oh!

Uh oh!

divyagandhisethi commented Mar 5, 2021

Uh oh!

tomkerkhove left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pragnagopa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pragnagopa Mar 8, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ahmelsayed commented Mar 9, 2021

Uh oh!

divyagandhisethi commented Mar 10, 2021

Uh oh!

divyagandhisethi commented Mar 11, 2021

Uh oh!

pragnagopa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tomkerkhove commented Mar 12, 2021

Uh oh!

tomkerkhove left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tomkerkhove Mar 12, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tomkerkhove Mar 12, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

pragnagopa left a comment

Choose a reason for hiding this comment

Uh oh!

divyagandhisethi commented Mar 5, 2021 •

edited

Loading

divyagandhisethi commented Aug 19, 2021 •

edited

Loading