Introduce Target Based Scaling by AmeyaRele · Pull Request #598 · Azure/azure-functions-sql-extension

AmeyaRele · 2022-12-21T05:32:31Z

This PR refactors the scaling logic from SqlTriggerListener class to SqlTriggerScaleMonitor and SqlTriggerMetricsProvider class.
It also introduces SqlTriggerTargetScaler for target based scaling logic.

Reference PRs:
Azure/azure-sdk-for-net#31367
Azure/azure-webjobs-sdk-extensions#806

ToDo

Add unit tests for SqlTriggerTargetScaler
Test with scale controller 3.x prototype

Local functions host with target based scaling enabled

…SqlTriggerScaleMonitor

nuget.config

…SqlTriggerUtils

Charles-Gagnon

Couple small comments but overall looks good once the package is updated.

src/TriggerBinding/SqlTriggerMetricsProvider.cs

src/TriggerBinding/SqlTriggerScaleMonitor.cs

src/TriggerBinding/SqlTriggerTargetScaler.cs

Co-authored-by: Charles Gagnon <chgagnon@microsoft.com>

…zure/azure-functions-sql-extension into ameyarele/target-based-scaling

Directory.Packages.props

chiangvincent

Thanks for the PR! Left minor comments and clarifications -- please also verify these changes with a locally running functions host, and paste screenshots of Target Based Scaling and Incremental Scaling logs from console. See here for reference.

src/TriggerBinding/SqlTriggerScaleMonitor.cs

src/TriggerBinding/SqlTriggerMetrics.cs

src/TriggerBinding/SqlTriggerScaleMonitor.cs

src/TriggerBinding/SqlTriggerTargetScaler.cs

src/TriggerBinding/SqlTriggerListener.cs

alrod · 2023-02-15T22:19:41Z

src/TriggerBinding/SqlTriggerMetricsProvider.cs

+
+                using (var connection = new SqlConnection(this._connectionString))
+                {
+                    this._logger.LogDebugWithThreadId("BEGIN OpenGetUnprocessedChangesConnection");


Where does the log go, kusto FunctionLogs table/customers appinsights? Do we want to log it on every call?

This is logging at the debug level so I would only expect it to be logged if the log level is set to that by the customer. And information like this is useful for helping track down issues (for example if the connection hangs on opening this would help indicate that).

yes, the AppInsights logs will no be published. I am concerning about kusto FunctionLogs table as we log the all levels there. I am wondering about the amount of the logs as the checks will be executed periodically. We can log if an operation time was exceeded some expected threshold or an exception was occurred.

There lot of "LogDebugWithThreadId" introduced in the PR. Can you try to deploy a function app + new SQL extension with TBS + RDS enabled and see how much new logs are generated in FunctionLogs kusto table.

src/TriggerBinding/SqlTriggerScaleMonitor.cs

src/TriggerBinding/SqlTriggerMetricsProvider.cs

src/TriggerBinding/SqlTriggerScaleMonitor.cs

src/TriggerBinding/SqlTriggerTargetScaler.cs

src/TriggerBinding/SqlTriggerMetricsProvider.cs

Co-authored-by: Charles Gagnon <chgagnon@microsoft.com>

Charles-Gagnon · 2023-03-08T17:18:44Z

@AmeyaRele - Please take a look at the open comments

Co-authored-by: Charles Gagnon <chgagnon@microsoft.com>

Charles-Gagnon · 2023-03-09T22:11:10Z

@AmeyaRele Please provide screenshots/logs as requested by @chiangvincent #598 (review)

AmeyaRele · 2023-03-13T11:34:37Z

Having some issues with running it on local functions host, syncing up with Vincent offline to fix them.

Charles-Gagnon · 2023-03-15T17:30:14Z

@AmeyaRele FYI - in preparation for input/output binding GA we've switched trigger stuff back to a separate branch (release/trigger). I've switched this to target that new branch instead so nothing should change, just letting you know in case you need to make any more changes.

AmeyaRele · 2023-03-16T11:36:51Z

@chiangvincent Please do take a look at the PR and the attached screenshot. I have addressed all your comments from the last review

docs/BindingsOverview.md

src/TriggerBinding/SqlTriggerTargetScaler.cs

chiangvincent

Looks good, thanks for the PR! Approving, and please also get approval from @alrod

alrod

My only concern is about debug logging.
#598 (comment)

TBS related stuff looks good.

Charles-Gagnon · 2023-04-06T21:43:50Z

@AmeyaRele - Chatted with Alexey offline about the logging and we're going to investigate and address that separately. So you're good to merge this in whenever you're ready to and we'll take care of following up about the logging stuff.

* Revert "Remove trigger binding for GA release (#732)" (#733) This reverts commit 9713259. * vBump release/trigger 2.* -> 3.* (#738) * Add JObject support for SQL trigger (#722) * add support for jobject and js, ps, python samples * use utils JsonSerializeObject + comment * Update src/TriggerBinding/SqlTriggerBindingProvider.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * fix unit test + pylint --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Fix trigger connection not being retried when connection is closed (#731) * Add Privacy Statement to README and telemetry message (#751) * Add link to privacy statement * Add privacy link to output message * add to output message * Fix typos (cherry picked from commit a9565f7) * enable trigger oop (#750) * add sqltrigger attribute * add SqlChange type * enable oop test * add check * fix the null reference * use GetLogger * Enable debug logging for Python and PS samples (#754) * enable debug logging for ps and python samples * enable debug logs for js samples * Add docs & logging information for Leases table info (#756) * Add ProductsTriggerWithValidation tests for Python, JS, PS (#753) * add js, py, ps ProductsTriggerWithValidation test * fix test + missing file * Further clarifications to trigger retry docs (#770) * Further clarifications to trigger retry docs * Update description * increase timeout to 120 minutes (#776) * Enable more tests for OOP trigger (#777) * add more oop test samples * update logger * bump jackson dependency (#782) * Add detailed trigger docs (#780) * Filter out more default telemetry properties (cherry picked from commit d6ea6d7) * bring back trigger text * Fixes * Add SQLTrigger annotation to java library (#783) * add SQLTrigger annotation to java library * remove default values * use enum for commandtype * Disable CSX trigger tests * fix regex for java library version (#788) * Adding rest of Trigger integration tests for Python, PS, JS (#763) * add js, py, ps ProductsTriggerWithValidation test * fix test + missing file * add tests * remove extra comments + powershell compress * update comment * Skip more trigger tests for CSX (#791) * Introduce Target Based Scaling (#598) * Refactor SqlTriggerListener scaling to SqlTriggerMetricsProvider and SqlTriggerScaleMonitor * Create SqlTriggerTargetScaler * Refactor unit tests * Refactor to include common queries for scaling and listener class to SqlTriggerUtils * Add doc comments for scaling classes * Update src/TriggerBinding/SqlTriggerTargetScaler.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Fix log statement * Update WebJobs package * Update nuget.config * Address review comments * Address review comments pt2 * Update src/TriggerBinding/SqlTriggerTargetScaler.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Address comments, test failures * Fix packages lock file * Fix error message * Address comments and test failures * Apply suggestions from code review Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Change in documentation * Fix log level --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Enable trace level logging for tests (#795) * Enable trace level logging for tests * one more * Add java trigger samples (#793) * add java trigger samples * update test-java * add SqlChangeProduct type * add rest of tests * fix polling interval test * Fix up performance tests & enable schedule (#799) * Minor trigger doc updates (#800) * Minor trigger doc updates * fix * enable rest of the trigger tests for oop (#798) * enable all trigger tests for OOP * undo * fix test * merge conflict * remove Java * add sample for python v2 model (#803) * Fix logs * fix compile * remove delegate (#809) Signed-off-by: Maddy Koripalli <makoripa@microsoft.com> * add csx trigger samples (#806) * add trigger samples for csx * enable csx tests * add copy trigger sample * use utils.josnserialize * separate utils file * reuse utils from sql extension * remove deleted references * Cleanup trigger logging (#808) * Cleanup trigger logging * more * fix compile * Default log level of debug * Fix scale monitor tests * [Trigger] Update from main (#820) * update java library 0.1.1 -> 1.0.0 (#804) * Pin Node version to 18.15 (#818) * Pin Node version to 18.15 * Update build pipeline * Set node modules path for all * Debug * Pass in env var * Update pipeline comments (#819) --------- Co-authored-by: Lucy Zhang <luczhan@microsoft.com> * [Trigger] Rename csharpscript and align host.json (#825) * Merge branch 'main' into release/trigger # Conflicts: # docs/SetupGuide_DotnetCSharpScript.md # samples/samples-csharp/host.json # test/Microsoft.Azure.WebJobs.Extensions.Sql.Tests.csproj * Move triggers * Fix java version in setup guide (#821) * Refactor integration tests (#813) * create database once * fix compat level test * fix compat + logs * start func host in the beginning * fix change tracking * fix tests * fix tests * add CSharpscript and fix change tracking * set port * update xunit analyzers & temp skip failling tests * update port from fixture * pass console writeline as logger * enable identity tests * fix case sensitive tests + temp disable identity * fix indent * AddProductWithIdentityColumnIncluded fix * AddProductWithIdentityColumnIncluded * cleanup * remove extra fixture * default port * fix csx * Clean csx samples (#812) * sql attribute not needed * simplify the folder structure for csx * add local.settings.json * rename folders in proj file * clean up logs and unused * remove TriggerSamples folder * update trigger docs for csx * Fix compilation error (#835) * Add retry for Trigger tests (#836) * add retry logic * fix the csx compilation error * use xRetry * replace with RetryTheory * add diagnostic messages to logs * Update Trigger Binding set up in SetupGuide_DotnetOutOfProc.md (#845) * Update SetupGuide_DotnetOutOfProc.md * fixes * Update Trigger Binding set up in SetupGuide_Java.md (#846) * Update SetupGuide_Java.md * fix annotation * Update SetupGuide_Javascript.md (#847) * Update SetupGuide_Javascript.md * fix toc * FIx perf pipeline (#838) * dont start func host in fixture for perf * don't use fixture in perf test * finally block * Update SetupGuide_PowerShell.md (#848) * Update SetupGuide_Python.md (#849) * enable test TriggerWithException (#801) * enable the TriggerWithException test * update comment * remove logger delegate * fix error * add retry * remove Java test and try --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Add additional check for closed/broken connection errors (#862) * add timestamp to global state table for lease table cleanup (#861) * add LastAccessTime column to GlobalState * update docs with clean up scripts * update createtable * add comments * refine comments * fix script error * add print * add test to check LastAccessTime column creation for existing triggers (#865) * add LastAccessTime column to GlobalState * update docs with clean up scripts * update createtable * add comments * refine comments * fix script error * add print * add test * fix test error * use constants * Merge latest from main into trigger (#866) * Fix the race condition by adding the extension reference to build first. (#839) * fix the correct path separator * add the extension ref to avoid race condition * format files * pack and copy the sql nupkg to local-packages after build (#853) * copy to local packages before build * AfterTargets since package is created after build * make sure pack runs after build * address comments * Remove doc/sample references to preview bundle (#744) * Remove doc/sample references to preview bundle * Few more * undo * Powershell -> PowerShell * fix version * use scriptdom nuget (#864) * Revert "Remove doc/sample references to preview bundle (#744)" This reverts commit d7cd728. * Fix nuget package versioning --------- Co-authored-by: Maddy <12754347+MaddyDev@users.noreply.github.com> Co-authored-by: Lucy Zhang <luczhan@microsoft.com> * Revert "Remove doc/sample references to preview bundle (#744)" This reverts commit d7cd728. * add troubleshooting information to the overview doc (#871) * add troubleshooting section * update format * Update docs/BindingsOverview.md Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Update docs/BindingsOverview.md Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * clarifying view change tracking command (#873) * Add sql trigger test for different data types (#876) * add productcolumntypestriggertest * try running only csharp * try running js * comment out column values check temporarily * skip date and byte checks * fix build error * skip byte check * remove date/time, add back binary * fix build error * remove only binary check * remove only nchar and nvarchar * check equality of nchar and nvarchar * fix tests * change nchar to test * Bump Version of Grpc.Net.Client (#877) * Bump version of Grpc.Net.Client * Bump grpc.net.client to latest * Update packages.lock * Add to Directory.Packages.props * Fix code coverage in trigger branch (#880) * run only csharp test task * comment out trigger tests * don't start func hosts in the beginning * start three functions in beginning * start only csharp host * fix build error * start both csharp and java hosts * start only csharp and oop hosts * add logging for disposefunctionhosts * start only csharp host * revert commented out code * add comment * Fix schema parse error when using reserved keys as table names (#881) * bracketed name for reserved words as table name * add test * add tests * fix csx test * enable test only for csharp * fix TableNotPresentTest * revert GetUserTableIdAsync change * refactor GetUserTableIdAsync * comment out Java test * refactor code to use SqlObject * Add LeasesTableName to SqlTriggerAttribute (#893) * addLeasesTableNameSetting to SqlTriggerAttribute * add provider test * add samples * remove setting * add new constructor * fix metrics provider * add integration test * fix oop * fix test * cleanup + pr comments * quote escape leasestablename * Add leasesTableName to SqlTrigger Java annotation (#897) * add leasestablename to java sqltrigger * add default value for commandtype * remove sample * update java library to 2.1.0-preview (#899) * Update docs with LeasesTableName (#898) * add leasestablename to docs * fix links * GetLeaseLockedRows for debug logging (#900) * add isLeaseLocked into the getchanges query * add log message * add debug logging for row counts * update log msg * create a separate query for getting the locked rows * update ver name * update * remove user table join * unused var * correct xml info * address comments * remove extra line * dont throw * Update src/TriggerBinding/SqlTableChangeMonitor.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * rename leasesTableName to bracketedLeasesTableName (#902) * rename leasesTableName to bracketedLeasesTableName * merge error * [Trigger] Sync with main (#906) * Bump Version of Grpc.Net.Client (#877) * Bump version of Grpc.Net.Client * Bump grpc.net.client to latest * Update packages.lock * Add to Directory.Packages.props * Update GetProducts.js (#884) Corrected input model so it works * Bump word-wrap from 1.2.3 to 1.2.4 in /samples/samples-js (#885) Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4. - [Release notes](https://github.com/jonschlinkert/word-wrap/releases) - [Commits](jonschlinkert/word-wrap@1.2.3...1.2.4) --- updated-dependencies: - dependency-name: word-wrap dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump word-wrap from 1.2.3 to 1.2.4 in /test/Integration/test-js (#886) Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4. - [Release notes](https://github.com/jonschlinkert/word-wrap/releases) - [Commits](jonschlinkert/word-wrap@1.2.3...1.2.4) --- updated-dependencies: - dependency-name: word-wrap dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update dependencies (#887) * update all dependencies to latest versions * revert xunit package update * undo * test * restoreArguments * --no-restore * force-evaluate * noCache * revert xunit update * undo exception change * Update docs with details about output binding columns (#889) * update sdk and xunit (#890) * package lock updates * restore packages * merge conflicts * exclusing oop test for timeout issue * remove retrytheory * add oop * use TrySetResult * revert back to RetryTheory --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Chris LaFreniere <40371649+chlafreniere@users.noreply.github.com> Co-authored-by: Taigh Hawkins <taigh.hawkins@cloud-trade.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * undo logging deletes * uno default * update perf.yml --------- Signed-off-by: Maddy Koripalli <makoripa@microsoft.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> Co-authored-by: Lucy Zhang <luczhan@microsoft.com> Co-authored-by: AmeyaRele <35621237+AmeyaRele@users.noreply.github.com> Co-authored-by: Drew Skwiers-Koballa <dzsquared@users.noreply.github.com> Co-authored-by: Chris LaFreniere <40371649+chlafreniere@users.noreply.github.com> Co-authored-by: Taigh Hawkins <taigh.hawkins@cloud-trade.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Revert "Remove trigger binding for GA release (#732)" (#733) This reverts commit 9713259. * vBump release/trigger 2.* -> 3.* (#738) * Add JObject support for SQL trigger (#722) * add support for jobject and js, ps, python samples * use utils JsonSerializeObject + comment * Update src/TriggerBinding/SqlTriggerBindingProvider.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * fix unit test + pylint --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Fix trigger connection not being retried when connection is closed (#731) * Add Privacy Statement to README and telemetry message (#751) * Add link to privacy statement * Add privacy link to output message * add to output message * Fix typos (cherry picked from commit a9565f7) * enable trigger oop (#750) * add sqltrigger attribute * add SqlChange type * enable oop test * add check * fix the null reference * use GetLogger * Enable debug logging for Python and PS samples (#754) * enable debug logging for ps and python samples * enable debug logs for js samples * Add docs & logging information for Leases table info (#756) * Add ProductsTriggerWithValidation tests for Python, JS, PS (#753) * add js, py, ps ProductsTriggerWithValidation test * fix test + missing file * Further clarifications to trigger retry docs (#770) * Further clarifications to trigger retry docs * Update description * increase timeout to 120 minutes (#776) * Enable more tests for OOP trigger (#777) * add more oop test samples * update logger * bump jackson dependency (#782) * Add detailed trigger docs (#780) * Filter out more default telemetry properties (cherry picked from commit d6ea6d7) * bring back trigger text * Fixes * Add SQLTrigger annotation to java library (#783) * add SQLTrigger annotation to java library * remove default values * use enum for commandtype * Disable CSX trigger tests * fix regex for java library version (#788) * Adding rest of Trigger integration tests for Python, PS, JS (#763) * add js, py, ps ProductsTriggerWithValidation test * fix test + missing file * add tests * remove extra comments + powershell compress * update comment * Skip more trigger tests for CSX (#791) * Introduce Target Based Scaling (#598) * Refactor SqlTriggerListener scaling to SqlTriggerMetricsProvider and SqlTriggerScaleMonitor * Create SqlTriggerTargetScaler * Refactor unit tests * Refactor to include common queries for scaling and listener class to SqlTriggerUtils * Add doc comments for scaling classes * Update src/TriggerBinding/SqlTriggerTargetScaler.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Fix log statement * Update WebJobs package * Update nuget.config * Address review comments * Address review comments pt2 * Update src/TriggerBinding/SqlTriggerTargetScaler.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Address comments, test failures * Fix packages lock file * Fix error message * Address comments and test failures * Apply suggestions from code review Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Change in documentation * Fix log level --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Enable trace level logging for tests (#795) * Enable trace level logging for tests * one more * Add java trigger samples (#793) * add java trigger samples * update test-java * add SqlChangeProduct type * add rest of tests * fix polling interval test * Fix up performance tests & enable schedule (#799) * Minor trigger doc updates (#800) * Minor trigger doc updates * fix * enable rest of the trigger tests for oop (#798) * enable all trigger tests for OOP * undo * fix test * merge conflict * remove Java * add sample for python v2 model (#803) * Fix logs * fix compile * remove delegate (#809) Signed-off-by: Maddy Koripalli <makoripa@microsoft.com> * add csx trigger samples (#806) * add trigger samples for csx * enable csx tests * add copy trigger sample * use utils.josnserialize * separate utils file * reuse utils from sql extension * remove deleted references * Cleanup trigger logging (#808) * Cleanup trigger logging * more * fix compile * Default log level of debug * Fix scale monitor tests * [Trigger] Update from main (#820) * update java library 0.1.1 -> 1.0.0 (#804) * Pin Node version to 18.15 (#818) * Pin Node version to 18.15 * Update build pipeline * Set node modules path for all * Debug * Pass in env var * Update pipeline comments (#819) --------- Co-authored-by: Lucy Zhang <luczhan@microsoft.com> * [Trigger] Rename csharpscript and align host.json (#825) * Merge branch 'main' into release/trigger # Conflicts: # docs/SetupGuide_DotnetCSharpScript.md # samples/samples-csharp/host.json # test/Microsoft.Azure.WebJobs.Extensions.Sql.Tests.csproj * Move triggers * Fix java version in setup guide (#821) * Refactor integration tests (#813) * create database once * fix compat level test * fix compat + logs * start func host in the beginning * fix change tracking * fix tests * fix tests * add CSharpscript and fix change tracking * set port * update xunit analyzers & temp skip failling tests * update port from fixture * pass console writeline as logger * enable identity tests * fix case sensitive tests + temp disable identity * fix indent * AddProductWithIdentityColumnIncluded fix * AddProductWithIdentityColumnIncluded * cleanup * remove extra fixture * default port * fix csx * Clean csx samples (#812) * sql attribute not needed * simplify the folder structure for csx * add local.settings.json * rename folders in proj file * clean up logs and unused * remove TriggerSamples folder * update trigger docs for csx * Fix compilation error (#835) * Add retry for Trigger tests (#836) * add retry logic * fix the csx compilation error * use xRetry * replace with RetryTheory * add diagnostic messages to logs * Update Trigger Binding set up in SetupGuide_DotnetOutOfProc.md (#845) * Update SetupGuide_DotnetOutOfProc.md * fixes * Update Trigger Binding set up in SetupGuide_Java.md (#846) * Update SetupGuide_Java.md * fix annotation * Update SetupGuide_Javascript.md (#847) * Update SetupGuide_Javascript.md * fix toc * FIx perf pipeline (#838) * dont start func host in fixture for perf * don't use fixture in perf test * finally block * Update SetupGuide_PowerShell.md (#848) * Update SetupGuide_Python.md (#849) * enable test TriggerWithException (#801) * enable the TriggerWithException test * update comment * remove logger delegate * fix error * add retry * remove Java test and try --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Add additional check for closed/broken connection errors (#862) * add timestamp to global state table for lease table cleanup (#861) * add LastAccessTime column to GlobalState * update docs with clean up scripts * update createtable * add comments * refine comments * fix script error * add print * add test to check LastAccessTime column creation for existing triggers (#865) * add LastAccessTime column to GlobalState * update docs with clean up scripts * update createtable * add comments * refine comments * fix script error * add print * add test * fix test error * use constants * Merge latest from main into trigger (#866) * Fix the race condition by adding the extension reference to build first. (#839) * fix the correct path separator * add the extension ref to avoid race condition * format files * pack and copy the sql nupkg to local-packages after build (#853) * copy to local packages before build * AfterTargets since package is created after build * make sure pack runs after build * address comments * Remove doc/sample references to preview bundle (#744) * Remove doc/sample references to preview bundle * Few more * undo * Powershell -> PowerShell * fix version * use scriptdom nuget (#864) * Revert "Remove doc/sample references to preview bundle (#744)" This reverts commit d7cd728. * Fix nuget package versioning --------- Co-authored-by: Maddy <12754347+MaddyDev@users.noreply.github.com> Co-authored-by: Lucy Zhang <luczhan@microsoft.com> * Revert "Remove doc/sample references to preview bundle (#744)" This reverts commit d7cd728. * add troubleshooting information to the overview doc (#871) * add troubleshooting section * update format * Update docs/BindingsOverview.md Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Update docs/BindingsOverview.md Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * clarifying view change tracking command (#873) * Add sql trigger test for different data types (#876) * add productcolumntypestriggertest * try running only csharp * try running js * comment out column values check temporarily * skip date and byte checks * fix build error * skip byte check * remove date/time, add back binary * fix build error * remove only binary check * remove only nchar and nvarchar * check equality of nchar and nvarchar * fix tests * change nchar to test * Bump Version of Grpc.Net.Client (#877) * Bump version of Grpc.Net.Client * Bump grpc.net.client to latest * Update packages.lock * Add to Directory.Packages.props * Fix code coverage in trigger branch (#880) * run only csharp test task * comment out trigger tests * don't start func hosts in the beginning * start three functions in beginning * start only csharp host * fix build error * start both csharp and java hosts * start only csharp and oop hosts * add logging for disposefunctionhosts * start only csharp host * revert commented out code * add comment * Fix schema parse error when using reserved keys as table names (#881) * bracketed name for reserved words as table name * add test * add tests * fix csx test * enable test only for csharp * fix TableNotPresentTest * revert GetUserTableIdAsync change * refactor GetUserTableIdAsync * comment out Java test * refactor code to use SqlObject * Add LeasesTableName to SqlTriggerAttribute (#893) * addLeasesTableNameSetting to SqlTriggerAttribute * add provider test * add samples * remove setting * add new constructor * fix metrics provider * add integration test * fix oop * fix test * cleanup + pr comments * quote escape leasestablename * Add leasesTableName to SqlTrigger Java annotation (#897) * add leasestablename to java sqltrigger * add default value for commandtype * remove sample * update java library to 2.1.0-preview (#899) * Update docs with LeasesTableName (#898) * add leasestablename to docs * fix links * GetLeaseLockedRows for debug logging (#900) * add isLeaseLocked into the getchanges query * add log message * add debug logging for row counts * update log msg * create a separate query for getting the locked rows * update ver name * update * remove user table join * unused var * correct xml info * address comments * remove extra line * dont throw * Update src/TriggerBinding/SqlTableChangeMonitor.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * rename leasesTableName to bracketedLeasesTableName (#902) * rename leasesTableName to bracketedLeasesTableName * merge error * update ReadAsync with Read * replace ExecuteReaderAsync * remove missed ExecuteReaderAsync * remove ReadAsync * remove async from updated methods * merge main * fix perf uml * resolve doc conflicts * resolve remaining conflicts * fix incomplete merge --------- Signed-off-by: Maddy Koripalli <makoripa@microsoft.com> Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> Co-authored-by: Lucy Zhang <luczhan@microsoft.com> Co-authored-by: AmeyaRele <35621237+AmeyaRele@users.noreply.github.com> Co-authored-by: Drew Skwiers-Koballa <dzsquared@users.noreply.github.com> Co-authored-by: Chris LaFreniere <40371649+chlafreniere@users.noreply.github.com>

* Revert "Remove trigger binding for GA release (Azure#732)" (Azure#733) This reverts commit 78f4853. * vBump release/trigger 2.* -> 3.* (Azure#738) * Add JObject support for SQL trigger (Azure#722) * add support for jobject and js, ps, python samples * use utils JsonSerializeObject + comment * Update src/TriggerBinding/SqlTriggerBindingProvider.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * fix unit test + pylint --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Fix trigger connection not being retried when connection is closed (Azure#731) * Add Privacy Statement to README and telemetry message (Azure#751) * Add link to privacy statement * Add privacy link to output message * add to output message * Fix typos (cherry picked from commit 623118f) * enable trigger oop (Azure#750) * add sqltrigger attribute * add SqlChange type * enable oop test * add check * fix the null reference * use GetLogger * Enable debug logging for Python and PS samples (Azure#754) * enable debug logging for ps and python samples * enable debug logs for js samples * Add docs & logging information for Leases table info (Azure#756) * Add ProductsTriggerWithValidation tests for Python, JS, PS (Azure#753) * add js, py, ps ProductsTriggerWithValidation test * fix test + missing file * Further clarifications to trigger retry docs (Azure#770) * Further clarifications to trigger retry docs * Update description * increase timeout to 120 minutes (Azure#776) * Enable more tests for OOP trigger (Azure#777) * add more oop test samples * update logger * bump jackson dependency (Azure#782) * Add detailed trigger docs (Azure#780) * Filter out more default telemetry properties (cherry picked from commit d6ea6d7) * bring back trigger text * Fixes * Add SQLTrigger annotation to java library (Azure#783) * add SQLTrigger annotation to java library * remove default values * use enum for commandtype * Disable CSX trigger tests * fix regex for java library version (Azure#788) * Adding rest of Trigger integration tests for Python, PS, JS (Azure#763) * add js, py, ps ProductsTriggerWithValidation test * fix test + missing file * add tests * remove extra comments + powershell compress * update comment * Skip more trigger tests for CSX (Azure#791) * Introduce Target Based Scaling (Azure#598) * Refactor SqlTriggerListener scaling to SqlTriggerMetricsProvider and SqlTriggerScaleMonitor * Create SqlTriggerTargetScaler * Refactor unit tests * Refactor to include common queries for scaling and listener class to SqlTriggerUtils * Add doc comments for scaling classes * Update src/TriggerBinding/SqlTriggerTargetScaler.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Fix log statement * Update WebJobs package * Update nuget.config * Address review comments * Address review comments pt2 * Update src/TriggerBinding/SqlTriggerTargetScaler.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Address comments, test failures * Fix packages lock file * Fix error message * Address comments and test failures * Apply suggestions from code review Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Change in documentation * Fix log level --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Enable trace level logging for tests (Azure#795) * Enable trace level logging for tests * one more * Add java trigger samples (Azure#793) * add java trigger samples * update test-java * add SqlChangeProduct type * add rest of tests * fix polling interval test * Fix up performance tests & enable schedule (Azure#799) * Minor trigger doc updates (Azure#800) * Minor trigger doc updates * fix * enable rest of the trigger tests for oop (Azure#798) * enable all trigger tests for OOP * undo * fix test * merge conflict * remove Java * add sample for python v2 model (Azure#803) * Fix logs * fix compile * remove delegate (Azure#809) Signed-off-by: Maddy Koripalli <makoripa@microsoft.com> * add csx trigger samples (Azure#806) * add trigger samples for csx * enable csx tests * add copy trigger sample * use utils.josnserialize * separate utils file * reuse utils from sql extension * remove deleted references * Cleanup trigger logging (Azure#808) * Cleanup trigger logging * more * fix compile * Default log level of debug * Fix scale monitor tests * [Trigger] Update from main (Azure#820) * update java library 0.1.1 -> 1.0.0 (Azure#804) * Pin Node version to 18.15 (Azure#818) * Pin Node version to 18.15 * Update build pipeline * Set node modules path for all * Debug * Pass in env var * Update pipeline comments (Azure#819) --------- Co-authored-by: Lucy Zhang <luczhan@microsoft.com> * [Trigger] Rename csharpscript and align host.json (Azure#825) * Merge branch 'main' into release/trigger # Conflicts: # docs/SetupGuide_DotnetCSharpScript.md # samples/samples-csharp/host.json # test/Microsoft.Azure.WebJobs.Extensions.Sql.Tests.csproj * Move triggers * Fix java version in setup guide (Azure#821) * Refactor integration tests (Azure#813) * create database once * fix compat level test * fix compat + logs * start func host in the beginning * fix change tracking * fix tests * fix tests * add CSharpscript and fix change tracking * set port * update xunit analyzers & temp skip failling tests * update port from fixture * pass console writeline as logger * enable identity tests * fix case sensitive tests + temp disable identity * fix indent * AddProductWithIdentityColumnIncluded fix * AddProductWithIdentityColumnIncluded * cleanup * remove extra fixture * default port * fix csx * Clean csx samples (Azure#812) * sql attribute not needed * simplify the folder structure for csx * add local.settings.json * rename folders in proj file * clean up logs and unused * remove TriggerSamples folder * update trigger docs for csx * Fix compilation error (Azure#835) * Add retry for Trigger tests (Azure#836) * add retry logic * fix the csx compilation error * use xRetry * replace with RetryTheory * add diagnostic messages to logs * Update Trigger Binding set up in SetupGuide_DotnetOutOfProc.md (Azure#845) * Update SetupGuide_DotnetOutOfProc.md * fixes * Update Trigger Binding set up in SetupGuide_Java.md (Azure#846) * Update SetupGuide_Java.md * fix annotation * Update SetupGuide_Javascript.md (Azure#847) * Update SetupGuide_Javascript.md * fix toc * FIx perf pipeline (Azure#838) * dont start func host in fixture for perf * don't use fixture in perf test * finally block * Update SetupGuide_PowerShell.md (Azure#848) * Update SetupGuide_Python.md (Azure#849) * enable test TriggerWithException (Azure#801) * enable the TriggerWithException test * update comment * remove logger delegate * fix error * add retry * remove Java test and try --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Add additional check for closed/broken connection errors (Azure#862) * add timestamp to global state table for lease table cleanup (Azure#861) * add LastAccessTime column to GlobalState * update docs with clean up scripts * update createtable * add comments * refine comments * fix script error * add print * add test to check LastAccessTime column creation for existing triggers (Azure#865) * add LastAccessTime column to GlobalState * update docs with clean up scripts * update createtable * add comments * refine comments * fix script error * add print * add test * fix test error * use constants * Merge latest from main into trigger (Azure#866) * Fix the race condition by adding the extension reference to build first. (Azure#839) * fix the correct path separator * add the extension ref to avoid race condition * format files * pack and copy the sql nupkg to local-packages after build (Azure#853) * copy to local packages before build * AfterTargets since package is created after build * make sure pack runs after build * address comments * Remove doc/sample references to preview bundle (Azure#744) * Remove doc/sample references to preview bundle * Few more * undo * Powershell -> PowerShell * fix version * use scriptdom nuget (Azure#864) * Revert "Remove doc/sample references to preview bundle (Azure#744)" This reverts commit b21fc0b. * Fix nuget package versioning --------- Co-authored-by: Maddy <12754347+MaddyDev@users.noreply.github.com> Co-authored-by: Lucy Zhang <luczhan@microsoft.com> * Revert "Remove doc/sample references to preview bundle (Azure#744)" This reverts commit b21fc0b. * add troubleshooting information to the overview doc (Azure#871) * add troubleshooting section * update format * Update docs/BindingsOverview.md Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Update docs/BindingsOverview.md Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * clarifying view change tracking command (Azure#873) * Add sql trigger test for different data types (Azure#876) * add productcolumntypestriggertest * try running only csharp * try running js * comment out column values check temporarily * skip date and byte checks * fix build error * skip byte check * remove date/time, add back binary * fix build error * remove only binary check * remove only nchar and nvarchar * check equality of nchar and nvarchar * fix tests * change nchar to test * Bump Version of Grpc.Net.Client (Azure#877) * Bump version of Grpc.Net.Client * Bump grpc.net.client to latest * Update packages.lock * Add to Directory.Packages.props * Fix code coverage in trigger branch (Azure#880) * run only csharp test task * comment out trigger tests * don't start func hosts in the beginning * start three functions in beginning * start only csharp host * fix build error * start both csharp and java hosts * start only csharp and oop hosts * add logging for disposefunctionhosts * start only csharp host * revert commented out code * add comment * Fix schema parse error when using reserved keys as table names (Azure#881) * bracketed name for reserved words as table name * add test * add tests * fix csx test * enable test only for csharp * fix TableNotPresentTest * revert GetUserTableIdAsync change * refactor GetUserTableIdAsync * comment out Java test * refactor code to use SqlObject * Add LeasesTableName to SqlTriggerAttribute (Azure#893) * addLeasesTableNameSetting to SqlTriggerAttribute * add provider test * add samples * remove setting * add new constructor * fix metrics provider * add integration test * fix oop * fix test * cleanup + pr comments * quote escape leasestablename * Add leasesTableName to SqlTrigger Java annotation (Azure#897) * add leasestablename to java sqltrigger * add default value for commandtype * remove sample * update java library to 2.1.0-preview (Azure#899) * Update docs with LeasesTableName (Azure#898) * add leasestablename to docs * fix links * GetLeaseLockedRows for debug logging (Azure#900) * add isLeaseLocked into the getchanges query * add log message * add debug logging for row counts * update log msg * create a separate query for getting the locked rows * update ver name * update * remove user table join * unused var * correct xml info * address comments * remove extra line * dont throw * Update src/TriggerBinding/SqlTableChangeMonitor.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * rename leasesTableName to bracketedLeasesTableName (Azure#902) * rename leasesTableName to bracketedLeasesTableName * merge error * [Trigger] Sync with main (Azure#906) * Bump Version of Grpc.Net.Client (Azure#877) * Bump version of Grpc.Net.Client * Bump grpc.net.client to latest * Update packages.lock * Add to Directory.Packages.props * Update GetProducts.js (Azure#884) Corrected input model so it works * Bump word-wrap from 1.2.3 to 1.2.4 in /samples/samples-js (Azure#885) Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4. - [Release notes](https://github.com/jonschlinkert/word-wrap/releases) - [Commits](jonschlinkert/word-wrap@1.2.3...1.2.4) --- updated-dependencies: - dependency-name: word-wrap dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Bump word-wrap from 1.2.3 to 1.2.4 in /test/Integration/test-js (Azure#886) Bumps [word-wrap](https://github.com/jonschlinkert/word-wrap) from 1.2.3 to 1.2.4. - [Release notes](https://github.com/jonschlinkert/word-wrap/releases) - [Commits](jonschlinkert/word-wrap@1.2.3...1.2.4) --- updated-dependencies: - dependency-name: word-wrap dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update dependencies (Azure#887) * update all dependencies to latest versions * revert xunit package update * undo * test * restoreArguments * --no-restore * force-evaluate * noCache * revert xunit update * undo exception change * Update docs with details about output binding columns (Azure#889) * update sdk and xunit (Azure#890) * package lock updates * restore packages * merge conflicts * exclusing oop test for timeout issue * remove retrytheory * add oop * use TrySetResult * revert back to RetryTheory --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Chris LaFreniere <40371649+chlafreniere@users.noreply.github.com> Co-authored-by: Taigh Hawkins <taigh.hawkins@cloud-trade.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * undo logging deletes * uno default * update perf.yml --------- Signed-off-by: Maddy Koripalli <makoripa@microsoft.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> Co-authored-by: Lucy Zhang <luczhan@microsoft.com> Co-authored-by: AmeyaRele <35621237+AmeyaRele@users.noreply.github.com> Co-authored-by: Drew Skwiers-Koballa <dzsquared@users.noreply.github.com> Co-authored-by: Chris LaFreniere <40371649+chlafreniere@users.noreply.github.com> Co-authored-by: Taigh Hawkins <taigh.hawkins@cloud-trade.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Revert "Remove trigger binding for GA release (Azure#732)" (Azure#733) This reverts commit 78f4853. * vBump release/trigger 2.* -> 3.* (Azure#738) * Add JObject support for SQL trigger (Azure#722) * add support for jobject and js, ps, python samples * use utils JsonSerializeObject + comment * Update src/TriggerBinding/SqlTriggerBindingProvider.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * fix unit test + pylint --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Fix trigger connection not being retried when connection is closed (Azure#731) * Add Privacy Statement to README and telemetry message (Azure#751) * Add link to privacy statement * Add privacy link to output message * add to output message * Fix typos (cherry picked from commit 623118f) * enable trigger oop (Azure#750) * add sqltrigger attribute * add SqlChange type * enable oop test * add check * fix the null reference * use GetLogger * Enable debug logging for Python and PS samples (Azure#754) * enable debug logging for ps and python samples * enable debug logs for js samples * Add docs & logging information for Leases table info (Azure#756) * Add ProductsTriggerWithValidation tests for Python, JS, PS (Azure#753) * add js, py, ps ProductsTriggerWithValidation test * fix test + missing file * Further clarifications to trigger retry docs (Azure#770) * Further clarifications to trigger retry docs * Update description * increase timeout to 120 minutes (Azure#776) * Enable more tests for OOP trigger (Azure#777) * add more oop test samples * update logger * bump jackson dependency (Azure#782) * Add detailed trigger docs (Azure#780) * Filter out more default telemetry properties (cherry picked from commit d6ea6d7) * bring back trigger text * Fixes * Add SQLTrigger annotation to java library (Azure#783) * add SQLTrigger annotation to java library * remove default values * use enum for commandtype * Disable CSX trigger tests * fix regex for java library version (Azure#788) * Adding rest of Trigger integration tests for Python, PS, JS (Azure#763) * add js, py, ps ProductsTriggerWithValidation test * fix test + missing file * add tests * remove extra comments + powershell compress * update comment * Skip more trigger tests for CSX (Azure#791) * Introduce Target Based Scaling (Azure#598) * Refactor SqlTriggerListener scaling to SqlTriggerMetricsProvider and SqlTriggerScaleMonitor * Create SqlTriggerTargetScaler * Refactor unit tests * Refactor to include common queries for scaling and listener class to SqlTriggerUtils * Add doc comments for scaling classes * Update src/TriggerBinding/SqlTriggerTargetScaler.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Fix log statement * Update WebJobs package * Update nuget.config * Address review comments * Address review comments pt2 * Update src/TriggerBinding/SqlTriggerTargetScaler.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Address comments, test failures * Fix packages lock file * Fix error message * Address comments and test failures * Apply suggestions from code review Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Change in documentation * Fix log level --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Enable trace level logging for tests (Azure#795) * Enable trace level logging for tests * one more * Add java trigger samples (Azure#793) * add java trigger samples * update test-java * add SqlChangeProduct type * add rest of tests * fix polling interval test * Fix up performance tests & enable schedule (Azure#799) * Minor trigger doc updates (Azure#800) * Minor trigger doc updates * fix * enable rest of the trigger tests for oop (Azure#798) * enable all trigger tests for OOP * undo * fix test * merge conflict * remove Java * add sample for python v2 model (Azure#803) * Fix logs * fix compile * remove delegate (Azure#809) Signed-off-by: Maddy Koripalli <makoripa@microsoft.com> * add csx trigger samples (Azure#806) * add trigger samples for csx * enable csx tests * add copy trigger sample * use utils.josnserialize * separate utils file * reuse utils from sql extension * remove deleted references * Cleanup trigger logging (Azure#808) * Cleanup trigger logging * more * fix compile * Default log level of debug * Fix scale monitor tests * [Trigger] Update from main (Azure#820) * update java library 0.1.1 -> 1.0.0 (Azure#804) * Pin Node version to 18.15 (Azure#818) * Pin Node version to 18.15 * Update build pipeline * Set node modules path for all * Debug * Pass in env var * Update pipeline comments (Azure#819) --------- Co-authored-by: Lucy Zhang <luczhan@microsoft.com> * [Trigger] Rename csharpscript and align host.json (Azure#825) * Merge branch 'main' into release/trigger # Conflicts: # docs/SetupGuide_DotnetCSharpScript.md # samples/samples-csharp/host.json # test/Microsoft.Azure.WebJobs.Extensions.Sql.Tests.csproj * Move triggers * Fix java version in setup guide (Azure#821) * Refactor integration tests (Azure#813) * create database once * fix compat level test * fix compat + logs * start func host in the beginning * fix change tracking * fix tests * fix tests * add CSharpscript and fix change tracking * set port * update xunit analyzers & temp skip failling tests * update port from fixture * pass console writeline as logger * enable identity tests * fix case sensitive tests + temp disable identity * fix indent * AddProductWithIdentityColumnIncluded fix * AddProductWithIdentityColumnIncluded * cleanup * remove extra fixture * default port * fix csx * Clean csx samples (Azure#812) * sql attribute not needed * simplify the folder structure for csx * add local.settings.json * rename folders in proj file * clean up logs and unused * remove TriggerSamples folder * update trigger docs for csx * Fix compilation error (Azure#835) * Add retry for Trigger tests (Azure#836) * add retry logic * fix the csx compilation error * use xRetry * replace with RetryTheory * add diagnostic messages to logs * Update Trigger Binding set up in SetupGuide_DotnetOutOfProc.md (Azure#845) * Update SetupGuide_DotnetOutOfProc.md * fixes * Update Trigger Binding set up in SetupGuide_Java.md (Azure#846) * Update SetupGuide_Java.md * fix annotation * Update SetupGuide_Javascript.md (Azure#847) * Update SetupGuide_Javascript.md * fix toc * FIx perf pipeline (Azure#838) * dont start func host in fixture for perf * don't use fixture in perf test * finally block * Update SetupGuide_PowerShell.md (Azure#848) * Update SetupGuide_Python.md (Azure#849) * enable test TriggerWithException (Azure#801) * enable the TriggerWithException test * update comment * remove logger delegate * fix error * add retry * remove Java test and try --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Add additional check for closed/broken connection errors (Azure#862) * add timestamp to global state table for lease table cleanup (Azure#861) * add LastAccessTime column to GlobalState * update docs with clean up scripts * update createtable * add comments * refine comments * fix script error * add print * add test to check LastAccessTime column creation for existing triggers (Azure#865) * add LastAccessTime column to GlobalState * update docs with clean up scripts * update createtable * add comments * refine comments * fix script error * add print * add test * fix test error * use constants * Merge latest from main into trigger (Azure#866) * Fix the race condition by adding the extension reference to build first. (Azure#839) * fix the correct path separator * add the extension ref to avoid race condition * format files * pack and copy the sql nupkg to local-packages after build (Azure#853) * copy to local packages before build * AfterTargets since package is created after build * make sure pack runs after build * address comments * Remove doc/sample references to preview bundle (Azure#744) * Remove doc/sample references to preview bundle * Few more * undo * Powershell -> PowerShell * fix version * use scriptdom nuget (Azure#864) * Revert "Remove doc/sample references to preview bundle (Azure#744)" This reverts commit b21fc0b. * Fix nuget package versioning --------- Co-authored-by: Maddy <12754347+MaddyDev@users.noreply.github.com> Co-authored-by: Lucy Zhang <luczhan@microsoft.com> * Revert "Remove doc/sample references to preview bundle (Azure#744)" This reverts commit b21fc0b. * add troubleshooting information to the overview doc (Azure#871) * add troubleshooting section * update format * Update docs/BindingsOverview.md Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * Update docs/BindingsOverview.md Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * clarifying view change tracking command (Azure#873) * Add sql trigger test for different data types (Azure#876) * add productcolumntypestriggertest * try running only csharp * try running js * comment out column values check temporarily * skip date and byte checks * fix build error * skip byte check * remove date/time, add back binary * fix build error * remove only binary check * remove only nchar and nvarchar * check equality of nchar and nvarchar * fix tests * change nchar to test * Bump Version of Grpc.Net.Client (Azure#877) * Bump version of Grpc.Net.Client * Bump grpc.net.client to latest * Update packages.lock * Add to Directory.Packages.props * Fix code coverage in trigger branch (Azure#880) * run only csharp test task * comment out trigger tests * don't start func hosts in the beginning * start three functions in beginning * start only csharp host * fix build error * start both csharp and java hosts * start only csharp and oop hosts * add logging for disposefunctionhosts * start only csharp host * revert commented out code * add comment * Fix schema parse error when using reserved keys as table names (Azure#881) * bracketed name for reserved words as table name * add test * add tests * fix csx test * enable test only for csharp * fix TableNotPresentTest * revert GetUserTableIdAsync change * refactor GetUserTableIdAsync * comment out Java test * refactor code to use SqlObject * Add LeasesTableName to SqlTriggerAttribute (Azure#893) * addLeasesTableNameSetting to SqlTriggerAttribute * add provider test * add samples * remove setting * add new constructor * fix metrics provider * add integration test * fix oop * fix test * cleanup + pr comments * quote escape leasestablename * Add leasesTableName to SqlTrigger Java annotation (Azure#897) * add leasestablename to java sqltrigger * add default value for commandtype * remove sample * update java library to 2.1.0-preview (Azure#899) * Update docs with LeasesTableName (Azure#898) * add leasestablename to docs * fix links * GetLeaseLockedRows for debug logging (Azure#900) * add isLeaseLocked into the getchanges query * add log message * add debug logging for row counts * update log msg * create a separate query for getting the locked rows * update ver name * update * remove user table join * unused var * correct xml info * address comments * remove extra line * dont throw * Update src/TriggerBinding/SqlTableChangeMonitor.cs Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> --------- Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> * rename leasesTableName to bracketedLeasesTableName (Azure#902) * rename leasesTableName to bracketedLeasesTableName * merge error * update ReadAsync with Read * replace ExecuteReaderAsync * remove missed ExecuteReaderAsync * remove ReadAsync * remove async from updated methods * merge main * fix perf uml * resolve doc conflicts * resolve remaining conflicts * fix incomplete merge --------- Signed-off-by: Maddy Koripalli <makoripa@microsoft.com> Co-authored-by: Charles Gagnon <chgagnon@microsoft.com> Co-authored-by: Lucy Zhang <luczhan@microsoft.com> Co-authored-by: AmeyaRele <35621237+AmeyaRele@users.noreply.github.com> Co-authored-by: Drew Skwiers-Koballa <dzsquared@users.noreply.github.com> Co-authored-by: Chris LaFreniere <40371649+chlafreniere@users.noreply.github.com>

AmeyaRele added 3 commits December 1, 2022 16:49

Refactor SqlTriggerListener scaling to SqlTriggerMetricsProvider and …

0410f88

…SqlTriggerScaleMonitor

Create SqlTriggerTargetScaler

4861d58

Refactor unit tests

6434a33

AmeyaRele requested a review from JatinSanghvi December 21, 2022 05:33

Charles-Gagnon reviewed Dec 21, 2022

View reviewed changes

nuget.config Outdated Show resolved Hide resolved

Refactor to include common queries for scaling and listener class to …

221c577

…SqlTriggerUtils

AmeyaRele marked this pull request as ready for review January 17, 2023 17:14

AmeyaRele requested review from MaddyDev and lucyzhang929 as code owners January 17, 2023 17:14

Charles-Gagnon reviewed Jan 17, 2023

View reviewed changes

Add doc comments for scaling classes

c5805e4

Charles-Gagnon reviewed Jan 19, 2023

View reviewed changes

src/TriggerBinding/SqlTriggerTargetScaler.cs Outdated Show resolved Hide resolved

AmeyaRele requested a review from VasuBhog as a code owner January 19, 2023 06:37

AmeyaRele and others added 3 commits January 19, 2023 12:07

Update src/TriggerBinding/SqlTriggerTargetScaler.cs

e6e7973

Co-authored-by: Charles Gagnon <chgagnon@microsoft.com>

Fix log statement

a437d2c

Merge branch 'ameyarele/target-based-scaling' of https://github.com/A…

f239b09

…zure/azure-functions-sql-extension into ameyarele/target-based-scaling

AmeyaRele requested review from alrod, chiangvincent, mathewc and pragnagopa January 24, 2023 09:45

pragnagopa reviewed Feb 14, 2023

View reviewed changes

Directory.Packages.props Outdated Show resolved Hide resolved

AmeyaRele and others added 2 commits February 15, 2023 14:38

Update WebJobs package

1621a09

Update nuget.config

3e5961c

chiangvincent requested changes Feb 15, 2023

View reviewed changes

alrod reviewed Feb 15, 2023

View reviewed changes

src/TriggerBinding/SqlTriggerScaleMonitor.cs Show resolved Hide resolved

Address review comments

a16a5ea

Charles-Gagnon reviewed Feb 16, 2023

View reviewed changes

AmeyaRele and others added 2 commits February 21, 2023 21:47

Address review comments pt2

0636741

Update src/TriggerBinding/SqlTriggerTargetScaler.cs

d9e1201

Co-authored-by: Charles Gagnon <chgagnon@microsoft.com>

AmeyaRele added 2 commits March 8, 2023 17:11

Fix packages lock file

dff5d1d

Fix error message

6cd9d88

AmeyaRele and others added 2 commits March 9, 2023 16:21

Address comments and test failures

cf1aacb

Apply suggestions from code review

bdc291b

Co-authored-by: Charles Gagnon <chgagnon@microsoft.com>

Charles-Gagnon requested a review from chiangvincent March 9, 2023 16:19

Merge branch 'main' into ameyarele/target-based-scaling

b821935

Charles-Gagnon approved these changes Mar 9, 2023

View reviewed changes

Charles-Gagnon changed the base branch from main to release/trigger March 15, 2023 17:29

Merge branch 'release/trigger' into ameyarele/target-based-scaling

79c1d9f

chiangvincent reviewed Mar 17, 2023

View reviewed changes

docs/BindingsOverview.md Outdated Show resolved Hide resolved

docs/BindingsOverview.md Outdated Show resolved Hide resolved

src/TriggerBinding/SqlTriggerTargetScaler.cs Outdated Show resolved Hide resolved

AmeyaRele and others added 2 commits March 22, 2023 15:57

Change in documentation

05a15e8

Merge branch 'release/trigger' into ameyarele/target-based-scaling

601c670

AmeyaRele requested a review from chiangvincent March 23, 2023 11:15

chiangvincent requested changes Mar 28, 2023

View reviewed changes

src/TriggerBinding/SqlTriggerTargetScaler.cs Outdated Show resolved Hide resolved

AmeyaRele requested a review from alrod March 28, 2023 16:48

Fix log level

10763d8

chiangvincent approved these changes Mar 28, 2023

View reviewed changes

alrod approved these changes Apr 6, 2023

View reviewed changes

AmeyaRele merged commit ff6e739 into release/trigger Apr 10, 2023

AmeyaRele deleted the ameyarele/target-based-scaling branch April 10, 2023 05:22

Conversation

AmeyaRele commented Dec 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Charles-Gagnon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chiangvincent left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

alrod Feb 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Charles-Gagnon Feb 15, 2023

Choose a reason for hiding this comment

Uh oh!

alrod Apr 6, 2023

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Charles-Gagnon commented Mar 8, 2023

Uh oh!

Charles-Gagnon commented Mar 9, 2023

Uh oh!

AmeyaRele commented Mar 13, 2023

Uh oh!

Charles-Gagnon commented Mar 15, 2023

Uh oh!

AmeyaRele commented Mar 16, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chiangvincent left a comment

Choose a reason for hiding this comment

Uh oh!

alrod left a comment

Choose a reason for hiding this comment

Uh oh!

Charles-Gagnon commented Apr 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

AmeyaRele commented Dec 21, 2022 •

edited

Loading

chiangvincent left a comment •

edited

Loading

alrod Feb 15, 2023 •

edited

Loading