From a14ccd49389ca41446acc3200e3e870cde15a68e Mon Sep 17 00:00:00 2001 From: AWSHurneyt <79280347+AWSHurneyt@users.noreply.github.com> Date: Wed, 30 Jun 2021 23:06:45 -0700 Subject: [PATCH 01/11] Added release notes for OpenSearch 1.0.0.0. (#123) --- ...ensearch-alerting.release-notes-1.0.0.0.md | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 release-notes/opensearch-alerting.release-notes-1.0.0.0.md diff --git a/release-notes/opensearch-alerting.release-notes-1.0.0.0.md b/release-notes/opensearch-alerting.release-notes-1.0.0.0.md new file mode 100644 index 000000000..f2f7f5696 --- /dev/null +++ b/release-notes/opensearch-alerting.release-notes-1.0.0.0.md @@ -0,0 +1,22 @@ +## Version 1.0.0.0 2021-06-30 + +Compatible with OpenSearch 1.0.0 + +### Infrastructure +* Upgrading the Ktlint Version and applying the formatting to the project ([#20](https://github.com/opensearch-project/alerting/pull/20)) +* Upgrade google-java-format to 1.10.0 to pick guava v30 ([#115](https://github.com/opensearch-project/alerting/pull/115)) +### Enhancements +* Adding check for security enabled ([#24](https://github.com/opensearch-project/alerting/pull/24)) +* Adding Workflow to test the Security Integration tests with the Security Plugin Installed on Opensearch Docker Image ([#25](https://github.com/opensearch-project/alerting/pull/25)) +* Adding Ktlint formatting check to the Gradle build task ([#26](https://github.com/opensearch-project/alerting/pull/26)) +* Updating UTs for the Destination Settings ([#102](https://github.com/opensearch-project/alerting/pull/102)) +* Adding additional null checks for URL not present ([#112](https://github.com/opensearch-project/alerting/pull/112)) +* Enable license headers check ([118](https://github.com/opensearch-project/alerting/pull/118)) +### Documentation +* Update issue template with multiple labels ([#13](https://github.com/opensearch-project/alerting/pull/13)) +* Update and add documentation files ([#117](https://github.com/opensearch-project/alerting/pull/117)) +### Maintenance +* Adding Rest APIs Backward Compatibility with ODFE ([#16](https://github.com/opensearch-project/alerting/pull/16)) +* Moving the ODFE Settings to Legacy Settings and adding the new settings compatible with Opensearch ([#18](https://github.com/opensearch-project/alerting/pull/18)) +### Refactoring +* Rename namespaces from com.amazon.opendistroforelasticsearch to org.opensearch ([#15](https://github.com/opensearch-project/alerting/pull/15)) \ No newline at end of file From af5b6c574435fa05d427aaa3ed6f2ea7201b8d58 Mon Sep 17 00:00:00 2001 From: Aditya Jindal <13850971+aditjind@users.noreply.github.com> Date: Thu, 2 Sep 2021 11:52:03 -0700 Subject: [PATCH 02/11] Merge commits from the main branch to the 1.x branch. (#133) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Added release notes for OpenSearch 1.0.0.0. (#123) (#124) Co-authored-by: AWSHurneyt <79280347+AWSHurneyt@users.noreply.github.com> * Add Integtest.sh for OpenSearch integtest setups (#121) * Add integtest script to the repo Signed-off-by: Peter Zhu * Add Alerting specific security param for integTest Signed-off-by: Peter Zhu * Remove default assignee (#127) Signed-off-by: Ashish Agrawal * Removing All Usages of Action Get Method Calls and adding the listeners (#130) Signed-off-by: Aditya Jindal * Fix snapshot build and increment to 1.1.0. (#142) Signed-off-by: dblock * Refactor MonitorRunner (#143) Signed-off-by: Mohammad Qureshi * Update Bucket-Level Alerting RFC (#145) Signed-off-by: Mohammad Qureshi * Add BucketSelector pipeline aggregation extension (#144) Signed-off-by: Mohammad Qureshi Co-authored-by: Rishabh Maurya * Add AggregationResultBucket (#148) Signed-off-by: Mohammad Qureshi Co-authored-by: Rishabh Maurya * Add ActionExecutionPolicy (#149) * Add ActionExecutionPolicy Signed-off-by: Mohammad Qureshi * Throw exception if there is an invalid field in PER_ALERT config when parsing Signed-off-by: Mohammad Qureshi * Don't allow throttle to be configured for PerExecutionActionScope at the data class level since it is not supported yet Signed-off-by: Mohammad Qureshi * Refactor Monitor and Trigger to split into Query-Level and Bucket-Lev… (#150) * Refactor Monitor and Trigger to split into Query-Level and Bucket-Level Monitors Signed-off-by: Mohammad Qureshi * Require condition to not be null when parsing Bucket-Level Trigger Signed-off-by: Mohammad Qureshi * Update InputService for Bucket-Level Alerting (#152) Signed-off-by: Mohammad Qureshi Co-authored-by: Rishabh Maurya * Update TriggerService for Bucket-Level Alerting (#153) * Update TriggerService for Bucket-Level Alerting Signed-off-by: Mohammad Qureshi * Remove client from TriggerService Signed-off-by: Mohammad Qureshi * Update AlertService for Bucket-Level Alerting (#154) * Update AlertService for Bucket-Level Alerting Signed-off-by: Mohammad Qureshi * Move Alert search size for Bucket-Level Monitors to a const Signed-off-by: Mohammad Qureshi * Add worksheets to help with testing (#151) Signed-off-by: Mohammad Qureshi * Update MonitorRunner for Bucket-Level Alerting (#155) * Update MonitorRunner for Bucket-Level Alerting Signed-off-by: Mohammad Qureshi * Update regressed comment in MonitorRunnerIT Signed-off-by: Mohammad Qureshi * Add TODO to break down runBucketLevelMonitor method in MonitorRunner Signed-off-by: Mohammad Qureshi * Fix ktlint formatting issues (#156) Signed-off-by: Mohammad Qureshi * Execute Actions on runTrigger exceptions for Bucket-Level Monitor (#157) Signed-off-by: Mohammad Qureshi * Skip execution of Actions on ACKNOWLEDGED Alerts for Bucket-Level Monitors (#158) Signed-off-by: Mohammad Qureshi * Return first page of input results in MonitorRunResult for Bucket-Level Monitor (#159) Signed-off-by: Mohammad Qureshi * Add setting to limit per alert action executions and don't save Alerts for test Bucket-Level Monitors (#161) Signed-off-by: Mohammad Qureshi * Fix bug in paginating multiple bucket paths for Bucket-Level Monitor (#163) * Fix bug in paginating multiple bucket paths for Bucket-Level Monitor Signed-off-by: Mohammad Qureshi * Change trigger after key conditionals to when statement Signed-off-by: Mohammad Qureshi * Various bug fixes pertaining to throttling on PER_ALERT, saving COMPLETED Alerts and rewriting input query for Bucket-Level Monitors (#164) Signed-off-by: Mohammad Qureshi * Return only monitors for /monitors/_search. (#162) * Return only monitors for /monitors/_search. * Added missing imports * Added additional check to the unit test * Resolve default for ActionExecutionPolicy at runtime (#165) Signed-off-by: Mohammad Qureshi Co-authored-by: AWSHurneyt <79280347+AWSHurneyt@users.noreply.github.com> Co-authored-by: Peter Zhu Co-authored-by: Ashish Agrawal Co-authored-by: Daniel Doubrovkine (dB.) Co-authored-by: Mohammad Qureshi <47198598+qreshi@users.noreply.github.com> Co-authored-by: Rishabh Maurya Co-authored-by: Sriram <59816283+skkosuri-amzn@users.noreply.github.com> --- .github/ISSUE_TEMPLATE/bug_report.md | 2 +- .github/ISSUE_TEMPLATE/feature_request.md | 2 +- .../workflows/multi-node-test-workflow.yml | 10 +- .github/workflows/test-workflow.yml | 12 +- .gitignore | 3 + alerting/build.gradle | 10 +- .../org/opensearch/alerting/AlertService.kt | 437 ++++++++++ .../org/opensearch/alerting/AlertingPlugin.kt | 46 +- .../org/opensearch/alerting/InputService.kt | 172 ++++ .../org/opensearch/alerting/MonitorRunner.kt | 796 ++++++++++-------- .../org/opensearch/alerting/TriggerService.kt | 110 +++ .../alerting/action/ExecuteMonitorResponse.kt | 4 +- .../BucketSelectorExtAggregationBuilder.kt | 256 ++++++ .../BucketSelectorExtAggregator.kt | 168 ++++ .../BucketSelectorExtFilter.kt | 149 ++++ .../BucketSelectorIndices.kt | 79 ++ .../alerting/alerts/AlertIndices.kt | 101 ++- .../alerting/model/AggregationResultBucket.kt | 95 +++ .../org/opensearch/alerting/model/Alert.kt | 85 +- .../alerting/model/BucketLevelTrigger.kt | 155 ++++ .../model/BucketLevelTriggerRunResult.kt | 63 ++ .../org/opensearch/alerting/model/Monitor.kt | 58 +- .../alerting/model/MonitorRunResult.kt | 81 +- .../alerting/model/QueryLevelTrigger.kt | 190 +++++ .../model/QueryLevelTriggerRunResult.kt | 72 ++ .../org/opensearch/alerting/model/Trigger.kt | 133 +-- .../alerting/model/TriggerRunResult.kt | 61 ++ .../alerting/model/action/Action.kt | 36 +- .../model/action/ActionExecutionPolicy.kt | 95 +++ .../model/action/ActionExecutionScope.kt | 182 ++++ .../resthandler/RestSearchMonitorAction.kt | 2 + .../BucketLevelTriggerExecutionContext.kt | 57 ++ .../QueryLevelTriggerExecutionContext.kt | 51 ++ .../script/TriggerExecutionContext.kt | 25 +- .../alerting/script/TriggerScript.kt | 2 +- .../alerting/settings/AlertingSettings.kt | 8 + .../alerting/settings/DestinationSettings.kt | 1 + .../LegacyOpenDistroDestinationSettings.kt | 4 +- .../TransportExecuteMonitorAction.kt | 7 +- .../alerting/util/AggregationQueryRewriter.kt | 120 +++ .../opensearch/alerting/util/AlertingUtils.kt | 41 + .../alerting/alerts/alert_mapping.json | 13 +- .../alerting/org.opensearch.alerting.txt | 18 +- .../org/opensearch/alerting/ADTestHelpers.kt | 11 +- .../opensearch/alerting/AlertServiceTests.kt | 215 +++++ .../alerting/AlertingRestTestCase.kt | 57 +- .../opensearch/alerting/MonitorRunnerIT.kt | 736 ++++++++++++++-- .../org/opensearch/alerting/MonitorTests.kt | 6 +- .../org/opensearch/alerting/TestHelpers.kt | 214 ++++- .../action/AcknowledgeAlertResponseTests.kt | 4 +- .../action/ExecuteMonitorRequestTests.kt | 4 +- .../action/ExecuteMonitorResponseTests.kt | 20 +- .../alerting/action/GetAlertsResponseTests.kt | 6 +- .../action/GetMonitorResponseTests.kt | 21 +- .../action/IndexMonitorRequestTests.kt | 6 +- .../action/IndexMonitorResponseTests.kt | 21 +- ...ucketSelectorExtAggregationBuilderTests.kt | 60 ++ .../BucketSelectorExtAggregatorTests.kt | 374 ++++++++ .../alerting/alerts/AlertIndicesIT.kt | 32 +- .../opensearch/alerting/model/AlertTests.kt | 26 + .../alerting/model/WriteableTests.kt | 75 +- .../alerting/model/XContentTests.kt | 181 +++- .../alerting/resthandler/MonitorRestApiIT.kt | 91 +- .../resthandler/SecureMonitorRestApiIT.kt | 27 +- .../util/AggregationQueryRewriterTests.kt | 335 ++++++++ .../util/AnomalyDetectionUtilsTests.kt | 12 +- build.gradle | 22 +- .../resources/mappings/scheduled-jobs.json | 72 +- ...ng-rfc.md => bucket-level-alerting-rfc.md} | 6 +- integtest.sh | 77 ++ worksheets/bucket_level_monitor_test.http | 104 +++ .../opensearch_dashboards_sample_data.http | 9 + 72 files changed, 6001 insertions(+), 835 deletions(-) create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/AlertService.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/InputService.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/TriggerService.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregationBuilder.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregator.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtFilter.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorIndices.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/model/AggregationResultBucket.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/model/BucketLevelTrigger.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/model/BucketLevelTriggerRunResult.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/model/QueryLevelTrigger.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/model/QueryLevelTriggerRunResult.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/model/TriggerRunResult.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/model/action/ActionExecutionPolicy.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/model/action/ActionExecutionScope.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/script/BucketLevelTriggerExecutionContext.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/script/QueryLevelTriggerExecutionContext.kt create mode 100644 alerting/src/main/kotlin/org/opensearch/alerting/util/AggregationQueryRewriter.kt create mode 100644 alerting/src/test/kotlin/org/opensearch/alerting/AlertServiceTests.kt create mode 100644 alerting/src/test/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregationBuilderTests.kt create mode 100644 alerting/src/test/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregatorTests.kt create mode 100644 alerting/src/test/kotlin/org/opensearch/alerting/util/AggregationQueryRewriterTests.kt rename docs/{document-level-alerting-rfc.md => bucket-level-alerting-rfc.md} (83%) create mode 100755 integtest.sh create mode 100644 worksheets/bucket_level_monitor_test.http create mode 100644 worksheets/opensearch_dashboards_sample_data.http diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 53e2c41e0..7b6750b61 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -3,7 +3,7 @@ name: Bug report about: Create a report to help us improve title: "[BUG]" labels: 'bug, untriaged, Beta' -assignees: skkosuri-amzn +assignees: --- diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index a962435cd..f89a20b43 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -3,7 +3,7 @@ name: Feature request about: Suggest an idea for this project title: '' labels: enhancement -assignees: skkosuri-amzn +assignees: --- diff --git a/.github/workflows/multi-node-test-workflow.yml b/.github/workflows/multi-node-test-workflow.yml index ab5bbbeeb..e0eb52ca7 100644 --- a/.github/workflows/multi-node-test-workflow.yml +++ b/.github/workflows/multi-node-test-workflow.yml @@ -29,20 +29,20 @@ jobs: with: repository: 'opensearch-project/OpenSearch' path: OpenSearch - ref: '1.0' + ref: '1.x' - name: Build OpenSearch working-directory: ./OpenSearch - run: ./gradlew publishToMavenLocal -Dbuild.snapshot=false + run: ./gradlew publishToMavenLocal # This step adds dependency, common-utils - name: Checkout common-utils uses: actions/checkout@v2 with: repository: 'opensearch-project/common-utils' path: common-utils - ref: '1.0' + ref: 'main' - name: Build common-utils working-directory: ./common-utils - run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0 + run: ./gradlew publishToMavenLocal -Dopensearch.version=1.1.0-SNAPSHOT # This step uses the checkout Github action: https://github.com/actions/checkout - name: Checkout Branch uses: actions/checkout@v2 @@ -52,7 +52,7 @@ jobs: with: java-version: 14 - name: Run integration tests with multi node config - run: ./gradlew integTest -PnumNodes=3 -Dopensearch.version=1.0.0 -Dbuild.snapshot=false + run: ./gradlew integTest -PnumNodes=3 -Dopensearch.version=1.1.0-SNAPSHOT - name: Pull and Run Docker run: | plugin=`ls alerting/build/distributions/*.zip` diff --git a/.github/workflows/test-workflow.yml b/.github/workflows/test-workflow.yml index 63e301404..6098fc7d0 100644 --- a/.github/workflows/test-workflow.yml +++ b/.github/workflows/test-workflow.yml @@ -35,10 +35,10 @@ jobs: with: repository: 'opensearch-project/OpenSearch' path: OpenSearch - ref: '1.0' + ref: '1.x' - name: Build OpenSearch working-directory: ./OpenSearch - run: ./gradlew publishToMavenLocal -Dbuild.snapshot=false + run: ./gradlew publishToMavenLocal # dependencies: common-utils - name: Checkout common-utils @@ -46,13 +46,13 @@ jobs: with: repository: 'opensearch-project/common-utils' path: common-utils - ref: '1.0' + ref: 'main' - name: Build common-utils working-directory: ./common-utils - run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0 + run: ./gradlew publishToMavenLocal -Dopensearch.version=1.1.0-SNAPSHOT - name: Build and run with Gradle - run: ./gradlew build -Dopensearch.version=1.0.0 + run: ./gradlew build -Dopensearch.version=1.1.0-SNAPSHOT # - name: Create Artifact Path # run: | @@ -71,4 +71,4 @@ jobs: # path: alerting-artifacts # Publish to local maven - name: Publish to Maven Local - run: ./gradlew publishToMavenLocal -Dopensearch.version=1.0.0 + run: ./gradlew publishToMavenLocal -Dopensearch.version=1.1.0-SNAPSHOT diff --git a/.gitignore b/.gitignore index 91b200679..a7be6eda6 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,6 @@ build/ .DS_Store *.log out/ +.project +.settings +.vscode \ No newline at end of file diff --git a/alerting/build.gradle b/alerting/build.gradle index c8b6b7c27..1f31293c0 100644 --- a/alerting/build.gradle +++ b/alerting/build.gradle @@ -35,17 +35,12 @@ apply plugin: 'jacoco' def usingRemoteCluster = System.properties.containsKey('tests.rest.cluster') || System.properties.containsKey('tests.cluster') def usingMultiNode = project.properties.containsKey('numNodes') - ext { projectSubstitutions = [:] licenseFile = rootProject.file('LICENSE.txt') noticeFile = rootProject.file('NOTICE.txt') } -if (isSnapshot) { - version += "-SNAPSHOT" -} - opensearchplugin { name 'opensearch-alerting' description 'Amazon OpenSearch alerting plugin' @@ -74,6 +69,10 @@ configurations.all { } } +configurations.testCompile { + exclude module: "securemock" +} + dependencies { compileOnly "org.opensearch.plugin:opensearch-scripting-painless-spi:${versions.opensearch}" @@ -87,6 +86,7 @@ dependencies { implementation "com.github.seancfoley:ipaddress:5.3.3" testImplementation "org.jetbrains.kotlin:kotlin-test:${kotlin_version}" + testCompile "org.mockito:mockito-core:2.23.0" } javadoc.enabled = false // turn off javadoc as it barfs on Kotlin code diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/AlertService.kt b/alerting/src/main/kotlin/org/opensearch/alerting/AlertService.kt new file mode 100644 index 000000000..197c5892c --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/AlertService.kt @@ -0,0 +1,437 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting + +import org.apache.logging.log4j.LogManager +import org.opensearch.ExceptionsHelper +import org.opensearch.action.DocWriteRequest +import org.opensearch.action.bulk.BackoffPolicy +import org.opensearch.action.bulk.BulkRequest +import org.opensearch.action.bulk.BulkResponse +import org.opensearch.action.delete.DeleteRequest +import org.opensearch.action.index.IndexRequest +import org.opensearch.action.search.SearchRequest +import org.opensearch.action.search.SearchResponse +import org.opensearch.alerting.alerts.AlertError +import org.opensearch.alerting.alerts.AlertIndices +import org.opensearch.alerting.elasticapi.firstFailureOrNull +import org.opensearch.alerting.elasticapi.retry +import org.opensearch.alerting.elasticapi.suspendUntil +import org.opensearch.alerting.model.ActionExecutionResult +import org.opensearch.alerting.model.ActionRunResult +import org.opensearch.alerting.model.AggregationResultBucket +import org.opensearch.alerting.model.Alert +import org.opensearch.alerting.model.BucketLevelTrigger +import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.QueryLevelTriggerRunResult +import org.opensearch.alerting.model.Trigger +import org.opensearch.alerting.model.action.AlertCategory +import org.opensearch.alerting.script.QueryLevelTriggerExecutionContext +import org.opensearch.alerting.util.IndexUtils +import org.opensearch.alerting.util.getBucketKeysHash +import org.opensearch.client.Client +import org.opensearch.common.bytes.BytesReference +import org.opensearch.common.xcontent.LoggingDeprecationHandler +import org.opensearch.common.xcontent.NamedXContentRegistry +import org.opensearch.common.xcontent.ToXContent +import org.opensearch.common.xcontent.XContentFactory +import org.opensearch.common.xcontent.XContentHelper +import org.opensearch.common.xcontent.XContentParser +import org.opensearch.common.xcontent.XContentParserUtils +import org.opensearch.common.xcontent.XContentType +import org.opensearch.index.query.QueryBuilders +import org.opensearch.rest.RestStatus +import org.opensearch.search.builder.SearchSourceBuilder +import java.time.Instant + +/** Service that handles CRUD operations for alerts */ +class AlertService( + val client: Client, + val xContentRegistry: NamedXContentRegistry, + val alertIndices: AlertIndices +) { + + companion object { + const val MAX_BUCKET_LEVEL_MONITOR_ALERT_SEARCH_COUNT = 500 + } + + private val logger = LogManager.getLogger(AlertService::class.java) + + suspend fun loadCurrentAlertsForQueryLevelMonitor(monitor: Monitor): Map { + val searchAlertsResponse: SearchResponse = searchAlerts( + monitorId = monitor.id, + size = monitor.triggers.size * 2 // We expect there to be only a single in-progress alert so fetch 2 to check + ) + + val foundAlerts = searchAlertsResponse.hits.map { Alert.parse(contentParser(it.sourceRef), it.id, it.version) } + .groupBy { it.triggerId } + foundAlerts.values.forEach { alerts -> + if (alerts.size > 1) { + logger.warn("Found multiple alerts for same trigger: $alerts") + } + } + + return monitor.triggers.associateWith { trigger -> + foundAlerts[trigger.id]?.firstOrNull() + } + } + + suspend fun loadCurrentAlertsForBucketLevelMonitor(monitor: Monitor): Map> { + val searchAlertsResponse: SearchResponse = searchAlerts( + monitorId = monitor.id, + // TODO: This should be limited based on a circuit breaker that limits Alerts + size = MAX_BUCKET_LEVEL_MONITOR_ALERT_SEARCH_COUNT + ) + + val foundAlerts = searchAlertsResponse.hits.map { Alert.parse(contentParser(it.sourceRef), it.id, it.version) } + .groupBy { it.triggerId } + + return monitor.triggers.associateWith { trigger -> + // Default to an empty map if there are no Alerts found for a Trigger to make Alert categorization logic easier + ( + foundAlerts[trigger.id]?.mapNotNull { alert -> + alert.aggregationResultBucket?.let { it.getBucketKeysHash() to alert } + }?.toMap() ?: mutableMapOf() + ) as MutableMap + } + } + + fun composeQueryLevelAlert( + ctx: QueryLevelTriggerExecutionContext, + result: QueryLevelTriggerRunResult, + alertError: AlertError? + ): Alert? { + val currentTime = Instant.now() + val currentAlert = ctx.alert + + val updatedActionExecutionResults = mutableListOf() + val currentActionIds = mutableSetOf() + if (currentAlert != null) { + // update current alert's action execution results + for (actionExecutionResult in currentAlert.actionExecutionResults) { + val actionId = actionExecutionResult.actionId + currentActionIds.add(actionId) + val actionRunResult = result.actionResults[actionId] + when { + actionRunResult == null -> updatedActionExecutionResults.add(actionExecutionResult) + actionRunResult.throttled -> + updatedActionExecutionResults.add( + actionExecutionResult.copy( + throttledCount = actionExecutionResult.throttledCount + 1 + ) + ) + else -> updatedActionExecutionResults.add(actionExecutionResult.copy(lastExecutionTime = actionRunResult.executionTime)) + } + } + // add action execution results which not exist in current alert + updatedActionExecutionResults.addAll( + result.actionResults.filter { !currentActionIds.contains(it.key) } + .map { ActionExecutionResult(it.key, it.value.executionTime, if (it.value.throttled) 1 else 0) } + ) + } else { + updatedActionExecutionResults.addAll( + result.actionResults.map { + ActionExecutionResult(it.key, it.value.executionTime, if (it.value.throttled) 1 else 0) + } + ) + } + + // Merge the alert's error message to the current alert's history + val updatedHistory = currentAlert?.errorHistory.update(alertError) + return if (alertError == null && !result.triggered) { + currentAlert?.copy( + state = Alert.State.COMPLETED, endTime = currentTime, errorMessage = null, + errorHistory = updatedHistory, actionExecutionResults = updatedActionExecutionResults, + schemaVersion = IndexUtils.alertIndexSchemaVersion + ) + } else if (alertError == null && currentAlert?.isAcknowledged() == true) { + null + } else if (currentAlert != null) { + val alertState = if (alertError == null) Alert.State.ACTIVE else Alert.State.ERROR + currentAlert.copy( + state = alertState, lastNotificationTime = currentTime, errorMessage = alertError?.message, + errorHistory = updatedHistory, actionExecutionResults = updatedActionExecutionResults, + schemaVersion = IndexUtils.alertIndexSchemaVersion + ) + } else { + val alertState = if (alertError == null) Alert.State.ACTIVE else Alert.State.ERROR + Alert( + monitor = ctx.monitor, trigger = ctx.trigger, startTime = currentTime, + lastNotificationTime = currentTime, state = alertState, errorMessage = alertError?.message, + errorHistory = updatedHistory, actionExecutionResults = updatedActionExecutionResults, + schemaVersion = IndexUtils.alertIndexSchemaVersion + ) + } + } + + fun updateActionResultsForBucketLevelAlert( + currentAlert: Alert, + actionResults: Map, + alertError: AlertError? + ): Alert { + val updatedActionExecutionResults = mutableListOf() + val currentActionIds = mutableSetOf() + // Update alert's existing action execution results + for (actionExecutionResult in currentAlert.actionExecutionResults) { + val actionId = actionExecutionResult.actionId + currentActionIds.add(actionId) + val actionRunResult = actionResults[actionId] + when { + actionRunResult == null -> updatedActionExecutionResults.add(actionExecutionResult) + actionRunResult.throttled -> + updatedActionExecutionResults.add( + actionExecutionResult.copy( + throttledCount = actionExecutionResult.throttledCount + 1 + ) + ) + else -> updatedActionExecutionResults.add(actionExecutionResult.copy(lastExecutionTime = actionRunResult.executionTime)) + } + } + + // Add action execution results not currently present in the alert + updatedActionExecutionResults.addAll( + actionResults.filter { !currentActionIds.contains(it.key) } + .map { ActionExecutionResult(it.key, it.value.executionTime, if (it.value.throttled) 1 else 0) } + ) + + val updatedErrorHistory = currentAlert.errorHistory.update(alertError) + return if (alertError == null) { + currentAlert.copy(errorHistory = updatedErrorHistory, actionExecutionResults = updatedActionExecutionResults) + } else { + currentAlert.copy( + state = Alert.State.ERROR, + errorMessage = alertError.message, + errorHistory = updatedErrorHistory, + actionExecutionResults = updatedActionExecutionResults + ) + } + } + + // TODO: Can change the parameters to use ctx: BucketLevelTriggerExecutionContext instead of monitor/trigger and + // result: AggTriggerRunResult for aggResultBuckets + // TODO: Can refactor this method to use Sets instead which can cleanup some of the categorization logic (like getting completed alerts) + fun getCategorizedAlertsForBucketLevelMonitor( + monitor: Monitor, + trigger: BucketLevelTrigger, + currentAlerts: MutableMap, + aggResultBuckets: List + ): Map> { + val dedupedAlerts = mutableListOf() + val newAlerts = mutableListOf() + val currentTime = Instant.now() + + aggResultBuckets.forEach { aggAlertBucket -> + val currentAlert = currentAlerts[aggAlertBucket.getBucketKeysHash()] + if (currentAlert != null) { + // De-duped Alert + dedupedAlerts.add(currentAlert.copy(aggregationResultBucket = aggAlertBucket)) + + // Remove de-duped Alert from currentAlerts since it is no longer a candidate for a potentially completed Alert + currentAlerts.remove(aggAlertBucket.getBucketKeysHash()) + } else { + // New Alert + val newAlert = Alert( + monitor = monitor, trigger = trigger, startTime = currentTime, + lastNotificationTime = null, state = Alert.State.ACTIVE, errorMessage = null, + errorHistory = mutableListOf(), actionExecutionResults = mutableListOf(), + schemaVersion = IndexUtils.alertIndexSchemaVersion, aggregationResultBucket = aggAlertBucket + ) + newAlerts.add(newAlert) + } + } + + return mapOf( + AlertCategory.DEDUPED to dedupedAlerts, + AlertCategory.NEW to newAlerts + ) + } + + fun convertToCompletedAlerts(currentAlerts: Map?): List { + val currentTime = Instant.now() + return currentAlerts?.map { + it.value.copy( + state = Alert.State.COMPLETED, endTime = currentTime, errorMessage = null, + schemaVersion = IndexUtils.alertIndexSchemaVersion + ) + } ?: listOf() + } + + suspend fun saveAlerts(alerts: List, retryPolicy: BackoffPolicy, allowUpdatingAcknowledgedAlert: Boolean = false) { + var requestsToRetry = alerts.flatMap { alert -> + // We don't want to set the version when saving alerts because the MonitorRunner has first priority when writing alerts. + // In the rare event that a user acknowledges an alert between when it's read and when it's written + // back we're ok if that acknowledgement is lost. It's easier to get the user to retry than for the runner to + // spend time reloading the alert and writing it back. + when (alert.state) { + Alert.State.ACTIVE, Alert.State.ERROR -> { + listOf>( + IndexRequest(AlertIndices.ALERT_INDEX) + .routing(alert.monitorId) + .source(alert.toXContent(XContentFactory.jsonBuilder(), ToXContent.EMPTY_PARAMS)) + .id(if (alert.id != Alert.NO_ID) alert.id else null) + ) + } + Alert.State.ACKNOWLEDGED -> { + // Allow ACKNOWLEDGED Alerts to be updated for Bucket-Level Monitors since de-duped Alerts can be ACKNOWLEDGED + // and updated by the MonitorRunner + if (allowUpdatingAcknowledgedAlert) { + listOf>( + IndexRequest(AlertIndices.ALERT_INDEX) + .routing(alert.monitorId) + .source(alert.toXContent(XContentFactory.jsonBuilder(), ToXContent.EMPTY_PARAMS)) + .id(if (alert.id != Alert.NO_ID) alert.id else null) + ) + } else { + throw IllegalStateException("Unexpected attempt to save ${alert.state} alert: $alert") + } + } + Alert.State.DELETED -> { + throw IllegalStateException("Unexpected attempt to save ${alert.state} alert: $alert") + } + Alert.State.COMPLETED -> { + listOfNotNull>( + DeleteRequest(AlertIndices.ALERT_INDEX, alert.id) + .routing(alert.monitorId), + // Only add completed alert to history index if history is enabled + if (alertIndices.isHistoryEnabled()) { + IndexRequest(AlertIndices.HISTORY_WRITE_INDEX) + .routing(alert.monitorId) + .source(alert.toXContent(XContentFactory.jsonBuilder(), ToXContent.EMPTY_PARAMS)) + .id(alert.id) + } else null + ) + } + } + } + + if (requestsToRetry.isEmpty()) return + // Retry Bulk requests if there was any 429 response + retryPolicy.retry(logger, listOf(RestStatus.TOO_MANY_REQUESTS)) { + val bulkRequest = BulkRequest().add(requestsToRetry) + val bulkResponse: BulkResponse = client.suspendUntil { client.bulk(bulkRequest, it) } + val failedResponses = (bulkResponse.items ?: arrayOf()).filter { it.isFailed } + requestsToRetry = failedResponses.filter { it.status() == RestStatus.TOO_MANY_REQUESTS } + .map { bulkRequest.requests()[it.itemId] as IndexRequest } + + if (requestsToRetry.isNotEmpty()) { + val retryCause = failedResponses.first { it.status() == RestStatus.TOO_MANY_REQUESTS }.failure.cause + throw ExceptionsHelper.convertToOpenSearchException(retryCause) + } + } + } + + /** + * This is a separate method created specifically for saving new Alerts during the Bucket-Level Monitor run. + * Alerts are saved in two batches during the execution of an Bucket-Level Monitor, once before the Actions are executed + * and once afterwards. This method saves Alerts to the [AlertIndices.ALERT_INDEX] but returns the same Alerts with their document IDs. + * + * The Alerts are required with their indexed ID so that when the new Alerts are updated after the Action execution, + * the ID is available for the index request so that the existing Alert can be updated, instead of creating a duplicate Alert document. + */ + suspend fun saveNewAlerts(alerts: List, retryPolicy: BackoffPolicy): List { + val savedAlerts = mutableListOf() + var alertsBeingIndexed = alerts + var requestsToRetry: MutableList = alerts.map { alert -> + if (alert.state != Alert.State.ACTIVE) { + throw IllegalStateException("Unexpected attempt to save new alert [$alert] with state [${alert.state}]") + } + if (alert.id != Alert.NO_ID) { + throw IllegalStateException("Unexpected attempt to save new alert [$alert] with an existing alert ID [${alert.id}]") + } + IndexRequest(AlertIndices.ALERT_INDEX) + .routing(alert.monitorId) + .source(alert.toXContent(XContentFactory.jsonBuilder(), ToXContent.EMPTY_PARAMS)) + }.toMutableList() + + if (requestsToRetry.isEmpty()) return listOf() + + // Retry Bulk requests if there was any 429 response. + // The responses of a bulk request will be in the same order as the individual requests. + // If the index request succeeded for an Alert, the document ID from the response is taken and saved in the Alert. + // If the index request is to be retried, the Alert is saved separately as well so that its relative ordering is maintained in + // relation to index request in the retried bulk request for when it eventually succeeds. + retryPolicy.retry(logger, listOf(RestStatus.TOO_MANY_REQUESTS)) { + val bulkRequest = BulkRequest().add(requestsToRetry) + val bulkResponse: BulkResponse = client.suspendUntil { client.bulk(bulkRequest, it) } + // TODO: This is only used to retrieve the retryCause, could instead fetch it from the bulkResponse iteration below + val failedResponses = (bulkResponse.items ?: arrayOf()).filter { it.isFailed } + + requestsToRetry = mutableListOf() + val alertsBeingRetried = mutableListOf() + bulkResponse.items.forEach { item -> + if (item.isFailed) { + // TODO: What if the failure cause was not TOO_MANY_REQUESTS, should these be saved and logged? + if (item.status() == RestStatus.TOO_MANY_REQUESTS) { + requestsToRetry.add(bulkRequest.requests()[item.itemId] as IndexRequest) + alertsBeingRetried.add(alertsBeingIndexed[item.itemId]) + } + } else { + // The ID of the BulkItemResponse in this case is the document ID resulting from the DocWriteRequest operation + savedAlerts.add(alertsBeingIndexed[item.itemId].copy(id = item.id)) + } + } + + alertsBeingIndexed = alertsBeingRetried + + if (requestsToRetry.isNotEmpty()) { + val retryCause = failedResponses.first { it.status() == RestStatus.TOO_MANY_REQUESTS }.failure.cause + throw ExceptionsHelper.convertToOpenSearchException(retryCause) + } + } + + return savedAlerts + } + + private fun contentParser(bytesReference: BytesReference): XContentParser { + val xcp = XContentHelper.createParser( + xContentRegistry, LoggingDeprecationHandler.INSTANCE, + bytesReference, XContentType.JSON + ) + XContentParserUtils.ensureExpectedToken(XContentParser.Token.START_OBJECT, xcp.nextToken(), xcp) + return xcp + } + + /** + * Searches for Alerts in the [AlertIndices.ALERT_INDEX]. + * + * @param monitorId The Monitor to get Alerts for + * @param size The number of search hits (Alerts) to return + */ + private suspend fun searchAlerts(monitorId: String, size: Int): SearchResponse { + val queryBuilder = QueryBuilders.boolQuery() + .filter(QueryBuilders.termQuery(Alert.MONITOR_ID_FIELD, monitorId)) + + val searchSourceBuilder = SearchSourceBuilder() + .size(size) + .query(queryBuilder) + + val searchRequest = SearchRequest(AlertIndices.ALERT_INDEX) + .routing(monitorId) + .source(searchSourceBuilder) + val searchResponse: SearchResponse = client.suspendUntil { client.search(searchRequest, it) } + if (searchResponse.status() != RestStatus.OK) { + throw (searchResponse.firstFailureOrNull()?.cause ?: RuntimeException("Unknown error loading alerts")) + } + + return searchResponse + } + + private fun List?.update(alertError: AlertError?): List { + return when { + this == null && alertError == null -> emptyList() + this != null && alertError == null -> this + this == null && alertError != null -> listOf(alertError) + this != null && alertError != null -> (listOf(alertError) + this).take(10) + else -> throw IllegalStateException("Unreachable code reached!") + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/AlertingPlugin.kt b/alerting/src/main/kotlin/org/opensearch/alerting/AlertingPlugin.kt index 2a2c75510..6c74c725e 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/AlertingPlugin.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/AlertingPlugin.kt @@ -45,6 +45,7 @@ import org.opensearch.alerting.action.IndexMonitorAction import org.opensearch.alerting.action.SearchEmailAccountAction import org.opensearch.alerting.action.SearchEmailGroupAction import org.opensearch.alerting.action.SearchMonitorAction +import org.opensearch.alerting.aggregation.bucketselectorext.BucketSelectorExtAggregationBuilder import org.opensearch.alerting.alerts.AlertIndices import org.opensearch.alerting.core.JobSweeper import org.opensearch.alerting.core.ScheduledJobIndices @@ -56,7 +57,9 @@ import org.opensearch.alerting.core.resthandler.RestScheduledJobStatsHandler import org.opensearch.alerting.core.schedule.JobScheduler import org.opensearch.alerting.core.settings.LegacyOpenDistroScheduledJobSettings import org.opensearch.alerting.core.settings.ScheduledJobSettings +import org.opensearch.alerting.model.BucketLevelTrigger import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.QueryLevelTrigger import org.opensearch.alerting.resthandler.RestAcknowledgeAlertAction import org.opensearch.alerting.resthandler.RestDeleteDestinationAction import org.opensearch.alerting.resthandler.RestDeleteEmailAccountAction @@ -103,12 +106,14 @@ import org.opensearch.cluster.metadata.IndexNameExpressionResolver import org.opensearch.cluster.node.DiscoveryNodes import org.opensearch.cluster.service.ClusterService import org.opensearch.common.io.stream.NamedWriteableRegistry +import org.opensearch.common.io.stream.StreamInput import org.opensearch.common.settings.ClusterSettings import org.opensearch.common.settings.IndexScopedSettings import org.opensearch.common.settings.Setting import org.opensearch.common.settings.Settings import org.opensearch.common.settings.SettingsFilter import org.opensearch.common.xcontent.NamedXContentRegistry +import org.opensearch.common.xcontent.XContentParser import org.opensearch.env.Environment import org.opensearch.env.NodeEnvironment import org.opensearch.index.IndexModule @@ -119,6 +124,7 @@ import org.opensearch.plugins.ActionPlugin import org.opensearch.plugins.Plugin import org.opensearch.plugins.ReloadablePlugin import org.opensearch.plugins.ScriptPlugin +import org.opensearch.plugins.SearchPlugin import org.opensearch.repositories.RepositoriesService import org.opensearch.rest.RestController import org.opensearch.rest.RestHandler @@ -131,10 +137,10 @@ import java.util.function.Supplier /** * Entry point of the OpenDistro for Elasticsearch alerting plugin * This class initializes the [RestGetMonitorAction], [RestDeleteMonitorAction], [RestIndexMonitorAction] rest handlers. - * It also adds [Monitor.XCONTENT_REGISTRY], [SearchInput.XCONTENT_REGISTRY] to the - * [NamedXContentRegistry] so that we are able to deserialize the custom named objects. + * It also adds [Monitor.XCONTENT_REGISTRY], [SearchInput.XCONTENT_REGISTRY], [QueryLevelTrigger.XCONTENT_REGISTRY], + * [BucketLevelTrigger.XCONTENT_REGISTRY] to the [NamedXContentRegistry] so that we are able to deserialize the custom named objects. */ -internal class AlertingPlugin : PainlessExtension, ActionPlugin, ScriptPlugin, ReloadablePlugin, Plugin() { +internal class AlertingPlugin : PainlessExtension, ActionPlugin, ScriptPlugin, ReloadablePlugin, SearchPlugin, Plugin() { override fun getContextWhitelists(): Map, List> { val whitelist = WhitelistLoader.loadFromResourceFiles(javaClass, "org.opensearch.alerting.txt") @@ -220,7 +226,12 @@ internal class AlertingPlugin : PainlessExtension, ActionPlugin, ScriptPlugin, R } override fun getNamedXContent(): List { - return listOf(Monitor.XCONTENT_REGISTRY, SearchInput.XCONTENT_REGISTRY) + return listOf( + Monitor.XCONTENT_REGISTRY, + SearchInput.XCONTENT_REGISTRY, + QueryLevelTrigger.XCONTENT_REGISTRY, + BucketLevelTrigger.XCONTENT_REGISTRY + ) } override fun createComponents( @@ -239,7 +250,19 @@ internal class AlertingPlugin : PainlessExtension, ActionPlugin, ScriptPlugin, R // Need to figure out how to use the OpenSearch DI classes rather than handwiring things here. val settings = environment.settings() alertIndices = AlertIndices(settings, client, threadPool, clusterService) - runner = MonitorRunner(settings, client, threadPool, scriptService, xContentRegistry, alertIndices, clusterService) + runner = MonitorRunner + .registerClusterService(clusterService) + .registerClient(client) + .registerNamedXContentRegistry(xContentRegistry) + .registerScriptService(scriptService) + .registerSettings(settings) + .registerThreadPool(threadPool) + .registerAlertIndices(alertIndices) + .registerInputService(InputService(client, scriptService, namedWriteableRegistry, xContentRegistry)) + .registerTriggerService(TriggerService(scriptService)) + .registerAlertService(AlertService(client, xContentRegistry, alertIndices)) + .registerConsumers() + .registerDestinationSettings() scheduledJobIndices = ScheduledJobIndices(client.admin(), clusterService) scheduler = JobScheduler(threadPool, runner) sweeper = JobSweeper(environment.settings(), client, clusterService, threadPool, xContentRegistry, scheduler, ALERTING_JOB_TYPES) @@ -278,6 +301,7 @@ internal class AlertingPlugin : PainlessExtension, ActionPlugin, ScriptPlugin, R AlertingSettings.REQUEST_TIMEOUT, AlertingSettings.MAX_ACTION_THROTTLE_VALUE, AlertingSettings.FILTER_BY_BACKEND_ROLES, + AlertingSettings.MAX_ACTIONABLE_ALERT_COUNT, LegacyOpenDistroAlertingSettings.INPUT_TIMEOUT, LegacyOpenDistroAlertingSettings.INDEX_TIMEOUT, LegacyOpenDistroAlertingSettings.BULK_TIMEOUT, @@ -318,4 +342,16 @@ internal class AlertingPlugin : PainlessExtension, ActionPlugin, ScriptPlugin, R override fun reload(settings: Settings) { runner.reloadDestinationSettings(settings) } + + override fun getPipelineAggregations(): List { + return listOf( + SearchPlugin.PipelineAggregationSpec( + BucketSelectorExtAggregationBuilder.NAME, + { sin: StreamInput -> BucketSelectorExtAggregationBuilder(sin) }, + { parser: XContentParser, agg_name: String -> + BucketSelectorExtAggregationBuilder.parse(agg_name, parser) + } + ) + ) + } } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/InputService.kt b/alerting/src/main/kotlin/org/opensearch/alerting/InputService.kt new file mode 100644 index 000000000..a389b8457 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/InputService.kt @@ -0,0 +1,172 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting + +import org.apache.logging.log4j.LogManager +import org.opensearch.action.search.SearchRequest +import org.opensearch.action.search.SearchResponse +import org.opensearch.alerting.core.model.SearchInput +import org.opensearch.alerting.elasticapi.convertToMap +import org.opensearch.alerting.elasticapi.suspendUntil +import org.opensearch.alerting.model.InputRunResults +import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.TriggerAfterKey +import org.opensearch.alerting.util.AggregationQueryRewriter +import org.opensearch.alerting.util.addUserBackendRolesFilter +import org.opensearch.client.Client +import org.opensearch.common.io.stream.BytesStreamOutput +import org.opensearch.common.io.stream.NamedWriteableAwareStreamInput +import org.opensearch.common.io.stream.NamedWriteableRegistry +import org.opensearch.common.xcontent.LoggingDeprecationHandler +import org.opensearch.common.xcontent.NamedXContentRegistry +import org.opensearch.common.xcontent.XContentType +import org.opensearch.script.Script +import org.opensearch.script.ScriptService +import org.opensearch.script.ScriptType +import org.opensearch.script.TemplateScript +import org.opensearch.search.builder.SearchSourceBuilder +import java.time.Instant + +/** Service that handles the collection of input results for Monitor executions */ +class InputService( + val client: Client, + val scriptService: ScriptService, + val namedWriteableRegistry: NamedWriteableRegistry, + val xContentRegistry: NamedXContentRegistry +) { + + private val logger = LogManager.getLogger(InputService::class.java) + + suspend fun collectInputResults( + monitor: Monitor, + periodStart: Instant, + periodEnd: Instant, + prevResult: InputRunResults? = null + ): InputRunResults { + return try { + val results = mutableListOf>() + val aggTriggerAfterKey: MutableMap = mutableMapOf() + + // TODO: If/when multiple input queries are supported for Bucket-Level Monitor execution, aggTriggerAfterKeys will + // need to be updated to account for it + monitor.inputs.forEach { input -> + when (input) { + is SearchInput -> { + // TODO: Figure out a way to use SearchTemplateRequest without bringing in the entire TransportClient + val searchParams = mapOf( + "period_start" to periodStart.toEpochMilli(), + "period_end" to periodEnd.toEpochMilli() + ) + // Deep copying query before passing it to rewriteQuery since otherwise, the monitor.input is modified directly + // which causes a strange bug where the rewritten query persists on the Monitor across executions + val rewrittenQuery = AggregationQueryRewriter.rewriteQuery(deepCopyQuery(input.query), prevResult, monitor.triggers) + val searchSource = scriptService.compile( + Script( + ScriptType.INLINE, Script.DEFAULT_TEMPLATE_LANG, + rewrittenQuery.toString(), searchParams + ), + TemplateScript.CONTEXT + ) + .newInstance(searchParams) + .execute() + + val searchRequest = SearchRequest().indices(*input.indices.toTypedArray()) + XContentType.JSON.xContent().createParser(xContentRegistry, LoggingDeprecationHandler.INSTANCE, searchSource).use { + searchRequest.source(SearchSourceBuilder.fromXContent(it)) + } + val searchResponse: SearchResponse = client.suspendUntil { client.search(searchRequest, it) } + aggTriggerAfterKey += AggregationQueryRewriter.getAfterKeysFromSearchResponse( + searchResponse, + monitor.triggers, + prevResult?.aggTriggersAfterKey + ) + results += searchResponse.convertToMap() + } + else -> { + throw IllegalArgumentException("Unsupported input type: ${input.name()}.") + } + } + } + InputRunResults(results.toList(), aggTriggersAfterKey = aggTriggerAfterKey) + } catch (e: Exception) { + logger.info("Error collecting inputs for monitor: ${monitor.id}", e) + InputRunResults(emptyList(), e) + } + } + + private fun deepCopyQuery(query: SearchSourceBuilder): SearchSourceBuilder { + val out = BytesStreamOutput() + query.writeTo(out) + val sin = NamedWriteableAwareStreamInput(out.bytes().streamInput(), namedWriteableRegistry) + return SearchSourceBuilder(sin) + } + + /** + * We moved anomaly result index to system index list. So common user could not directly query + * this index any more. This method will stash current thread context to pass security check. + * So monitor job can access anomaly result index. We will add monitor user roles filter in + * search query to only return documents the monitor user can access. + * + * On alerting Kibana, monitor users can only see detectors that they have read access. So they + * can't create monitor on other user's detector which they have no read access. Even they know + * other user's detector id and use it to create monitor, this method will only return anomaly + * results they can read. + */ + suspend fun collectInputResultsForADMonitor(monitor: Monitor, periodStart: Instant, periodEnd: Instant): InputRunResults { + return try { + val results = mutableListOf>() + val input = monitor.inputs[0] as SearchInput + + val searchParams = mapOf("period_start" to periodStart.toEpochMilli(), "period_end" to periodEnd.toEpochMilli()) + val searchSource = scriptService.compile( + Script( + ScriptType.INLINE, Script.DEFAULT_TEMPLATE_LANG, + input.query.toString(), searchParams + ), + TemplateScript.CONTEXT + ) + .newInstance(searchParams) + .execute() + + val searchRequest = SearchRequest().indices(*input.indices.toTypedArray()) + XContentType.JSON.xContent().createParser(xContentRegistry, LoggingDeprecationHandler.INSTANCE, searchSource).use { + searchRequest.source(SearchSourceBuilder.fromXContent(it)) + } + + // Add user role filter for AD result + client.threadPool().threadContext.stashContext().use { + // Currently we have no way to verify if user has AD read permission or not. So we always add user + // role filter here no matter AD backend role filter enabled or not. If we don't add user role filter + // when AD backend filter disabled, user can run monitor on any detector and get anomaly data even + // they have no AD read permission. So if domain disabled AD backend role filter, monitor runner + // still can't get AD result with different user backend role, even the monitor user has permission + // to read AD result. This is a short term solution to trade off between user experience and security. + // + // Possible long term solution: + // 1.Use secure rest client to send request to AD search result API. If no permission exception, + // that mean user has read access on AD result. Then don't need to add user role filter when query + // AD result if AD backend role filter is disabled. + // 2.Security provide some transport action to verify if user has permission to search AD result. + // Monitor runner will send transport request to check permission first. If security plugin response + // is yes, user has permission to query AD result. If AD role filter enabled, we will add user role + // filter to protect data at user role level; otherwise, user can query any AD result. + addUserBackendRolesFilter(monitor.user, searchRequest.source()) + val searchResponse: SearchResponse = client.suspendUntil { client.search(searchRequest, it) } + results += searchResponse.convertToMap() + } + InputRunResults(results.toList()) + } catch (e: Exception) { + logger.info("Error collecting anomaly result inputs for monitor: ${monitor.id}", e) + InputRunResults(emptyList(), e) + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/MonitorRunner.kt b/alerting/src/main/kotlin/org/opensearch/alerting/MonitorRunner.kt index a688d6822..152cd102e 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/MonitorRunner.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/MonitorRunner.kt @@ -34,125 +34,183 @@ import kotlinx.coroutines.launch import kotlinx.coroutines.runBlocking import kotlinx.coroutines.withContext import org.apache.logging.log4j.LogManager -import org.opensearch.ExceptionsHelper -import org.opensearch.action.DocWriteRequest import org.opensearch.action.bulk.BackoffPolicy -import org.opensearch.action.bulk.BulkRequest -import org.opensearch.action.bulk.BulkResponse -import org.opensearch.action.delete.DeleteRequest -import org.opensearch.action.index.IndexRequest -import org.opensearch.action.search.SearchRequest -import org.opensearch.action.search.SearchResponse -import org.opensearch.alerting.alerts.AlertError import org.opensearch.alerting.alerts.AlertIndices import org.opensearch.alerting.alerts.moveAlerts import org.opensearch.alerting.core.JobRunner import org.opensearch.alerting.core.model.ScheduledJob -import org.opensearch.alerting.core.model.SearchInput import org.opensearch.alerting.elasticapi.InjectorContextElement -import org.opensearch.alerting.elasticapi.convertToMap -import org.opensearch.alerting.elasticapi.firstFailureOrNull import org.opensearch.alerting.elasticapi.retry -import org.opensearch.alerting.elasticapi.suspendUntil -import org.opensearch.alerting.model.ActionExecutionResult import org.opensearch.alerting.model.ActionRunResult import org.opensearch.alerting.model.Alert -import org.opensearch.alerting.model.Alert.State.ACKNOWLEDGED -import org.opensearch.alerting.model.Alert.State.ACTIVE -import org.opensearch.alerting.model.Alert.State.COMPLETED -import org.opensearch.alerting.model.Alert.State.DELETED -import org.opensearch.alerting.model.Alert.State.ERROR import org.opensearch.alerting.model.AlertingConfigAccessor +import org.opensearch.alerting.model.BucketLevelTrigger +import org.opensearch.alerting.model.BucketLevelTriggerRunResult import org.opensearch.alerting.model.InputRunResults import org.opensearch.alerting.model.Monitor import org.opensearch.alerting.model.MonitorRunResult -import org.opensearch.alerting.model.Trigger -import org.opensearch.alerting.model.TriggerRunResult +import org.opensearch.alerting.model.QueryLevelTrigger +import org.opensearch.alerting.model.QueryLevelTriggerRunResult import org.opensearch.alerting.model.action.Action import org.opensearch.alerting.model.action.Action.Companion.MESSAGE import org.opensearch.alerting.model.action.Action.Companion.MESSAGE_ID import org.opensearch.alerting.model.action.Action.Companion.SUBJECT +import org.opensearch.alerting.model.action.ActionExecutionScope +import org.opensearch.alerting.model.action.AlertCategory +import org.opensearch.alerting.model.action.PerAlertActionScope +import org.opensearch.alerting.model.action.PerExecutionActionScope import org.opensearch.alerting.model.destination.DestinationContextFactory +import org.opensearch.alerting.script.BucketLevelTriggerExecutionContext +import org.opensearch.alerting.script.QueryLevelTriggerExecutionContext import org.opensearch.alerting.script.TriggerExecutionContext -import org.opensearch.alerting.script.TriggerScript import org.opensearch.alerting.settings.AlertingSettings.Companion.ALERT_BACKOFF_COUNT import org.opensearch.alerting.settings.AlertingSettings.Companion.ALERT_BACKOFF_MILLIS +import org.opensearch.alerting.settings.AlertingSettings.Companion.DEFAULT_MAX_ACTIONABLE_ALERT_COUNT +import org.opensearch.alerting.settings.AlertingSettings.Companion.MAX_ACTIONABLE_ALERT_COUNT import org.opensearch.alerting.settings.AlertingSettings.Companion.MOVE_ALERTS_BACKOFF_COUNT import org.opensearch.alerting.settings.AlertingSettings.Companion.MOVE_ALERTS_BACKOFF_MILLIS +import org.opensearch.alerting.settings.DestinationSettings import org.opensearch.alerting.settings.DestinationSettings.Companion.ALLOW_LIST +import org.opensearch.alerting.settings.DestinationSettings.Companion.ALLOW_LIST_NONE import org.opensearch.alerting.settings.DestinationSettings.Companion.HOST_DENY_LIST import org.opensearch.alerting.settings.DestinationSettings.Companion.loadDestinationSettings -import org.opensearch.alerting.util.IndexUtils -import org.opensearch.alerting.util.addUserBackendRolesFilter +import org.opensearch.alerting.settings.LegacyOpenDistroDestinationSettings.Companion.HOST_DENY_LIST_NONE +import org.opensearch.alerting.util.getActionExecutionPolicy +import org.opensearch.alerting.util.getBucketKeysHash +import org.opensearch.alerting.util.getCombinedTriggerRunResult import org.opensearch.alerting.util.isADMonitor import org.opensearch.alerting.util.isAllowed +import org.opensearch.alerting.util.isBucketLevelMonitor import org.opensearch.client.Client import org.opensearch.cluster.service.ClusterService import org.opensearch.common.Strings -import org.opensearch.common.bytes.BytesReference import org.opensearch.common.component.AbstractLifecycleComponent import org.opensearch.common.settings.Settings -import org.opensearch.common.xcontent.LoggingDeprecationHandler import org.opensearch.common.xcontent.NamedXContentRegistry -import org.opensearch.common.xcontent.ToXContent -import org.opensearch.common.xcontent.XContentFactory -import org.opensearch.common.xcontent.XContentHelper -import org.opensearch.common.xcontent.XContentParser -import org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken -import org.opensearch.common.xcontent.XContentType -import org.opensearch.index.query.QueryBuilders -import org.opensearch.rest.RestStatus import org.opensearch.script.Script import org.opensearch.script.ScriptService -import org.opensearch.script.ScriptType import org.opensearch.script.TemplateScript -import org.opensearch.search.builder.SearchSourceBuilder import org.opensearch.threadpool.ThreadPool import java.time.Instant import kotlin.coroutines.CoroutineContext -class MonitorRunner( - private val settings: Settings, - private val client: Client, - private val threadPool: ThreadPool, - private val scriptService: ScriptService, - private val xContentRegistry: NamedXContentRegistry, - private val alertIndices: AlertIndices, - clusterService: ClusterService -) : JobRunner, CoroutineScope, AbstractLifecycleComponent() { +object MonitorRunner : JobRunner, CoroutineScope, AbstractLifecycleComponent() { - private val logger = LogManager.getLogger(MonitorRunner::class.java) + private val logger = LogManager.getLogger(javaClass) + + private lateinit var clusterService: ClusterService + private lateinit var client: Client + private lateinit var xContentRegistry: NamedXContentRegistry + private lateinit var scriptService: ScriptService + private lateinit var settings: Settings + private lateinit var threadPool: ThreadPool + private lateinit var alertIndices: AlertIndices + private lateinit var inputService: InputService + private lateinit var triggerService: TriggerService + private lateinit var alertService: AlertService + + @Volatile private lateinit var retryPolicy: BackoffPolicy + @Volatile private lateinit var moveAlertsRetryPolicy: BackoffPolicy + + @Volatile private var allowList = ALLOW_LIST_NONE + @Volatile private var hostDenyList = HOST_DENY_LIST_NONE + + @Volatile private lateinit var destinationSettings: Map + @Volatile private lateinit var destinationContextFactory: DestinationContextFactory + + @Volatile private var maxActionableAlertCount = DEFAULT_MAX_ACTIONABLE_ALERT_COUNT private lateinit var runnerSupervisor: Job override val coroutineContext: CoroutineContext get() = Dispatchers.Default + runnerSupervisor - @Volatile private var retryPolicy = - BackoffPolicy.constantBackoff(ALERT_BACKOFF_MILLIS.get(settings), ALERT_BACKOFF_COUNT.get(settings)) - @Volatile private var moveAlertsRetryPolicy = - BackoffPolicy.exponentialBackoff(MOVE_ALERTS_BACKOFF_MILLIS.get(settings), MOVE_ALERTS_BACKOFF_COUNT.get(settings)) - @Volatile private var allowList = ALLOW_LIST.get(settings) + fun registerClusterService(clusterService: ClusterService): MonitorRunner { + this.clusterService = clusterService + return this + } - @Volatile private var hostDenyList = HOST_DENY_LIST.get(settings) + fun registerClient(client: Client): MonitorRunner { + this.client = client + return this + } - @Volatile private var destinationSettings = loadDestinationSettings(settings) - @Volatile private var destinationContextFactory = DestinationContextFactory(client, xContentRegistry, destinationSettings) + fun registerNamedXContentRegistry(xContentRegistry: NamedXContentRegistry): MonitorRunner { + this.xContentRegistry = xContentRegistry + return this + } - init { - clusterService.clusterSettings.addSettingsUpdateConsumer(ALERT_BACKOFF_MILLIS, ALERT_BACKOFF_COUNT) { - millis, count -> + fun registerScriptService(scriptService: ScriptService): MonitorRunner { + this.scriptService = scriptService + return this + } + + fun registerSettings(settings: Settings): MonitorRunner { + this.settings = settings + return this + } + + fun registerThreadPool(threadPool: ThreadPool): MonitorRunner { + this.threadPool = threadPool + return this + } + + fun registerAlertIndices(alertIndices: AlertIndices): MonitorRunner { + this.alertIndices = alertIndices + return this + } + + fun registerInputService(inputService: InputService): MonitorRunner { + this.inputService = inputService + return this + } + + fun registerTriggerService(triggerService: TriggerService): MonitorRunner { + this.triggerService = triggerService + return this + } + + fun registerAlertService(alertService: AlertService): MonitorRunner { + this.alertService = alertService + return this + } + + // Must be called after registerClusterService and registerSettings in AlertingPlugin + fun registerConsumers(): MonitorRunner { + retryPolicy = BackoffPolicy.constantBackoff(ALERT_BACKOFF_MILLIS.get(settings), ALERT_BACKOFF_COUNT.get(settings)) + clusterService.clusterSettings.addSettingsUpdateConsumer(ALERT_BACKOFF_MILLIS, ALERT_BACKOFF_COUNT) { millis, count -> retryPolicy = BackoffPolicy.constantBackoff(millis, count) } - clusterService.clusterSettings.addSettingsUpdateConsumer(MOVE_ALERTS_BACKOFF_MILLIS, MOVE_ALERTS_BACKOFF_COUNT) { - millis, count -> + + moveAlertsRetryPolicy = + BackoffPolicy.exponentialBackoff(MOVE_ALERTS_BACKOFF_MILLIS.get(settings), MOVE_ALERTS_BACKOFF_COUNT.get(settings)) + clusterService.clusterSettings.addSettingsUpdateConsumer(MOVE_ALERTS_BACKOFF_MILLIS, MOVE_ALERTS_BACKOFF_COUNT) { millis, count -> moveAlertsRetryPolicy = BackoffPolicy.exponentialBackoff(millis, count) } + + allowList = ALLOW_LIST.get(settings) clusterService.clusterSettings.addSettingsUpdateConsumer(ALLOW_LIST) { allowList = it } + + // Host deny list is not a dynamic setting so no consumer is registered but the variable is set here + hostDenyList = HOST_DENY_LIST.get(settings) + + maxActionableAlertCount = MAX_ACTIONABLE_ALERT_COUNT.get(settings) + clusterService.clusterSettings.addSettingsUpdateConsumer(MAX_ACTIONABLE_ALERT_COUNT) { + maxActionableAlertCount = it + } + + return this + } + + // To be safe, call this last as it depends on a number of other components being registered beforehand (client, settings, etc.) + fun registerDestinationSettings(): MonitorRunner { + destinationSettings = loadDestinationSettings(settings) + destinationContextFactory = DestinationContextFactory(client, xContentRegistry, destinationSettings) + return this } - /** Update destination settings when the reload API is called so that new keystore values are visible */ + // Updates destination settings when the reload API is called so that new keystore values are visible fun reloadDestinationSettings(settings: Settings) { destinationSettings = loadDestinationSettings(settings) @@ -207,35 +265,29 @@ class MonitorRunner( throw IllegalArgumentException("Invalid job type") } - launch { runMonitor(job, periodStart, periodEnd) } + launch { + if (job.isBucketLevelMonitor()) { + runBucketLevelMonitor(job, periodStart, periodEnd) + } else { + runQueryLevelMonitor(job, periodStart, periodEnd) + } + } } - suspend fun runMonitor(monitor: Monitor, periodStart: Instant, periodEnd: Instant, dryrun: Boolean = false): MonitorRunResult { - /* - * We need to handle 3 cases: - * 1. Monitors created by older versions and never updated. These monitors wont have User details in the - * monitor object. `monitor.user` will be null. Insert `all_access, AmazonES_all_access` role. - * 2. Monitors are created when security plugin is disabled, these will have empty User object. - * (`monitor.user.name`, `monitor.user.roles` are empty ) - * 3. Monitors are created when security plugin is enabled, these will have an User object. - */ - var roles = if (monitor.user == null) { - // fixme: discuss and remove hardcoded to settings? - settings.getAsList("", listOf("all_access", "AmazonES_all_access")) - } else { - monitor.user.roles - } + suspend fun runQueryLevelMonitor(monitor: Monitor, periodStart: Instant, periodEnd: Instant, dryrun: Boolean = false): + MonitorRunResult { + val roles = getRolesForMonitor(monitor) logger.debug("Running monitor: ${monitor.name} with roles: $roles Thread: ${Thread.currentThread().name}") if (periodStart == periodEnd) { logger.warn("Start and end time are the same: $periodStart. This monitor will probably only run once.") } - var monitorResult = MonitorRunResult(monitor.name, periodStart, periodEnd) + var monitorResult = MonitorRunResult(monitor.name, periodStart, periodEnd) val currentAlerts = try { alertIndices.createOrUpdateAlertIndex() alertIndices.createOrUpdateInitialHistoryIndex() - loadCurrentAlerts(monitor) + alertService.loadCurrentAlertsForQueryLevelMonitor(monitor) } catch (e: Exception) { // We can't save ERROR alerts to the index here as we don't know if there are existing ACTIVE alerts val id = if (monitor.id.trim().isEmpty()) "_na_" else monitor.id @@ -244,312 +296,338 @@ class MonitorRunner( } if (!isADMonitor(monitor)) { runBlocking(InjectorContextElement(monitor.id, settings, threadPool.threadContext, roles)) { - monitorResult = monitorResult.copy(inputResults = collectInputResults(monitor, periodStart, periodEnd)) + monitorResult = monitorResult.copy(inputResults = inputService.collectInputResults(monitor, periodStart, periodEnd)) } } else { - monitorResult = monitorResult.copy(inputResults = collectInputResultsForADMonitor(monitor, periodStart, periodEnd)) + monitorResult = monitorResult.copy(inputResults = inputService.collectInputResultsForADMonitor(monitor, periodStart, periodEnd)) } val updatedAlerts = mutableListOf() - val triggerResults = mutableMapOf() + val triggerResults = mutableMapOf() for (trigger in monitor.triggers) { val currentAlert = currentAlerts[trigger] - val triggerCtx = TriggerExecutionContext(monitor, trigger, monitorResult, currentAlert) - val triggerResult = runTrigger(monitor, trigger, triggerCtx) + val triggerCtx = QueryLevelTriggerExecutionContext(monitor, trigger as QueryLevelTrigger, monitorResult, currentAlert) + val triggerResult = triggerService.runQueryLevelTrigger(monitor, trigger, triggerCtx) triggerResults[trigger.id] = triggerResult - if (isTriggerActionable(triggerCtx, triggerResult)) { + if (triggerService.isQueryLevelTriggerActionable(triggerCtx, triggerResult)) { val actionCtx = triggerCtx.copy(error = monitorResult.error ?: triggerResult.error) for (action in trigger.actions) { triggerResult.actionResults[action.id] = runAction(action, actionCtx, dryrun) } } - val updatedAlert = composeAlert(triggerCtx, triggerResult, monitorResult.alertError() ?: triggerResult.alertError()) + val updatedAlert = alertService.composeQueryLevelAlert( + triggerCtx, triggerResult, + monitorResult.alertError() ?: triggerResult.alertError() + ) if (updatedAlert != null) updatedAlerts += updatedAlert } // Don't save alerts if this is a test monitor if (!dryrun && monitor.id != Monitor.NO_ID) { - saveAlerts(updatedAlerts) + alertService.saveAlerts(updatedAlerts, retryPolicy) } return monitorResult.copy(triggerResults = triggerResults) } - private fun currentTime() = Instant.ofEpochMilli(threadPool.absoluteTimeInMillis()) + // TODO: This method has grown very large with all the business logic that has been added. + // Revisit this during refactoring and break it down to be more manageable. + suspend fun runBucketLevelMonitor( + monitor: Monitor, + periodStart: Instant, + periodEnd: Instant, + dryrun: Boolean = false + ): MonitorRunResult { + val roles = getRolesForMonitor(monitor) + logger.debug("Running monitor: ${monitor.name} with roles: $roles Thread: ${Thread.currentThread().name}") - private fun composeAlert(ctx: TriggerExecutionContext, result: TriggerRunResult, alertError: AlertError?): Alert? { - val currentTime = currentTime() - val currentAlert = ctx.alert - - val updatedActionExecutionResults = mutableListOf() - val currentActionIds = mutableSetOf() - if (currentAlert != null) { - // update current alert's action execution results - for (actionExecutionResult in currentAlert.actionExecutionResults) { - val actionId = actionExecutionResult.actionId - currentActionIds.add(actionId) - val actionRunResult = result.actionResults[actionId] - when { - actionRunResult == null -> updatedActionExecutionResults.add(actionExecutionResult) - actionRunResult.throttled -> - updatedActionExecutionResults.add( - actionExecutionResult.copy( - throttledCount = actionExecutionResult.throttledCount + 1 - ) - ) - else -> updatedActionExecutionResults.add(actionExecutionResult.copy(lastExecutionTime = actionRunResult.executionTime)) + if (periodStart == periodEnd) { + logger.warn("Start and end time are the same: $periodStart. This monitor will probably only run once.") + } + + var monitorResult = MonitorRunResult(monitor.name, periodStart, periodEnd) + val currentAlerts = try { + alertIndices.createOrUpdateAlertIndex() + alertIndices.createOrUpdateInitialHistoryIndex() + alertService.loadCurrentAlertsForBucketLevelMonitor(monitor) + } catch (e: Exception) { + // We can't save ERROR alerts to the index here as we don't know if there are existing ACTIVE alerts + val id = if (monitor.id.trim().isEmpty()) "_na_" else monitor.id + logger.error("Error loading alerts for monitor: $id", e) + return monitorResult.copy(error = e) + } + + /* + * Since the aggregation query can consist of multiple pages, each iteration of the do-while loop only has partial results + * from the runBucketLevelTrigger results whereas the currentAlerts has a complete view of existing Alerts. This means that + * it can be confirmed if an Alert is new or de-duped local to the do-while loop if a key appears or doesn't appear in + * the currentAlerts. However, it cannot be guaranteed that an existing Alert is COMPLETED until all pages have been + * iterated over (since a bucket that did not appear in one page of the aggregation results, could appear in a later page). + * + * To solve for this, the currentAlerts will be acting as a list of "potentially completed alerts" throughout the execution. + * When categorizing the Alerts in each iteration, de-duped Alerts will be removed from the currentAlerts map + * (for the Trigger being executed) and the Alerts left in currentAlerts after all pages have been iterated through can + * be marked as COMPLETED since they were never de-duped. + * + * Meanwhile, the nextAlerts map will contain Alerts that will exist at the end of this Monitor execution. It is a compilation + * across Triggers because in the case of executing actions at a PER_EXECUTION frequency, all the Alerts are needed before executing + * Actions which can only be done once all of the aggregation results (and Triggers given the pagination logic) have been evaluated. + */ + val triggerResults = mutableMapOf() + val triggerContexts = mutableMapOf() + val nextAlerts = mutableMapOf>>() + var firstIteration = true + var firstPageOfInputResults = InputRunResults(listOf(), null) + do { + // TODO: Since a composite aggregation is being used for the input query, the total bucket count cannot be determined. + // If a setting is imposed that limits buckets that can be processed for Bucket-Level Monitors, we'd need to iterate over + // the buckets until we hit that threshold. In that case, we'd want to exit the execution without creating any alerts since the + // buckets we iterate over before hitting the limit is not deterministic. Is there a better way to fail faster in this case? + runBlocking(InjectorContextElement(monitor.id, settings, threadPool.threadContext, roles)) { + // Storing the first page of results in the case of pagination input results to prevent empty results + // in the final output of monitorResult which occurs when all pages have been exhausted. + // If it's favorable to return the last page, will need to check how to accomplish that with multiple aggregation paths + // with different page counts. + val inputResults = inputService.collectInputResults(monitor, periodStart, periodEnd, monitorResult.inputResults) + if (firstIteration) { + firstPageOfInputResults = inputResults + firstIteration = false } + monitorResult = monitorResult.copy(inputResults = inputResults) } - // add action execution results which not exist in current alert - updatedActionExecutionResults.addAll( - result.actionResults.filter { it -> !currentActionIds.contains(it.key) } - .map { it -> ActionExecutionResult(it.key, it.value.executionTime, if (it.value.throttled) 1 else 0) } - ) - } else { - updatedActionExecutionResults.addAll( - result.actionResults.map { it -> - ActionExecutionResult( - it.key, it.value.executionTime, - if (it.value.throttled) 1 else 0 + + for (trigger in monitor.triggers) { + // The currentAlerts map is formed by iterating over the Monitor's Triggers as keys so null should not be returned here + val currentAlertsForTrigger = currentAlerts[trigger]!! + val triggerCtx = BucketLevelTriggerExecutionContext(monitor, trigger as BucketLevelTrigger, monitorResult) + triggerContexts[trigger.id] = triggerCtx + val triggerResult = triggerService.runBucketLevelTrigger(monitor, trigger, triggerCtx) + triggerResults[trigger.id] = triggerResult.getCombinedTriggerRunResult(triggerResults[trigger.id]) + + /* + * If an error was encountered when running the trigger, it means that something went wrong when parsing the input results + * for the filtered buckets returned from the pipeline bucket selector injected into the input query. + * + * In this case, the returned aggregation result buckets are empty so the categorization of the Alerts that happens below + * should be skipped/invalidated since comparing the current Alerts to an empty result will lead the execution to believe + * that all Alerts have been COMPLETED. Not doing so would mean it would not be possible to propagate the error into the + * existing Alerts in a way the user can easily view them since they will have all been moved to the history index. + */ + if (triggerResults[trigger.id]?.error != null) continue + + // TODO: Should triggerResult's aggregationResultBucket be a list? If not, getCategorizedAlertsForBucketLevelMonitor can + // be refactored to use a map instead + val categorizedAlerts = alertService.getCategorizedAlertsForBucketLevelMonitor( + monitor, trigger, currentAlertsForTrigger, triggerResult.aggregationResultBuckets.values.toList() + ).toMutableMap() + val dedupedAlerts = categorizedAlerts.getOrDefault(AlertCategory.DEDUPED, emptyList()) + var newAlerts = categorizedAlerts.getOrDefault(AlertCategory.NEW, emptyList()) + + /* + * Index de-duped and new Alerts here (if it's not a test Monitor) so they are available at the time the Actions are executed. + * + * The new Alerts have to be returned and saved back with their indexed doc ID to prevent duplicate documents + * when the Alerts are updated again after Action execution. + * + * Note: Index operations can fail for various reasons (such as write blocks on cluster), in such a case, the Actions + * will still execute with the Alert information in the ctx but the Alerts may not be visible. + */ + if (!dryrun && monitor.id != Monitor.NO_ID) { + alertService.saveAlerts(dedupedAlerts, retryPolicy, allowUpdatingAcknowledgedAlert = true) + newAlerts = alertService.saveNewAlerts(newAlerts, retryPolicy) + } + + // Store deduped and new Alerts to accumulate across pages + if (!nextAlerts.containsKey(trigger.id)) { + nextAlerts[trigger.id] = mutableMapOf( + AlertCategory.DEDUPED to mutableListOf(), + AlertCategory.NEW to mutableListOf(), + AlertCategory.COMPLETED to mutableListOf() ) } - ) + nextAlerts[trigger.id]?.get(AlertCategory.DEDUPED)?.addAll(dedupedAlerts) + nextAlerts[trigger.id]?.get(AlertCategory.NEW)?.addAll(newAlerts) + } + } while (monitorResult.inputResults.afterKeysPresent()) + + // The completed Alerts are whatever are left in the currentAlerts. + // However, this operation will only be done if there was no trigger error, since otherwise the nextAlerts were not collected + // in favor of just using the currentAlerts as-is. + currentAlerts.forEach { (trigger, keysToAlertsMap) -> + if (triggerResults[trigger.id]?.error == null) + nextAlerts[trigger.id]?.get(AlertCategory.COMPLETED)?.addAll(alertService.convertToCompletedAlerts(keysToAlertsMap)) } - // Merge the alert's error message to the current alert's history - val updatedHistory = currentAlert?.errorHistory.update(alertError) - return if (alertError == null && !result.triggered) { - currentAlert?.copy( - state = COMPLETED, endTime = currentTime, errorMessage = null, - errorHistory = updatedHistory, actionExecutionResults = updatedActionExecutionResults, - schemaVersion = IndexUtils.alertIndexSchemaVersion - ) - } else if (alertError == null && currentAlert?.isAcknowledged() == true) { - null - } else if (currentAlert != null) { - val alertState = if (alertError == null) ACTIVE else ERROR - currentAlert.copy( - state = alertState, lastNotificationTime = currentTime, errorMessage = alertError?.message, - errorHistory = updatedHistory, actionExecutionResults = updatedActionExecutionResults, - schemaVersion = IndexUtils.alertIndexSchemaVersion - ) - } else { - val alertState = if (alertError == null) ACTIVE else ERROR - Alert( - monitor = ctx.monitor, trigger = ctx.trigger, startTime = currentTime, - lastNotificationTime = currentTime, state = alertState, errorMessage = alertError?.message, - errorHistory = updatedHistory, actionExecutionResults = updatedActionExecutionResults, - schemaVersion = IndexUtils.alertIndexSchemaVersion + for (trigger in monitor.triggers) { + val alertsToUpdate = mutableSetOf() + val completedAlertsToUpdate = mutableSetOf() + // Filter ACKNOWLEDGED Alerts from the deduped list so they do not have Actions executed for them. + // New Alerts are ignored since they cannot be acknowledged yet. + val dedupedAlerts = nextAlerts[trigger.id]?.get(AlertCategory.DEDUPED) + ?.filterNot { it.state == Alert.State.ACKNOWLEDGED }?.toMutableList() + ?: mutableListOf() + // Update nextAlerts so the filtered DEDUPED Alerts are reflected for PER_ALERT Action execution + nextAlerts[trigger.id]?.set(AlertCategory.DEDUPED, dedupedAlerts) + val newAlerts = nextAlerts[trigger.id]?.get(AlertCategory.NEW) ?: mutableListOf() + val completedAlerts = nextAlerts[trigger.id]?.get(AlertCategory.COMPLETED) ?: mutableListOf() + + // Adding all the COMPLETED Alerts to a separate set and removing them if they get added + // to alertsToUpdate to ensure the Alert doc is updated at the end in either case + completedAlertsToUpdate.addAll(completedAlerts) + + // All trigger contexts and results should be available at this point since all triggers were evaluated in the main do-while loop + val triggerCtx = triggerContexts[trigger.id]!! + val triggerResult = triggerResults[trigger.id]!! + val monitorOrTriggerError = monitorResult.error ?: triggerResult.error + val shouldDefaultToPerExecution = defaultToPerExecutionAction( + monitorId = monitor.id, + triggerId = trigger.id, + totalActionableAlertCount = dedupedAlerts.size + newAlerts.size + completedAlerts.size, + monitorOrTriggerError = monitorOrTriggerError ) - } - } - - private suspend fun collectInputResults(monitor: Monitor, periodStart: Instant, periodEnd: Instant): InputRunResults { - return try { - val results = mutableListOf>() - monitor.inputs.forEach { input -> - when (input) { - is SearchInput -> { - // TODO: Figure out a way to use SearchTemplateRequest without bringing in the entire TransportClient - val searchParams = mapOf( - "period_start" to periodStart.toEpochMilli(), - "period_end" to periodEnd.toEpochMilli() - ) - val searchSource = scriptService.compile( - Script( - ScriptType.INLINE, Script.DEFAULT_TEMPLATE_LANG, - input.query.toString(), searchParams - ), - TemplateScript.CONTEXT - ) - .newInstance(searchParams) - .execute() - - val searchRequest = SearchRequest().indices(*input.indices.toTypedArray()) - XContentType.JSON.xContent().createParser(xContentRegistry, LoggingDeprecationHandler.INSTANCE, searchSource).use { - searchRequest.source(SearchSourceBuilder.fromXContent(it)) + for (action in trigger.actions) { + // ActionExecutionPolicy should not be null for Bucket-Level Monitors since it has a default config when not set explicitly + val actionExecutionScope = action.getActionExecutionPolicy(monitor)!!.actionExecutionScope + if (actionExecutionScope is PerAlertActionScope && !shouldDefaultToPerExecution) { + for (alertCategory in actionExecutionScope.actionableAlerts) { + val alertsToExecuteActionsFor = nextAlerts[trigger.id]?.get(alertCategory) ?: mutableListOf() + for (alert in alertsToExecuteActionsFor) { + val actionCtx = getActionContextForAlertCategory( + alertCategory, alert, triggerCtx, monitorOrTriggerError + ) + // AggregationResultBucket should not be null here + val alertBucketKeysHash = alert.aggregationResultBucket!!.getBucketKeysHash() + if (!triggerResult.actionResultsMap.containsKey(alertBucketKeysHash)) { + triggerResult.actionResultsMap[alertBucketKeysHash] = mutableMapOf() + } + + // Keeping the throttled response separate from runAction for now since + // throttling is not supported for PER_EXECUTION + val actionResult = if (isActionActionable(action, alert)) { + runAction(action, actionCtx, dryrun) + } else { + ActionRunResult(action.id, action.name, mapOf(), true, null, null) + } + + triggerResult.actionResultsMap[alertBucketKeysHash]?.set(action.id, actionResult) + alertsToUpdate.add(alert) + // Remove the alert from completedAlertsToUpdate in case it is present there since + // its update will be handled in the alertsToUpdate batch + completedAlertsToUpdate.remove(alert) } - val searchResponse: SearchResponse = client.suspendUntil { client.search(searchRequest, it) } - results += searchResponse.convertToMap() } - else -> { - throw IllegalArgumentException("Unsupported input type: ${input.name()}.") + } else if (actionExecutionScope is PerExecutionActionScope || shouldDefaultToPerExecution) { + // If all categories of Alerts are empty, there is nothing to message on and we can skip the Action. + // If the error is not null, this is disregarded and the Action is executed anyway so the user can be notified. + if (monitorOrTriggerError == null && dedupedAlerts.isEmpty() && newAlerts.isEmpty() && completedAlerts.isEmpty()) + continue + + val actionCtx = triggerCtx.copy( + dedupedAlerts = dedupedAlerts, + newAlerts = newAlerts, + completedAlerts = completedAlerts, + error = monitorResult.error ?: triggerResult.error + ) + val actionResult = runAction(action, actionCtx, dryrun) + // If there was an error during trigger execution then the Alerts to be updated are the current Alerts since the state + // was not changed. Otherwise, the Alerts to be updated are the sum of the deduped, new and completed Alerts. + val alertsToIterate = if (monitorOrTriggerError == null) { + (dedupedAlerts + newAlerts + completedAlerts) + } else currentAlerts[trigger]?.map { it.value } ?: listOf() + // Save the Action run result for every Alert + for (alert in alertsToIterate) { + val alertBucketKeysHash = alert.aggregationResultBucket!!.getBucketKeysHash() + if (!triggerResult.actionResultsMap.containsKey(alertBucketKeysHash)) { + triggerResult.actionResultsMap[alertBucketKeysHash] = mutableMapOf() + } + triggerResult.actionResultsMap[alertBucketKeysHash]?.set(action.id, actionResult) + alertsToUpdate.add(alert) + // Remove the alert from completedAlertsToUpdate in case it is present there since + // its update will be handled in the alertsToUpdate batch + completedAlertsToUpdate.remove(alert) } } } - InputRunResults(results.toList()) - } catch (e: Exception) { - logger.info("Error collecting inputs for monitor: ${monitor.id}", e) - InputRunResults(emptyList(), e) - } - } - /** - * We moved anomaly result index to system index list. So common user could not directly query - * this index any more. This method will stash current thread context to pass security check. - * So monitor job can access anomaly result index. We will add monitor user roles filter in - * search query to only return documents the monitor user can access. - * - * On alerting Kibana, monitor users can only see detectors that they have read access. So they - * can't create monitor on other user's detector which they have no read access. Even they know - * other user's detector id and use it to create monitor, this method will only return anomaly - * results they can read. - */ - private suspend fun collectInputResultsForADMonitor(monitor: Monitor, periodStart: Instant, periodEnd: Instant): InputRunResults { - return try { - val results = mutableListOf>() - val input = monitor.inputs[0] as SearchInput - - val searchParams = mapOf("period_start" to periodStart.toEpochMilli(), "period_end" to periodEnd.toEpochMilli()) - val searchSource = scriptService.compile( - Script( - ScriptType.INLINE, Script.DEFAULT_TEMPLATE_LANG, - input.query.toString(), searchParams - ), - TemplateScript.CONTEXT - ) - .newInstance(searchParams) - .execute() - - val searchRequest = SearchRequest().indices(*input.indices.toTypedArray()) - XContentType.JSON.xContent().createParser(xContentRegistry, LoggingDeprecationHandler.INSTANCE, searchSource).use { - searchRequest.source(SearchSourceBuilder.fromXContent(it)) + // Alerts are only added to alertsToUpdate after Action execution meaning the action results for it should be present + // in the actionResultsMap but returning a default value when accessing the map to be safe. + val updatedAlerts = alertsToUpdate.map { alert -> + val bucketKeysHash = alert.aggregationResultBucket!!.getBucketKeysHash() + val actionResults = triggerResult.actionResultsMap.getOrDefault(bucketKeysHash, emptyMap()) + alertService.updateActionResultsForBucketLevelAlert( + alert.copy(lastNotificationTime = currentTime()), + actionResults, + // TODO: Update BucketLevelTriggerRunResult.alertError() to retrieve error based on the first failed Action + monitorResult.alertError() ?: triggerResult.alertError() + ) } - // Add user role filter for AD result - client.threadPool().threadContext.stashContext().use { - // Currently we have no way to verify if user has AD read permission or not. So we always add user - // role filter here no matter AD backend role filter enabled or not. If we don't add user role filter - // when AD backend filter disabled, user can run monitor on any detector and get anomaly data even - // they have no AD read permission. So if domain disabled AD backend role filter, monitor runner - // still can't get AD result with different user backend role, even the monitor user has permission - // to read AD result. This is a short term solution to trade off between user experience and security. - // - // Possible long term solution: - // 1.Use secure rest client to send request to AD search result API. If no permission exception, - // that mean user has read access on AD result. Then don't need to add user role filter when query - // AD result if AD backend role filter is disabled. - // 2.Security provide some transport action to verify if user has permission to search AD result. - // Monitor runner will send transport request to check permission first. If security plugin response - // is yes, user has permission to query AD result. If AD role filter enabled, we will add user role - // filter to protect data at user role level; otherwise, user can query any AD result. - addUserBackendRolesFilter(monitor.user, searchRequest.source()) - val searchResponse: SearchResponse = client.suspendUntil { client.search(searchRequest, it) } - results += searchResponse.convertToMap() + // Update Alerts with action execution results (if it's not a test Monitor). + // ACKNOWLEDGED Alerts should not be saved here since actions are not executed for them. + if (!dryrun && monitor.id != Monitor.NO_ID) { + alertService.saveAlerts(updatedAlerts, retryPolicy, allowUpdatingAcknowledgedAlert = false) + // Save any COMPLETED Alerts that were not covered in updatedAlerts + alertService.saveAlerts(completedAlertsToUpdate.toList(), retryPolicy, allowUpdatingAcknowledgedAlert = false) } - InputRunResults(results.toList()) - } catch (e: Exception) { - logger.info("Error collecting anomaly result inputs for monitor: ${monitor.id}", e) - InputRunResults(emptyList(), e) } - } - private fun runTrigger(monitor: Monitor, trigger: Trigger, ctx: TriggerExecutionContext): TriggerRunResult { - return try { - val triggered = scriptService.compile(trigger.condition, TriggerScript.CONTEXT) - .newInstance(trigger.condition.params) - .execute(ctx) - TriggerRunResult(trigger.name, triggered, null) - } catch (e: Exception) { - logger.info("Error running script for monitor ${monitor.id}, trigger: ${trigger.id}", e) - // if the script fails we need to send an alert so set triggered = true - TriggerRunResult(trigger.name, true, e) - } + return monitorResult.copy(inputResults = firstPageOfInputResults, triggerResults = triggerResults) } - private suspend fun loadCurrentAlerts(monitor: Monitor): Map { - val request = SearchRequest(AlertIndices.ALERT_INDEX) - .routing(monitor.id) - .source(alertQuery(monitor)) - val response: SearchResponse = client.suspendUntil { client.search(request, it) } - if (response.status() != RestStatus.OK) { - throw (response.firstFailureOrNull()?.cause ?: RuntimeException("Unknown error loading alerts")) + private fun defaultToPerExecutionAction( + monitorId: String, + triggerId: String, + totalActionableAlertCount: Int, + monitorOrTriggerError: Exception? + ): Boolean { + // If the monitorId or triggerResult has an error, then also default to PER_EXECUTION to communicate the error + if (monitorOrTriggerError != null) { + logger.debug( + "Trigger [$triggerId] in monitor [$monitorId] encountered an error. Defaulting to " + + "[${ActionExecutionScope.Type.PER_EXECUTION}] for action execution to communicate error." + ) + return true } - val foundAlerts = response.hits.map { Alert.parse(contentParser(it.sourceRef), it.id, it.version) } - .groupBy { it.triggerId } - foundAlerts.values.forEach { alerts -> - if (alerts.size > 1) { - logger.warn("Found multiple alerts for same trigger: $alerts") - } - } + // If the MAX_ACTIONABLE_ALERT_COUNT is set to -1, consider it unbounded and proceed regardless of actionable Alert count + if (maxActionableAlertCount < 0) return false - return monitor.triggers.associate { trigger -> - trigger to (foundAlerts[trigger.id]?.firstOrNull()) + // If the total number of Alerts to execute Actions on exceeds the MAX_ACTIONABLE_ALERT_COUNT setting then default to + // PER_EXECUTION for less intrusive Actions + if (totalActionableAlertCount > maxActionableAlertCount) { + logger.debug( + "The total actionable alerts for trigger [$triggerId] in monitor [$monitorId] is [$totalActionableAlertCount] " + + "which exceeds the maximum of [$maxActionableAlertCount]. Defaulting to [${ActionExecutionScope.Type.PER_EXECUTION}] " + + "for action execution." + ) + return true } - } - - private fun contentParser(bytesReference: BytesReference): XContentParser { - val xcp = XContentHelper.createParser( - xContentRegistry, LoggingDeprecationHandler.INSTANCE, - bytesReference, XContentType.JSON - ) - ensureExpectedToken(XContentParser.Token.START_OBJECT, xcp.nextToken(), xcp) - return xcp - } - private fun alertQuery(monitor: Monitor): SearchSourceBuilder { - return SearchSourceBuilder.searchSource() - .size(monitor.triggers.size * 2) // We expect there to be only a single in-progress alert so fetch 2 to check - .query(QueryBuilders.termQuery(Alert.MONITOR_ID_FIELD, monitor.id)) + return false } - private suspend fun saveAlerts(alerts: List) { - var requestsToRetry = alerts.flatMap { alert -> - // we don't want to set the version when saving alerts because the Runner has first priority when writing alerts. - // In the rare event that a user acknowledges an alert between when it's read and when it's written - // back we're ok if that acknowledgement is lost. It's easier to get the user to retry than for the runner to - // spend time reloading the alert and writing it back. - when (alert.state) { - ACTIVE, ERROR -> { - listOf>( - IndexRequest(AlertIndices.ALERT_INDEX) - .routing(alert.monitorId) - .source(alert.toXContent(XContentFactory.jsonBuilder(), ToXContent.EMPTY_PARAMS)) - .id(if (alert.id != Alert.NO_ID) alert.id else null) - ) - } - ACKNOWLEDGED, DELETED -> { - throw IllegalStateException("Unexpected attempt to save ${alert.state} alert: $alert") - } - COMPLETED -> { - listOfNotNull>( - DeleteRequest(AlertIndices.ALERT_INDEX, alert.id) - .routing(alert.monitorId), - // Only add completed alert to history index if history is enabled - if (alertIndices.isHistoryEnabled()) { - IndexRequest(AlertIndices.HISTORY_WRITE_INDEX) - .routing(alert.monitorId) - .source(alert.toXContent(XContentFactory.jsonBuilder(), ToXContent.EMPTY_PARAMS)) - .id(alert.id) - } else null - ) - } - } - } - - if (requestsToRetry.isEmpty()) return - // Retry Bulk requests if there was any 429 response - retryPolicy.retry(logger, listOf(RestStatus.TOO_MANY_REQUESTS)) { - val bulkRequest = BulkRequest().add(requestsToRetry) - val bulkResponse: BulkResponse = client.suspendUntil { client.bulk(bulkRequest, it) } - val failedResponses = (bulkResponse.items ?: arrayOf()).filter { it.isFailed } - requestsToRetry = failedResponses.filter { it.status() == RestStatus.TOO_MANY_REQUESTS } - .map { bulkRequest.requests()[it.itemId] as IndexRequest } - - if (requestsToRetry.isNotEmpty()) { - val retryCause = failedResponses.first { it.status() == RestStatus.TOO_MANY_REQUESTS }.failure.cause - throw ExceptionsHelper.convertToOpenSearchException(retryCause) - } + private fun getRolesForMonitor(monitor: Monitor): List { + /* + * We need to handle 3 cases: + * 1. Monitors created by older versions and never updated. These monitors wont have User details in the + * monitor object. `monitor.user` will be null. Insert `all_access, AmazonES_all_access` role. + * 2. Monitors are created when security plugin is disabled, these will have empty User object. + * (`monitor.user.name`, `monitor.user.roles` are empty ) + * 3. Monitors are created when security plugin is enabled, these will have an User object. + */ + return if (monitor.user == null) { + // fixme: discuss and remove hardcoded to settings? + // TODO: Remove "AmazonES_all_access" role? + settings.getAsList("", listOf("all_access", "AmazonES_all_access")) + } else { + monitor.user.roles } } - private fun isTriggerActionable(ctx: TriggerExecutionContext, result: TriggerRunResult): Boolean { - // Suppress actions if the current alert is acknowledged and there are no errors. - val suppress = ctx.alert?.state == ACKNOWLEDGED && result.error == null && ctx.error == null - return result.triggered && !suppress - } + // TODO: Can this be updated to just use 'Instant.now()'? + // 'threadPool.absoluteTimeInMillis()' is referring to a cached value of System.currentTimeMillis() that by default updates every 200ms + private fun currentTime() = Instant.ofEpochMilli(threadPool.absoluteTimeInMillis()) private fun isActionActionable(action: Action, alert: Alert?): Boolean { if (alert == null || action.throttle == null) { @@ -564,7 +642,23 @@ class MonitorRunner( return true } - private suspend fun runAction(action: Action, ctx: TriggerExecutionContext, dryrun: Boolean): ActionRunResult { + private fun getActionContextForAlertCategory( + alertCategory: AlertCategory, + alert: Alert, + ctx: BucketLevelTriggerExecutionContext, + error: Exception? + ): BucketLevelTriggerExecutionContext { + return when (alertCategory) { + AlertCategory.DEDUPED -> + ctx.copy(dedupedAlerts = listOf(alert), newAlerts = emptyList(), completedAlerts = emptyList(), error = error) + AlertCategory.NEW -> + ctx.copy(dedupedAlerts = emptyList(), newAlerts = listOf(alert), completedAlerts = emptyList(), error = error) + AlertCategory.COMPLETED -> + ctx.copy(dedupedAlerts = emptyList(), newAlerts = emptyList(), completedAlerts = listOf(alert), error = error) + } + } + + private suspend fun runAction(action: Action, ctx: QueryLevelTriggerExecutionContext, dryrun: Boolean): ActionRunResult { return try { if (!isActionActionable(action, ctx.alert)) { return ActionRunResult(action.id, action.name, mapOf(), true, null, null) @@ -597,19 +691,41 @@ class MonitorRunner( } } + // TODO: This is largely a duplicate of runAction above for BucketLevelTriggerExecutionContext for now. + // After suppression logic implementation, if this remains mostly the same, it can be refactored. + private suspend fun runAction(action: Action, ctx: BucketLevelTriggerExecutionContext, dryrun: Boolean): ActionRunResult { + return try { + val actionOutput = mutableMapOf() + actionOutput[SUBJECT] = if (action.subjectTemplate != null) compileTemplate(action.subjectTemplate, ctx) else "" + actionOutput[MESSAGE] = compileTemplate(action.messageTemplate, ctx) + if (Strings.isNullOrEmpty(actionOutput[MESSAGE])) { + throw IllegalStateException("Message content missing in the Destination with id: ${action.destinationId}") + } + if (!dryrun) { + withContext(Dispatchers.IO) { + val destination = AlertingConfigAccessor.getDestinationInfo(client, xContentRegistry, action.destinationId) + if (!destination.isAllowed(allowList)) { + throw IllegalStateException("Monitor contains a Destination type that is not allowed: ${destination.type}") + } + + val destinationCtx = destinationContextFactory.getDestinationContext(destination) + actionOutput[MESSAGE_ID] = destination.publish( + actionOutput[SUBJECT], + actionOutput[MESSAGE]!!, + destinationCtx, + hostDenyList + ) + } + } + ActionRunResult(action.id, action.name, actionOutput, false, currentTime(), null) + } catch (e: Exception) { + ActionRunResult(action.id, action.name, mapOf(), false, currentTime(), e) + } + } + private fun compileTemplate(template: Script, ctx: TriggerExecutionContext): String { return scriptService.compile(template, TemplateScript.CONTEXT) .newInstance(template.params + mapOf("ctx" to ctx.asTemplateArg())) .execute() } - - private fun List?.update(alertError: AlertError?): List { - return when { - this == null && alertError == null -> emptyList() - this != null && alertError == null -> this - this == null && alertError != null -> listOf(alertError) - this != null && alertError != null -> (listOf(alertError) + this).take(10) - else -> throw IllegalStateException("Unreachable code reached!") - } - } } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/TriggerService.kt b/alerting/src/main/kotlin/org/opensearch/alerting/TriggerService.kt new file mode 100644 index 000000000..6fda9e107 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/TriggerService.kt @@ -0,0 +1,110 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting + +import org.apache.logging.log4j.LogManager +import org.opensearch.alerting.aggregation.bucketselectorext.BucketSelectorIndices.Fields.BUCKET_INDICES +import org.opensearch.alerting.aggregation.bucketselectorext.BucketSelectorIndices.Fields.PARENT_BUCKET_PATH +import org.opensearch.alerting.model.AggregationResultBucket +import org.opensearch.alerting.model.Alert +import org.opensearch.alerting.model.BucketLevelTrigger +import org.opensearch.alerting.model.BucketLevelTriggerRunResult +import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.QueryLevelTrigger +import org.opensearch.alerting.model.QueryLevelTriggerRunResult +import org.opensearch.alerting.script.BucketLevelTriggerExecutionContext +import org.opensearch.alerting.script.QueryLevelTriggerExecutionContext +import org.opensearch.alerting.script.TriggerScript +import org.opensearch.alerting.util.getBucketKeysHash +import org.opensearch.script.ScriptService +import org.opensearch.search.aggregations.Aggregation +import org.opensearch.search.aggregations.Aggregations +import org.opensearch.search.aggregations.support.AggregationPath +import java.lang.IllegalArgumentException + +/** Service that handles executing Triggers */ +class TriggerService(val scriptService: ScriptService) { + + private val logger = LogManager.getLogger(TriggerService::class.java) + + fun isQueryLevelTriggerActionable(ctx: QueryLevelTriggerExecutionContext, result: QueryLevelTriggerRunResult): Boolean { + // Suppress actions if the current alert is acknowledged and there are no errors. + val suppress = ctx.alert?.state == Alert.State.ACKNOWLEDGED && result.error == null && ctx.error == null + return result.triggered && !suppress + } + + fun runQueryLevelTrigger( + monitor: Monitor, + trigger: QueryLevelTrigger, + ctx: QueryLevelTriggerExecutionContext + ): QueryLevelTriggerRunResult { + return try { + val triggered = scriptService.compile(trigger.condition, TriggerScript.CONTEXT) + .newInstance(trigger.condition.params) + .execute(ctx) + QueryLevelTriggerRunResult(trigger.name, triggered, null) + } catch (e: Exception) { + logger.info("Error running script for monitor ${monitor.id}, trigger: ${trigger.id}", e) + // if the script fails we need to send an alert so set triggered = true + QueryLevelTriggerRunResult(trigger.name, true, e) + } + } + + @Suppress("UNCHECKED_CAST") + fun runBucketLevelTrigger( + monitor: Monitor, + trigger: BucketLevelTrigger, + ctx: BucketLevelTriggerExecutionContext + ): BucketLevelTriggerRunResult { + return try { + val bucketIndices = + ((ctx.results[0][Aggregations.AGGREGATIONS_FIELD] as HashMap<*, *>)[trigger.id] as HashMap<*, *>)[BUCKET_INDICES] as List<*> + val parentBucketPath = ( + (ctx.results[0][Aggregations.AGGREGATIONS_FIELD] as HashMap<*, *>) + .get(trigger.id) as HashMap<*, *> + )[PARENT_BUCKET_PATH] as String + val aggregationPath = AggregationPath.parse(parentBucketPath) + // TODO test this part by passing sub-aggregation path + var parentAgg = (ctx.results[0][Aggregations.AGGREGATIONS_FIELD] as HashMap<*, *>) + aggregationPath.pathElementsAsStringList.forEach { sub_agg -> + parentAgg = (parentAgg[sub_agg] as HashMap<*, *>) + } + val buckets = parentAgg[Aggregation.CommonFields.BUCKETS.preferredName] as List<*> + val selectedBuckets = mutableMapOf() + for (bucketIndex in bucketIndices) { + val bucketDict = buckets[bucketIndex as Int] as Map + val bucketKeyValuesList = getBucketKeyValuesList(bucketDict) + val aggResultBucket = AggregationResultBucket(parentBucketPath, bucketKeyValuesList, bucketDict) + selectedBuckets[aggResultBucket.getBucketKeysHash()] = aggResultBucket + } + BucketLevelTriggerRunResult(trigger.name, null, selectedBuckets) + } catch (e: Exception) { + logger.info("Error running trigger [${trigger.id}] for monitor [${monitor.id}]", e) + BucketLevelTriggerRunResult(trigger.name, e, emptyMap()) + } + } + + @Suppress("UNCHECKED_CAST") + private fun getBucketKeyValuesList(bucket: Map): List { + val keyField = Aggregation.CommonFields.KEY.preferredName + val keyValuesList = mutableListOf() + when { + bucket[keyField] is String -> keyValuesList.add(bucket[keyField] as String) + // In the case where the key field is an object with multiple values (such as a composite aggregation with more than one source) + // the values will be iterated through and converted into a string + bucket[keyField] is Map<*, *> -> (bucket[keyField] as Map).values.map { keyValuesList.add(it as String) } + else -> throw IllegalArgumentException("Unexpected format for key in bucket [$bucket]") + } + + return keyValuesList + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/action/ExecuteMonitorResponse.kt b/alerting/src/main/kotlin/org/opensearch/alerting/action/ExecuteMonitorResponse.kt index b00733205..44b3064f3 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/action/ExecuteMonitorResponse.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/action/ExecuteMonitorResponse.kt @@ -37,9 +37,9 @@ import java.io.IOException class ExecuteMonitorResponse : ActionResponse, ToXContentObject { - val monitorRunResult: MonitorRunResult + val monitorRunResult: MonitorRunResult<*> - constructor(monitorRunResult: MonitorRunResult) : super() { + constructor(monitorRunResult: MonitorRunResult<*>) : super() { this.monitorRunResult = monitorRunResult } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregationBuilder.kt b/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregationBuilder.kt new file mode 100644 index 000000000..9962b9d58 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregationBuilder.kt @@ -0,0 +1,256 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.aggregation.bucketselectorext + +import org.opensearch.alerting.aggregation.bucketselectorext.BucketSelectorExtFilter.Companion.BUCKET_SELECTOR_COMPOSITE_AGG_FILTER +import org.opensearch.alerting.aggregation.bucketselectorext.BucketSelectorExtFilter.Companion.BUCKET_SELECTOR_FILTER +import org.opensearch.common.ParseField +import org.opensearch.common.ParsingException +import org.opensearch.common.io.stream.StreamInput +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.xcontent.ToXContent.Params +import org.opensearch.common.xcontent.XContentBuilder +import org.opensearch.common.xcontent.XContentParser +import org.opensearch.script.Script +import org.opensearch.search.aggregations.pipeline.AbstractPipelineAggregationBuilder +import org.opensearch.search.aggregations.pipeline.BucketHelpers.GapPolicy +import org.opensearch.search.aggregations.pipeline.PipelineAggregator +import java.io.IOException +import java.util.Objects + +class BucketSelectorExtAggregationBuilder : + AbstractPipelineAggregationBuilder { + private val bucketsPathsMap: Map + val parentBucketPath: String + val script: Script + val filter: BucketSelectorExtFilter? + private var gapPolicy = GapPolicy.SKIP + + constructor( + name: String, + bucketsPathsMap: Map, + script: Script, + parentBucketPath: String, + filter: BucketSelectorExtFilter? + ) : super(name, NAME.preferredName, listOf(parentBucketPath).toTypedArray()) { + this.bucketsPathsMap = bucketsPathsMap + this.script = script + this.parentBucketPath = parentBucketPath + this.filter = filter + } + + @Throws(IOException::class) + @Suppress("UNCHECKED_CAST") + constructor(sin: StreamInput) : super(sin, NAME.preferredName) { + bucketsPathsMap = sin.readMap() as MutableMap + script = Script(sin) + gapPolicy = GapPolicy.readFrom(sin) + parentBucketPath = sin.readString() + filter = if (sin.readBoolean()) { + BucketSelectorExtFilter(sin) + } else { + null + } + } + + @Throws(IOException::class) + override fun doWriteTo(out: StreamOutput) { + out.writeMap(bucketsPathsMap) + script.writeTo(out) + gapPolicy.writeTo(out) + out.writeString(parentBucketPath) + if (filter != null) { + out.writeBoolean(true) + filter.writeTo(out) + } else { + out.writeBoolean(false) + } + } + + /** + * Sets the gap policy to use for this aggregation. + */ + fun gapPolicy(gapPolicy: GapPolicy?): BucketSelectorExtAggregationBuilder { + requireNotNull(gapPolicy) { "[gapPolicy] must not be null: [$name]" } + this.gapPolicy = gapPolicy + return this + } + + override fun createInternal(metaData: Map?): PipelineAggregator { + return BucketSelectorExtAggregator(name, bucketsPathsMap, parentBucketPath, script, gapPolicy, filter, metaData) + } + + @Throws(IOException::class) + public override fun internalXContent(builder: XContentBuilder, params: Params): XContentBuilder { + builder.field(PipelineAggregator.Parser.BUCKETS_PATH.preferredName, bucketsPathsMap as Map?) + .field(PARENT_BUCKET_PATH.preferredName, parentBucketPath) + .field(Script.SCRIPT_PARSE_FIELD.preferredName, script) + .field(PipelineAggregator.Parser.GAP_POLICY.preferredName, gapPolicy.getName()) + if (filter != null) { + if (filter.isCompositeAggregation) { + builder.startObject(BUCKET_SELECTOR_COMPOSITE_AGG_FILTER.preferredName) + .value(filter) + .endObject() + } else { + builder.startObject(BUCKET_SELECTOR_FILTER.preferredName) + .value(filter) + .endObject() + } + } + return builder + } + + override fun overrideBucketsPath(): Boolean { + return true + } + + override fun validate(context: ValidationContext) { + // Nothing to check + } + + override fun hashCode(): Int { + return Objects.hash(super.hashCode(), bucketsPathsMap, script, gapPolicy) + } + + override fun equals(other: Any?): Boolean { + if (this === other) return true + if (other == null || javaClass != other.javaClass) return false + if (!super.equals(other)) return false + val otherCast = other as BucketSelectorExtAggregationBuilder + return ( + bucketsPathsMap == otherCast.bucketsPathsMap && + script == otherCast.script && + gapPolicy == otherCast.gapPolicy + ) + } + + override fun getWriteableName(): String { + return NAME.preferredName + } + + companion object { + val NAME = ParseField("bucket_selector_ext") + val PARENT_BUCKET_PATH = ParseField("parent_bucket_path") + + @Throws(IOException::class) + fun parse(reducerName: String, parser: XContentParser): BucketSelectorExtAggregationBuilder { + var token: XContentParser.Token + var script: Script? = null + var currentFieldName: String? = null + var bucketsPathsMap: MutableMap? = null + var gapPolicy: GapPolicy? = null + var parentBucketPath: String? = null + var filter: BucketSelectorExtFilter? = null + while (parser.nextToken().also { token = it } !== XContentParser.Token.END_OBJECT) { + if (token === XContentParser.Token.FIELD_NAME) { + currentFieldName = parser.currentName() + } else if (token === XContentParser.Token.VALUE_STRING) { + when { + PipelineAggregator.Parser.BUCKETS_PATH.match(currentFieldName, parser.deprecationHandler) -> { + bucketsPathsMap = HashMap() + bucketsPathsMap["_value"] = parser.text() + } + PipelineAggregator.Parser.GAP_POLICY.match(currentFieldName, parser.deprecationHandler) -> { + gapPolicy = GapPolicy.parse(parser.text(), parser.tokenLocation) + } + Script.SCRIPT_PARSE_FIELD.match(currentFieldName, parser.deprecationHandler) -> { + script = Script.parse(parser) + } + PARENT_BUCKET_PATH.match(currentFieldName, parser.deprecationHandler) -> { + parentBucketPath = parser.text() + } + else -> { + throw ParsingException( + parser.tokenLocation, + "Unknown key for a $token in [$reducerName]: [$currentFieldName]." + ) + } + } + } else if (token === XContentParser.Token.START_ARRAY) { + if (PipelineAggregator.Parser.BUCKETS_PATH.match(currentFieldName, parser.deprecationHandler)) { + val paths: MutableList = ArrayList() + while (parser.nextToken().also { token = it } !== XContentParser.Token.END_ARRAY) { + val path = parser.text() + paths.add(path) + } + bucketsPathsMap = HashMap() + for (i in paths.indices) { + bucketsPathsMap["_value$i"] = paths[i] + } + } else { + throw ParsingException( + parser.tokenLocation, + "Unknown key for a $token in [$reducerName]: [$currentFieldName]." + ) + } + } else if (token === XContentParser.Token.START_OBJECT) { + when { + Script.SCRIPT_PARSE_FIELD.match(currentFieldName, parser.deprecationHandler) -> { + script = Script.parse(parser) + } + PipelineAggregator.Parser.BUCKETS_PATH.match(currentFieldName, parser.deprecationHandler) -> { + val map = parser.map() + bucketsPathsMap = HashMap() + for ((key, value) in map) { + bucketsPathsMap[key] = value.toString() + } + } + BUCKET_SELECTOR_FILTER.match(currentFieldName, parser.deprecationHandler) -> { + filter = BucketSelectorExtFilter.parse(reducerName, false, parser) + } + BUCKET_SELECTOR_COMPOSITE_AGG_FILTER.match( + currentFieldName, + parser.deprecationHandler + ) -> { + filter = BucketSelectorExtFilter.parse(reducerName, true, parser) + } + else -> { + throw ParsingException( + parser.tokenLocation, + "Unknown key for a $token in [$reducerName]: [$currentFieldName]." + ) + } + } + } else { + throw ParsingException(parser.tokenLocation, "Unexpected token $token in [$reducerName].") + } + } + if (bucketsPathsMap == null) { + throw ParsingException( + parser.tokenLocation, + "Missing required field [" + PipelineAggregator.Parser.BUCKETS_PATH.preferredName + + "] for bucket_selector aggregation [" + reducerName + "]" + ) + } + if (script == null) { + throw ParsingException( + parser.tokenLocation, + "Missing required field [" + Script.SCRIPT_PARSE_FIELD.preferredName + + "] for bucket_selector aggregation [" + reducerName + "]" + ) + } + + if (parentBucketPath == null) { + throw ParsingException( + parser.tokenLocation, + "Missing required field [" + PARENT_BUCKET_PATH + + "] for bucket_selector aggregation [" + reducerName + "]" + ) + } + val factory = BucketSelectorExtAggregationBuilder(reducerName, bucketsPathsMap, script, parentBucketPath, filter) + if (gapPolicy != null) { + factory.gapPolicy(gapPolicy) + } + return factory + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregator.kt b/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregator.kt new file mode 100644 index 000000000..59b65cd55 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregator.kt @@ -0,0 +1,168 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.aggregation.bucketselectorext + +import org.apache.lucene.util.BytesRef +import org.opensearch.alerting.aggregation.bucketselectorext.BucketSelectorExtAggregationBuilder.Companion.NAME +import org.opensearch.common.io.stream.StreamInput +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.script.BucketAggregationSelectorScript +import org.opensearch.script.Script +import org.opensearch.search.DocValueFormat +import org.opensearch.search.aggregations.Aggregations +import org.opensearch.search.aggregations.InternalAggregation +import org.opensearch.search.aggregations.InternalAggregation.ReduceContext +import org.opensearch.search.aggregations.InternalMultiBucketAggregation +import org.opensearch.search.aggregations.bucket.SingleBucketAggregation +import org.opensearch.search.aggregations.bucket.composite.InternalComposite +import org.opensearch.search.aggregations.bucket.terms.IncludeExclude +import org.opensearch.search.aggregations.pipeline.BucketHelpers +import org.opensearch.search.aggregations.pipeline.BucketHelpers.GapPolicy +import org.opensearch.search.aggregations.pipeline.SiblingPipelineAggregator +import org.opensearch.search.aggregations.support.AggregationPath +import java.io.IOException + +class BucketSelectorExtAggregator : SiblingPipelineAggregator { + private var name: String? = null + private var bucketsPathsMap: Map + private var parentBucketPath: String + private var script: Script + private var gapPolicy: GapPolicy + private var bucketSelectorExtFilter: BucketSelectorExtFilter? = null + + constructor( + name: String?, + bucketsPathsMap: Map, + parentBucketPath: String, + script: Script, + gapPolicy: GapPolicy, + filter: BucketSelectorExtFilter?, + metadata: Map? + ) : super(name, bucketsPathsMap.values.toTypedArray(), metadata) { + this.bucketsPathsMap = bucketsPathsMap + this.parentBucketPath = parentBucketPath + this.script = script + this.gapPolicy = gapPolicy + this.bucketSelectorExtFilter = filter + } + + /** + * Read from a stream. + */ + @Suppress("UNCHECKED_CAST") + @Throws(IOException::class) + constructor(sin: StreamInput) : super(sin.readString(), null, null) { + script = Script(sin) + gapPolicy = GapPolicy.readFrom(sin) + bucketsPathsMap = sin.readMap() as Map + parentBucketPath = sin.readString() + if (sin.readBoolean()) { + bucketSelectorExtFilter = BucketSelectorExtFilter(sin) + } else { + bucketSelectorExtFilter = null + } + } + + @Throws(IOException::class) + override fun doWriteTo(out: StreamOutput) { + out.writeString(name) + script.writeTo(out) + gapPolicy.writeTo(out) + out.writeGenericValue(bucketsPathsMap) + out.writeString(parentBucketPath) + if (bucketSelectorExtFilter != null) { + out.writeBoolean(true) + bucketSelectorExtFilter!!.writeTo(out) + } else { + out.writeBoolean(false) + } + } + + override fun getWriteableName(): String { + return NAME.preferredName + } + + override fun doReduce(aggregations: Aggregations, reduceContext: ReduceContext): InternalAggregation { + val parentBucketPathList = AggregationPath.parse(parentBucketPath).pathElementsAsStringList + var subAggregations: Aggregations = aggregations + for (i in 0 until parentBucketPathList.size - 1) { + subAggregations = subAggregations.get(parentBucketPathList[0]).aggregations + } + val originalAgg = subAggregations.get(parentBucketPathList.last()) as InternalMultiBucketAggregation<*, *> + val buckets = originalAgg.buckets + val factory = reduceContext.scriptService().compile(script, BucketAggregationSelectorScript.CONTEXT) + val selectedBucketsIndex: MutableList = ArrayList() + for (i in buckets.indices) { + val bucket = buckets[i] + if (bucketSelectorExtFilter != null) { + var accepted = true + if (bucketSelectorExtFilter!!.isCompositeAggregation) { + val compBucketKeyObj = (bucket as InternalComposite.InternalBucket).key + val filtersMap: HashMap? = bucketSelectorExtFilter!!.filtersMap + for (sourceKey in compBucketKeyObj.keys) { + if (filtersMap != null) { + if (filtersMap.containsKey(sourceKey)) { + val obj = compBucketKeyObj[sourceKey] + accepted = isAccepted(obj!!, filtersMap[sourceKey]) + if (!accepted) break + } else { + accepted = false + break + } + } + } + } else { + accepted = isAccepted(bucket.key, bucketSelectorExtFilter!!.filters) + } + if (!accepted) continue + } + + val vars: MutableMap = HashMap() + if (script.params != null) { + vars.putAll(script.params) + } + for ((varName, bucketsPath) in bucketsPathsMap) { + val value = BucketHelpers.resolveBucketValue(originalAgg, bucket, bucketsPath, gapPolicy) + vars[varName] = value + } + val executableScript = factory.newInstance(vars) + // TODO: can we use one instance of the script for all buckets? it should be stateless? + if (executableScript.execute()) { + selectedBucketsIndex.add(i) + } + } + + return BucketSelectorIndices( + name(), parentBucketPath, selectedBucketsIndex, originalAgg.metadata + ) + } + + private fun isAccepted(obj: Any, filter: IncludeExclude?): Boolean { + return when (obj.javaClass) { + String::class.java -> { + val stringFilter = filter!!.convertToStringFilter(DocValueFormat.RAW) + stringFilter.accept(BytesRef(obj as String)) + } + java.lang.Long::class.java, Long::class.java -> { + val longFilter = filter!!.convertToLongFilter(DocValueFormat.RAW) + longFilter.accept(obj as Long) + } + java.lang.Double::class.java, Double::class.java -> { + val doubleFilter = filter!!.convertToDoubleFilter() + doubleFilter.accept(obj as Long) + } + else -> { + throw IllegalStateException("Object is not comparable. Please use one of String, Long or Double type.") + } + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtFilter.kt b/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtFilter.kt new file mode 100644 index 000000000..49eff3a66 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtFilter.kt @@ -0,0 +1,149 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.aggregation.bucketselectorext + +import org.opensearch.common.ParseField +import org.opensearch.common.ParsingException +import org.opensearch.common.io.stream.StreamInput +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.io.stream.Writeable +import org.opensearch.common.xcontent.ToXContent.Params +import org.opensearch.common.xcontent.ToXContentObject +import org.opensearch.common.xcontent.XContentBuilder +import org.opensearch.common.xcontent.XContentParser +import org.opensearch.search.aggregations.bucket.terms.IncludeExclude +import java.io.IOException + +class BucketSelectorExtFilter : ToXContentObject, Writeable { + // used for composite aggregations + val filtersMap: HashMap? + // used for filtering string term aggregation + val filters: IncludeExclude? + + constructor(filters: IncludeExclude?) { + filtersMap = null + this.filters = filters + } + + constructor(filtersMap: HashMap?) { + this.filtersMap = filtersMap + filters = null + } + + constructor(sin: StreamInput) { + if (sin.readBoolean()) { + val size: Int = sin.readVInt() + filtersMap = java.util.HashMap() + for (i in 0 until size) { + filtersMap[sin.readString()] = IncludeExclude(sin) + } + filters = null + } else { + filters = IncludeExclude(sin) + filtersMap = null + } + } + + @Throws(IOException::class) + override fun writeTo(out: StreamOutput) { + val isCompAgg = isCompositeAggregation + out.writeBoolean(isCompAgg) + if (isCompAgg) { + out.writeVInt(filtersMap!!.size) + for ((key, value) in filtersMap) { + out.writeString(key) + value.writeTo(out) + } + } else { + filters!!.writeTo(out) + } + } + + @Throws(IOException::class) + override fun toXContent(builder: XContentBuilder, params: Params): XContentBuilder { + if (isCompositeAggregation) { + for ((key, filter) in filtersMap!!) { + builder.startObject(key) + filter.toXContent(builder, params) + builder.endObject() + } + } else { + filters!!.toXContent(builder, params) + } + return builder + } + + val isCompositeAggregation: Boolean + get() = if (filtersMap != null && filters == null) { + true + } else if (filtersMap == null && filters != null) { + false + } else { + throw IllegalStateException("Type of selector cannot be determined") + } + + companion object { + const val NAME = "filter" + var BUCKET_SELECTOR_FILTER = ParseField("filter") + var BUCKET_SELECTOR_COMPOSITE_AGG_FILTER = ParseField("composite_agg_filter") + + @Throws(IOException::class) + fun parse(reducerName: String, isCompositeAggregation: Boolean, parser: XContentParser): BucketSelectorExtFilter { + var token: XContentParser.Token + return if (isCompositeAggregation) { + val filtersMap = HashMap() + while (parser.nextToken().also { token = it } !== XContentParser.Token.END_OBJECT) { + if (token === XContentParser.Token.FIELD_NAME) { + val sourceKey = parser.currentName() + token = parser.nextToken() + filtersMap[sourceKey] = parseIncludeExclude(reducerName, parser) + } else { + throw ParsingException( + parser.tokenLocation, + "Unknown key for a " + token + " in [" + reducerName + "]: [" + parser.currentName() + "]." + ) + } + } + BucketSelectorExtFilter(filtersMap) + } else { + BucketSelectorExtFilter(parseIncludeExclude(reducerName, parser)) + } + } + + @Throws(IOException::class) + private fun parseIncludeExclude(reducerName: String, parser: XContentParser): IncludeExclude { + var token: XContentParser.Token + var include: IncludeExclude? = null + var exclude: IncludeExclude? = null + while (parser.nextToken().also { token = it } !== XContentParser.Token.END_OBJECT) { + val fieldName = parser.currentName() + when { + IncludeExclude.INCLUDE_FIELD.match(fieldName, parser.deprecationHandler) -> { + parser.nextToken() + include = IncludeExclude.parseInclude(parser) + } + IncludeExclude.EXCLUDE_FIELD.match(fieldName, parser.deprecationHandler) -> { + parser.nextToken() + exclude = IncludeExclude.parseExclude(parser) + } + else -> { + throw ParsingException( + parser.tokenLocation, + "Unknown key for a $token in [$reducerName]: [$fieldName]." + ) + } + } + } + return IncludeExclude.merge(include, exclude) + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorIndices.kt b/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorIndices.kt new file mode 100644 index 000000000..4e636e360 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorIndices.kt @@ -0,0 +1,79 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.aggregation.bucketselectorext + +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.xcontent.ToXContent.Params +import org.opensearch.common.xcontent.XContentBuilder +import org.opensearch.search.aggregations.InternalAggregation +import java.io.IOException +import java.util.Objects + +open class BucketSelectorIndices( + name: String?, + private var parentBucketPath: String, + var bucketIndices: List, + metaData: Map? +) : InternalAggregation(name, metaData) { + + @Throws(IOException::class) + override fun doWriteTo(out: StreamOutput) { + out.writeString(parentBucketPath) + out.writeIntArray(bucketIndices.stream().mapToInt { i: Int? -> i!! }.toArray()) + } + + override fun getWriteableName(): String { + return name + } + + override fun reduce(aggregations: List, reduceContext: ReduceContext): BucketSelectorIndices { + throw UnsupportedOperationException("Not supported") + } + + override fun mustReduceOnSingleInternalAgg(): Boolean { + return false + } + + override fun getProperty(path: MutableList?): Any { + throw UnsupportedOperationException("Not supported") + } + + internal object Fields { + const val PARENT_BUCKET_PATH = "parent_bucket_path" + const val BUCKET_INDICES = "bucket_indices" + } + + @Throws(IOException::class) + override fun doXContentBody(builder: XContentBuilder, params: Params): XContentBuilder { + builder.field(Fields.PARENT_BUCKET_PATH, parentBucketPath) + builder.field(Fields.BUCKET_INDICES, bucketIndices) + otherStatsToXContent(builder) + return builder + } + + @Throws(IOException::class) + protected fun otherStatsToXContent(builder: XContentBuilder): XContentBuilder { + return builder + } + + override fun hashCode(): Int { + return Objects.hash(super.hashCode(), parentBucketPath) + } + + override fun equals(other: Any?): Boolean { + if (this === other) return true + if (other == null || javaClass != other.javaClass) return false + if (!super.equals(other)) return false + val otherCast = other as BucketSelectorIndices + return name == otherCast.name && parentBucketPath == otherCast.parentBucketPath + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/alerts/AlertIndices.kt b/alerting/src/main/kotlin/org/opensearch/alerting/alerts/AlertIndices.kt index a4c7950b8..6665b8773 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/alerts/AlertIndices.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/alerts/AlertIndices.kt @@ -27,9 +27,10 @@ package org.opensearch.alerting.alerts import org.apache.logging.log4j.LogManager -import org.apache.lucene.index.IndexNotFoundException import org.opensearch.ResourceAlreadyExistsException +import org.opensearch.action.ActionListener import org.opensearch.action.admin.cluster.state.ClusterStateRequest +import org.opensearch.action.admin.cluster.state.ClusterStateResponse import org.opensearch.action.admin.indices.alias.Alias import org.opensearch.action.admin.indices.create.CreateIndexRequest import org.opensearch.action.admin.indices.create.CreateIndexResponse @@ -38,6 +39,7 @@ import org.opensearch.action.admin.indices.exists.indices.IndicesExistsRequest import org.opensearch.action.admin.indices.exists.indices.IndicesExistsResponse import org.opensearch.action.admin.indices.mapping.put.PutMappingRequest import org.opensearch.action.admin.indices.rollover.RolloverRequest +import org.opensearch.action.admin.indices.rollover.RolloverResponse import org.opensearch.action.support.IndicesOptions import org.opensearch.action.support.master.AcknowledgedResponse import org.opensearch.alerting.alerts.AlertIndices.Companion.ALERT_INDEX @@ -145,7 +147,6 @@ class AlertIndices( fun onMaster() { try { - // TODO: Change current actionGet requests within rolloverHistoryIndex() rolloverAndDeleteHistoryIndices() to use suspendUntil // try to rollover immediately as we might be restarting the cluster rolloverHistoryIndex() // schedule the next rollover for approx MAX_AGE later @@ -279,9 +280,9 @@ class AlertIndices( deleteOldHistoryIndices() } - private fun rolloverHistoryIndex(): Boolean { + private fun rolloverHistoryIndex() { if (!historyIndexInitialized) { - return false + return } // We have to pass null for newIndexName in order to get Elastic to increment the index count. @@ -291,17 +292,24 @@ class AlertIndices( .settings(Settings.builder().put("index.hidden", true).build()) request.addMaxIndexDocsCondition(historyMaxDocs) request.addMaxIndexAgeCondition(historyMaxAge) - val response = client.admin().indices().rolloverIndex(request).actionGet(requestTimeout) - if (!response.isRolledOver) { - logger.info("$HISTORY_WRITE_INDEX not rolled over. Conditions were: ${response.conditionStatus}") - } else { - lastRolloverTime = TimeValue.timeValueMillis(threadPool.absoluteTimeInMillis()) - } - return response.isRolledOver + client.admin().indices().rolloverIndex( + request, + object : ActionListener { + override fun onResponse(response: RolloverResponse) { + if (!response.isRolledOver) { + logger.info("$HISTORY_WRITE_INDEX not rolled over. Conditions were: ${response.conditionStatus}") + } else { + lastRolloverTime = TimeValue.timeValueMillis(threadPool.absoluteTimeInMillis()) + } + } + override fun onFailure(e: Exception) { + logger.error("$HISTORY_WRITE_INDEX not roll over failed.") + } + } + ) } private fun deleteOldHistoryIndices() { - val indicesToDelete = mutableListOf() val clusterStateRequest = ClusterStateRequest() .clear() @@ -310,8 +318,27 @@ class AlertIndices( .local(true) .indicesOptions(IndicesOptions.strictExpand()) - val clusterStateResponse = client.admin().cluster().state(clusterStateRequest).actionGet() + client.admin().cluster().state( + clusterStateRequest, + object : ActionListener { + override fun onResponse(clusterStateResponse: ClusterStateResponse) { + if (!clusterStateResponse.state.metadata.indices.isEmpty) { + val indicesToDelete = getIndicesToDelete(clusterStateResponse) + logger.info("Deleting old history indices viz $indicesToDelete") + deleteAllOldHistoryIndices(indicesToDelete) + } else { + logger.info("No Old History Indices to delete") + } + } + override fun onFailure(e: Exception) { + logger.error("Error fetching cluster state") + } + } + ) + } + private fun getIndicesToDelete(clusterStateResponse: ClusterStateResponse): List { + val indicesToDelete = mutableListOf() for (entry in clusterStateResponse.state.metadata.indices) { val indexMetaData = entry.value val creationTime = indexMetaData.creationDate @@ -331,26 +358,48 @@ class AlertIndices( indicesToDelete.add(indexMetaData.index.name) } } + return indicesToDelete + } + private fun deleteAllOldHistoryIndices(indicesToDelete: List) { if (indicesToDelete.isNotEmpty()) { val deleteIndexRequest = DeleteIndexRequest(*indicesToDelete.toTypedArray()) - val deleteIndexResponse = client.admin().indices().delete(deleteIndexRequest).actionGet() - - if (!deleteIndexResponse.isAcknowledged) { - logger.error("Could not delete one or more Alerting history indices: $indicesToDelete. Retrying one by one.") - for (index in indicesToDelete) { - try { - val singleDeleteRequest = DeleteIndexRequest(*indicesToDelete.toTypedArray()) - val singleDeleteResponse = client.admin().indices().delete(singleDeleteRequest).actionGet() + client.admin().indices().delete( + deleteIndexRequest, + object : ActionListener { + override fun onResponse(deleteIndicesResponse: AcknowledgedResponse) { + if (!deleteIndicesResponse.isAcknowledged) { + logger.error("Could not delete one or more Alerting history indices: $indicesToDelete. Retrying one by one.") + deleteOldHistoryIndex(indicesToDelete) + } + } + override fun onFailure(e: Exception) { + logger.error("Delete for Alerting History Indices $indicesToDelete Failed. Retrying one By one.") + deleteOldHistoryIndex(indicesToDelete) + } + } + ) + } + } - if (!singleDeleteResponse.isAcknowledged) { - logger.error("Could not delete one or more Alerting history indices: $index") + private fun deleteOldHistoryIndex(indicesToDelete: List) { + for (index in indicesToDelete) { + val singleDeleteRequest = DeleteIndexRequest(*indicesToDelete.toTypedArray()) + client.admin().indices().delete( + singleDeleteRequest, + object : ActionListener { + override fun onResponse(acknowledgedResponse: AcknowledgedResponse?) { + if (acknowledgedResponse != null) { + if (!acknowledgedResponse.isAcknowledged) { + logger.error("Could not delete one or more Alerting history indices: $index") + } } - } catch (e: IndexNotFoundException) { - logger.debug("$index was already deleted. ${e.message}") + } + override fun onFailure(e: Exception) { + logger.debug("Exception ${e.message} while deleting the index $index") } } - } + ) } } } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/AggregationResultBucket.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/AggregationResultBucket.kt new file mode 100644 index 000000000..e31942c30 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/AggregationResultBucket.kt @@ -0,0 +1,95 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.model + +import org.opensearch.common.ParsingException +import org.opensearch.common.io.stream.StreamInput +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.io.stream.Writeable +import org.opensearch.common.xcontent.ToXContent +import org.opensearch.common.xcontent.ToXContentObject +import org.opensearch.common.xcontent.XContentBuilder +import org.opensearch.common.xcontent.XContentParser +import org.opensearch.common.xcontent.XContentParser.Token +import org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken +import java.io.IOException +import java.util.Locale + +data class AggregationResultBucket( + val parentBucketPath: String?, + val bucketKeys: List, + val bucket: Map? // TODO: Should reduce contents to only top-level to not include sub-aggs here +) : Writeable, ToXContentObject { + + @Throws(IOException::class) + constructor(sin: StreamInput) : this(sin.readString(), sin.readStringList(), sin.readMap()) + + override fun writeTo(out: StreamOutput) { + out.writeString(parentBucketPath) + out.writeStringCollection(bucketKeys) + out.writeMap(bucket) + } + + override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { + builder.startObject() + innerXContent(builder) + return builder.endObject() + } + + fun innerXContent(builder: XContentBuilder): XContentBuilder { + builder.startObject(CONFIG_NAME) + .field(PARENTS_BUCKET_PATH, parentBucketPath) + .field(BUCKET_KEYS, bucketKeys.toTypedArray()) + .field(BUCKET, bucket) + .endObject() + return builder + } + + companion object { + const val CONFIG_NAME = "agg_alert_content" + const val PARENTS_BUCKET_PATH = "parent_bucket_path" + const val BUCKET_KEYS = "bucket_keys" + private const val BUCKET = "bucket" + + fun parse(xcp: XContentParser): AggregationResultBucket { + var parentBucketPath: String? = null + var bucketKeys = mutableListOf() + var bucket: MutableMap? = null + ensureExpectedToken(XContentParser.Token.START_OBJECT, xcp.currentToken(), xcp) + + if (CONFIG_NAME != xcp.currentName()) { + throw ParsingException( + xcp.tokenLocation, + String.format( + Locale.ROOT, "Failed to parse object: expecting token with name [%s] but found [%s]", + CONFIG_NAME, xcp.currentName() + ) + ) + } + while (xcp.nextToken() != Token.END_OBJECT) { + val fieldName = xcp.currentName() + xcp.nextToken() + when (fieldName) { + PARENTS_BUCKET_PATH -> parentBucketPath = xcp.text() + BUCKET_KEYS -> { + ensureExpectedToken(Token.START_ARRAY, xcp.currentToken(), xcp) + while (xcp.nextToken() != Token.END_ARRAY) { + bucketKeys.add(xcp.text()) + } + } + BUCKET -> bucket = xcp.map() + } + } + return AggregationResultBucket(parentBucketPath, bucketKeys, bucket) + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/Alert.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/Alert.kt index f1053d8ee..22ba715e7 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/model/Alert.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/Alert.kt @@ -61,7 +61,8 @@ data class Alert( val errorMessage: String? = null, val errorHistory: List, val severity: String, - val actionExecutionResults: List + val actionExecutionResults: List, + val aggregationResultBucket: AggregationResultBucket? = null ) : Writeable, ToXContent { init { @@ -72,7 +73,7 @@ data class Alert( constructor( monitor: Monitor, - trigger: Trigger, + trigger: QueryLevelTrigger, startTime: Instant, lastNotificationTime: Instant?, state: State = State.ACTIVE, @@ -84,7 +85,45 @@ data class Alert( monitorId = monitor.id, monitorName = monitor.name, monitorVersion = monitor.version, monitorUser = monitor.user, triggerId = trigger.id, triggerName = trigger.name, state = state, startTime = startTime, lastNotificationTime = lastNotificationTime, errorMessage = errorMessage, errorHistory = errorHistory, - severity = trigger.severity, actionExecutionResults = actionExecutionResults, schemaVersion = schemaVersion + severity = trigger.severity, actionExecutionResults = actionExecutionResults, schemaVersion = schemaVersion, + aggregationResultBucket = null + ) + + constructor( + monitor: Monitor, + trigger: BucketLevelTrigger, + startTime: Instant, + lastNotificationTime: Instant?, + state: State = State.ACTIVE, + errorMessage: String? = null, + errorHistory: List = mutableListOf(), + actionExecutionResults: List = mutableListOf(), + schemaVersion: Int = NO_SCHEMA_VERSION + ) : this( + monitorId = monitor.id, monitorName = monitor.name, monitorVersion = monitor.version, monitorUser = monitor.user, + triggerId = trigger.id, triggerName = trigger.name, state = state, startTime = startTime, + lastNotificationTime = lastNotificationTime, errorMessage = errorMessage, errorHistory = errorHistory, + severity = trigger.severity, actionExecutionResults = actionExecutionResults, schemaVersion = schemaVersion, + aggregationResultBucket = null + ) + + constructor( + monitor: Monitor, + trigger: BucketLevelTrigger, + startTime: Instant, + lastNotificationTime: Instant?, + state: State = State.ACTIVE, + errorMessage: String? = null, + errorHistory: List = mutableListOf(), + actionExecutionResults: List = mutableListOf(), + schemaVersion: Int = NO_SCHEMA_VERSION, + aggregationResultBucket: AggregationResultBucket + ) : this( + monitorId = monitor.id, monitorName = monitor.name, monitorVersion = monitor.version, monitorUser = monitor.user, + triggerId = trigger.id, triggerName = trigger.name, state = state, startTime = startTime, + lastNotificationTime = lastNotificationTime, errorMessage = errorMessage, errorHistory = errorHistory, + severity = trigger.severity, actionExecutionResults = actionExecutionResults, schemaVersion = schemaVersion, + aggregationResultBucket = aggregationResultBucket ) enum class State { @@ -112,7 +151,8 @@ data class Alert( errorMessage = sin.readOptionalString(), errorHistory = sin.readList(::AlertError), severity = sin.readString(), - actionExecutionResults = sin.readList(::ActionExecutionResult) + actionExecutionResults = sin.readList(::ActionExecutionResult), + aggregationResultBucket = if (sin.readBoolean()) AggregationResultBucket(sin) else null ) fun isAcknowledged(): Boolean = (state == State.ACKNOWLEDGED) @@ -138,6 +178,12 @@ data class Alert( out.writeCollection(errorHistory) out.writeString(severity) out.writeCollection(actionExecutionResults) + if (aggregationResultBucket != null) { + out.writeBoolean(true) + aggregationResultBucket.writeTo(out) + } else { + out.writeBoolean(false) + } } companion object { @@ -160,7 +206,8 @@ data class Alert( const val ALERT_HISTORY_FIELD = "alert_history" const val SEVERITY_FIELD = "severity" const val ACTION_EXECUTION_RESULTS_FIELD = "action_execution_results" - + const val BUCKET_KEYS = AggregationResultBucket.BUCKET_KEYS + const val PARENTS_BUCKET_PATH = AggregationResultBucket.PARENTS_BUCKET_PATH const val NO_ID = "" const val NO_VERSION = Versions.NOT_FOUND @@ -183,8 +230,8 @@ data class Alert( var acknowledgedTime: Instant? = null var errorMessage: String? = null val errorHistory: MutableList = mutableListOf() - var actionExecutionResults: MutableList = mutableListOf() - + val actionExecutionResults: MutableList = mutableListOf() + var aggAlertBucket: AggregationResultBucket? = null ensureExpectedToken(XContentParser.Token.START_OBJECT, xcp.currentToken(), xcp) while (xcp.nextToken() != XContentParser.Token.END_OBJECT) { val fieldName = xcp.currentName() @@ -217,6 +264,17 @@ data class Alert( actionExecutionResults.add(ActionExecutionResult.parse(xcp)) } } + AggregationResultBucket.CONFIG_NAME -> { + // If an Alert with aggAlertBucket contents is indexed into the alerts index first, then + // that field will be added to the mappings. + // In this case, that field will default to null when it isn't present for Alerts created by Query-Level Monitors + // (even though the toXContent doesn't output the field) so null is being accounted for here. + aggAlertBucket = if (xcp.currentToken() == XContentParser.Token.VALUE_NULL) { + null + } else { + AggregationResultBucket.parse(xcp) + } + } } } @@ -227,7 +285,7 @@ data class Alert( state = requireNotNull(state), startTime = requireNotNull(startTime), endTime = endTime, lastNotificationTime = lastNotificationTime, acknowledgedTime = acknowledgedTime, errorMessage = errorMessage, errorHistory = errorHistory, severity = severity, - actionExecutionResults = actionExecutionResults + actionExecutionResults = actionExecutionResults, aggregationResultBucket = aggAlertBucket ) } @@ -239,7 +297,7 @@ data class Alert( } override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { - return builder.startObject() + builder.startObject() .field(ALERT_ID_FIELD, id) .field(ALERT_VERSION_FIELD, version) .field(MONITOR_ID_FIELD, monitorId) @@ -258,7 +316,9 @@ data class Alert( .optionalTimeField(LAST_NOTIFICATION_TIME_FIELD, lastNotificationTime) .optionalTimeField(END_TIME_FIELD, endTime) .optionalTimeField(ACKNOWLEDGED_TIME_FIELD, acknowledgedTime) - .endObject() + aggregationResultBucket?.innerXContent(builder) + builder.endObject() + return builder } fun asTemplateArg(): Map { @@ -271,7 +331,10 @@ data class Alert( LAST_NOTIFICATION_TIME_FIELD to lastNotificationTime?.toEpochMilli(), SEVERITY_FIELD to severity, START_TIME_FIELD to startTime.toEpochMilli(), - STATE_FIELD to state.toString() + STATE_FIELD to state.toString(), + // Converting bucket keys to comma separated String to avoid manipulation in Action mustache templates + BUCKET_KEYS to aggregationResultBucket?.bucketKeys?.joinToString(","), + PARENTS_BUCKET_PATH to aggregationResultBucket?.parentBucketPath ) } } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/BucketLevelTrigger.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/BucketLevelTrigger.kt new file mode 100644 index 000000000..750a4d7fb --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/BucketLevelTrigger.kt @@ -0,0 +1,155 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.model + +import org.opensearch.alerting.aggregation.bucketselectorext.BucketSelectorExtAggregationBuilder +import org.opensearch.alerting.model.Trigger.Companion.ACTIONS_FIELD +import org.opensearch.alerting.model.Trigger.Companion.ID_FIELD +import org.opensearch.alerting.model.Trigger.Companion.NAME_FIELD +import org.opensearch.alerting.model.Trigger.Companion.SEVERITY_FIELD +import org.opensearch.alerting.model.action.Action +import org.opensearch.common.CheckedFunction +import org.opensearch.common.ParseField +import org.opensearch.common.UUIDs +import org.opensearch.common.io.stream.StreamInput +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.xcontent.NamedXContentRegistry +import org.opensearch.common.xcontent.ToXContent +import org.opensearch.common.xcontent.XContentBuilder +import org.opensearch.common.xcontent.XContentParser +import org.opensearch.common.xcontent.XContentParser.Token +import org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken +import java.io.IOException + +/** + * A multi-alert Trigger available with Bucket-Level Monitors that filters aggregation buckets via a pipeline + * aggregator. + */ +data class BucketLevelTrigger( + override val id: String = UUIDs.base64UUID(), + override val name: String, + override val severity: String, + val bucketSelector: BucketSelectorExtAggregationBuilder, + override val actions: List +) : Trigger { + + @Throws(IOException::class) + constructor(sin: StreamInput) : this( + sin.readString(), // id + sin.readString(), // name + sin.readString(), // severity + BucketSelectorExtAggregationBuilder(sin), // condition + sin.readList(::Action) // actions + ) + + override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { + builder.startObject() + .startObject(BUCKET_LEVEL_TRIGGER_FIELD) + .field(ID_FIELD, id) + .field(NAME_FIELD, name) + .field(SEVERITY_FIELD, severity) + .startObject(CONDITION_FIELD) + bucketSelector.internalXContent(builder, params) + builder.endObject() + .field(ACTIONS_FIELD, actions.toTypedArray()) + .endObject() + .endObject() + return builder + } + + override fun name(): String { + return BUCKET_LEVEL_TRIGGER_FIELD + } + + @Throws(IOException::class) + override fun writeTo(out: StreamOutput) { + out.writeString(id) + out.writeString(name) + out.writeString(severity) + bucketSelector.writeTo(out) + out.writeCollection(actions) + } + + fun asTemplateArg(): Map { + return mapOf( + ID_FIELD to id, + NAME_FIELD to name, + SEVERITY_FIELD to severity, + ACTIONS_FIELD to actions.map { it.asTemplateArg() }, + PARENT_BUCKET_PATH to getParentBucketPath() + ) + } + + fun getParentBucketPath(): String { + return bucketSelector.parentBucketPath + } + + companion object { + const val BUCKET_LEVEL_TRIGGER_FIELD = "bucket_level_trigger" + const val CONDITION_FIELD = "condition" + const val PARENT_BUCKET_PATH = "parentBucketPath" + + val XCONTENT_REGISTRY = NamedXContentRegistry.Entry( + Trigger::class.java, ParseField(BUCKET_LEVEL_TRIGGER_FIELD), + CheckedFunction { parseInner(it) } + ) + + @JvmStatic + @Throws(IOException::class) + fun parseInner(xcp: XContentParser): BucketLevelTrigger { + var id = UUIDs.base64UUID() // assign a default triggerId if one is not specified + lateinit var name: String + lateinit var severity: String + val actions: MutableList = mutableListOf() + ensureExpectedToken(Token.START_OBJECT, xcp.currentToken(), xcp) + lateinit var bucketSelector: BucketSelectorExtAggregationBuilder + + while (xcp.nextToken() != Token.END_OBJECT) { + val fieldName = xcp.currentName() + + xcp.nextToken() + when (fieldName) { + ID_FIELD -> id = xcp.text() + NAME_FIELD -> name = xcp.text() + SEVERITY_FIELD -> severity = xcp.text() + CONDITION_FIELD -> { + // Using the trigger id as the name in the bucket selector since it is validated for uniqueness within Monitors. + // The contents of the trigger definition are round-tripped through parse and toXContent during Monitor creation + // ensuring that the id is available here in the version of the Monitor object that will be executed, even if the + // user submitted a custom trigger id after the condition definition. + bucketSelector = BucketSelectorExtAggregationBuilder.parse(id, xcp) + } + ACTIONS_FIELD -> { + ensureExpectedToken(Token.START_ARRAY, xcp.currentToken(), xcp) + while (xcp.nextToken() != Token.END_ARRAY) { + actions.add(Action.parse(xcp)) + } + } + } + } + + return BucketLevelTrigger( + id = requireNotNull(id) { "Trigger id is null." }, + name = requireNotNull(name) { "Trigger name is null" }, + severity = requireNotNull(severity) { "Trigger severity is null" }, + bucketSelector = requireNotNull(bucketSelector) { "Trigger condition is null" }, + actions = requireNotNull(actions) { "Trigger actions are null" } + ) + } + + @JvmStatic + @Throws(IOException::class) + fun readFrom(sin: StreamInput): BucketLevelTrigger { + return BucketLevelTrigger(sin) + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/BucketLevelTriggerRunResult.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/BucketLevelTriggerRunResult.kt new file mode 100644 index 000000000..23c9a4af7 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/BucketLevelTriggerRunResult.kt @@ -0,0 +1,63 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.model + +import org.opensearch.common.io.stream.StreamInput +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.xcontent.ToXContent +import org.opensearch.common.xcontent.XContentBuilder +import java.io.IOException + +data class BucketLevelTriggerRunResult( + override var triggerName: String, + override var error: Exception? = null, + var aggregationResultBuckets: Map, + var actionResultsMap: MutableMap> = mutableMapOf() +) : TriggerRunResult(triggerName, error) { + + @Throws(IOException::class) + @Suppress("UNCHECKED_CAST") + constructor(sin: StreamInput) : this( + sin.readString(), + sin.readException() as Exception?, // error + sin.readMap(StreamInput::readString, ::AggregationResultBucket), + sin.readMap() as MutableMap> + ) + + override fun internalXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { + return builder + .field(AGG_RESULT_BUCKETS, aggregationResultBuckets) + .field(ACTIONS_RESULTS, actionResultsMap as Map) + } + + @Throws(IOException::class) + @Suppress("UNCHECKED_CAST") + override fun writeTo(out: StreamOutput) { + super.writeTo(out) + out.writeMap(aggregationResultBuckets, StreamOutput::writeString) { + valueOut: StreamOutput, aggResultBucket: AggregationResultBucket -> + aggResultBucket.writeTo(valueOut) + } + out.writeMap(actionResultsMap as Map) + } + + companion object { + const val AGG_RESULT_BUCKETS = "agg_result_buckets" + const val ACTIONS_RESULTS = "action_results" + + @JvmStatic + @Throws(IOException::class) + fun readFrom(sin: StreamInput): TriggerRunResult { + return BucketLevelTriggerRunResult(sin) + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/Monitor.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/Monitor.kt index 9df01b0c4..8fe3e8ce5 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/model/Monitor.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/Monitor.kt @@ -39,6 +39,7 @@ import org.opensearch.alerting.settings.AlertingSettings.Companion.MONITOR_MAX_T import org.opensearch.alerting.util.IndexUtils.Companion.NO_SCHEMA_VERSION import org.opensearch.alerting.util._ID import org.opensearch.alerting.util._VERSION +import org.opensearch.alerting.util.isBucketLevelMonitor import org.opensearch.common.CheckedFunction import org.opensearch.common.ParseField import org.opensearch.common.io.stream.StreamInput @@ -52,6 +53,7 @@ import org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken import org.opensearch.commons.authuser.User import java.io.IOException import java.time.Instant +import java.util.Locale /** * A value object that represents a Monitor. Monitors are used to periodically execute a source query and check the @@ -65,6 +67,9 @@ data class Monitor( override val schedule: Schedule, override val lastUpdateTime: Instant, override val enabledTime: Instant?, + // TODO: Check how this behaves during rolling upgrade/multi-version cluster + // Can read/write and parsing break if it's done from an old -> new version of the plugin? + val monitorType: MonitorType, val user: User?, val schemaVersion: Int = NO_SCHEMA_VERSION, val inputs: List, @@ -79,6 +84,13 @@ data class Monitor( val triggerIds = mutableSetOf() triggers.forEach { trigger -> require(triggerIds.add(trigger.id)) { "Duplicate trigger id: ${trigger.id}. Trigger ids must be unique." } + // Verify Trigger type based on Monitor type + when (monitorType) { + MonitorType.QUERY_LEVEL_MONITOR -> + require(trigger is QueryLevelTrigger) { "Incompatible trigger [$trigger.id] for monitor type [$monitorType]" } + MonitorType.BUCKET_LEVEL_MONITOR -> + require(trigger is BucketLevelTrigger) { "Incompatible trigger [$trigger.id] for monitor type [$monitorType]" } + } } if (enabled) { requireNotNull(enabledTime) @@ -87,6 +99,16 @@ data class Monitor( } require(inputs.size <= MONITOR_MAX_INPUTS) { "Monitors can only have $MONITOR_MAX_INPUTS search input." } require(triggers.size <= MONITOR_MAX_TRIGGERS) { "Monitors can only support up to $MONITOR_MAX_TRIGGERS triggers." } + if (this.isBucketLevelMonitor()) { + inputs.forEach { input -> + require(input is SearchInput) { "Unsupported input [$input] for Monitor" } + // TODO: Keeping query validation simple for now, only term aggregations have full support for the "group by" on the + // initial release. Should either add tests for other aggregation types or add validation to prevent using them. + require(input.query.aggregations() != null && !input.query.aggregations().aggregatorFactories.isEmpty()) { + "At least one aggregation is required for the input [$input]" + } + } + } } @Throws(IOException::class) @@ -98,14 +120,27 @@ data class Monitor( schedule = Schedule.readFrom(sin), lastUpdateTime = sin.readInstant(), enabledTime = sin.readOptionalInstant(), + monitorType = sin.readEnum(MonitorType::class.java), user = if (sin.readBoolean()) { User(sin) } else null, schemaVersion = sin.readInt(), inputs = sin.readList(::SearchInput), - triggers = sin.readList(::Trigger), + triggers = sin.readList((Trigger)::readFrom), uiMetadata = suppressWarning(sin.readMap()) ) + + // This enum classifies different Monitors + // This is different from 'type' which denotes the Scheduled Job type + enum class MonitorType(val value: String) { + QUERY_LEVEL_MONITOR("query_level_monitor"), + BUCKET_LEVEL_MONITOR("bucket_level_monitor"); + + override fun toString(): String { + return value + } + } + fun toXContent(builder: XContentBuilder): XContentBuilder { return toXContent(builder, ToXContent.EMPTY_PARAMS) } @@ -121,6 +156,7 @@ data class Monitor( builder.field(TYPE_FIELD, type) .field(SCHEMA_VERSION_FIELD, schemaVersion) .field(NAME_FIELD, name) + .field(MONITOR_TYPE_FIELD, monitorType) .optionalUserField(USER_FIELD, user) .field(ENABLED_FIELD, enabled) .optionalTimeField(ENABLED_TIME_FIELD, enabledTime) @@ -149,17 +185,25 @@ data class Monitor( schedule.writeTo(out) out.writeInstant(lastUpdateTime) out.writeOptionalInstant(enabledTime) + out.writeEnum(monitorType) out.writeBoolean(user != null) user?.writeTo(out) out.writeInt(schemaVersion) out.writeCollection(inputs) - out.writeCollection(triggers) + // Outputting type with each Trigger so that the generic Trigger.readFrom() can read it + out.writeVInt(triggers.size) + triggers.forEach { + if (it is QueryLevelTrigger) out.writeEnum(Trigger.Type.QUERY_LEVEL_TRIGGER) + else out.writeEnum(Trigger.Type.BUCKET_LEVEL_TRIGGER) + it.writeTo(out) + } out.writeMap(uiMetadata) } companion object { const val MONITOR_TYPE = "monitor" const val TYPE_FIELD = "type" + const val MONITOR_TYPE_FIELD = "monitor_type" const val SCHEMA_VERSION_FIELD = "schema_version" const val NAME_FIELD = "name" const val USER_FIELD = "user" @@ -186,6 +230,8 @@ data class Monitor( @Throws(IOException::class) fun parse(xcp: XContentParser, id: String = NO_ID, version: Long = NO_VERSION): Monitor { lateinit var name: String + // Default to QUERY_LEVEL_MONITOR to cover Monitors that existed before the addition of MonitorType + var monitorType: String = MonitorType.QUERY_LEVEL_MONITOR.toString() var user: User? = null lateinit var schedule: Schedule var lastUpdateTime: Instant? = null @@ -204,6 +250,13 @@ data class Monitor( when (fieldName) { SCHEMA_VERSION_FIELD -> schemaVersion = xcp.intValue() NAME_FIELD -> name = xcp.text() + MONITOR_TYPE_FIELD -> { + monitorType = xcp.text() + val allowedTypes = MonitorType.values().map { it.value } + if (!allowedTypes.contains(monitorType)) { + throw IllegalStateException("Monitor type should be one of $allowedTypes") + } + } USER_FIELD -> user = if (xcp.currentToken() == Token.VALUE_NULL) null else User.parse(xcp) ENABLED_FIELD -> enabled = xcp.booleanValue() SCHEDULE_FIELD -> schedule = Schedule.parse(xcp) @@ -241,6 +294,7 @@ data class Monitor( requireNotNull(schedule) { "Monitor schedule is null" }, lastUpdateTime ?: Instant.now(), enabledTime, + MonitorType.valueOf(monitorType.toUpperCase(Locale.ROOT)), user, schemaVersion, inputs.toList(), diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/MonitorRunResult.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/MonitorRunResult.kt index 28a6dec05..d3fa97a1b 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/model/MonitorRunResult.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/MonitorRunResult.kt @@ -39,23 +39,24 @@ import org.opensearch.script.ScriptException import java.io.IOException import java.time.Instant -data class MonitorRunResult( +data class MonitorRunResult( val monitorName: String, val periodStart: Instant, val periodEnd: Instant, val error: Exception? = null, val inputResults: InputRunResults = InputRunResults(), - val triggerResults: Map = mapOf() + val triggerResults: Map = mapOf() ) : Writeable, ToXContent { @Throws(IOException::class) + @Suppress("UNCHECKED_CAST") constructor(sin: StreamInput) : this( sin.readString(), // monitorName sin.readInstant(), // periodStart sin.readInstant(), // periodEnd sin.readException(), // error InputRunResults.readFrom(sin), // inputResults - suppressWarning(sin.readMap()) // triggerResults + suppressWarning(sin.readMap()) as Map // triggerResults ) override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { @@ -88,7 +89,7 @@ data class MonitorRunResult( companion object { @JvmStatic @Throws(IOException::class) - fun readFrom(sin: StreamInput): MonitorRunResult { + fun readFrom(sin: StreamInput): MonitorRunResult { return MonitorRunResult(sin) } @@ -109,7 +110,11 @@ data class MonitorRunResult( } } -data class InputRunResults(val results: List> = listOf(), val error: Exception? = null) : Writeable, ToXContent { +data class InputRunResults( + val results: List> = listOf(), + val error: Exception? = null, + val aggTriggersAfterKey: MutableMap? = null +) : Writeable, ToXContent { override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { return builder.startObject() @@ -144,69 +149,19 @@ data class InputRunResults(val results: List> = listOf(), val e return map as Map } } -} - -data class TriggerRunResult( - val triggerName: String, - val triggered: Boolean, - val error: Exception? = null, - val actionResults: MutableMap = mutableMapOf() -) : Writeable, ToXContent { - - @Throws(IOException::class) - constructor(sin: StreamInput) : this( - sin.readString(), // triggerName - sin.readBoolean(), // triggered - sin.readException(), // error - suppressWarning(sin.readMap()) // actionResults - ) - - override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { - var msg = error?.message - if (error is ScriptException) msg = error.toJsonString() - return builder.startObject() - .field("name", triggerName) - .field("triggered", triggered) - .field("error", msg) - .field("action_results", actionResults as Map) - .endObject() - } - /** Returns error information to store in the Alert. Currently it's just the stack trace but it can be more */ - fun alertError(): AlertError? { - if (error != null) { - return AlertError(Instant.now(), "Failed evaluating trigger:\n${error.userErrorMessage()}") - } - for (actionResult in actionResults.values) { - if (actionResult.error != null) { - return AlertError(Instant.now(), "Failed running action:\n${actionResult.error.userErrorMessage()}") + fun afterKeysPresent(): Boolean { + aggTriggersAfterKey?.forEach { + if (it.value.afterKey != null && !it.value.lastPage) { + return true } } - return null - } - - @Throws(IOException::class) - override fun writeTo(out: StreamOutput) { - out.writeString(triggerName) - out.writeBoolean(triggered) - out.writeException(error) - out.writeMap(actionResults as Map) - } - - companion object { - @JvmStatic - @Throws(IOException::class) - fun readFrom(sin: StreamInput): TriggerRunResult { - return TriggerRunResult(sin) - } - - @Suppress("UNCHECKED_CAST") - fun suppressWarning(map: MutableMap?): MutableMap { - return map as MutableMap - } + return false } } +data class TriggerAfterKey(val afterKey: Map?, val lastPage: Boolean) + data class ActionRunResult( val actionId: String, val actionName: String, @@ -264,7 +219,7 @@ data class ActionRunResult( private val logger = LogManager.getLogger(MonitorRunResult::class.java) /** Constructs an error message from an exception suitable for human consumption. */ -private fun Throwable.userErrorMessage(): String { +fun Throwable.userErrorMessage(): String { return when { this is ScriptException -> this.scriptStack.joinToString(separator = "\n", limit = 100) this is OpenSearchException -> this.detailedMessage diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/QueryLevelTrigger.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/QueryLevelTrigger.kt new file mode 100644 index 000000000..67cc8c2e0 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/QueryLevelTrigger.kt @@ -0,0 +1,190 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.model + +import org.opensearch.alerting.model.Trigger.Companion.ACTIONS_FIELD +import org.opensearch.alerting.model.Trigger.Companion.ID_FIELD +import org.opensearch.alerting.model.Trigger.Companion.NAME_FIELD +import org.opensearch.alerting.model.Trigger.Companion.SEVERITY_FIELD +import org.opensearch.alerting.model.action.Action +import org.opensearch.common.CheckedFunction +import org.opensearch.common.ParseField +import org.opensearch.common.UUIDs +import org.opensearch.common.io.stream.StreamInput +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.xcontent.NamedXContentRegistry +import org.opensearch.common.xcontent.ToXContent +import org.opensearch.common.xcontent.XContentBuilder +import org.opensearch.common.xcontent.XContentParser +import org.opensearch.common.xcontent.XContentParser.Token +import org.opensearch.common.xcontent.XContentParserUtils +import org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken +import org.opensearch.script.Script +import java.io.IOException + +/** + * A single-alert Trigger that uses Painless scripts which execute on the response of the Monitor input query to define + * alerting conditions. + */ +data class QueryLevelTrigger( + override val id: String = UUIDs.base64UUID(), + override val name: String, + override val severity: String, + override val actions: List, + val condition: Script +) : Trigger { + + @Throws(IOException::class) + constructor(sin: StreamInput) : this( + sin.readString(), // id + sin.readString(), // name + sin.readString(), // severity + sin.readList(::Action), // actions + Script(sin) // condition + ) + + override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { + builder.startObject() + .startObject(QUERY_LEVEL_TRIGGER_FIELD) + .field(ID_FIELD, id) + .field(NAME_FIELD, name) + .field(SEVERITY_FIELD, severity) + .startObject(CONDITION_FIELD) + .field(SCRIPT_FIELD, condition) + .endObject() + .field(ACTIONS_FIELD, actions.toTypedArray()) + .endObject() + .endObject() + return builder + } + + override fun name(): String { + return QUERY_LEVEL_TRIGGER_FIELD + } + + /** Returns a representation of the trigger suitable for passing into painless and mustache scripts. */ + fun asTemplateArg(): Map { + return mapOf( + ID_FIELD to id, NAME_FIELD to name, SEVERITY_FIELD to severity, + ACTIONS_FIELD to actions.map { it.asTemplateArg() } + ) + } + + @Throws(IOException::class) + override fun writeTo(out: StreamOutput) { + out.writeString(id) + out.writeString(name) + out.writeString(severity) + out.writeCollection(actions) + condition.writeTo(out) + } + + companion object { + const val QUERY_LEVEL_TRIGGER_FIELD = "query_level_trigger" + const val CONDITION_FIELD = "condition" + const val SCRIPT_FIELD = "script" + + val XCONTENT_REGISTRY = NamedXContentRegistry.Entry( + Trigger::class.java, ParseField(QUERY_LEVEL_TRIGGER_FIELD), + CheckedFunction { parseInner(it) } + ) + + /** + * This parse method needs to account for both the old and new Trigger format. + * In the old format, only one Trigger existed (which is now QueryLevelTrigger) and it was + * not a named object. + * + * The parse() method in the Trigger interface needs to consume the outer START_OBJECT to be able + * to infer whether it is dealing with the old or new Trigger format. This means that the currentToken at + * the time this parseInner method is called could differ based on which format is being dealt with. + * + * Old Format + * ---------- + * { + * "id": ..., + * ^ + * Current token starts here + * "name" ..., + * ... + * } + * + * New Format + * ---------- + * { + * "query_level_trigger": { + * "id": ..., ^ Current token starts here + * "name": ..., + * ... + * } + * } + * + * It isn't typically conventional but this parse method will account for both START_OBJECT + * and FIELD_NAME as the starting token to cover both cases. + */ + @JvmStatic @Throws(IOException::class) + fun parseInner(xcp: XContentParser): QueryLevelTrigger { + var id = UUIDs.base64UUID() // assign a default triggerId if one is not specified + lateinit var name: String + lateinit var severity: String + lateinit var condition: Script + val actions: MutableList = mutableListOf() + + if (xcp.currentToken() != Token.START_OBJECT && xcp.currentToken() != Token.FIELD_NAME) { + XContentParserUtils.throwUnknownToken(xcp.currentToken(), xcp.tokenLocation) + } + + // If the parser began on START_OBJECT, move to the next token so that the while loop enters on + // the fieldName (or END_OBJECT if it's empty). + if (xcp.currentToken() == Token.START_OBJECT) xcp.nextToken() + + while (xcp.currentToken() != Token.END_OBJECT) { + val fieldName = xcp.currentName() + + xcp.nextToken() + when (fieldName) { + ID_FIELD -> id = xcp.text() + NAME_FIELD -> name = xcp.text() + SEVERITY_FIELD -> severity = xcp.text() + CONDITION_FIELD -> { + xcp.nextToken() + condition = Script.parse(xcp) + require(condition.lang == Script.DEFAULT_SCRIPT_LANG) { + "Invalid script language. Allowed languages are [${Script.DEFAULT_SCRIPT_LANG}]" + } + xcp.nextToken() + } + ACTIONS_FIELD -> { + ensureExpectedToken(Token.START_ARRAY, xcp.currentToken(), xcp) + while (xcp.nextToken() != Token.END_ARRAY) { + actions.add(Action.parse(xcp)) + } + } + } + xcp.nextToken() + } + + return QueryLevelTrigger( + name = requireNotNull(name) { "Trigger name is null" }, + severity = requireNotNull(severity) { "Trigger severity is null" }, + condition = requireNotNull(condition) { "Trigger condition is null" }, + actions = requireNotNull(actions) { "Trigger actions are null" }, + id = requireNotNull(id) { "Trigger id is null." } + ) + } + + @JvmStatic + @Throws(IOException::class) + fun readFrom(sin: StreamInput): QueryLevelTrigger { + return QueryLevelTrigger(sin) + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/QueryLevelTriggerRunResult.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/QueryLevelTriggerRunResult.kt new file mode 100644 index 000000000..0ed7f9b1a --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/QueryLevelTriggerRunResult.kt @@ -0,0 +1,72 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.model + +import org.opensearch.alerting.alerts.AlertError +import org.opensearch.common.io.stream.StreamInput +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.xcontent.ToXContent +import org.opensearch.common.xcontent.XContentBuilder +import org.opensearch.script.ScriptException +import java.io.IOException +import java.time.Instant + +data class QueryLevelTriggerRunResult( + override var triggerName: String, + var triggered: Boolean, + override var error: Exception?, + var actionResults: MutableMap = mutableMapOf() +) : TriggerRunResult(triggerName, error) { + + @Throws(IOException::class) + @Suppress("UNCHECKED_CAST") + constructor(sin: StreamInput) : this( + triggerName = sin.readString(), + error = sin.readException(), + triggered = sin.readBoolean(), + actionResults = sin.readMap() as MutableMap + ) + + override fun alertError(): AlertError? { + if (error != null) { + return AlertError(Instant.now(), "Failed evaluating trigger:\n${error!!.userErrorMessage()}") + } + for (actionResult in actionResults.values) { + if (actionResult.error != null) { + return AlertError(Instant.now(), "Failed running action:\n${actionResult.error.userErrorMessage()}") + } + } + return null + } + + override fun internalXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { + if (error is ScriptException) error = Exception((error as ScriptException).toJsonString(), error) + return builder + .field("triggered", triggered) + .field("action_results", actionResults as Map) + } + + @Throws(IOException::class) + override fun writeTo(out: StreamOutput) { + super.writeTo(out) + out.writeBoolean(triggered) + out.writeMap(actionResults as Map) + } + + companion object { + @JvmStatic + @Throws(IOException::class) + fun readFrom(sin: StreamInput): TriggerRunResult { + return QueryLevelTriggerRunResult(sin) + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/Trigger.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/Trigger.kt index 6cee62bfc..127240748 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/model/Trigger.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/Trigger.kt @@ -26,120 +26,77 @@ package org.opensearch.alerting.model +import org.opensearch.alerting.core.model.ScheduledJob.Companion.SCHEDULED_JOBS_INDEX import org.opensearch.alerting.model.action.Action -import org.opensearch.common.UUIDs import org.opensearch.common.io.stream.StreamInput -import org.opensearch.common.io.stream.StreamOutput import org.opensearch.common.io.stream.Writeable -import org.opensearch.common.xcontent.ToXContent -import org.opensearch.common.xcontent.XContentBuilder +import org.opensearch.common.xcontent.ToXContentObject import org.opensearch.common.xcontent.XContentParser import org.opensearch.common.xcontent.XContentParser.Token import org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken -import org.opensearch.script.Script import java.io.IOException -data class Trigger( - val name: String, - val severity: String, - val condition: Script, - val actions: List, - val id: String = UUIDs.base64UUID() -) : Writeable, ToXContent { +interface Trigger : Writeable, ToXContentObject { - @Throws(IOException::class) - constructor(sin: StreamInput) : this( - sin.readString(), // name - sin.readString(), // severity - Script(sin), // condition - sin.readList(::Action), // actions - sin.readString() // id - ) - override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { - builder.startObject() - .field(ID_FIELD, id) - .field(NAME_FIELD, name) - .field(SEVERITY_FIELD, severity) - .startObject(CONDITION_FIELD) - .field(SCRIPT_FIELD, condition) - .endObject() - .field(ACTIONS_FIELD, actions.toTypedArray()) - .endObject() - return builder - } - - /** Returns a representation of the trigger suitable for passing into painless and mustache scripts. */ - fun asTemplateArg(): Map { - return mapOf( - ID_FIELD to id, NAME_FIELD to name, SEVERITY_FIELD to severity, - ACTIONS_FIELD to actions.map { it.asTemplateArg() } - ) - } + enum class Type(val value: String) { + QUERY_LEVEL_TRIGGER(QueryLevelTrigger.QUERY_LEVEL_TRIGGER_FIELD), + BUCKET_LEVEL_TRIGGER(BucketLevelTrigger.BUCKET_LEVEL_TRIGGER_FIELD); - @Throws(IOException::class) - override fun writeTo(out: StreamOutput) { - out.writeString(name) - out.writeString(severity) - condition.writeTo(out) - out.writeCollection(actions) - out.writeString(id) + override fun toString(): String { + return value + } } companion object { const val ID_FIELD = "id" const val NAME_FIELD = "name" const val SEVERITY_FIELD = "severity" - const val CONDITION_FIELD = "condition" const val ACTIONS_FIELD = "actions" - const val SCRIPT_FIELD = "script" - @JvmStatic @Throws(IOException::class) + @Throws(IOException::class) fun parse(xcp: XContentParser): Trigger { - var id = UUIDs.base64UUID() // assign a default triggerId if one is not specified - lateinit var name: String - lateinit var severity: String - lateinit var condition: Script - val actions: MutableList = mutableListOf() - ensureExpectedToken(Token.START_OBJECT, xcp.currentToken(), xcp) + val trigger: Trigger - while (xcp.nextToken() != Token.END_OBJECT) { - val fieldName = xcp.currentName() - - xcp.nextToken() - when (fieldName) { - ID_FIELD -> id = xcp.text() - NAME_FIELD -> name = xcp.text() - SEVERITY_FIELD -> severity = xcp.text() - CONDITION_FIELD -> { - xcp.nextToken() - condition = Script.parse(xcp) - require(condition.lang == Script.DEFAULT_SCRIPT_LANG) { - "Invalid script language. Allowed languages are [${Script.DEFAULT_SCRIPT_LANG}]" - } - xcp.nextToken() - } - ACTIONS_FIELD -> { - ensureExpectedToken(Token.START_ARRAY, xcp.currentToken(), xcp) - while (xcp.nextToken() != Token.END_ARRAY) { - actions.add(Action.parse(xcp)) - } - } - } + ensureExpectedToken(Token.START_OBJECT, xcp.currentToken(), xcp) + ensureExpectedToken(Token.FIELD_NAME, xcp.nextToken(), xcp) + val triggerTypeNames = Type.values().map { it.toString() } + if (triggerTypeNames.contains(xcp.currentName())) { + ensureExpectedToken(Token.START_OBJECT, xcp.nextToken(), xcp) + trigger = xcp.namedObject(Trigger::class.java, xcp.currentName(), null) + ensureExpectedToken(Token.END_OBJECT, xcp.nextToken(), xcp) + } else { + // Infer the old Trigger (now called QueryLevelTrigger) when it is not defined as a named + // object to remain backwards compatible when parsing the old format + trigger = QueryLevelTrigger.parseInner(xcp) + ensureExpectedToken(Token.END_OBJECT, xcp.currentToken(), xcp) } - - return Trigger( - name = requireNotNull(name) { "Trigger name is null" }, - severity = requireNotNull(severity) { "Trigger severity is null" }, - condition = requireNotNull(condition) { "Trigger is null" }, - actions = requireNotNull(actions) { "Trigger actions are null" }, - id = requireNotNull(id) { "Trigger id is null." } - ) + return trigger } @JvmStatic @Throws(IOException::class) fun readFrom(sin: StreamInput): Trigger { - return Trigger(sin) + return when (val type = sin.readEnum(Trigger.Type::class.java)) { + Type.QUERY_LEVEL_TRIGGER -> QueryLevelTrigger(sin) + Type.BUCKET_LEVEL_TRIGGER -> BucketLevelTrigger(sin) + // This shouldn't be reachable but ensuring exhaustiveness as Kotlin warns + // enum can be null in Java + else -> throw IllegalStateException("Unexpected input [$type] when reading Trigger") + } } } + + /** The id of the Trigger in the [SCHEDULED_JOBS_INDEX] */ + val id: String + + /** The name of the Trigger */ + val name: String + + /** The severity of the Trigger, used to classify the subsequent Alert */ + val severity: String + + /** The actions executed if the Trigger condition evaluates to true */ + val actions: List + + fun name(): String } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/TriggerRunResult.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/TriggerRunResult.kt new file mode 100644 index 000000000..cd05bcdff --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/TriggerRunResult.kt @@ -0,0 +1,61 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.model + +import org.opensearch.alerting.alerts.AlertError +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.io.stream.Writeable +import org.opensearch.common.xcontent.ToXContent +import org.opensearch.common.xcontent.XContentBuilder +import java.io.IOException +import java.time.Instant + +abstract class TriggerRunResult( + open var triggerName: String, + open var error: Exception? = null +) : Writeable, ToXContent { + + override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { + builder.startObject() + .field("name", triggerName) + + internalXContent(builder, params) + val msg = error?.message + + builder.field("error", msg) + .endObject() + return builder + } + + abstract fun internalXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder + + /** Returns error information to store in the Alert. Currently it's just the stack trace but it can be more */ + open fun alertError(): AlertError? { + if (error != null) { + return AlertError(Instant.now(), "Failed evaluating trigger:\n${error!!.userErrorMessage()}") + } + return null + } + + @Throws(IOException::class) + override fun writeTo(out: StreamOutput) { + out.writeString(triggerName) + out.writeException(error) + } + + companion object { + @Suppress("UNCHECKED_CAST") + fun suppressWarning(map: MutableMap?): MutableMap { + return map as MutableMap + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/action/Action.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/action/Action.kt index e05398d22..e3f030175 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/model/action/Action.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/action/Action.kt @@ -48,14 +48,19 @@ data class Action( val messageTemplate: Script, val throttleEnabled: Boolean, val throttle: Throttle?, - val id: String = UUIDs.base64UUID() + val id: String = UUIDs.base64UUID(), + val actionExecutionPolicy: ActionExecutionPolicy? = null ) : Writeable, ToXContentObject { init { if (subjectTemplate != null) { - require(subjectTemplate.lang == Action.MUSTACHE) { "subject_template must be a mustache script" } + require(subjectTemplate.lang == MUSTACHE) { "subject_template must be a mustache script" } + } + require(messageTemplate.lang == MUSTACHE) { "message_template must be a mustache script" } + + if (actionExecutionPolicy?.actionExecutionScope is PerExecutionActionScope) { + require(throttle == null) { "Throttle is currently not supported for per execution action scope" } } - require(messageTemplate.lang == Action.MUSTACHE) { "message_template must be a mustache script" } } @Throws(IOException::class) @@ -66,7 +71,8 @@ data class Action( Script(sin), // messageTemplate sin.readBoolean(), // throttleEnabled sin.readOptionalWriteable(::Throttle), // throttle - sin.readString() // id + sin.readString(), // id + sin.readOptionalWriteable(::ActionExecutionPolicy) // actionExecutionPolicy ) override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { @@ -82,6 +88,9 @@ data class Action( if (throttle != null) { xContentBuilder.field(THROTTLE_FIELD, throttle) } + if (actionExecutionPolicy != null) { + xContentBuilder.field(ACTION_EXECUTION_POLICY_FIELD, actionExecutionPolicy) + } return xContentBuilder.endObject() } @@ -108,6 +117,12 @@ data class Action( out.writeBoolean(false) } out.writeString(id) + if (actionExecutionPolicy != null) { + out.writeBoolean(true) + actionExecutionPolicy.writeTo(out) + } else { + out.writeBoolean(false) + } } companion object { @@ -118,6 +133,7 @@ data class Action( const val MESSAGE_TEMPLATE_FIELD = "message_template" const val THROTTLE_ENABLED_FIELD = "throttle_enabled" const val THROTTLE_FIELD = "throttle" + const val ACTION_EXECUTION_POLICY_FIELD = "action_execution_policy" const val MUSTACHE = "mustache" const val SUBJECT = "subject" const val MESSAGE = "message" @@ -133,6 +149,7 @@ data class Action( lateinit var messageTemplate: Script var throttleEnabled = false var throttle: Throttle? = null + var actionExecutionPolicy: ActionExecutionPolicy? = null XContentParserUtils.ensureExpectedToken(XContentParser.Token.START_OBJECT, xcp.currentToken(), xcp) while (xcp.nextToken() != XContentParser.Token.END_OBJECT) { @@ -153,7 +170,13 @@ data class Action( THROTTLE_ENABLED_FIELD -> { throttleEnabled = xcp.booleanValue() } - + ACTION_EXECUTION_POLICY_FIELD -> { + actionExecutionPolicy = if (xcp.currentToken() == XContentParser.Token.VALUE_NULL) { + null + } else { + ActionExecutionPolicy.parse(xcp) + } + } else -> { throw IllegalStateException("Unexpected field: $fieldName, while parsing action") } @@ -171,7 +194,8 @@ data class Action( requireNotNull(messageTemplate) { "Action message template is null" }, throttleEnabled, throttle, - id = requireNotNull(id) + id = requireNotNull(id), + actionExecutionPolicy = actionExecutionPolicy ) } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/action/ActionExecutionPolicy.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/action/ActionExecutionPolicy.kt new file mode 100644 index 000000000..23bc294e0 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/action/ActionExecutionPolicy.kt @@ -0,0 +1,95 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.model.action + +import org.opensearch.common.io.stream.StreamInput +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.io.stream.Writeable +import org.opensearch.common.xcontent.ToXContent +import org.opensearch.common.xcontent.ToXContentObject +import org.opensearch.common.xcontent.XContentBuilder +import org.opensearch.common.xcontent.XContentParser +import org.opensearch.common.xcontent.XContentParser.Token +import org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken +import java.io.IOException + +/** + * This class represents the container for various configurations which control Action behavior. + */ +data class ActionExecutionPolicy( + val actionExecutionScope: ActionExecutionScope +) : Writeable, ToXContentObject { + + @Throws(IOException::class) + constructor(sin: StreamInput) : this ( + ActionExecutionScope.readFrom(sin) // actionExecutionScope + ) + + override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { + builder.startObject() + .field(ACTION_EXECUTION_SCOPE, actionExecutionScope) + return builder.endObject() + } + + @Throws(IOException::class) + override fun writeTo(out: StreamOutput) { + if (actionExecutionScope is PerAlertActionScope) { + out.writeEnum(ActionExecutionScope.Type.PER_ALERT) + } else { + out.writeEnum(ActionExecutionScope.Type.PER_EXECUTION) + } + actionExecutionScope.writeTo(out) + } + + companion object { + const val ACTION_EXECUTION_SCOPE = "action_execution_scope" + + @JvmStatic + @Throws(IOException::class) + fun parse(xcp: XContentParser): ActionExecutionPolicy { + lateinit var actionExecutionScope: ActionExecutionScope + + ensureExpectedToken(Token.START_OBJECT, xcp.currentToken(), xcp) + while (xcp.nextToken() != Token.END_OBJECT) { + val fieldName = xcp.currentName() + xcp.nextToken() + + when (fieldName) { + ACTION_EXECUTION_SCOPE -> actionExecutionScope = ActionExecutionScope.parse(xcp) + } + } + + return ActionExecutionPolicy( + requireNotNull(actionExecutionScope) { "Action execution scope is null" } + ) + } + + @JvmStatic + @Throws(IOException::class) + fun readFrom(sin: StreamInput): ActionExecutionPolicy { + return ActionExecutionPolicy(sin) + } + + /** + * The default [ActionExecutionPolicy] configuration for Bucket-Level Monitors. + * + * If Query-Level Monitors integrate the use of [ActionExecutionPolicy] then a separate default configuration + * will need to be made depending on the desired behavior. + */ + fun getDefaultConfigurationForBucketLevelMonitor(): ActionExecutionPolicy { + val defaultActionExecutionScope = PerAlertActionScope( + actionableAlerts = setOf(AlertCategory.DEDUPED, AlertCategory.NEW) + ) + return ActionExecutionPolicy(actionExecutionScope = defaultActionExecutionScope) + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/model/action/ActionExecutionScope.kt b/alerting/src/main/kotlin/org/opensearch/alerting/model/action/ActionExecutionScope.kt new file mode 100644 index 000000000..e1dcd1fd5 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/model/action/ActionExecutionScope.kt @@ -0,0 +1,182 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.model.action + +import org.opensearch.common.io.stream.StreamInput +import org.opensearch.common.io.stream.StreamOutput +import org.opensearch.common.io.stream.Writeable +import org.opensearch.common.xcontent.ToXContent +import org.opensearch.common.xcontent.ToXContentObject +import org.opensearch.common.xcontent.XContentBuilder +import org.opensearch.common.xcontent.XContentParser +import org.opensearch.common.xcontent.XContentParser.Token +import org.opensearch.common.xcontent.XContentParserUtils.ensureExpectedToken +import java.io.IOException +import java.lang.IllegalArgumentException + +/** + * This class represents configurations used to control the scope of Action executions when Alerts are created. + */ +sealed class ActionExecutionScope : Writeable, ToXContentObject { + + enum class Type { PER_ALERT, PER_EXECUTION } + + companion object { + const val PER_ALERT_FIELD = "per_alert" + const val PER_EXECUTION_FIELD = "per_execution" + const val ACTIONABLE_ALERTS_FIELD = "actionable_alerts" + + @JvmStatic + @Throws(IOException::class) + fun parse(xcp: XContentParser): ActionExecutionScope { + var type: Type? = null + var actionExecutionScope: ActionExecutionScope? = null + val alertFilter = mutableSetOf() + + ensureExpectedToken(Token.START_OBJECT, xcp.currentToken(), xcp) + while (xcp.nextToken() != Token.END_OBJECT) { + val fieldName = xcp.currentName() + xcp.nextToken() + + // If the type field has already been set, the user has provided more than one type of schedule + if (type != null) { + throw IllegalArgumentException("You can only specify one type of action execution scope.") + } + + when (fieldName) { + PER_ALERT_FIELD -> { + type = Type.PER_ALERT + while (xcp.nextToken() != Token.END_OBJECT) { + val perAlertFieldName = xcp.currentName() + xcp.nextToken() + when (perAlertFieldName) { + ACTIONABLE_ALERTS_FIELD -> { + ensureExpectedToken(Token.START_ARRAY, xcp.currentToken(), xcp) + val allowedCategories = AlertCategory.values().map { it.toString() } + while (xcp.nextToken() != Token.END_ARRAY) { + val alertCategory = xcp.text() + if (!allowedCategories.contains(alertCategory)) { + throw IllegalStateException("Actionable alerts should be one of $allowedCategories") + } + alertFilter.add(AlertCategory.valueOf(alertCategory)) + } + } + else -> throw IllegalArgumentException( + "Invalid field [$perAlertFieldName] found in per alert action execution scope." + ) + } + } + } + PER_EXECUTION_FIELD -> { + type = Type.PER_EXECUTION + while (xcp.nextToken() != Token.END_OBJECT) {} + } + else -> throw IllegalArgumentException("Invalid field [$fieldName] found in action execution scope.") + } + } + + if (type == Type.PER_ALERT) { + actionExecutionScope = PerAlertActionScope(alertFilter) + } else if (type == Type.PER_EXECUTION) { + actionExecutionScope = PerExecutionActionScope() + } + + return requireNotNull(actionExecutionScope) { "Action execution scope is null." } + } + + @JvmStatic + @Throws(IOException::class) + fun readFrom(sin: StreamInput): ActionExecutionScope { + val type = sin.readEnum(ActionExecutionScope.Type::class.java) + return if (type == Type.PER_ALERT) { + PerAlertActionScope(sin) + } else { + PerExecutionActionScope(sin) + } + } + } + + abstract fun getExecutionScope(): Type +} + +data class PerAlertActionScope( + val actionableAlerts: Set +) : ActionExecutionScope() { + + @Throws(IOException::class) + constructor(sin: StreamInput) : this( + sin.readSet { si -> si.readEnum(AlertCategory::class.java) } // alertFilter + ) + + override fun getExecutionScope(): Type = Type.PER_ALERT + + override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { + builder.startObject() + .startObject(PER_ALERT_FIELD) + .field(ACTIONABLE_ALERTS_FIELD, actionableAlerts.toTypedArray()) + .endObject() + return builder.endObject() + } + + @Throws(IOException::class) + override fun writeTo(out: StreamOutput) { + out.writeCollection(actionableAlerts) { o, v -> o.writeEnum(v) } + } + + companion object { + @JvmStatic + @Throws(IOException::class) + fun readFrom(sin: StreamInput): PerAlertActionScope { + return PerAlertActionScope(sin) + } + } +} + +class PerExecutionActionScope() : ActionExecutionScope() { + + @Throws(IOException::class) + constructor(sin: StreamInput) : this() + + override fun hashCode(): Int { + return javaClass.hashCode() + } + + // Creating an equals method that just checks class type rather than reference since this is currently stateless. + // Otherwise, it would have been a dataclass which would have handled this. + override fun equals(other: Any?): Boolean { + if (this === other) return true + if (other?.javaClass != javaClass) return false + return true + } + + override fun getExecutionScope(): Type = Type.PER_EXECUTION + + override fun toXContent(builder: XContentBuilder, params: ToXContent.Params): XContentBuilder { + builder.startObject() + .startObject(PER_EXECUTION_FIELD) + .endObject() + return builder.endObject() + } + + @Throws(IOException::class) + override fun writeTo(out: StreamOutput) {} + + companion object { + @JvmStatic + @Throws(IOException::class) + fun readFrom(sin: StreamInput): PerExecutionActionScope { + return PerExecutionActionScope(sin) + } + } +} + +enum class AlertCategory { DEDUPED, NEW, COMPLETED } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/resthandler/RestSearchMonitorAction.kt b/alerting/src/main/kotlin/org/opensearch/alerting/resthandler/RestSearchMonitorAction.kt index ea67a66f3..d19cd6560 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/resthandler/RestSearchMonitorAction.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/resthandler/RestSearchMonitorAction.kt @@ -33,6 +33,7 @@ import org.opensearch.alerting.action.SearchMonitorAction import org.opensearch.alerting.action.SearchMonitorRequest import org.opensearch.alerting.core.model.ScheduledJob import org.opensearch.alerting.core.model.ScheduledJob.Companion.SCHEDULED_JOBS_INDEX +import org.opensearch.alerting.model.Monitor import org.opensearch.alerting.settings.AlertingSettings import org.opensearch.alerting.util.context import org.opensearch.client.node.NodeClient @@ -112,6 +113,7 @@ class RestSearchMonitorAction( searchSourceBuilder.fetchSource(context(request)) val queryBuilder = QueryBuilders.boolQuery().must(searchSourceBuilder.query()) + queryBuilder.filter(QueryBuilders.existsQuery(Monitor.MONITOR_TYPE)) searchSourceBuilder.query(queryBuilder) .seqNoAndPrimaryTerm(true) diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/script/BucketLevelTriggerExecutionContext.kt b/alerting/src/main/kotlin/org/opensearch/alerting/script/BucketLevelTriggerExecutionContext.kt new file mode 100644 index 000000000..d79f7d5ea --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/script/BucketLevelTriggerExecutionContext.kt @@ -0,0 +1,57 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.script + +import org.opensearch.alerting.model.Alert +import org.opensearch.alerting.model.BucketLevelTrigger +import org.opensearch.alerting.model.BucketLevelTriggerRunResult +import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.MonitorRunResult +import java.time.Instant + +data class BucketLevelTriggerExecutionContext( + override val monitor: Monitor, + val trigger: BucketLevelTrigger, + override val results: List>, + override val periodStart: Instant, + override val periodEnd: Instant, + val dedupedAlerts: List = listOf(), + val newAlerts: List = listOf(), + val completedAlerts: List = listOf(), + override val error: Exception? = null +) : TriggerExecutionContext(monitor, results, periodStart, periodEnd, error) { + + constructor( + monitor: Monitor, + trigger: BucketLevelTrigger, + monitorRunResult: MonitorRunResult, + dedupedAlerts: List = listOf(), + newAlerts: List = listOf(), + completedAlerts: List = listOf() + ) : this( + monitor, trigger, monitorRunResult.inputResults.results, monitorRunResult.periodStart, monitorRunResult.periodEnd, + dedupedAlerts, newAlerts, completedAlerts, monitorRunResult.scriptContextError(trigger) + ) + + /** + * Mustache templates need special permissions to reflectively introspect field names. To avoid doing this we + * translate the context to a Map of Strings to primitive types, which can be accessed without reflection. + */ + override fun asTemplateArg(): Map { + val tempArg = super.asTemplateArg().toMutableMap() + tempArg["trigger"] = trigger.asTemplateArg() + tempArg["dedupedAlerts"] = dedupedAlerts.map { it.asTemplateArg() } + tempArg["newAlerts"] = newAlerts.map { it.asTemplateArg() } + tempArg["completedAlerts"] = completedAlerts.map { it.asTemplateArg() } + return tempArg + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/script/QueryLevelTriggerExecutionContext.kt b/alerting/src/main/kotlin/org/opensearch/alerting/script/QueryLevelTriggerExecutionContext.kt new file mode 100644 index 000000000..e047da7f5 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/script/QueryLevelTriggerExecutionContext.kt @@ -0,0 +1,51 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.script + +import org.opensearch.alerting.model.Alert +import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.MonitorRunResult +import org.opensearch.alerting.model.QueryLevelTrigger +import org.opensearch.alerting.model.QueryLevelTriggerRunResult +import java.time.Instant + +data class QueryLevelTriggerExecutionContext( + override val monitor: Monitor, + val trigger: QueryLevelTrigger, + override val results: List>, + override val periodStart: Instant, + override val periodEnd: Instant, + val alert: Alert? = null, + override val error: Exception? = null +) : TriggerExecutionContext(monitor, results, periodStart, periodEnd, error) { + + constructor( + monitor: Monitor, + trigger: QueryLevelTrigger, + monitorRunResult: MonitorRunResult, + alert: Alert? = null + ) : this( + monitor, trigger, monitorRunResult.inputResults.results, monitorRunResult.periodStart, monitorRunResult.periodEnd, + alert, monitorRunResult.scriptContextError(trigger) + ) + + /** + * Mustache templates need special permissions to reflectively introspect field names. To avoid doing this we + * translate the context to a Map of Strings to primitive types, which can be accessed without reflection. + */ + override fun asTemplateArg(): Map { + val tempArg = super.asTemplateArg().toMutableMap() + tempArg["trigger"] = trigger.asTemplateArg() + tempArg["alert"] = alert?.asTemplateArg() + return tempArg + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/script/TriggerExecutionContext.kt b/alerting/src/main/kotlin/org/opensearch/alerting/script/TriggerExecutionContext.kt index 0f6c65d6f..d5ac7c35a 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/script/TriggerExecutionContext.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/script/TriggerExecutionContext.kt @@ -26,40 +26,35 @@ package org.opensearch.alerting.script -import org.opensearch.alerting.model.Alert import org.opensearch.alerting.model.Monitor import org.opensearch.alerting.model.MonitorRunResult import org.opensearch.alerting.model.Trigger import java.time.Instant -data class TriggerExecutionContext( - val monitor: Monitor, - val trigger: Trigger, - val results: List>, - val periodStart: Instant, - val periodEnd: Instant, - val alert: Alert? = null, - val error: Exception? = null +abstract class TriggerExecutionContext( + open val monitor: Monitor, + open val results: List>, + open val periodStart: Instant, + open val periodEnd: Instant, + open val error: Exception? = null ) { - constructor(monitor: Monitor, trigger: Trigger, monitorRunResult: MonitorRunResult, alert: Alert? = null) : + constructor(monitor: Monitor, trigger: Trigger, monitorRunResult: MonitorRunResult<*>) : this( - monitor, trigger, monitorRunResult.inputResults.results, monitorRunResult.periodStart, - monitorRunResult.periodEnd, alert, monitorRunResult.scriptContextError(trigger) + monitor, monitorRunResult.inputResults.results, monitorRunResult.periodStart, + monitorRunResult.periodEnd, monitorRunResult.scriptContextError(trigger) ) /** * Mustache templates need special permissions to reflectively introspect field names. To avoid doing this we * translate the context to a Map of Strings to primitive types, which can be accessed without reflection. */ - fun asTemplateArg(): Map { + open fun asTemplateArg(): Map { return mapOf( "monitor" to monitor.asTemplateArg(), - "trigger" to trigger.asTemplateArg(), "results" to results, "periodStart" to periodStart, "periodEnd" to periodEnd, - "alert" to alert?.asTemplateArg(), "error" to error ) } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/script/TriggerScript.kt b/alerting/src/main/kotlin/org/opensearch/alerting/script/TriggerScript.kt index 722f8c9a0..3cf345be6 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/script/TriggerScript.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/script/TriggerScript.kt @@ -58,7 +58,7 @@ abstract class TriggerScript(_scriptParams: Map) { * * @param ctx - the trigger execution context */ - abstract fun execute(ctx: TriggerExecutionContext): Boolean + abstract fun execute(ctx: QueryLevelTriggerExecutionContext): Boolean interface Factory { fun newInstance(scriptParams: Map): TriggerScript diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/settings/AlertingSettings.kt b/alerting/src/main/kotlin/org/opensearch/alerting/settings/AlertingSettings.kt index f00a43aa2..c314770dd 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/settings/AlertingSettings.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/settings/AlertingSettings.kt @@ -38,6 +38,7 @@ class AlertingSettings { const val MONITOR_MAX_INPUTS = 1 const val MONITOR_MAX_TRIGGERS = 10 + const val DEFAULT_MAX_ACTIONABLE_ALERT_COUNT = 50L val ALERTING_MAX_MONITORS = Setting.intSetting( "plugins.alerting.monitor.max_monitors", @@ -134,5 +135,12 @@ class AlertingSettings { LegacyOpenDistroAlertingSettings.FILTER_BY_BACKEND_ROLES, Setting.Property.NodeScope, Setting.Property.Dynamic ) + + val MAX_ACTIONABLE_ALERT_COUNT = Setting.longSetting( + "plugins.alerting.max_actionable_alert_count", + DEFAULT_MAX_ACTIONABLE_ALERT_COUNT, + -1L, + Setting.Property.NodeScope, Setting.Property.Dynamic + ) } } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/settings/DestinationSettings.kt b/alerting/src/main/kotlin/org/opensearch/alerting/settings/DestinationSettings.kt index 8533d75b5..01dd8ea2b 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/settings/DestinationSettings.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/settings/DestinationSettings.kt @@ -41,6 +41,7 @@ class DestinationSettings { const val DESTINATION_SETTING_PREFIX = "plugins.alerting.destination." const val EMAIL_DESTINATION_SETTING_PREFIX = DESTINATION_SETTING_PREFIX + "email." + val ALLOW_LIST_NONE = emptyList() val ALLOW_LIST: Setting> = Setting.listSetting( DESTINATION_SETTING_PREFIX + "allow_list", diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/settings/LegacyOpenDistroDestinationSettings.kt b/alerting/src/main/kotlin/org/opensearch/alerting/settings/LegacyOpenDistroDestinationSettings.kt index 48a8ce7eb..c4f63ee57 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/settings/LegacyOpenDistroDestinationSettings.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/settings/LegacyOpenDistroDestinationSettings.kt @@ -29,7 +29,7 @@ class LegacyOpenDistroDestinationSettings { const val DESTINATION_SETTING_PREFIX = "opendistro.alerting.destination." const val EMAIL_DESTINATION_SETTING_PREFIX = DESTINATION_SETTING_PREFIX + "email." val ALLOW_LIST_ALL = DestinationType.values().toList().map { it.value } - val DENY_LIST_NONE = emptyList() + val HOST_DENY_LIST_NONE = emptyList() val ALLOW_LIST: Setting> = Setting.listSetting( DESTINATION_SETTING_PREFIX + "allow_list", @@ -56,7 +56,7 @@ class LegacyOpenDistroDestinationSettings { val HOST_DENY_LIST: Setting> = Setting.listSetting( "opendistro.destination.host.deny_list", - DENY_LIST_NONE, + HOST_DENY_LIST_NONE, Function.identity(), Setting.Property.NodeScope, Setting.Property.Final, diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/transport/TransportExecuteMonitorAction.kt b/alerting/src/main/kotlin/org/opensearch/alerting/transport/TransportExecuteMonitorAction.kt index 7243b033c..799b33eac 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/transport/TransportExecuteMonitorAction.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/transport/TransportExecuteMonitorAction.kt @@ -28,6 +28,7 @@ import org.opensearch.alerting.action.ExecuteMonitorResponse import org.opensearch.alerting.core.model.ScheduledJob import org.opensearch.alerting.model.Monitor import org.opensearch.alerting.util.AlertingException +import org.opensearch.alerting.util.isBucketLevelMonitor import org.opensearch.client.Client import org.opensearch.common.inject.Inject import org.opensearch.common.xcontent.LoggingDeprecationHandler @@ -68,7 +69,11 @@ class TransportExecuteMonitorAction @Inject constructor( val (periodStart, periodEnd) = monitor.schedule.getPeriodEndingAt(Instant.ofEpochMilli(execMonitorRequest.requestEnd.millis)) try { - val monitorRunResult = runner.runMonitor(monitor, periodStart, periodEnd, execMonitorRequest.dryrun) + val monitorRunResult = if (monitor.isBucketLevelMonitor()) { + runner.runBucketLevelMonitor(monitor, periodStart, periodEnd, execMonitorRequest.dryrun) + } else { + runner.runQueryLevelMonitor(monitor, periodStart, periodEnd, execMonitorRequest.dryrun) + } withContext(Dispatchers.IO) { actionListener.onResponse(ExecuteMonitorResponse(monitorRunResult)) } diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/util/AggregationQueryRewriter.kt b/alerting/src/main/kotlin/org/opensearch/alerting/util/AggregationQueryRewriter.kt new file mode 100644 index 000000000..22b787836 --- /dev/null +++ b/alerting/src/main/kotlin/org/opensearch/alerting/util/AggregationQueryRewriter.kt @@ -0,0 +1,120 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.util + +import org.opensearch.action.search.SearchResponse +import org.opensearch.alerting.model.BucketLevelTrigger +import org.opensearch.alerting.model.InputRunResults +import org.opensearch.alerting.model.Trigger +import org.opensearch.alerting.model.TriggerAfterKey +import org.opensearch.search.aggregations.AggregationBuilder +import org.opensearch.search.aggregations.AggregatorFactories +import org.opensearch.search.aggregations.bucket.SingleBucketAggregation +import org.opensearch.search.aggregations.bucket.composite.CompositeAggregation +import org.opensearch.search.aggregations.bucket.composite.CompositeAggregationBuilder +import org.opensearch.search.aggregations.support.AggregationPath +import org.opensearch.search.builder.SearchSourceBuilder + +class AggregationQueryRewriter { + + companion object { + /** + * Add the bucket selector conditions for each trigger in input query. It also adds afterKeys from previous result + * for each trigger. + */ + fun rewriteQuery(query: SearchSourceBuilder, prevResult: InputRunResults?, triggers: List): SearchSourceBuilder { + triggers.forEach { trigger -> + if (trigger is BucketLevelTrigger) { + // add bucket selector pipeline aggregation for each trigger in query + query.aggregation(trigger.bucketSelector) + // if this request is processing the subsequent pages of input query result, then add after key + if (prevResult?.aggTriggersAfterKey?.get(trigger.id) != null) { + val parentBucketPath = AggregationPath.parse(trigger.bucketSelector.parentBucketPath) + var aggBuilders = (query.aggregations() as AggregatorFactories.Builder).aggregatorFactories + var factory: AggregationBuilder? = null + for (i in 0 until parentBucketPath.pathElements.size) { + factory = null + for (aggFactory in aggBuilders) { + if (aggFactory.name.equals(parentBucketPath.pathElements[i].name)) { + aggBuilders = aggFactory.subAggregations + factory = aggFactory + break + } + } + if (factory == null) { + throw IllegalArgumentException("ParentBucketPath: $parentBucketPath not found in input query results") + } + } + if (factory is CompositeAggregationBuilder) { + // if the afterKey from previous result is null, what does it signify? + // A) result set exhausted OR B) first page ? + val afterKey = prevResult.aggTriggersAfterKey[trigger.id]!!.afterKey + factory.aggregateAfter(afterKey) + } else { + throw IllegalStateException("AfterKeys are not expected to be present in non CompositeAggregationBuilder") + } + } + } + } + + return query + } + + /** + * For each trigger, returns the after keys if present in query result. + */ + fun getAfterKeysFromSearchResponse( + searchResponse: SearchResponse, + triggers: List, + prevBucketLevelTriggerAfterKeys: Map? + ): Map { + val bucketLevelTriggerAfterKeys = mutableMapOf() + triggers.forEach { trigger -> + if (trigger is BucketLevelTrigger) { + val parentBucketPath = AggregationPath.parse(trigger.bucketSelector.parentBucketPath) + var aggs = searchResponse.aggregations + // assuming all intermediate aggregations as SingleBucketAggregation + for (i in 0 until parentBucketPath.pathElements.size - 1) { + aggs = (aggs.asMap()[parentBucketPath.pathElements[i].name] as SingleBucketAggregation).aggregations + } + val lastAgg = aggs.asMap[parentBucketPath.pathElements.last().name] + // if leaf is CompositeAggregation, then fetch afterKey if present + if (lastAgg is CompositeAggregation) { + /* + * Bucket-Level Triggers can have different parent bucket paths that they are tracking for condition evaluation. + * These different bucket paths could have different page sizes, meaning one could be exhausted while another + * bucket path still has pages to iterate in the query responses. + * + * To ensure that these can be exhausted and tracked independently, the after key that led to the last page (which + * should be an empty result for the bucket path) will be saved when the last page is hit and will be continued + * to be passed on for that bucket path if there are still other bucket paths being paginated. + */ + val afterKey = lastAgg.afterKey() + val prevTriggerAfterKey = prevBucketLevelTriggerAfterKeys?.get(trigger.id) + bucketLevelTriggerAfterKeys[trigger.id] = when { + // If the previous TriggerAfterKey was null, this should be the first page + prevTriggerAfterKey == null -> TriggerAfterKey(afterKey, afterKey == null) + // If the previous TriggerAfterKey already hit the last page, pass along the after key it used to get there + prevTriggerAfterKey.lastPage -> prevTriggerAfterKey + // If the previous TriggerAfterKey had not reached the last page and the after key for the current result + // is null, then the last page has been reached so the after key that was used to get there is stored + afterKey == null -> TriggerAfterKey(prevTriggerAfterKey.afterKey, true) + // Otherwise, update the after key to the current one + else -> TriggerAfterKey(afterKey, false) + } + } + } + } + return bucketLevelTriggerAfterKeys + } + } +} diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/util/AlertingUtils.kt b/alerting/src/main/kotlin/org/opensearch/alerting/util/AlertingUtils.kt index 0f5a74b5c..61f4dd9e0 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/util/AlertingUtils.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/util/AlertingUtils.kt @@ -30,6 +30,11 @@ import inet.ipaddr.IPAddressString import org.opensearch.OpenSearchStatusException import org.opensearch.action.ActionListener import org.opensearch.alerting.destination.message.BaseMessage +import org.opensearch.alerting.model.AggregationResultBucket +import org.opensearch.alerting.model.BucketLevelTriggerRunResult +import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.action.Action +import org.opensearch.alerting.model.action.ActionExecutionPolicy import org.opensearch.alerting.model.destination.Destination import org.opensearch.alerting.settings.DestinationSettings import org.opensearch.commons.authuser.User @@ -139,3 +144,39 @@ fun checkUserFilterByPermissions( } return true } + +fun Monitor.isBucketLevelMonitor(): Boolean = this.monitorType == Monitor.MonitorType.BUCKET_LEVEL_MONITOR + +/** + * Since buckets can have multi-value keys, this converts the bucket key values to a string that can be used + * as the key for a HashMap to easily retrieve [AggregationResultBucket] based on the bucket key values. + */ +fun AggregationResultBucket.getBucketKeysHash(): String = this.bucketKeys.joinToString(separator = "#") + +fun Action.getActionExecutionPolicy(monitor: Monitor): ActionExecutionPolicy? { + // When the ActionExecutionPolicy is null for an Action, the default is resolved at runtime + // so it can be chosen based on the Monitor type at that time. + // The Action config is not aware of the Monitor type which is why the default was not stored during + // the parse. + return this.actionExecutionPolicy ?: if (monitor.isBucketLevelMonitor()) { + ActionExecutionPolicy.getDefaultConfigurationForBucketLevelMonitor() + } else { + null + } +} + +fun BucketLevelTriggerRunResult.getCombinedTriggerRunResult( + prevTriggerRunResult: BucketLevelTriggerRunResult? +): BucketLevelTriggerRunResult { + if (prevTriggerRunResult == null) return this + + // The aggregation results and action results across to two trigger run results should not have overlapping keys + // since they represent different pages of aggregations so a simple concatenation will combine them + val mergedAggregationResultBuckets = prevTriggerRunResult.aggregationResultBuckets + this.aggregationResultBuckets + val mergedActionResultsMap = (prevTriggerRunResult.actionResultsMap + this.actionResultsMap).toMutableMap() + + // Update to the most recent error if it's not null, otherwise keep the old one + val error = this.error ?: prevTriggerRunResult.error + + return this.copy(aggregationResultBuckets = mergedAggregationResultBuckets, actionResultsMap = mergedActionResultsMap, error = error) +} diff --git a/alerting/src/main/resources/org/opensearch/alerting/alerts/alert_mapping.json b/alerting/src/main/resources/org/opensearch/alerting/alerts/alert_mapping.json index c30dfbfd1..4ebba38fa 100644 --- a/alerting/src/main/resources/org/opensearch/alerting/alerts/alert_mapping.json +++ b/alerting/src/main/resources/org/opensearch/alerting/alerts/alert_mapping.json @@ -4,7 +4,7 @@ "required": true }, "_meta" : { - "schema_version": 2 + "schema_version": 3 }, "properties": { "schema_version": { @@ -125,6 +125,17 @@ "type": "integer" } } + }, + "agg_alert_content": { + "dynamic": true, + "properties": { + "parent_bucket_path": { + "type": "text" + }, + "bucket_key": { + "type": "text" + } + } } } } \ No newline at end of file diff --git a/alerting/src/main/resources/org/opensearch/alerting/org.opensearch.alerting.txt b/alerting/src/main/resources/org/opensearch/alerting/org.opensearch.alerting.txt index a5d7b9cea..8359204ca 100644 --- a/alerting/src/main/resources/org/opensearch/alerting/org.opensearch.alerting.txt +++ b/alerting/src/main/resources/org/opensearch/alerting/org.opensearch.alerting.txt @@ -4,7 +4,7 @@ class org.opensearch.alerting.script.TriggerScript { Map getParams() - boolean execute(TriggerExecutionContext) + boolean execute(QueryLevelTriggerExecutionContext) String[] PARAMETERS } @@ -14,7 +14,15 @@ class org.opensearch.alerting.script.TriggerScript$Factory { class org.opensearch.alerting.script.TriggerExecutionContext { Monitor getMonitor() - Trigger getTrigger() + List getResults() + java.time.Instant getPeriodStart() + java.time.Instant getPeriodEnd() + Exception getError() +} + +class org.opensearch.alerting.script.QueryLevelTriggerExecutionContext { + Monitor getMonitor() + QueryLevelTrigger getTrigger() List getResults() java.time.Instant getPeriodStart() java.time.Instant getPeriodEnd() @@ -29,17 +37,13 @@ class org.opensearch.alerting.model.Monitor { boolean getEnabled() } -class org.opensearch.alerting.model.Trigger { +class org.opensearch.alerting.model.QueryLevelTrigger { String getId() String getName() String getSeverity() List getActions() } -class org.opensearch.alerting.model.action.Action { - String getName() -} - class org.opensearch.alerting.model.Alert { String getId() long getVersion() diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/ADTestHelpers.kt b/alerting/src/test/kotlin/org/opensearch/alerting/ADTestHelpers.kt index 996419411..ea7b4d7c2 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/ADTestHelpers.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/ADTestHelpers.kt @@ -30,6 +30,7 @@ import org.opensearch.alerting.core.model.IntervalSchedule import org.opensearch.alerting.core.model.Schedule import org.opensearch.alerting.core.model.SearchInput import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.QueryLevelTrigger import org.opensearch.alerting.model.Trigger import org.opensearch.commons.authuser.User import org.opensearch.index.query.BoolQueryBuilder @@ -490,12 +491,12 @@ fun maxAnomalyGradeSearchInput( return SearchInput(indices = listOf(adResultIndex), query = searchSourceBuilder) } -fun adMonitorTrigger(): Trigger { +fun adMonitorTrigger(): QueryLevelTrigger { val triggerScript = """ return ctx.results[0].aggregations.max_anomaly_grade.value != null && ctx.results[0].aggregations.max_anomaly_grade.value > 0.7 """.trimIndent() - return randomTrigger(condition = Script(triggerScript)) + return randomQueryLevelTrigger(condition = Script(triggerScript)) } fun adSearchInput(detectorId: String): SearchInput { @@ -508,14 +509,14 @@ fun randomADMonitor( inputs: List = listOf(adSearchInput("test_detector_id")), schedule: Schedule = IntervalSchedule(interval = 5, unit = ChronoUnit.MINUTES), enabled: Boolean = OpenSearchTestCase.randomBoolean(), - triggers: List = (1..OpenSearchTestCase.randomInt(10)).map { randomTrigger() }, + triggers: List = (1..OpenSearchTestCase.randomInt(10)).map { randomQueryLevelTrigger() }, enabledTime: Instant? = if (enabled) Instant.now().truncatedTo(ChronoUnit.MILLIS) else null, lastUpdateTime: Instant = Instant.now().truncatedTo(ChronoUnit.MILLIS), withMetadata: Boolean = false ): Monitor { return Monitor( - name = name, enabled = enabled, inputs = inputs, schedule = schedule, triggers = triggers, - enabledTime = enabledTime, lastUpdateTime = lastUpdateTime, + name = name, monitorType = Monitor.MonitorType.QUERY_LEVEL_MONITOR, enabled = enabled, inputs = inputs, + schedule = schedule, triggers = triggers, enabledTime = enabledTime, lastUpdateTime = lastUpdateTime, user = user, uiMetadata = if (withMetadata) mapOf("foo" to "bar") else mapOf() ) } diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/AlertServiceTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/AlertServiceTests.kt new file mode 100644 index 000000000..b2dccf921 --- /dev/null +++ b/alerting/src/test/kotlin/org/opensearch/alerting/AlertServiceTests.kt @@ -0,0 +1,215 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting + +import org.junit.Before +import org.mockito.Mockito +import org.opensearch.Version +import org.opensearch.alerting.alerts.AlertIndices +import org.opensearch.alerting.model.AggregationResultBucket +import org.opensearch.alerting.model.Alert +import org.opensearch.alerting.model.BucketLevelTrigger +import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.action.AlertCategory +import org.opensearch.alerting.settings.AlertingSettings +import org.opensearch.alerting.util.getBucketKeysHash +import org.opensearch.client.Client +import org.opensearch.cluster.node.DiscoveryNode +import org.opensearch.cluster.service.ClusterService +import org.opensearch.common.settings.ClusterSettings +import org.opensearch.common.settings.Setting +import org.opensearch.common.settings.Settings +import org.opensearch.common.xcontent.NamedXContentRegistry +import org.opensearch.test.ClusterServiceUtils +import org.opensearch.test.OpenSearchTestCase +import org.opensearch.threadpool.ThreadPool +import java.time.Instant +import java.time.temporal.ChronoUnit + +class AlertServiceTests : OpenSearchTestCase() { + + private lateinit var client: Client + private lateinit var xContentRegistry: NamedXContentRegistry + private lateinit var settings: Settings + private lateinit var threadPool: ThreadPool + private lateinit var clusterService: ClusterService + + private lateinit var alertIndices: AlertIndices + private lateinit var alertService: AlertService + + @Before + fun setup() { + // TODO: If more *Service unit tests are added, this configuration can be moved to some base class for each service test class to use + client = Mockito.mock(Client::class.java) + xContentRegistry = Mockito.mock(NamedXContentRegistry::class.java) + threadPool = Mockito.mock(ThreadPool::class.java) + clusterService = Mockito.mock(ClusterService::class.java) + + settings = Settings.builder().build() + val settingSet = hashSetOf>() + settingSet.addAll(ClusterSettings.BUILT_IN_CLUSTER_SETTINGS) + settingSet.add(AlertingSettings.ALERT_HISTORY_ENABLED) + settingSet.add(AlertingSettings.ALERT_HISTORY_MAX_DOCS) + settingSet.add(AlertingSettings.ALERT_HISTORY_INDEX_MAX_AGE) + settingSet.add(AlertingSettings.ALERT_HISTORY_ROLLOVER_PERIOD) + settingSet.add(AlertingSettings.ALERT_HISTORY_RETENTION_PERIOD) + settingSet.add(AlertingSettings.REQUEST_TIMEOUT) + val discoveryNode = DiscoveryNode("node", buildNewFakeTransportAddress(), Version.CURRENT) + val clusterSettings = ClusterSettings(settings, settingSet) + val testClusterService = ClusterServiceUtils.createClusterService(threadPool, discoveryNode, clusterSettings) + clusterService = Mockito.spy(testClusterService) + + alertIndices = AlertIndices(settings, client, threadPool, clusterService) + alertService = AlertService(client, xContentRegistry, alertIndices) + } + + fun `test getting categorized alerts for bucket-level monitor with no current alerts`() { + val trigger = randomBucketLevelTrigger() + val monitor = randomBucketLevelMonitor(triggers = listOf(trigger)) + + val currentAlerts = mutableMapOf() + val aggResultBuckets = createAggregationResultBucketsFromBucketKeys( + listOf( + listOf("a"), + listOf("b") + ) + ) + + val categorizedAlerts = alertService.getCategorizedAlertsForBucketLevelMonitor(monitor, trigger, currentAlerts, aggResultBuckets) + // Completed Alerts are what remains in currentAlerts after categorization + val completedAlerts = currentAlerts.values.toList() + assertEquals(listOf(), categorizedAlerts[AlertCategory.DEDUPED]) + assertAlertsExistForBucketKeys( + listOf( + listOf("a"), + listOf("b") + ), + categorizedAlerts[AlertCategory.NEW] ?: error("New alerts not found") + ) + assertEquals(listOf(), completedAlerts) + } + + fun `test getting categorized alerts for bucket-level monitor with de-duped alerts`() { + val trigger = randomBucketLevelTrigger() + val monitor = randomBucketLevelMonitor(triggers = listOf(trigger)) + + val currentAlerts = createCurrentAlertsFromBucketKeys( + monitor, trigger, + listOf( + listOf("a"), + listOf("b") + ) + ) + val aggResultBuckets = createAggregationResultBucketsFromBucketKeys( + listOf( + listOf("a"), + listOf("b") + ) + ) + + val categorizedAlerts = alertService.getCategorizedAlertsForBucketLevelMonitor(monitor, trigger, currentAlerts, aggResultBuckets) + // Completed Alerts are what remains in currentAlerts after categorization + val completedAlerts = currentAlerts.values.toList() + assertAlertsExistForBucketKeys( + listOf( + listOf("a"), + listOf("b") + ), + categorizedAlerts[AlertCategory.DEDUPED] ?: error("Deduped alerts not found") + ) + assertEquals(listOf(), categorizedAlerts[AlertCategory.NEW]) + assertEquals(listOf(), completedAlerts) + } + + fun `test getting categorized alerts for bucket-level monitor with completed alerts`() { + val trigger = randomBucketLevelTrigger() + val monitor = randomBucketLevelMonitor(triggers = listOf(trigger)) + + val currentAlerts = createCurrentAlertsFromBucketKeys( + monitor, trigger, + listOf( + listOf("a"), + listOf("b") + ) + ) + val aggResultBuckets = listOf() + + val categorizedAlerts = alertService.getCategorizedAlertsForBucketLevelMonitor(monitor, trigger, currentAlerts, aggResultBuckets) + // Completed Alerts are what remains in currentAlerts after categorization + val completedAlerts = currentAlerts.values.toList() + assertEquals(listOf(), categorizedAlerts[AlertCategory.DEDUPED]) + assertEquals(listOf(), categorizedAlerts[AlertCategory.NEW]) + assertAlertsExistForBucketKeys( + listOf( + listOf("a"), + listOf("b") + ), + completedAlerts + ) + } + + fun `test getting categorized alerts for bucket-level monitor with de-duped and completed alerts`() { + val trigger = randomBucketLevelTrigger() + val monitor = randomBucketLevelMonitor(triggers = listOf(trigger)) + + val currentAlerts = createCurrentAlertsFromBucketKeys( + monitor, trigger, + listOf( + listOf("a"), + listOf("b") + ) + ) + val aggResultBuckets = createAggregationResultBucketsFromBucketKeys( + listOf( + listOf("b"), + listOf("c") + ) + ) + + val categorizedAlerts = alertService.getCategorizedAlertsForBucketLevelMonitor(monitor, trigger, currentAlerts, aggResultBuckets) + // Completed Alerts are what remains in currentAlerts after categorization + val completedAlerts = currentAlerts.values.toList() + assertAlertsExistForBucketKeys(listOf(listOf("b")), categorizedAlerts[AlertCategory.DEDUPED] ?: error("Deduped alerts not found")) + assertAlertsExistForBucketKeys(listOf(listOf("c")), categorizedAlerts[AlertCategory.NEW] ?: error("New alerts not found")) + assertAlertsExistForBucketKeys(listOf(listOf("a")), completedAlerts) + } + + private fun createCurrentAlertsFromBucketKeys( + monitor: Monitor, + trigger: BucketLevelTrigger, + bucketKeysList: List> + ): MutableMap { + return bucketKeysList.map { bucketKeys -> + val aggResultBucket = AggregationResultBucket("parent_bucket_path", bucketKeys, mapOf()) + val alert = Alert( + monitor, trigger, Instant.now().truncatedTo(ChronoUnit.MILLIS), null, + actionExecutionResults = listOf(randomActionExecutionResult()), aggregationResultBucket = aggResultBucket + ) + aggResultBucket.getBucketKeysHash() to alert + }.toMap() as MutableMap + } + + private fun createAggregationResultBucketsFromBucketKeys(bucketKeysList: List>): List { + return bucketKeysList.map { AggregationResultBucket("parent_bucket_path", it, mapOf()) } + } + + private fun assertAlertsExistForBucketKeys(bucketKeysList: List>, alerts: List) { + // Check if size is equals first for sanity and since bucketKeysList should have unique entries, + // this ensures there shouldn't be duplicates in the alerts + assertEquals(bucketKeysList.size, alerts.size) + val expectedBucketKeyHashes = bucketKeysList.map { it.joinToString(separator = "#") }.toSet() + alerts.forEach { alert -> + assertNotNull(alert.aggregationResultBucket) + assertTrue(expectedBucketKeyHashes.contains(alert.aggregationResultBucket!!.getBucketKeysHash())) + } + } +} diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/AlertingRestTestCase.kt b/alerting/src/test/kotlin/org/opensearch/alerting/AlertingRestTestCase.kt index 703d2c58e..d39542a02 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/AlertingRestTestCase.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/AlertingRestTestCase.kt @@ -43,7 +43,9 @@ import org.opensearch.alerting.core.model.SearchInput import org.opensearch.alerting.core.settings.ScheduledJobSettings import org.opensearch.alerting.elasticapi.string import org.opensearch.alerting.model.Alert +import org.opensearch.alerting.model.BucketLevelTrigger import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.QueryLevelTrigger import org.opensearch.alerting.model.destination.Destination import org.opensearch.alerting.model.destination.email.EmailAccount import org.opensearch.alerting.model.destination.email.EmailGroup @@ -72,6 +74,9 @@ import java.net.URLEncoder import java.nio.file.Files import java.nio.file.Path import java.time.Instant +import java.time.ZonedDateTime +import java.time.format.DateTimeFormatter +import java.time.temporal.ChronoUnit import java.util.Locale import javax.management.MBeanServerInvocationHandler import javax.management.ObjectName @@ -88,9 +93,10 @@ abstract class AlertingRestTestCase : ODFERestTestCase() { return NamedXContentRegistry( mutableListOf( Monitor.XCONTENT_REGISTRY, - SearchInput.XCONTENT_REGISTRY - ) + - SearchModule(Settings.EMPTY, false, emptyList()).namedXContents + SearchInput.XCONTENT_REGISTRY, + QueryLevelTrigger.XCONTENT_REGISTRY, + BucketLevelTrigger.XCONTENT_REGISTRY + ) + SearchModule(Settings.EMPTY, false, emptyList()).namedXContents ) } @@ -148,7 +154,7 @@ abstract class AlertingRestTestCase : ODFERestTestCase() { emptyMap(), destination.toHttpEntity() ) - assertEquals("Unable to create a new destination", RestStatus.OK, response.restStatus()) + assertEquals("Unable to delete destination", RestStatus.OK, response.restStatus()) return response } @@ -398,7 +404,7 @@ abstract class AlertingRestTestCase : ODFERestTestCase() { } protected fun createRandomMonitor(refresh: Boolean = false, withMetadata: Boolean = false): Monitor { - val monitor = randomMonitor(withMetadata = withMetadata) + val monitor = randomQueryLevelMonitor(withMetadata = withMetadata) val monitorId = createMonitor(monitor, refresh).id if (withMetadata) { return getMonitor(monitorId = monitorId, header = BasicHeader(HttpHeaders.USER_AGENT, "OpenSearch-Dashboards")) @@ -533,14 +539,22 @@ abstract class AlertingRestTestCase : ODFERestTestCase() { return response } + protected fun deleteDoc(index: String, id: String, refresh: Boolean = true): Response { + val params = if (refresh) mapOf("refresh" to "true") else mapOf() + val response = client().makeRequest("DELETE", "$index/_doc/$id", params) + assertTrue("Unable to delete doc with ID $id in index: '$index'", listOf(RestStatus.OK).contains(response.restStatus())) + return response + } + /** A test index that can be used across tests. Feel free to add new fields but don't remove any. */ protected fun createTestIndex(index: String = randomAlphaOfLength(10).toLowerCase(Locale.ROOT)): String { createIndex( index, Settings.EMPTY, """ - "properties" : { - "test_strict_date_time" : { "type" : "date", "format" : "strict_date_time" } - } + "properties" : { + "test_strict_date_time" : { "type" : "date", "format" : "strict_date_time" }, + "test_field" : { "type" : "keyword" } + } """.trimIndent() ) return index @@ -556,9 +570,9 @@ abstract class AlertingRestTestCase : ODFERestTestCase() { createIndex( index, Settings.builder().build(), """ - "properties" : { - "test_strict_date_time" : { "type" : "date", "format" : "strict_date_time" } - } + "properties" : { + "test_strict_date_time" : { "type" : "date", "format" : "strict_date_time" } + } """.trimIndent() ) } catch (ex: WarningFailureException) { @@ -567,6 +581,27 @@ abstract class AlertingRestTestCase : ODFERestTestCase() { return index } + protected fun insertSampleTimeSerializedData(index: String, data: List) { + data.forEachIndexed { i, value -> + val twoMinsAgo = ZonedDateTime.now().minus(2, ChronoUnit.MINUTES).truncatedTo(ChronoUnit.MILLIS) + val testTime = DateTimeFormatter.ISO_OFFSET_DATE_TIME.format(twoMinsAgo) + val testDoc = """ + { + "test_strict_date_time": "$testTime", + "test_field": "$value" + } + """.trimIndent() + // Indexing documents with deterministic doc id to allow for easy selected deletion during testing + indexDoc(index, (i + 1).toString(), testDoc) + } + } + + protected fun deleteDataWithDocIds(index: String, docIds: List) { + docIds.forEach { + deleteDoc(index, it) + } + } + fun putAlertMappings(mapping: String? = null) { val mappingHack = if (mapping != null) mapping else AlertIndices.alertMapping().trimStart('{').trimEnd('}') val encodedHistoryIndex = URLEncoder.encode(AlertIndices.HISTORY_INDEX_PATTERN, Charsets.UTF_8.toString()) diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/MonitorRunnerIT.kt b/alerting/src/test/kotlin/org/opensearch/alerting/MonitorRunnerIT.kt index 2d4d0eb39..6194191e6 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/MonitorRunnerIT.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/MonitorRunnerIT.kt @@ -27,6 +27,7 @@ package org.opensearch.alerting import org.junit.Assert +import org.opensearch.alerting.aggregation.bucketselectorext.BucketSelectorExtAggregationBuilder import org.opensearch.alerting.alerts.AlertError import org.opensearch.alerting.alerts.AlertIndices import org.opensearch.alerting.core.model.IntervalSchedule @@ -38,12 +39,17 @@ import org.opensearch.alerting.model.Alert.State.ACTIVE import org.opensearch.alerting.model.Alert.State.COMPLETED import org.opensearch.alerting.model.Alert.State.ERROR import org.opensearch.alerting.model.Monitor +import org.opensearch.alerting.model.action.ActionExecutionPolicy +import org.opensearch.alerting.model.action.AlertCategory +import org.opensearch.alerting.model.action.PerAlertActionScope +import org.opensearch.alerting.model.action.PerExecutionActionScope import org.opensearch.alerting.model.action.Throttle import org.opensearch.alerting.model.destination.CustomWebhook import org.opensearch.alerting.model.destination.Destination import org.opensearch.alerting.model.destination.email.Email import org.opensearch.alerting.model.destination.email.Recipient import org.opensearch.alerting.util.DestinationType +import org.opensearch.alerting.util.getBucketKeysHash import org.opensearch.client.ResponseException import org.opensearch.client.WarningFailureException import org.opensearch.common.settings.Settings @@ -51,6 +57,8 @@ import org.opensearch.commons.authuser.User import org.opensearch.index.query.QueryBuilders import org.opensearch.rest.RestStatus import org.opensearch.script.Script +import org.opensearch.search.aggregations.bucket.composite.CompositeAggregationBuilder +import org.opensearch.search.aggregations.bucket.composite.TermsValuesSourceBuilder import org.opensearch.search.builder.SearchSourceBuilder import java.net.URLEncoder import java.time.Instant @@ -66,7 +74,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test execute monitor with dryrun`() { val action = randomAction(template = randomTemplateScript("Hello {{ctx.monitor.name}}"), destinationId = createDestination().id) - val monitor = randomMonitor(triggers = listOf(randomTrigger(condition = ALWAYS_RUN, actions = listOf(action)))) + val monitor = randomQueryLevelMonitor( + triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = listOf(action))) + ) val response = executeMonitor(monitor, params = DRYRUN_MONITOR) @@ -101,8 +111,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { return ctx.results[0].hits.hits.size() == 1 """.trimIndent() - val trigger = randomTrigger(condition = Script(triggerScript)) - val monitor = randomMonitor(inputs = listOf(input), triggers = listOf(trigger)) + val trigger = randomQueryLevelTrigger(condition = Script(triggerScript)) + val monitor = randomQueryLevelMonitor(inputs = listOf(input), triggers = listOf(trigger)) val response = executeMonitor(monitor, params = DRYRUN_MONITOR) val output = entityAsMap(response) @@ -116,7 +126,7 @@ class MonitorRunnerIT : AlertingRestTestCase() { } fun `test execute monitor not triggered`() { - val monitor = randomMonitor(triggers = listOf(randomTrigger(condition = NEVER_RUN))) + val monitor = randomQueryLevelMonitor(triggers = listOf(randomQueryLevelTrigger(condition = NEVER_RUN))) val response = executeMonitor(monitor) @@ -132,7 +142,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test active alert is updated on each run`() { val monitor = createMonitor( - randomMonitor(triggers = listOf(randomTrigger(condition = ALWAYS_RUN, destinationId = createDestination().id))) + randomQueryLevelMonitor( + triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN, destinationId = createDestination().id)) + ) ) executeMonitor(monitor.id) @@ -158,9 +170,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { createIndex("foo", Settings.EMPTY) val input = SearchInput(indices = listOf("foo"), query = SearchSourceBuilder().query(QueryBuilders.matchAllQuery())) val monitor = createMonitor( - randomMonitor( + randomQueryLevelMonitor( inputs = listOf(input), - triggers = listOf(randomTrigger(condition = NEVER_RUN)) + triggers = listOf(randomQueryLevelTrigger(condition = NEVER_RUN)) ) ) @@ -183,9 +195,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { createIndex("foo", Settings.EMPTY) val input = SearchInput(indices = listOf("foo"), query = SearchSourceBuilder().query(QueryBuilders.matchAllQuery())) val monitor = createMonitor( - randomMonitor( + randomQueryLevelMonitor( inputs = listOf(input), - triggers = listOf(randomTrigger(condition = NEVER_RUN)) + triggers = listOf(randomQueryLevelTrigger(condition = NEVER_RUN)) ) ) @@ -204,9 +216,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { createIndex("foo", Settings.EMPTY) val input = SearchInput(indices = listOf("foo"), query = SearchSourceBuilder().query(QueryBuilders.matchAllQuery())) val monitor = createMonitor( - randomMonitor( + randomQueryLevelMonitor( inputs = listOf(input), - triggers = listOf(randomTrigger(condition = ALWAYS_RUN, destinationId = destinationId)) + triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN, destinationId = destinationId)) ) ) @@ -231,7 +243,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test acknowledged alert is not updated unnecessarily`() { val monitor = createMonitor( - randomMonitor(triggers = listOf(randomTrigger(condition = ALWAYS_RUN, destinationId = createDestination().id))) + randomQueryLevelMonitor( + triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN, destinationId = createDestination().id)) + ) ) executeMonitor(monitor.id) acknowledgeAlerts(monitor, searchAlerts(monitor).single()) @@ -253,8 +267,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { } fun `test alert completion`() { - val trigger = randomTrigger(condition = Script("ctx.alert == null"), destinationId = createDestination().id) - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) + val trigger = randomQueryLevelTrigger(condition = Script("ctx.alert == null"), destinationId = createDestination().id) + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) executeMonitor(monitor.id) val activeAlert = searchAlerts(monitor).single() @@ -268,8 +282,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test execute monitor script error`() { // This painless script should cause a syntax error - val trigger = randomTrigger(condition = Script("foo bar baz")) - val monitor = randomMonitor(triggers = listOf(trigger)) + val trigger = randomQueryLevelTrigger(condition = Script("foo bar baz")) + val monitor = randomQueryLevelMonitor(triggers = listOf(trigger)) val response = executeMonitor(monitor) @@ -286,7 +300,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test execute action template error`() { // Intentional syntax error in mustache template val action = randomAction(template = randomTemplateScript("Hello {{ctx.monitor.name")) - val monitor = randomMonitor(triggers = listOf(randomTrigger(condition = ALWAYS_RUN, actions = listOf(action)))) + val monitor = randomQueryLevelMonitor( + triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = listOf(action))) + ) val response = executeMonitor(monitor) @@ -320,8 +336,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { return ctx.results[0].hits.hits.size() > 0 """.trimIndent() val destinationId = createDestination().id - val trigger = randomTrigger(condition = Script(triggerScript), destinationId = destinationId) - val monitor = createMonitor(randomMonitor(inputs = listOf(input), triggers = listOf(trigger))) + val trigger = randomQueryLevelTrigger(condition = Script(triggerScript), destinationId = destinationId) + val monitor = createMonitor(randomQueryLevelMonitor(inputs = listOf(input), triggers = listOf(trigger))) val response = executeMonitor(monitor.id) @@ -355,8 +371,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { // make sure there is exactly one hit return ctx.results[0].hits.hits.size() == 1 """.trimIndent() - val trigger = randomTrigger(condition = Script(triggerScript)) - val monitor = randomMonitor(inputs = listOf(input), triggers = listOf(trigger)) + val trigger = randomQueryLevelTrigger(condition = Script(triggerScript)) + val monitor = randomQueryLevelMonitor(inputs = listOf(input), triggers = listOf(trigger)) val response = executeMonitor(monitor) @@ -397,8 +413,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { // make sure there is exactly one hit return ctx.results[0].hits.hits.size() == 1 """.trimIndent() - val trigger = randomTrigger(condition = Script(triggerScript)) - val monitor = randomMonitor(inputs = listOf(input), triggers = listOf(trigger)) + val trigger = randomQueryLevelTrigger(condition = Script(triggerScript)) + val monitor = randomQueryLevelMonitor(inputs = listOf(input), triggers = listOf(trigger)) val response = executeMonitor(monitor, params = DRYRUN_MONITOR) @@ -413,14 +429,19 @@ class MonitorRunnerIT : AlertingRestTestCase() { } fun `test monitor with one bad action and one good action`() { - val goodAction = randomAction(template = randomTemplateScript("Hello {{ctx.monitor.name}}"), destinationId = createDestination().id) + val goodAction = randomAction( + template = randomTemplateScript("Hello {{ctx.monitor.name}}"), + destinationId = createDestination().id + ) val syntaxErrorAction = randomAction( name = "bad syntax", template = randomTemplateScript("{{foo"), destinationId = createDestination().id ) val actions = listOf(goodAction, syntaxErrorAction) - val monitor = createMonitor(randomMonitor(triggers = listOf(randomTrigger(condition = ALWAYS_RUN, actions = actions)))) + val monitor = createMonitor( + randomQueryLevelMonitor(triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = actions))) + ) val output = entityAsMap(executeMonitor(monitor.id)) @@ -447,8 +468,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { putAlertMappings() // Required as we do not have a create alert API. // This template script has a parsing error to purposefully create an errorMessage during runMonitor val action = randomAction(template = randomTemplateScript("Hello {{ctx.monitor.name")) - val trigger = randomTrigger(condition = ALWAYS_RUN, actions = listOf(action)) - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) + val trigger = randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = listOf(action)) + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) val listOfFiveErrorMessages = (1..5).map { i -> AlertError(timestamp = Instant.now(), message = "error message $i") } val activeAlert = createAlert( randomAlert(monitor).copy( @@ -473,7 +494,7 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test latest error is not lost when alert is completed`() { // Creates an active alert the first time it's run and completes it the second time the monitor is run. - val trigger = randomTrigger( + val trigger = randomQueryLevelTrigger( condition = Script( """ if (ctx.alert == null) { @@ -484,7 +505,7 @@ class MonitorRunnerIT : AlertingRestTestCase() { """.trimIndent() ) ) - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) executeMonitor(monitor.id) val errorAlert = searchAlerts(monitor).single() @@ -501,14 +522,14 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test throw script exception`() { // Creates an active alert the first time it's run and completes it the second time the monitor is run. - val trigger = randomTrigger( + val trigger = randomQueryLevelTrigger( condition = Script( """ param[0]; return true """.trimIndent() ) ) - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) executeMonitor(monitor.id) val errorAlert = searchAlerts(monitor).single() @@ -524,8 +545,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { putAlertMappings() // Required as we do not have a create alert API. // This template script has a parsing error to purposefully create an errorMessage during runMonitor val action = randomAction(template = randomTemplateScript("Hello {{ctx.monitor.name")) - val trigger = randomTrigger(condition = ALWAYS_RUN, actions = listOf(action)) - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) + val trigger = randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = listOf(action)) + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) val listOfTenErrorMessages = (1..10).map { i -> AlertError(timestamp = Instant.now(), message = "error message $i") } val activeAlert = createAlert( randomAlert(monitor).copy( @@ -551,8 +572,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test execute monitor creates alert for trigger with no actions`() { putAlertMappings() // Required as we do not have a create alert API. - val trigger = randomTrigger(condition = ALWAYS_RUN, actions = emptyList(), destinationId = createDestination().id) - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) + val trigger = randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = emptyList(), destinationId = createDestination().id) + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) executeMonitor(monitor.id) @@ -563,9 +584,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test execute monitor non-dryrun`() { val monitor = createMonitor( - randomMonitor( + randomQueryLevelMonitor( triggers = listOf( - randomTrigger( + randomQueryLevelTrigger( condition = ALWAYS_RUN, actions = listOf(randomAction(destinationId = createDestination().id)) ) @@ -583,9 +604,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test execute monitor with already active alert`() { val monitor = createMonitor( - randomMonitor( + randomQueryLevelMonitor( triggers = listOf( - randomTrigger( + randomQueryLevelTrigger( condition = ALWAYS_RUN, actions = listOf(randomAction(destinationId = createDestination().id)) ) @@ -612,7 +633,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { putAlertMappings() val newMonitor = createMonitor( - randomMonitor(triggers = listOf(randomTrigger(condition = NEVER_RUN, actions = listOf(randomAction())))) + randomQueryLevelMonitor( + triggers = listOf(randomQueryLevelTrigger(condition = NEVER_RUN, actions = listOf(randomAction()))) + ) ) val deleteNewMonitorResponse = client().makeRequest("DELETE", "$ALERTING_BASE_URI/${newMonitor.id}") @@ -620,7 +643,7 @@ class MonitorRunnerIT : AlertingRestTestCase() { } fun `test update monitor stays on schedule`() { - val monitor = createMonitor(randomMonitor(enabled = true)) + val monitor = createMonitor(randomQueryLevelMonitor(enabled = true)) updateMonitor(monitor.copy(enabledTime = Instant.now())) @@ -629,19 +652,19 @@ class MonitorRunnerIT : AlertingRestTestCase() { } fun `test enabled time by disabling and re-enabling monitor`() { - val monitor = createMonitor(randomMonitor(enabled = true)) + val monitor = createMonitor(randomQueryLevelMonitor(enabled = true)) assertNotNull("Enabled time is null on a enabled monitor.", getMonitor(monitor.id).enabledTime) - val disabledMonitor = updateMonitor(randomMonitor(enabled = false).copy(id = monitor.id)) + val disabledMonitor = updateMonitor(randomQueryLevelMonitor(enabled = false).copy(id = monitor.id)) assertNull("Enabled time is not null on a disabled monitor.", disabledMonitor.enabledTime) - val enabledMonitor = updateMonitor(randomMonitor(enabled = true).copy(id = monitor.id)) + val enabledMonitor = updateMonitor(randomQueryLevelMonitor(enabled = true).copy(id = monitor.id)) assertNotNull("Enabled time is null on a enabled monitor.", enabledMonitor.enabledTime) } fun `test enabled time by providing enabled time`() { val enabledTime = Instant.ofEpochSecond(1538164858L) // This is 2018-09-27 20:00:58 GMT - val monitor = createMonitor(randomMonitor(enabled = true, enabledTime = enabledTime)) + val monitor = createMonitor(randomQueryLevelMonitor(enabled = true, enabledTime = enabledTime)) val retrievedMonitor = getMonitor(monitorId = monitor.id) assertTrue("Monitor is not enabled", retrievedMonitor.enabled) @@ -661,8 +684,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { ) val actions = listOf(actionThrottleEnabled, actionThrottleNotEnabled) val monitor = createMonitor( - randomMonitor( - triggers = listOf(randomTrigger(condition = ALWAYS_RUN, actions = actions)), + randomQueryLevelMonitor( + triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = actions)), schedule = IntervalSchedule(interval = 1, unit = ChronoUnit.MINUTES) ) ) @@ -716,9 +739,9 @@ class MonitorRunnerIT : AlertingRestTestCase() { throttleEnabled = true, throttle = Throttle(value = 5, unit = MINUTES) ) val actions = listOf(actionThrottleEnabled) - val trigger = randomTrigger(condition = ALWAYS_RUN, actions = actions) + val trigger = randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = actions) val monitor = createMonitor( - randomMonitor( + randomQueryLevelMonitor( triggers = listOf(trigger), schedule = IntervalSchedule(interval = 1, unit = ChronoUnit.MINUTES) ) @@ -751,7 +774,7 @@ class MonitorRunnerIT : AlertingRestTestCase() { ) } - fun `test execute monitor with email destination creates alerts in error state`() { + fun `test execute monitor with email destination creates alert in error state`() { putAlertMappings() // Required as we do not have a create alert API. val emailAccount = createRandomEmailAccount() @@ -777,8 +800,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { ) ) val action = randomAction(destinationId = destination.id) - val trigger = randomTrigger(condition = ALWAYS_RUN, actions = listOf(action)) - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) + val trigger = randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = listOf(action)) + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) executeMonitor(monitor.id) @@ -803,8 +826,8 @@ class MonitorRunnerIT : AlertingRestTestCase() { ) ) val action = randomAction(destinationId = destination.id) - val trigger = randomTrigger(condition = ALWAYS_RUN, actions = listOf(action)) - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) + val trigger = randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = listOf(action)) + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) executeMonitor(adminClient(), monitor.id) val alerts = searchAlerts(monitor) @@ -815,33 +838,30 @@ class MonitorRunnerIT : AlertingRestTestCase() { fun `test execute monitor with custom webhook destination and denied host`() { - // TODO: change to REST API call to test security enabled case - if (!securityEnabled()) { - listOf("http://10.1.1.1", "127.0.0.1").forEach { - val customWebhook = CustomWebhook(it, null, null, 80, null, "PUT", emptyMap(), emptyMap(), null, null) - val destination = createDestination( - Destination( - type = DestinationType.CUSTOM_WEBHOOK, - name = "testDesination", - user = randomUser(), - lastUpdateTime = Instant.now(), - chime = null, - slack = null, - customWebhook = customWebhook, - email = null - ) + listOf("http://10.1.1.1", "127.0.0.1").forEach { + val customWebhook = CustomWebhook(it, null, null, 80, null, "PUT", emptyMap(), emptyMap(), null, null) + val destination = createDestination( + Destination( + type = DestinationType.CUSTOM_WEBHOOK, + name = "testDesination", + user = randomUser(), + lastUpdateTime = Instant.now(), + chime = null, + slack = null, + customWebhook = customWebhook, + email = null ) - val action = randomAction(destinationId = destination.id) - val trigger = randomTrigger(condition = ALWAYS_RUN, actions = listOf(action)) - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) - executeMonitor(adminClient(), monitor.id) + ) + val action = randomAction(destinationId = destination.id) + val trigger = randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = listOf(action)) + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) + executeMonitor(adminClient(), monitor.id) - val alerts = searchAlerts(monitor) - assertEquals("Alert not saved", 1, alerts.size) - verifyAlert(alerts.single(), monitor, ERROR) + val alerts = searchAlerts(monitor) + assertEquals("Alert not saved", 1, alerts.size) + verifyAlert(alerts.single(), monitor, ERROR) - Assert.assertTrue(alerts.single().errorMessage?.contains("The destination address is invalid") as Boolean) - } + Assert.assertTrue(alerts.single().errorMessage?.contains("The destination address is invalid") as Boolean) } } @@ -954,6 +974,548 @@ class MonitorRunnerIT : AlertingRestTestCase() { } } + fun `test execute bucket-level monitor returns search result`() { + val testIndex = createTestIndex() + insertSampleTimeSerializedData( + testIndex, + listOf( + "test_value_1", + "test_value_1", // adding duplicate to verify aggregation + "test_value_2" + ) + ) + + val query = QueryBuilders.rangeQuery("test_strict_date_time") + .gt("{{period_end}}||-10d") + .lte("{{period_end}}") + .format("epoch_millis") + val compositeSources = listOf( + TermsValuesSourceBuilder("test_field").field("test_field") + ) + val compositeAgg = CompositeAggregationBuilder("composite_agg", compositeSources) + val input = SearchInput(indices = listOf(testIndex), query = SearchSourceBuilder().size(0).query(query).aggregation(compositeAgg)) + val triggerScript = """ + params.docCount > 0 + """.trimIndent() + + var trigger = randomBucketLevelTrigger() + trigger = trigger.copy( + bucketSelector = BucketSelectorExtAggregationBuilder( + name = trigger.id, + bucketsPathsMap = mapOf("docCount" to "_count"), + script = Script(triggerScript), + parentBucketPath = "composite_agg", + filter = null + ) + ) + val monitor = createMonitor(randomBucketLevelMonitor(inputs = listOf(input), enabled = false, triggers = listOf(trigger))) + val response = executeMonitor(monitor.id, params = DRYRUN_MONITOR) + val output = entityAsMap(response) + // print("Output is: $output") + + assertEquals(monitor.name, output["monitor_name"]) + @Suppress("UNCHECKED_CAST") + val searchResult = (output.objectMap("input_results")["results"] as List>).first() + @Suppress("UNCHECKED_CAST") + val buckets = searchResult.stringMap("aggregations")?.stringMap("composite_agg")?.get("buckets") as List> + assertEquals("Incorrect search result", 2, buckets.size) + } + + fun `test bucket-level monitor alert creation and completion`() { + val testIndex = createTestIndex() + insertSampleTimeSerializedData( + testIndex, + listOf( + "test_value_1", + "test_value_1", // adding duplicate to verify aggregation + "test_value_2" + ) + ) + + val query = QueryBuilders.rangeQuery("test_strict_date_time") + .gt("{{period_end}}||-10d") + .lte("{{period_end}}") + .format("epoch_millis") + val compositeSources = listOf( + TermsValuesSourceBuilder("test_field").field("test_field") + ) + val compositeAgg = CompositeAggregationBuilder("composite_agg", compositeSources) + val input = SearchInput(indices = listOf(testIndex), query = SearchSourceBuilder().size(0).query(query).aggregation(compositeAgg)) + val triggerScript = """ + params.docCount > 0 + """.trimIndent() + + var trigger = randomBucketLevelTrigger() + trigger = trigger.copy( + bucketSelector = BucketSelectorExtAggregationBuilder( + name = trigger.id, + bucketsPathsMap = mapOf("docCount" to "_count"), + script = Script(triggerScript), + parentBucketPath = "composite_agg", + filter = null + ) + ) + val monitor = createMonitor(randomBucketLevelMonitor(inputs = listOf(input), enabled = false, triggers = listOf(trigger))) + executeMonitor(monitor.id) + + // Check created alerts + var alerts = searchAlerts(monitor) + assertEquals("Alerts not saved", 2, alerts.size) + alerts.forEach { + verifyAlert(it, monitor, ACTIVE) + } + + // Delete documents of a particular value + deleteDataWithDocIds( + testIndex, + listOf( + "1", // test_value_1 + "2" // test_value_1 + ) + ) + + // Execute monitor again + executeMonitor(monitor.id) + + // Verify expected alert was completed + alerts = searchAlerts(monitor, AlertIndices.ALL_INDEX_PATTERN) + val activeAlerts = alerts.filter { it.state == ACTIVE } + val completedAlerts = alerts.filter { it.state == COMPLETED } + assertEquals("Incorrect number of active alerts", 1, activeAlerts.size) + assertEquals("Incorrect number of completed alerts", 1, completedAlerts.size) + } + + fun `test bucket-level monitor with acknowledged alert`() { + val testIndex = createTestIndex() + insertSampleTimeSerializedData( + testIndex, + listOf( + "test_value_1", + "test_value_2" + ) + ) + + val query = QueryBuilders.rangeQuery("test_strict_date_time") + .gt("{{period_end}}||-10d") + .lte("{{period_end}}") + .format("epoch_millis") + val compositeSources = listOf( + TermsValuesSourceBuilder("test_field").field("test_field") + ) + val compositeAgg = CompositeAggregationBuilder("composite_agg", compositeSources) + val input = SearchInput(indices = listOf(testIndex), query = SearchSourceBuilder().size(0).query(query).aggregation(compositeAgg)) + val triggerScript = """ + params.docCount > 0 + """.trimIndent() + + var trigger = randomBucketLevelTrigger() + trigger = trigger.copy( + bucketSelector = BucketSelectorExtAggregationBuilder( + name = trigger.id, + bucketsPathsMap = mapOf("docCount" to "_count"), + script = Script(triggerScript), + parentBucketPath = "composite_agg", + filter = null + ) + ) + val monitor = createMonitor(randomBucketLevelMonitor(inputs = listOf(input), enabled = false, triggers = listOf(trigger))) + executeMonitor(monitor.id) + + // Check created Alerts + var currentAlerts = searchAlerts(monitor) + assertEquals("Alerts not saved", 2, currentAlerts.size) + currentAlerts.forEach { + verifyAlert(it, monitor, ACTIVE) + } + + // Acknowledge one of the Alerts + val alertToAcknowledge = currentAlerts.single { it.aggregationResultBucket?.getBucketKeysHash().equals("test_value_1") } + acknowledgeAlerts(monitor, alertToAcknowledge) + currentAlerts = searchAlerts(monitor) + val acknowledgedAlert = currentAlerts.single { it.state == ACKNOWLEDGED } + val activeAlert = currentAlerts.single { it.state == ACTIVE } + + // Runner uses ThreadPool.CachedTimeThread thread which only updates once every 200 ms. Wait a bit to + // let lastNotificationTime change. W/o this sleep the test can result in a false negative. + Thread.sleep(200) + executeMonitor(monitor.id) + + // Check that the lastNotification time of the acknowledged Alert wasn't updated and the active Alert's was + currentAlerts = searchAlerts(monitor) + val acknowledgedAlert2 = currentAlerts.single { it.state == ACKNOWLEDGED } + val activeAlert2 = currentAlerts.single { it.state == ACTIVE } + assertEquals("Acknowledged alert was updated", acknowledgedAlert.lastNotificationTime, acknowledgedAlert2.lastNotificationTime) + assertTrue("Active alert was not updated", activeAlert2.lastNotificationTime!! > activeAlert.lastNotificationTime) + + // Remove data so that both Alerts are moved into completed + deleteDataWithDocIds( + testIndex, + listOf( + "1", // test_value_1 + "2" // test_value_2 + ) + ) + + // Execute Monitor and check that both Alerts were updated + Thread.sleep(200) + executeMonitor(monitor.id) + currentAlerts = searchAlerts(monitor, AlertIndices.ALL_INDEX_PATTERN) + val completedAlerts = currentAlerts.filter { it.state == COMPLETED } + assertEquals("Incorrect number of completed alerts", 2, completedAlerts.size) + val previouslyAcknowledgedAlert = completedAlerts.single { it.aggregationResultBucket?.getBucketKeysHash().equals("test_value_1") } + val previouslyActiveAlert = completedAlerts.single { it.aggregationResultBucket?.getBucketKeysHash().equals("test_value_2") } + // Note: Given the randomization of the Actions and ActionExecutionPolicy for the Bucket-Level Monitor + // there is a very small chance we could end up with COMPLETED Alerts that never had lastNotificationTime updated + // (This would occur if the Trigger contained Actions with ActionExecutionScope of PER_ALERT that all somehow excluded the + // same Alert categories being tested in this test) + // In such a rare case, the tests can just be rerun + assertTrue( + "Previously acknowledged alert was not updated when it moved to completed", + previouslyAcknowledgedAlert.lastNotificationTime!! > acknowledgedAlert2.lastNotificationTime + ) + assertTrue( + "Previously active alert was not updated when it moved to completed", + previouslyActiveAlert.lastNotificationTime!! > activeAlert2.lastNotificationTime + ) + } + + @Suppress("UNCHECKED_CAST") + fun `test bucket-level monitor with one good action and one bad action`() { + val testIndex = createTestIndex() + insertSampleTimeSerializedData( + testIndex, + listOf( + "test_value_1", + "test_value_1", + "test_value_3", + "test_value_2", + "test_value_2" + ) + ) + + val query = QueryBuilders.rangeQuery("test_strict_date_time") + .gt("{{period_end}}||-10d") + .lte("{{period_end}}") + .format("epoch_millis") + val compositeSources = listOf( + TermsValuesSourceBuilder("test_field").field("test_field") + ) + val compositeAgg = CompositeAggregationBuilder("composite_agg", compositeSources) + val input = SearchInput(indices = listOf(testIndex), query = SearchSourceBuilder().size(0).query(query).aggregation(compositeAgg)) + // Trigger script should only create Alerts for 'test_value_1' and 'test_value_2' + val triggerScript = """ + params.docCount > 1 + """.trimIndent() + + val goodAction = randomAction(template = randomTemplateScript("Hello {{ctx.monitor.name}}"), destinationId = createDestination().id) + val syntaxErrorAction = randomAction( + name = "bad syntax", + template = randomTemplateScript("{{foo"), + destinationId = createDestination().id + ) + val actions = listOf(goodAction, syntaxErrorAction) + + var trigger = randomBucketLevelTrigger(actions = actions) + trigger = trigger.copy( + bucketSelector = BucketSelectorExtAggregationBuilder( + name = trigger.id, + bucketsPathsMap = mapOf("docCount" to "_count"), + script = Script(triggerScript), + parentBucketPath = "composite_agg", + filter = null + ) + ) + val monitor = createMonitor(randomBucketLevelMonitor(inputs = listOf(input), enabled = false, triggers = listOf(trigger))) + + val output = entityAsMap(executeMonitor(monitor.id)) + // The 'events' in this case are the bucketKeys hashes representing the Alert events + val expectedEvents = setOf("test_value_1", "test_value_2") + + assertEquals(monitor.name, output["monitor_name"]) + for (triggerResult in output.objectMap("trigger_results").values) { + for (alertEvent in triggerResult.objectMap("action_results")) { + assertTrue(expectedEvents.contains(alertEvent.key)) + val actionResults = alertEvent.value.values as Collection> + for (actionResult in actionResults) { + val actionOutput = actionResult["output"] as Map + if (actionResult["name"] == goodAction.name) { + assertEquals("Hello ${monitor.name}", actionOutput["message"]) + } else if (actionResult["name"] == syntaxErrorAction.name) { + assertTrue("Missing action error message", (actionResult["error"] as String).isNotEmpty()) + } else { + fail("Unknown action: ${actionResult["name"]}") + } + } + } + } + + // Check created alerts + val alerts = searchAlerts(monitor) + assertEquals("Alerts not saved", 2, alerts.size) + alerts.forEach { + verifyAlert(it, monitor, ACTIVE) + } + } + + @Suppress("UNCHECKED_CAST") + fun `test bucket-level monitor with per execution action scope`() { + val testIndex = createTestIndex() + insertSampleTimeSerializedData( + testIndex, + listOf( + "test_value_1", + "test_value_1", + "test_value_3", + "test_value_2", + "test_value_2" + ) + ) + + val query = QueryBuilders.rangeQuery("test_strict_date_time") + .gt("{{period_end}}||-10d") + .lte("{{period_end}}") + .format("epoch_millis") + val compositeSources = listOf( + TermsValuesSourceBuilder("test_field").field("test_field") + ) + val compositeAgg = CompositeAggregationBuilder("composite_agg", compositeSources) + val input = SearchInput(indices = listOf(testIndex), query = SearchSourceBuilder().size(0).query(query).aggregation(compositeAgg)) + // Trigger script should only create Alerts for 'test_value_1' and 'test_value_2' + val triggerScript = """ + params.docCount > 1 + """.trimIndent() + + val action = randomActionWithPolicy( + template = randomTemplateScript("Hello {{ctx.monitor.name}}"), + destinationId = createDestination().id, + actionExecutionPolicy = ActionExecutionPolicy(PerExecutionActionScope()) + ) + var trigger = randomBucketLevelTrigger(actions = listOf(action)) + trigger = trigger.copy( + bucketSelector = BucketSelectorExtAggregationBuilder( + name = trigger.id, + bucketsPathsMap = mapOf("docCount" to "_count"), + script = Script(triggerScript), + parentBucketPath = "composite_agg", + filter = null + ) + ) + val monitor = createMonitor(randomBucketLevelMonitor(inputs = listOf(input), enabled = false, triggers = listOf(trigger))) + + val output = entityAsMap(executeMonitor(monitor.id)) + // The 'events' in this case are the bucketKeys hashes representing the Alert events + val expectedEvents = setOf("test_value_1", "test_value_2") + + assertEquals(monitor.name, output["monitor_name"]) + for (triggerResult in output.objectMap("trigger_results").values) { + for (alertEvent in triggerResult.objectMap("action_results")) { + assertTrue(expectedEvents.contains(alertEvent.key)) + val actionResults = alertEvent.value.values as Collection> + for (actionResult in actionResults) { + val actionOutput = actionResult["output"] as Map + assertEquals("Unknown action: ${actionResult["name"]}", action.name, actionResult["name"]) + assertEquals("Hello ${monitor.name}", actionOutput["message"]) + } + } + } + + // Check created alerts + val alerts = searchAlerts(monitor) + assertEquals("Alerts not saved", 2, alerts.size) + alerts.forEach { + verifyAlert(it, monitor, ACTIVE) + } + } + + fun `test bucket-level monitor with per alert action scope saves completed alerts even if not actionable`() { + val testIndex = createTestIndex() + insertSampleTimeSerializedData( + testIndex, + listOf( + "test_value_1", + "test_value_1", + "test_value_2", + "test_value_2" + ) + ) + + val query = QueryBuilders.rangeQuery("test_strict_date_time") + .gt("{{period_end}}||-10d") + .lte("{{period_end}}") + .format("epoch_millis") + val compositeSources = listOf( + TermsValuesSourceBuilder("test_field").field("test_field") + ) + val compositeAgg = CompositeAggregationBuilder("composite_agg", compositeSources) + val input = SearchInput(indices = listOf(testIndex), query = SearchSourceBuilder().size(0).query(query).aggregation(compositeAgg)) + val triggerScript = """ + params.docCount > 1 + """.trimIndent() + + val action = randomActionWithPolicy( + template = randomTemplateScript("Hello {{ctx.monitor.name}}"), + destinationId = createDestination().id, + actionExecutionPolicy = ActionExecutionPolicy(PerAlertActionScope(setOf(AlertCategory.DEDUPED, AlertCategory.NEW))) + ) + var trigger = randomBucketLevelTrigger(actions = listOf(action)) + trigger = trigger.copy( + bucketSelector = BucketSelectorExtAggregationBuilder( + name = trigger.id, + bucketsPathsMap = mapOf("docCount" to "_count"), + script = Script(triggerScript), + parentBucketPath = "composite_agg", + filter = null + ) + ) + val monitor = createMonitor(randomBucketLevelMonitor(inputs = listOf(input), enabled = false, triggers = listOf(trigger))) + executeMonitor(monitor.id) + + // Check created Alerts + var currentAlerts = searchAlerts(monitor) + assertEquals("Alerts not saved", 2, currentAlerts.size) + currentAlerts.forEach { + verifyAlert(it, monitor, ACTIVE) + } + + // Remove data so that both Alerts are moved into completed + deleteDataWithDocIds( + testIndex, + listOf( + "1", // test_value_1 + "2", // test_value_1 + "3", // test_value_2 + "4" // test_value_2 + ) + ) + + // Execute Monitor and check that both Alerts were moved to COMPLETED + executeMonitor(monitor.id) + currentAlerts = searchAlerts(monitor, AlertIndices.ALL_INDEX_PATTERN) + val completedAlerts = currentAlerts.filter { it.state == COMPLETED } + assertEquals("Incorrect number of completed alerts", 2, completedAlerts.size) + } + + @Suppress("UNCHECKED_CAST") + fun `test bucket-level monitor throttling with per alert action scope`() { + val testIndex = createTestIndex() + insertSampleTimeSerializedData( + testIndex, + listOf( + "test_value_1", + "test_value_2" + ) + ) + + val query = QueryBuilders.rangeQuery("test_strict_date_time") + .gt("{{period_end}}||-10d") + .lte("{{period_end}}") + .format("epoch_millis") + val compositeSources = listOf( + TermsValuesSourceBuilder("test_field").field("test_field") + ) + val compositeAgg = CompositeAggregationBuilder("composite_agg", compositeSources) + val input = SearchInput(indices = listOf(testIndex), query = SearchSourceBuilder().size(0).query(query).aggregation(compositeAgg)) + val triggerScript = """ + params.docCount > 0 + """.trimIndent() + + val actionThrottleEnabled = randomActionWithPolicy( + template = randomTemplateScript("Hello {{ctx.monitor.name}}"), + destinationId = createDestination().id, + throttleEnabled = true, + throttle = Throttle(value = 5, unit = MINUTES), + actionExecutionPolicy = ActionExecutionPolicy( + actionExecutionScope = PerAlertActionScope(setOf(AlertCategory.DEDUPED, AlertCategory.NEW)) + ) + ) + val actionThrottleNotEnabled = randomActionWithPolicy( + template = randomTemplateScript("Hello {{ctx.monitor.name}}"), + destinationId = createDestination().id, + throttleEnabled = false, + throttle = Throttle(value = 5, unit = MINUTES), + actionExecutionPolicy = ActionExecutionPolicy( + actionExecutionScope = PerAlertActionScope(setOf(AlertCategory.DEDUPED, AlertCategory.NEW)) + ) + ) + val actions = listOf(actionThrottleEnabled, actionThrottleNotEnabled) + var trigger = randomBucketLevelTrigger(actions = actions) + trigger = trigger.copy( + bucketSelector = BucketSelectorExtAggregationBuilder( + name = trigger.id, + bucketsPathsMap = mapOf("docCount" to "_count"), + script = Script(triggerScript), + parentBucketPath = "composite_agg", + filter = null + ) + ) + val monitor = createMonitor(randomBucketLevelMonitor(inputs = listOf(input), enabled = false, triggers = listOf(trigger))) + + val monitorRunResultNotThrottled = entityAsMap(executeMonitor(monitor.id)) + verifyActionThrottleResultsForBucketLevelMonitor( + monitorRunResult = monitorRunResultNotThrottled, + expectedEvents = setOf("test_value_1", "test_value_2"), + expectedActionResults = mapOf( + Pair(actionThrottleEnabled.id, false), + Pair(actionThrottleNotEnabled.id, false) + ) + ) + + val notThrottledAlerts = searchAlerts(monitor) + assertEquals("Alerts may not have been saved correctly", 2, notThrottledAlerts.size) + val previousAlertExecutionTime: MutableMap> = mutableMapOf() + notThrottledAlerts.forEach { + verifyAlert(it, monitor, ACTIVE) + val notThrottledActionResults = verifyActionExecutionResultInAlert( + it, + mutableMapOf(Pair(actionThrottleEnabled.id, 0), Pair(actionThrottleNotEnabled.id, 0)) + ) + assertEquals(notThrottledActionResults.size, 2) + // Save the lastExecutionTimes of the actions for the Alert to be compared later against + // the next Monitor execution run + previousAlertExecutionTime[it.id] = mutableMapOf() + previousAlertExecutionTime[it.id]!![actionThrottleEnabled.id] = + notThrottledActionResults[actionThrottleEnabled.id]!!.lastExecutionTime + previousAlertExecutionTime[it.id]!![actionThrottleNotEnabled.id] = + notThrottledActionResults[actionThrottleNotEnabled.id]!!.lastExecutionTime + } + + // Runner uses ThreadPool.CachedTimeThread thread which only updates once every 200 ms. Wait a bit to + // let Action executionTime change. W/o this sleep the test can result in a false negative. + Thread.sleep(200) + val monitorRunResultThrottled = entityAsMap(executeMonitor(monitor.id)) + verifyActionThrottleResultsForBucketLevelMonitor( + monitorRunResult = monitorRunResultThrottled, + expectedEvents = setOf("test_value_1", "test_value_2"), + expectedActionResults = mapOf( + Pair(actionThrottleEnabled.id, true), + Pair(actionThrottleNotEnabled.id, false) + ) + ) + + val throttledAlerts = searchAlerts(monitor) + assertEquals("Alerts may not have been saved correctly", 2, throttledAlerts.size) + throttledAlerts.forEach { + verifyAlert(it, monitor, ACTIVE) + val throttledActionResults = verifyActionExecutionResultInAlert( + it, + mutableMapOf(Pair(actionThrottleEnabled.id, 1), Pair(actionThrottleNotEnabled.id, 0)) + ) + assertEquals(throttledActionResults.size, 2) + + val prevthrottledActionLastExecutionTime = previousAlertExecutionTime[it.id]!![actionThrottleEnabled.id] + val prevNotThrottledActionLastExecutionTime = previousAlertExecutionTime[it.id]!![actionThrottleNotEnabled.id] + assertEquals( + "Last execution time of a throttled action was updated for one of the Alerts", + prevthrottledActionLastExecutionTime, + throttledActionResults[actionThrottleEnabled.id]!!.lastExecutionTime + ) + assertTrue( + "Last execution time of a non-throttled action was not updated for one of the Alerts", + throttledActionResults[actionThrottleNotEnabled.id]!!.lastExecutionTime!! > prevNotThrottledActionLastExecutionTime + ) + } + } + private fun prepareTestAnomalyResult(detectorId: String, user: User) { val adResultIndex = ".opendistro-anomaly-results-history-2020.10.17" try { @@ -1014,6 +1576,24 @@ class MonitorRunnerIT : AlertingRestTestCase() { } } + @Suppress("UNCHECKED_CAST") + private fun verifyActionThrottleResultsForBucketLevelMonitor( + monitorRunResult: MutableMap, + expectedEvents: Set, + expectedActionResults: Map + ) { + for (triggerResult in monitorRunResult.objectMap("trigger_results").values) { + for (alertEvent in triggerResult.objectMap("action_results")) { + assertTrue(expectedEvents.contains(alertEvent.key)) + val actionResults = alertEvent.value.values as Collection> + for (actionResult in actionResults) { + val expected = expectedActionResults[actionResult["id"]] + assertEquals(expected, actionResult["throttled"]) + } + } + } + } + private fun verifyAlert(alert: Alert, monitor: Monitor, expectedState: Alert.State = ACTIVE) { assertNotNull(alert.id) assertNotNull(alert.startTime) diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/MonitorTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/MonitorTests.kt index 46c0bde8e..0e8f05ec0 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/MonitorTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/MonitorTests.kt @@ -34,7 +34,7 @@ import java.time.Instant class MonitorTests : OpenSearchTestCase() { fun `test enabled time`() { - val monitor = randomMonitor() + val monitor = randomQueryLevelMonitor() val enabledMonitor = monitor.copy(enabled = true, enabledTime = Instant.now()) try { enabledMonitor.copy(enabled = false) @@ -52,11 +52,11 @@ class MonitorTests : OpenSearchTestCase() { } fun `test max triggers`() { - val monitor = randomMonitor() + val monitor = randomQueryLevelMonitor() val tooManyTriggers = mutableListOf() for (i in 0..10) { - tooManyTriggers.add(randomTrigger()) + tooManyTriggers.add(randomQueryLevelTrigger()) } try { diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/TestHelpers.kt b/alerting/src/test/kotlin/org/opensearch/alerting/TestHelpers.kt index ed62473df..c576ba104 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/TestHelpers.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/TestHelpers.kt @@ -27,6 +27,8 @@ package org.opensearch.alerting import org.apache.http.Header import org.apache.http.HttpEntity +import org.opensearch.alerting.aggregation.bucketselectorext.BucketSelectorExtAggregationBuilder +import org.opensearch.alerting.aggregation.bucketselectorext.BucketSelectorExtFilter import org.opensearch.alerting.core.model.Input import org.opensearch.alerting.core.model.IntervalSchedule import org.opensearch.alerting.core.model.Schedule @@ -34,17 +36,27 @@ import org.opensearch.alerting.core.model.SearchInput import org.opensearch.alerting.elasticapi.string import org.opensearch.alerting.model.ActionExecutionResult import org.opensearch.alerting.model.ActionRunResult +import org.opensearch.alerting.model.AggregationResultBucket import org.opensearch.alerting.model.Alert +import org.opensearch.alerting.model.BucketLevelTrigger +import org.opensearch.alerting.model.BucketLevelTriggerRunResult import org.opensearch.alerting.model.InputRunResults import org.opensearch.alerting.model.Monitor import org.opensearch.alerting.model.MonitorRunResult +import org.opensearch.alerting.model.QueryLevelTrigger +import org.opensearch.alerting.model.QueryLevelTriggerRunResult import org.opensearch.alerting.model.Trigger -import org.opensearch.alerting.model.TriggerRunResult import org.opensearch.alerting.model.action.Action +import org.opensearch.alerting.model.action.ActionExecutionPolicy +import org.opensearch.alerting.model.action.ActionExecutionScope +import org.opensearch.alerting.model.action.AlertCategory +import org.opensearch.alerting.model.action.PerAlertActionScope +import org.opensearch.alerting.model.action.PerExecutionActionScope import org.opensearch.alerting.model.action.Throttle import org.opensearch.alerting.model.destination.email.EmailAccount import org.opensearch.alerting.model.destination.email.EmailEntry import org.opensearch.alerting.model.destination.email.EmailGroup +import org.opensearch.alerting.util.getBucketKeysHash import org.opensearch.client.Request import org.opensearch.client.RequestOptions import org.opensearch.client.Response @@ -64,59 +76,84 @@ import org.opensearch.index.query.QueryBuilders import org.opensearch.script.Script import org.opensearch.script.ScriptType import org.opensearch.search.SearchModule +import org.opensearch.search.aggregations.bucket.terms.IncludeExclude +import org.opensearch.search.aggregations.bucket.terms.TermsAggregationBuilder import org.opensearch.search.builder.SearchSourceBuilder -import org.opensearch.test.OpenSearchTestCase +import org.opensearch.test.OpenSearchTestCase.randomBoolean import org.opensearch.test.OpenSearchTestCase.randomInt import org.opensearch.test.OpenSearchTestCase.randomIntBetween import org.opensearch.test.rest.OpenSearchRestTestCase import java.time.Instant import java.time.temporal.ChronoUnit -fun randomMonitor( +fun randomQueryLevelMonitor( name: String = OpenSearchRestTestCase.randomAlphaOfLength(10), user: User = randomUser(), inputs: List = listOf(SearchInput(emptyList(), SearchSourceBuilder().query(QueryBuilders.matchAllQuery()))), schedule: Schedule = IntervalSchedule(interval = 5, unit = ChronoUnit.MINUTES), - enabled: Boolean = OpenSearchTestCase.randomBoolean(), - triggers: List = (1..randomInt(10)).map { randomTrigger() }, + enabled: Boolean = randomBoolean(), + triggers: List = (1..randomInt(10)).map { randomQueryLevelTrigger() }, enabledTime: Instant? = if (enabled) Instant.now().truncatedTo(ChronoUnit.MILLIS) else null, lastUpdateTime: Instant = Instant.now().truncatedTo(ChronoUnit.MILLIS), withMetadata: Boolean = false ): Monitor { return Monitor( - name = name, enabled = enabled, inputs = inputs, schedule = schedule, triggers = triggers, - enabledTime = enabledTime, lastUpdateTime = lastUpdateTime, - user = user, uiMetadata = if (withMetadata) mapOf("foo" to "bar") else mapOf() + name = name, monitorType = Monitor.MonitorType.QUERY_LEVEL_MONITOR, enabled = enabled, inputs = inputs, + schedule = schedule, triggers = triggers, enabledTime = enabledTime, lastUpdateTime = lastUpdateTime, user = user, + uiMetadata = if (withMetadata) mapOf("foo" to "bar") else mapOf() ) } // Monitor of older versions without security. -fun randomMonitorWithoutUser( +fun randomQueryLevelMonitorWithoutUser( name: String = OpenSearchRestTestCase.randomAlphaOfLength(10), inputs: List = listOf(SearchInput(emptyList(), SearchSourceBuilder().query(QueryBuilders.matchAllQuery()))), schedule: Schedule = IntervalSchedule(interval = 5, unit = ChronoUnit.MINUTES), - enabled: Boolean = OpenSearchTestCase.randomBoolean(), - triggers: List = (1..randomInt(10)).map { randomTrigger() }, + enabled: Boolean = randomBoolean(), + triggers: List = (1..randomInt(10)).map { randomQueryLevelTrigger() }, enabledTime: Instant? = if (enabled) Instant.now().truncatedTo(ChronoUnit.MILLIS) else null, lastUpdateTime: Instant = Instant.now().truncatedTo(ChronoUnit.MILLIS), withMetadata: Boolean = false ): Monitor { return Monitor( - name = name, enabled = enabled, inputs = inputs, schedule = schedule, triggers = triggers, - enabledTime = enabledTime, lastUpdateTime = lastUpdateTime, - user = null, uiMetadata = if (withMetadata) mapOf("foo" to "bar") else mapOf() + name = name, monitorType = Monitor.MonitorType.QUERY_LEVEL_MONITOR, enabled = enabled, inputs = inputs, + schedule = schedule, triggers = triggers, enabledTime = enabledTime, lastUpdateTime = lastUpdateTime, user = null, + uiMetadata = if (withMetadata) mapOf("foo" to "bar") else mapOf() ) } -fun randomTrigger( +fun randomBucketLevelMonitor( + name: String = OpenSearchRestTestCase.randomAlphaOfLength(10), + user: User = randomUser(), + inputs: List = listOf( + SearchInput( + emptyList(), + SearchSourceBuilder().query(QueryBuilders.matchAllQuery()).aggregation(TermsAggregationBuilder("test_agg")) + ) + ), + schedule: Schedule = IntervalSchedule(interval = 5, unit = ChronoUnit.MINUTES), + enabled: Boolean = randomBoolean(), + triggers: List = (1..randomInt(10)).map { randomBucketLevelTrigger() }, + enabledTime: Instant? = if (enabled) Instant.now().truncatedTo(ChronoUnit.MILLIS) else null, + lastUpdateTime: Instant = Instant.now().truncatedTo(ChronoUnit.MILLIS), + withMetadata: Boolean = false +): Monitor { + return Monitor( + name = name, monitorType = Monitor.MonitorType.BUCKET_LEVEL_MONITOR, enabled = enabled, inputs = inputs, + schedule = schedule, triggers = triggers, enabledTime = enabledTime, lastUpdateTime = lastUpdateTime, user = user, + uiMetadata = if (withMetadata) mapOf("foo" to "bar") else mapOf() + ) +} + +fun randomQueryLevelTrigger( id: String = UUIDs.base64UUID(), name: String = OpenSearchRestTestCase.randomAlphaOfLength(10), severity: String = "1", condition: Script = randomScript(), actions: List = mutableListOf(), destinationId: String = "" -): Trigger { - return Trigger( +): QueryLevelTrigger { + return QueryLevelTrigger( id = id, name = name, severity = severity, @@ -125,6 +162,40 @@ fun randomTrigger( ) } +fun randomBucketLevelTrigger( + id: String = UUIDs.base64UUID(), + name: String = OpenSearchRestTestCase.randomAlphaOfLength(10), + severity: String = "1", + bucketSelector: BucketSelectorExtAggregationBuilder = randomBucketSelectorExtAggregationBuilder(name = id), + actions: List = mutableListOf(), + destinationId: String = "" +): BucketLevelTrigger { + return BucketLevelTrigger( + id = id, + name = name, + severity = severity, + bucketSelector = bucketSelector, + actions = if (actions.isEmpty()) (0..randomInt(10)).map { randomActionWithPolicy(destinationId = destinationId) } else actions + ) +} + +fun randomBucketSelectorExtAggregationBuilder( + name: String = OpenSearchRestTestCase.randomAlphaOfLength(10), + bucketsPathsMap: MutableMap = mutableMapOf("avg" to "10"), + script: Script = randomBucketSelectorScript(params = bucketsPathsMap), + parentBucketPath: String = "testPath", + filter: BucketSelectorExtFilter = BucketSelectorExtFilter(IncludeExclude("foo*", "bar*")) +): BucketSelectorExtAggregationBuilder { + return BucketSelectorExtAggregationBuilder(name, bucketsPathsMap, script, parentBucketPath, filter) +} + +fun randomBucketSelectorScript( + idOrCode: String = "params.avg >= 0", + params: Map = mutableMapOf("avg" to "10") +): Script { + return Script(Script.DEFAULT_SCRIPT_TYPE, Script.DEFAULT_SCRIPT_LANG, idOrCode, emptyMap(), params) +} + fun randomEmailAccount( name: String = OpenSearchRestTestCase.randomAlphaOfLength(10), email: String = OpenSearchRestTestCase.randomAlphaOfLength(5) + "@email.com", @@ -173,15 +244,44 @@ fun randomAction( destinationId: String = "", throttleEnabled: Boolean = false, throttle: Throttle = randomThrottle() -) = Action(name, destinationId, template, template, throttleEnabled, throttle) +) = Action(name, destinationId, template, template, throttleEnabled, throttle, actionExecutionPolicy = null) + +fun randomActionWithPolicy( + name: String = OpenSearchRestTestCase.randomUnicodeOfLength(10), + template: Script = randomTemplateScript("Hello World"), + destinationId: String = "", + throttleEnabled: Boolean = false, + throttle: Throttle = randomThrottle(), + actionExecutionPolicy: ActionExecutionPolicy? = randomActionExecutionPolicy() +): Action { + return if (actionExecutionPolicy?.actionExecutionScope is PerExecutionActionScope) { + // Return null for throttle when using PerExecutionActionScope since throttling is currently not supported for it + Action(name, destinationId, template, template, throttleEnabled, null, actionExecutionPolicy = actionExecutionPolicy) + } else { + Action(name, destinationId, template, template, throttleEnabled, throttle, actionExecutionPolicy = actionExecutionPolicy) + } +} fun randomThrottle( value: Int = randomIntBetween(60, 120), unit: ChronoUnit = ChronoUnit.MINUTES ) = Throttle(value, unit) -fun randomAlert(monitor: Monitor = randomMonitor()): Alert { - val trigger = randomTrigger() +fun randomActionExecutionPolicy( + actionExecutionScope: ActionExecutionScope = randomActionExecutionScope() +) = ActionExecutionPolicy(actionExecutionScope) + +fun randomActionExecutionScope(): ActionExecutionScope { + return if (randomBoolean()) { + val alertCategories = AlertCategory.values() + PerAlertActionScope(actionableAlerts = (1..randomInt(alertCategories.size)).map { alertCategories[it - 1] }.toSet()) + } else { + PerExecutionActionScope() + } +} + +fun randomAlert(monitor: Monitor = randomQueryLevelMonitor()): Alert { + val trigger = randomQueryLevelTrigger() val actionExecutionResults = mutableListOf(randomActionExecutionResult(), randomActionExecutionResult()) return Alert( monitor, trigger, Instant.now().truncatedTo(ChronoUnit.MILLIS), null, @@ -189,6 +289,20 @@ fun randomAlert(monitor: Monitor = randomMonitor()): Alert { ) } +fun randomAlertWithAggregationResultBucket(monitor: Monitor = randomBucketLevelMonitor()): Alert { + val trigger = randomBucketLevelTrigger() + val actionExecutionResults = mutableListOf(randomActionExecutionResult(), randomActionExecutionResult()) + return Alert( + monitor, trigger, Instant.now().truncatedTo(ChronoUnit.MILLIS), null, + actionExecutionResults = actionExecutionResults, + aggregationResultBucket = AggregationResultBucket( + "parent_bucket_path_1", + listOf("bucket_key_1"), + mapOf("k1" to "val1", "k2" to "val2") + ) + ) +} + fun randomEmailAccountMethod(): EmailAccount.MethodType { val methodValues = EmailAccount.MethodType.values().map { it.value } val randomValue = methodValues[randomInt(methodValues.size - 1)] @@ -201,9 +315,24 @@ fun randomActionExecutionResult( throttledCount: Int = randomInt() ) = ActionExecutionResult(actionId, lastExecutionTime, throttledCount) -fun randomMonitorRunResult(): MonitorRunResult { - val triggerResults = mutableMapOf() - val triggerRunResult = randomTriggerRunResult() +fun randomQueryLevelMonitorRunResult(): MonitorRunResult { + val triggerResults = mutableMapOf() + val triggerRunResult = randomQueryLevelTriggerRunResult() + triggerResults.plus(Pair("test", triggerRunResult)) + + return MonitorRunResult( + "test-monitor", + Instant.now(), + Instant.now(), + null, + randomInputRunResults(), + triggerResults + ) +} + +fun randomBucketLevelMonitorRunResult(): MonitorRunResult { + val triggerResults = mutableMapOf() + val triggerRunResult = randomBucketLevelTriggerRunResult() triggerResults.plus(Pair("test", triggerRunResult)) return MonitorRunResult( @@ -220,11 +349,41 @@ fun randomInputRunResults(): InputRunResults { return InputRunResults(listOf(), null) } -fun randomTriggerRunResult(): TriggerRunResult { +fun randomQueryLevelTriggerRunResult(): QueryLevelTriggerRunResult { val map = mutableMapOf() map.plus(Pair("key1", randomActionRunResult())) map.plus(Pair("key2", randomActionRunResult())) - return TriggerRunResult("trigger-name", true, null, map) + return QueryLevelTriggerRunResult("trigger-name", true, null, map) +} + +fun randomBucketLevelTriggerRunResult(): BucketLevelTriggerRunResult { + val map = mutableMapOf() + map.plus(Pair("key1", randomActionRunResult())) + map.plus(Pair("key2", randomActionRunResult())) + + val aggBucket1 = AggregationResultBucket( + "parent_bucket_path_1", + listOf("bucket_key_1"), + mapOf("k1" to "val1", "k2" to "val2") + ) + val aggBucket2 = AggregationResultBucket( + "parent_bucket_path_2", + listOf("bucket_key_2"), + mapOf("k1" to "val1", "k2" to "val2") + ) + + val actionResultsMap: MutableMap> = mutableMapOf() + actionResultsMap[aggBucket1.getBucketKeysHash()] = map + actionResultsMap[aggBucket2.getBucketKeysHash()] = map + + return BucketLevelTriggerRunResult( + "trigger-name", null, + mapOf( + aggBucket1.getBucketKeysHash() to aggBucket1, + aggBucket2.getBucketKeysHash() to aggBucket2 + ), + actionResultsMap + ) } fun randomActionRunResult(): ActionRunResult { @@ -331,8 +490,7 @@ fun parser(xc: String): XContentParser { fun xContentRegistry(): NamedXContentRegistry { return NamedXContentRegistry( listOf( - SearchInput.XCONTENT_REGISTRY - ) + - SearchModule(Settings.EMPTY, false, emptyList()).namedXContents + SearchInput.XCONTENT_REGISTRY, QueryLevelTrigger.XCONTENT_REGISTRY, BucketLevelTrigger.XCONTENT_REGISTRY + ) + SearchModule(Settings.EMPTY, false, emptyList()).namedXContents ) } diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/action/AcknowledgeAlertResponseTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/action/AcknowledgeAlertResponseTests.kt index 4b95a9e7c..c816b2248 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/action/AcknowledgeAlertResponseTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/action/AcknowledgeAlertResponseTests.kt @@ -45,7 +45,7 @@ class AcknowledgeAlertResponseTests : OpenSearchTestCase() { "1234", 0L, 1, "monitor-1234", "test-monitor", 0L, randomUser(), "trigger-14", "test-trigger", Alert.State.ACKNOWLEDGED, Instant.now(), Instant.now(), Instant.now(), Instant.now(), null, ArrayList(), - "sev-2", ArrayList() + "sev-2", ArrayList(), null ) ) val failed = mutableListOf( @@ -53,7 +53,7 @@ class AcknowledgeAlertResponseTests : OpenSearchTestCase() { "1234", 0L, 1, "monitor-1234", "test-monitor", 0L, randomUser(), "trigger-14", "test-trigger", Alert.State.ERROR, Instant.now(), Instant.now(), Instant.now(), Instant.now(), null, mutableListOf(AlertError(Instant.now(), "Error msg")), - "sev-2", mutableListOf(ActionExecutionResult("7890", null, 0)) + "sev-2", mutableListOf(ActionExecutionResult("7890", null, 0)), null ) ) val missing = mutableListOf("1", "2", "3", "4") diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/action/ExecuteMonitorRequestTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/action/ExecuteMonitorRequestTests.kt index ebd2373f8..2e7f95e19 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/action/ExecuteMonitorRequestTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/action/ExecuteMonitorRequestTests.kt @@ -27,7 +27,7 @@ package org.opensearch.alerting.action import org.opensearch.alerting.core.model.SearchInput -import org.opensearch.alerting.randomMonitor +import org.opensearch.alerting.randomQueryLevelMonitor import org.opensearch.common.io.stream.BytesStreamOutput import org.opensearch.common.io.stream.StreamInput import org.opensearch.common.unit.TimeValue @@ -52,7 +52,7 @@ class ExecuteMonitorRequestTests : OpenSearchTestCase() { } fun `test execute monitor request with monitor`() { - val monitor = randomMonitor().copy(inputs = listOf(SearchInput(emptyList(), SearchSourceBuilder()))) + val monitor = randomQueryLevelMonitor().copy(inputs = listOf(SearchInput(emptyList(), SearchSourceBuilder()))) val req = ExecuteMonitorRequest(false, TimeValue.timeValueSeconds(100L), null, monitor) assertNotNull(req.monitor) diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/action/ExecuteMonitorResponseTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/action/ExecuteMonitorResponseTests.kt index 7002bc098..d33127183 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/action/ExecuteMonitorResponseTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/action/ExecuteMonitorResponseTests.kt @@ -27,15 +27,29 @@ package org.opensearch.alerting.action import org.junit.Assert -import org.opensearch.alerting.randomMonitorRunResult +import org.opensearch.alerting.randomBucketLevelMonitorRunResult +import org.opensearch.alerting.randomQueryLevelMonitorRunResult import org.opensearch.common.io.stream.BytesStreamOutput import org.opensearch.common.io.stream.StreamInput import org.opensearch.test.OpenSearchTestCase class ExecuteMonitorResponseTests : OpenSearchTestCase() { - fun `test exec monitor response`() { - val req = ExecuteMonitorResponse(randomMonitorRunResult()) + fun `test exec query-level monitor response`() { + val req = ExecuteMonitorResponse(randomQueryLevelMonitorRunResult()) + Assert.assertNotNull(req) + + val out = BytesStreamOutput() + req.writeTo(out) + val sin = StreamInput.wrap(out.bytes().toBytesRef().bytes) + val newReq = ExecuteMonitorResponse(sin) + assertNotNull(newReq.monitorRunResult) + assertEquals("test-monitor", newReq.monitorRunResult.monitorName) + assertNotNull(newReq.monitorRunResult.inputResults) + } + + fun `test exec bucket-level monitor response`() { + val req = ExecuteMonitorResponse(randomBucketLevelMonitorRunResult()) Assert.assertNotNull(req) val out = BytesStreamOutput() diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/action/GetAlertsResponseTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/action/GetAlertsResponseTests.kt index afacfbd36..66931a415 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/action/GetAlertsResponseTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/action/GetAlertsResponseTests.kt @@ -72,7 +72,8 @@ class GetAlertsResponseTests : OpenSearchTestCase() { null, Collections.emptyList(), "severity", - Collections.emptyList() + Collections.emptyList(), + null ) val req = GetAlertsResponse(listOf(alert), 1) assertNotNull(req) @@ -107,7 +108,8 @@ class GetAlertsResponseTests : OpenSearchTestCase() { null, Collections.emptyList(), "severity", - Collections.emptyList() + Collections.emptyList(), + null ) val req = GetAlertsResponse(listOf(alert), 1) var actualXContentString = req.toXContent(builder(), ToXContent.EMPTY_PARAMS).string() diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/action/GetMonitorResponseTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/action/GetMonitorResponseTests.kt index 7516fea75..f2023fd6d 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/action/GetMonitorResponseTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/action/GetMonitorResponseTests.kt @@ -57,13 +57,22 @@ class GetMonitorResponseTests : OpenSearchTestCase() { val testInstance = Instant.ofEpochSecond(1538164858L) val cronSchedule = CronSchedule(cronExpression, ZoneId.of("Asia/Kolkata"), testInstance) - val req = GetMonitorResponse( - "1234", 1L, 2L, 0L, RestStatus.OK, - Monitor( - "123", 0L, "test-monitor", true, cronSchedule, Instant.now(), - Instant.now(), randomUser(), 0, mutableListOf(), mutableListOf(), mutableMapOf() - ) + val monitor = Monitor( + id = "123", + version = 0L, + name = "test-monitor", + enabled = true, + schedule = cronSchedule, + lastUpdateTime = Instant.now(), + enabledTime = Instant.now(), + monitorType = Monitor.MonitorType.QUERY_LEVEL_MONITOR, + user = randomUser(), + schemaVersion = 0, + inputs = mutableListOf(), + triggers = mutableListOf(), + uiMetadata = mutableMapOf() ) + val req = GetMonitorResponse("1234", 1L, 2L, 0L, RestStatus.OK, monitor) assertNotNull(req) val out = BytesStreamOutput() diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/action/IndexMonitorRequestTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/action/IndexMonitorRequestTests.kt index b38f2aee1..5c9ce3b78 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/action/IndexMonitorRequestTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/action/IndexMonitorRequestTests.kt @@ -28,7 +28,7 @@ package org.opensearch.alerting.action import org.opensearch.action.support.WriteRequest import org.opensearch.alerting.core.model.SearchInput -import org.opensearch.alerting.randomMonitor +import org.opensearch.alerting.randomQueryLevelMonitor import org.opensearch.common.io.stream.BytesStreamOutput import org.opensearch.common.io.stream.StreamInput import org.opensearch.rest.RestRequest @@ -41,7 +41,7 @@ class IndexMonitorRequestTests : OpenSearchTestCase() { val req = IndexMonitorRequest( "1234", 1L, 2L, WriteRequest.RefreshPolicy.IMMEDIATE, RestRequest.Method.POST, - randomMonitor().copy(inputs = listOf(SearchInput(emptyList(), SearchSourceBuilder()))) + randomQueryLevelMonitor().copy(inputs = listOf(SearchInput(emptyList(), SearchSourceBuilder()))) ) assertNotNull(req) @@ -60,7 +60,7 @@ class IndexMonitorRequestTests : OpenSearchTestCase() { val req = IndexMonitorRequest( "1234", 1L, 2L, WriteRequest.RefreshPolicy.IMMEDIATE, RestRequest.Method.PUT, - randomMonitor().copy(inputs = listOf(SearchInput(emptyList(), SearchSourceBuilder()))) + randomQueryLevelMonitor().copy(inputs = listOf(SearchInput(emptyList(), SearchSourceBuilder()))) ) assertNotNull(req) diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/action/IndexMonitorResponseTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/action/IndexMonitorResponseTests.kt index 444cf2d94..4d1d138de 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/action/IndexMonitorResponseTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/action/IndexMonitorResponseTests.kt @@ -43,13 +43,22 @@ class IndexMonitorResponseTests : OpenSearchTestCase() { val testInstance = Instant.ofEpochSecond(1538164858L) val cronSchedule = CronSchedule(cronExpression, ZoneId.of("Asia/Kolkata"), testInstance) - val req = IndexMonitorResponse( - "1234", 1L, 2L, 0L, RestStatus.OK, - Monitor( - "123", 0L, "test-monitor", true, cronSchedule, Instant.now(), - Instant.now(), randomUser(), 0, mutableListOf(), mutableListOf(), mutableMapOf() - ) + val monitor = Monitor( + id = "123", + version = 0L, + name = "test-monitor", + enabled = true, + schedule = cronSchedule, + lastUpdateTime = Instant.now(), + enabledTime = Instant.now(), + monitorType = Monitor.MonitorType.QUERY_LEVEL_MONITOR, + user = randomUser(), + schemaVersion = 0, + inputs = mutableListOf(), + triggers = mutableListOf(), + uiMetadata = mutableMapOf() ) + val req = IndexMonitorResponse("1234", 1L, 2L, 0L, RestStatus.OK, monitor) assertNotNull(req) val out = BytesStreamOutput() diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregationBuilderTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregationBuilderTests.kt new file mode 100644 index 000000000..15c15546c --- /dev/null +++ b/alerting/src/test/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregationBuilderTests.kt @@ -0,0 +1,60 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.aggregation.bucketselectorext + +import org.opensearch.alerting.AlertingPlugin +import org.opensearch.plugins.SearchPlugin +import org.opensearch.script.Script +import org.opensearch.script.ScriptType +import org.opensearch.search.aggregations.BasePipelineAggregationTestCase +import org.opensearch.search.aggregations.bucket.terms.IncludeExclude +import org.opensearch.search.aggregations.pipeline.BucketHelpers.GapPolicy + +class BucketSelectorExtAggregationBuilderTests : BasePipelineAggregationTestCase() { + override fun plugins(): List { + return listOf(AlertingPlugin()) + } + + override fun createTestAggregatorFactory(): BucketSelectorExtAggregationBuilder { + val name = randomAlphaOfLengthBetween(3, 20) + val bucketsPaths: MutableMap = HashMap() + val numBucketPaths = randomIntBetween(1, 10) + for (i in 0 until numBucketPaths) { + bucketsPaths[randomAlphaOfLengthBetween(1, 20)] = randomAlphaOfLengthBetween(1, 40) + } + val script: Script + if (randomBoolean()) { + script = mockScript("script") + } else { + val params: MutableMap = HashMap() + if (randomBoolean()) { + params["foo"] = "bar" + } + val type = randomFrom(*ScriptType.values()) + script = Script( + type, + if (type == ScriptType.STORED) null else randomFrom("my_lang", Script.DEFAULT_SCRIPT_LANG), + "script", params + ) + } + val parentBucketPath = randomAlphaOfLengthBetween(3, 20) + val filter = BucketSelectorExtFilter(IncludeExclude("foo.*", "bar.*")) + val factory = BucketSelectorExtAggregationBuilder( + name, bucketsPaths, + script, parentBucketPath, filter + ) + if (randomBoolean()) { + factory.gapPolicy(randomFrom(*GapPolicy.values())) + } + return factory + } +} diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregatorTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregatorTests.kt new file mode 100644 index 000000000..1a89bcbca --- /dev/null +++ b/alerting/src/test/kotlin/org/opensearch/alerting/aggregation/bucketselectorext/BucketSelectorExtAggregatorTests.kt @@ -0,0 +1,374 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.aggregation.bucketselectorext + +import org.apache.lucene.document.Document +import org.apache.lucene.document.SortedNumericDocValuesField +import org.apache.lucene.document.SortedSetDocValuesField +import org.apache.lucene.index.DirectoryReader +import org.apache.lucene.index.RandomIndexWriter +import org.apache.lucene.search.MatchAllDocsQuery +import org.apache.lucene.search.Query +import org.apache.lucene.util.BytesRef +import org.hamcrest.CoreMatchers +import org.opensearch.common.CheckedConsumer +import org.opensearch.common.settings.Settings +import org.opensearch.index.mapper.KeywordFieldMapper.KeywordFieldType +import org.opensearch.index.mapper.MappedFieldType +import org.opensearch.index.mapper.NumberFieldMapper +import org.opensearch.index.mapper.NumberFieldMapper.NumberFieldType +import org.opensearch.index.query.MatchAllQueryBuilder +import org.opensearch.script.MockScriptEngine +import org.opensearch.script.Script +import org.opensearch.script.ScriptEngine +import org.opensearch.script.ScriptModule +import org.opensearch.script.ScriptService +import org.opensearch.script.ScriptType +import org.opensearch.search.aggregations.Aggregation +import org.opensearch.search.aggregations.Aggregator +import org.opensearch.search.aggregations.AggregatorTestCase +import org.opensearch.search.aggregations.bucket.filter.FilterAggregationBuilder +import org.opensearch.search.aggregations.bucket.filter.FiltersAggregationBuilder +import org.opensearch.search.aggregations.bucket.filter.InternalFilter +import org.opensearch.search.aggregations.bucket.filter.InternalFilters +import org.opensearch.search.aggregations.bucket.terms.IncludeExclude +import org.opensearch.search.aggregations.bucket.terms.TermsAggregationBuilder +import org.opensearch.search.aggregations.metrics.AvgAggregationBuilder +import org.opensearch.search.aggregations.metrics.ValueCountAggregationBuilder +import java.io.IOException +import java.util.Collections +import java.util.function.Consumer +import java.util.function.Function + +class BucketSelectorExtAggregatorTests : AggregatorTestCase() { + + private var SCRIPTNAME = "bucket_selector_script" + private var paramName = "the_avg" + private var paramValue = 19.0 + + override fun getMockScriptService(): ScriptService { + + val scriptEngine = MockScriptEngine( + MockScriptEngine.NAME, + Collections.singletonMap( + SCRIPTNAME, + Function, Any> { script: Map -> + script[paramName].toString().toDouble() == paramValue + } + ), + emptyMap() + ) + val engines: Map = Collections.singletonMap(scriptEngine.type, scriptEngine) + return ScriptService(Settings.EMPTY, engines, ScriptModule.CORE_CONTEXTS) + } + + @Throws(Exception::class) + fun `test bucket selector script`() { + val fieldType: MappedFieldType = NumberFieldType("number_field", NumberFieldMapper.NumberType.INTEGER) + val fieldType1: MappedFieldType = KeywordFieldType("the_field") + + val filters: FiltersAggregationBuilder = FiltersAggregationBuilder("placeholder", MatchAllQueryBuilder()) + .subAggregation( + TermsAggregationBuilder("the_terms").field("the_field") + .subAggregation(AvgAggregationBuilder("the_avg").field("number_field")) + ) + .subAggregation( + BucketSelectorExtAggregationBuilder( + "test_bucket_selector_ext", + Collections.singletonMap("the_avg", "the_avg.value"), + Script(ScriptType.INLINE, MockScriptEngine.NAME, SCRIPTNAME, emptyMap()), + "the_terms", + null + ) + ) + paramName = "the_avg" + paramValue = 19.0 + testCase( + filters, MatchAllDocsQuery(), + CheckedConsumer { iw: RandomIndexWriter -> + var doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test1"))) + doc.add(SortedNumericDocValuesField("number_field", 20)) + iw.addDocument(doc) + doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test2"))) + doc.add(SortedNumericDocValuesField("number_field", 19)) + iw.addDocument(doc) + }, + Consumer { f: InternalFilters -> + assertThat( + (f.buckets[0].aggregations.get("test_bucket_selector_ext") as BucketSelectorIndices).bucketIndices[0], + CoreMatchers.equalTo(1) + ) + }, + fieldType, fieldType1 + ) + } + + @Throws(Exception::class) + fun `test bucket selector filter include`() { + val fieldType: MappedFieldType = NumberFieldType("number_field", NumberFieldMapper.NumberType.INTEGER) + val fieldType1: MappedFieldType = KeywordFieldType("the_field") + + val selectorAgg1: FiltersAggregationBuilder = FiltersAggregationBuilder("placeholder", MatchAllQueryBuilder()) + .subAggregation( + TermsAggregationBuilder("the_terms").field("the_field") + .subAggregation(AvgAggregationBuilder("the_avg").field("number_field")) + ) + .subAggregation( + BucketSelectorExtAggregationBuilder( + "test_bucket_selector_ext", + Collections.singletonMap("the_avg", "the_avg.value"), + Script(ScriptType.INLINE, MockScriptEngine.NAME, SCRIPTNAME, emptyMap()), + "the_terms", + BucketSelectorExtFilter(IncludeExclude(arrayOf("test1"), arrayOf())) + ) + ) + + val selectorAgg2: FiltersAggregationBuilder = FiltersAggregationBuilder("placeholder", MatchAllQueryBuilder()) + .subAggregation( + TermsAggregationBuilder("the_terms").field("the_field") + .subAggregation(AvgAggregationBuilder("the_avg").field("number_field")) + ) + .subAggregation( + BucketSelectorExtAggregationBuilder( + "test_bucket_selector_ext", + Collections.singletonMap("the_avg", "the_avg.value"), + Script(ScriptType.INLINE, MockScriptEngine.NAME, SCRIPTNAME, emptyMap()), + "the_terms", + BucketSelectorExtFilter(IncludeExclude(arrayOf("test2"), arrayOf())) + ) + ) + + paramName = "the_avg" + paramValue = 19.0 + + testCase( + selectorAgg1, MatchAllDocsQuery(), + CheckedConsumer { iw: RandomIndexWriter -> + var doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test1"))) + doc.add(SortedNumericDocValuesField("number_field", 20)) + iw.addDocument(doc) + doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test2"))) + doc.add(SortedNumericDocValuesField("number_field", 19)) + iw.addDocument(doc) + }, + Consumer { f: InternalFilters -> + assertThat( + (f.buckets[0].aggregations.get("test_bucket_selector_ext") as BucketSelectorIndices).bucketIndices.size, + CoreMatchers.equalTo(0) + ) + }, + fieldType, fieldType1 + ) + + testCase( + selectorAgg2, MatchAllDocsQuery(), + CheckedConsumer { iw: RandomIndexWriter -> + var doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test1"))) + doc.add(SortedNumericDocValuesField("number_field", 20)) + iw.addDocument(doc) + doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test2"))) + doc.add(SortedNumericDocValuesField("number_field", 19)) + iw.addDocument(doc) + }, + Consumer { f: InternalFilters -> + assertThat( + (f.buckets[0].aggregations.get("test_bucket_selector_ext") as BucketSelectorIndices).bucketIndices[0], + CoreMatchers.equalTo(1) + ) + }, + fieldType, fieldType1 + ) + } + + @Throws(Exception::class) + fun `test bucket selector filter exclude`() { + val fieldType: MappedFieldType = NumberFieldType("number_field", NumberFieldMapper.NumberType.INTEGER) + val fieldType1: MappedFieldType = KeywordFieldType("the_field") + + val selectorAgg1: FiltersAggregationBuilder = FiltersAggregationBuilder("placeholder", MatchAllQueryBuilder()) + .subAggregation( + TermsAggregationBuilder("the_terms").field("the_field") + .subAggregation(AvgAggregationBuilder("the_avg").field("number_field")) + ) + .subAggregation( + BucketSelectorExtAggregationBuilder( + "test_bucket_selector_ext", + Collections.singletonMap("the_avg", "the_avg.value"), + Script(ScriptType.INLINE, MockScriptEngine.NAME, SCRIPTNAME, emptyMap()), + "the_terms", + BucketSelectorExtFilter(IncludeExclude(arrayOf(), arrayOf("test2"))) + ) + ) + paramName = "the_avg" + paramValue = 19.0 + testCase( + selectorAgg1, MatchAllDocsQuery(), + CheckedConsumer { iw: RandomIndexWriter -> + var doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test1"))) + doc.add(SortedNumericDocValuesField("number_field", 20)) + iw.addDocument(doc) + doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test2"))) + doc.add(SortedNumericDocValuesField("number_field", 19)) + iw.addDocument(doc) + }, + Consumer { f: InternalFilters -> + assertThat( + (f.buckets[0].aggregations.get("test_bucket_selector_ext") as BucketSelectorIndices).bucketIndices.size, + CoreMatchers.equalTo(0) + ) + }, + fieldType, fieldType1 + ) + } + + @Throws(Exception::class) + fun `test bucket selector filter numeric key`() { + val fieldType: MappedFieldType = NumberFieldType("number_field", NumberFieldMapper.NumberType.INTEGER) + val fieldType1: MappedFieldType = KeywordFieldType("the_field") + + val selectorAgg1: FiltersAggregationBuilder = FiltersAggregationBuilder("placeholder", MatchAllQueryBuilder()) + .subAggregation( + TermsAggregationBuilder("number_agg").field("number_field") + .subAggregation(ValueCountAggregationBuilder("count").field("number_field")) + ) + .subAggregation( + BucketSelectorExtAggregationBuilder( + "test_bucket_selector_ext", + Collections.singletonMap("count", "count"), + Script(ScriptType.INLINE, MockScriptEngine.NAME, SCRIPTNAME, emptyMap()), + "number_agg", + BucketSelectorExtFilter(IncludeExclude(doubleArrayOf(19.0), doubleArrayOf())) + ) + ) + + paramName = "count" + paramValue = 1.0 + testCase( + selectorAgg1, MatchAllDocsQuery(), + CheckedConsumer { iw: RandomIndexWriter -> + var doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test1"))) + doc.add(SortedNumericDocValuesField("number_field", 20)) + iw.addDocument(doc) + doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test2"))) + doc.add(SortedNumericDocValuesField("number_field", 19)) + iw.addDocument(doc) + }, + Consumer { f: InternalFilters -> + assertThat( + (f.buckets[0].aggregations.get("test_bucket_selector_ext") as BucketSelectorIndices).bucketIndices[0], + CoreMatchers.equalTo(0) + ) + }, + fieldType, fieldType1 + ) + } + + @Throws(Exception::class) + fun `test bucket selector nested parent path`() { + val fieldType: MappedFieldType = NumberFieldType("number_field", NumberFieldMapper.NumberType.INTEGER) + val fieldType1: MappedFieldType = KeywordFieldType("the_field") + + val selectorAgg1: FilterAggregationBuilder = FilterAggregationBuilder("placeholder", MatchAllQueryBuilder()) + .subAggregation( + FilterAggregationBuilder("parent_agg", MatchAllQueryBuilder()) + .subAggregation( + TermsAggregationBuilder("term_agg").field("the_field") + .subAggregation(AvgAggregationBuilder("the_avg").field("number_field")) + ) + ) + .subAggregation( + BucketSelectorExtAggregationBuilder( + "test_bucket_selector_ext", + Collections.singletonMap("the_avg", "the_avg.value"), + Script(ScriptType.INLINE, MockScriptEngine.NAME, SCRIPTNAME, emptyMap()), + "parent_agg>term_agg", + null + ) + ) + paramName = "the_avg" + paramValue = 19.0 + testCaseInternalFilter( + selectorAgg1, MatchAllDocsQuery(), + CheckedConsumer { iw: RandomIndexWriter -> + var doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test1"))) + + doc.add(SortedNumericDocValuesField("number_field", 20)) + iw.addDocument(doc) + doc = Document() + doc.add(SortedSetDocValuesField("the_field", BytesRef("test2"))) + + doc.add(SortedNumericDocValuesField("number_field", 19)) + iw.addDocument(doc) + }, + Consumer { f: InternalFilter -> + assertThat( + (f.aggregations.get("test_bucket_selector_ext") as BucketSelectorIndices).bucketIndices[0], + CoreMatchers.equalTo(1) + ) + }, + fieldType, fieldType1 + ) + } + + @Throws(IOException::class) + private fun testCase( + aggregationBuilder: FiltersAggregationBuilder, + query: Query, + buildIndex: CheckedConsumer, + verify: Consumer, + vararg fieldType: MappedFieldType + ) { + newDirectory().use { directory -> + val indexWriter = RandomIndexWriter(random(), directory) + buildIndex.accept(indexWriter) + indexWriter.close() + DirectoryReader.open(directory).use { indexReader -> + val indexSearcher = newIndexSearcher(indexReader) + val filters: InternalFilters + filters = searchAndReduce(indexSearcher, query, aggregationBuilder, *fieldType) + verify.accept(filters) + } + } + } + + @Throws(IOException::class) + private fun testCaseInternalFilter( + aggregationBuilder: FilterAggregationBuilder, + query: Query, + buildIndex: CheckedConsumer, + verify: Consumer, + vararg fieldType: MappedFieldType + ) { + newDirectory().use { directory -> + val indexWriter = RandomIndexWriter(random(), directory) + buildIndex.accept(indexWriter) + indexWriter.close() + DirectoryReader.open(directory).use { indexReader -> + val indexSearcher = newIndexSearcher(indexReader) + val filters: InternalFilter + filters = searchAndReduce(indexSearcher, query, aggregationBuilder, *fieldType) + verify.accept(filters) + } + } + } +} diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/alerts/AlertIndicesIT.kt b/alerting/src/test/kotlin/org/opensearch/alerting/alerts/AlertIndicesIT.kt index c7ebcdd84..948066f4d 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/alerts/AlertIndicesIT.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/alerts/AlertIndicesIT.kt @@ -34,8 +34,8 @@ import org.opensearch.alerting.AlertingRestTestCase import org.opensearch.alerting.NEVER_RUN import org.opensearch.alerting.core.model.ScheduledJob import org.opensearch.alerting.makeRequest -import org.opensearch.alerting.randomMonitor -import org.opensearch.alerting.randomTrigger +import org.opensearch.alerting.randomQueryLevelMonitor +import org.opensearch.alerting.randomQueryLevelTrigger import org.opensearch.alerting.settings.AlertingSettings import org.opensearch.common.xcontent.XContentType import org.opensearch.common.xcontent.json.JsonXContent.jsonXContent @@ -44,7 +44,7 @@ import org.opensearch.rest.RestStatus class AlertIndicesIT : AlertingRestTestCase() { fun `test create alert index`() { - executeMonitor(randomMonitor(triggers = listOf(randomTrigger(condition = ALWAYS_RUN)))) + executeMonitor(randomQueryLevelMonitor(triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN)))) assertIndexExists(AlertIndices.ALERT_INDEX) assertIndexExists(AlertIndices.HISTORY_WRITE_INDEX) @@ -57,7 +57,7 @@ class AlertIndicesIT : AlertingRestTestCase() { putAlertMappings( AlertIndices.alertMapping().trimStart('{').trimEnd('}') - .replace("\"schema_version\": 2", "\"schema_version\": 0") + .replace("\"schema_version\": 3", "\"schema_version\": 0") ) assertIndexExists(AlertIndices.ALERT_INDEX) assertIndexExists(AlertIndices.HISTORY_WRITE_INDEX) @@ -67,15 +67,15 @@ class AlertIndicesIT : AlertingRestTestCase() { executeMonitor(createRandomMonitor()) assertIndexExists(AlertIndices.ALERT_INDEX) assertIndexExists(AlertIndices.HISTORY_WRITE_INDEX) - verifyIndexSchemaVersion(ScheduledJob.SCHEDULED_JOBS_INDEX, 3) - verifyIndexSchemaVersion(AlertIndices.ALERT_INDEX, 2) - verifyIndexSchemaVersion(AlertIndices.HISTORY_WRITE_INDEX, 2) + verifyIndexSchemaVersion(ScheduledJob.SCHEDULED_JOBS_INDEX, 4) + verifyIndexSchemaVersion(AlertIndices.ALERT_INDEX, 3) + verifyIndexSchemaVersion(AlertIndices.HISTORY_WRITE_INDEX, 3) } fun `test alert index gets recreated automatically if deleted`() { wipeAllODFEIndices() assertIndexDoesNotExist(AlertIndices.ALERT_INDEX) - val trueMonitor = randomMonitor(triggers = listOf(randomTrigger(condition = ALWAYS_RUN))) + val trueMonitor = randomQueryLevelMonitor(triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN))) executeMonitor(trueMonitor) assertIndexExists(AlertIndices.ALERT_INDEX) @@ -95,7 +95,7 @@ class AlertIndicesIT : AlertingRestTestCase() { client().updateSettings(AlertingSettings.ALERT_HISTORY_ROLLOVER_PERIOD.key, "1s") client().updateSettings(AlertingSettings.ALERT_HISTORY_INDEX_MAX_AGE.key, "1s") - val trueMonitor = randomMonitor(triggers = listOf(randomTrigger(condition = ALWAYS_RUN))) + val trueMonitor = randomQueryLevelMonitor(triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN))) executeMonitor(trueMonitor) // Allow for a rollover index. @@ -106,8 +106,8 @@ class AlertIndicesIT : AlertingRestTestCase() { fun `test history disabled`() { resetHistorySettings() - val trigger1 = randomTrigger(condition = ALWAYS_RUN) - val monitor1 = createMonitor(randomMonitor(triggers = listOf(trigger1))) + val trigger1 = randomQueryLevelTrigger(condition = ALWAYS_RUN) + val monitor1 = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger1))) executeMonitor(monitor1.id) // Check if alert is active @@ -126,8 +126,8 @@ class AlertIndicesIT : AlertingRestTestCase() { // Disable alert history client().updateSettings(AlertingSettings.ALERT_HISTORY_ENABLED.key, "false") - val trigger2 = randomTrigger(condition = ALWAYS_RUN) - val monitor2 = createMonitor(randomMonitor(triggers = listOf(trigger2))) + val trigger2 = randomQueryLevelTrigger(condition = ALWAYS_RUN) + val monitor2 = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger2))) executeMonitor(monitor2.id) // Check if second alert is active @@ -151,8 +151,8 @@ class AlertIndicesIT : AlertingRestTestCase() { resetHistorySettings() // Create monitor and execute - val trigger = randomTrigger(condition = ALWAYS_RUN) - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) + val trigger = randomQueryLevelTrigger(condition = ALWAYS_RUN) + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) executeMonitor(monitor.id) // Check if alert is active and alert index is created @@ -179,7 +179,7 @@ class AlertIndicesIT : AlertingRestTestCase() { client().updateSettings(AlertingSettings.ALERT_HISTORY_RETENTION_PERIOD.key, "1s") // Give some time for history to be rolled over and cleared - Thread.sleep(2000) + Thread.sleep(5000) // Given the max_docs and retention settings above, the history index will rollover and the non-write index will be deleted. // This leaves two indices: alert index and an empty history write index diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/model/AlertTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/model/AlertTests.kt index 77e1dbef6..de8f8b98c 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/model/AlertTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/model/AlertTests.kt @@ -26,7 +26,9 @@ package org.opensearch.alerting.model +import org.junit.Assert import org.opensearch.alerting.randomAlert +import org.opensearch.alerting.randomAlertWithAggregationResultBucket import org.opensearch.test.OpenSearchTestCase class AlertTests : OpenSearchTestCase() { @@ -46,6 +48,30 @@ class AlertTests : OpenSearchTestCase() { assertEquals("Template args severity does not match", templateArgs[Alert.SEVERITY_FIELD], alert.severity) } + fun `test agg alert as template args`() { + val alert = randomAlertWithAggregationResultBucket().copy(acknowledgedTime = null, lastNotificationTime = null) + + val templateArgs = alert.asTemplateArg() + + assertEquals("Template args id does not match", templateArgs[Alert.ALERT_ID_FIELD], alert.id) + assertEquals("Template args version does not match", templateArgs[Alert.ALERT_VERSION_FIELD], alert.version) + assertEquals("Template args state does not match", templateArgs[Alert.STATE_FIELD], alert.state.toString()) + assertEquals("Template args error message does not match", templateArgs[Alert.ERROR_MESSAGE_FIELD], alert.errorMessage) + assertEquals("Template args acknowledged time does not match", templateArgs[Alert.ACKNOWLEDGED_TIME_FIELD], null) + assertEquals("Template args end time does not", templateArgs[Alert.END_TIME_FIELD], alert.endTime?.toEpochMilli()) + assertEquals("Template args start time does not", templateArgs[Alert.START_TIME_FIELD], alert.startTime.toEpochMilli()) + assertEquals("Template args last notification time does not match", templateArgs[Alert.LAST_NOTIFICATION_TIME_FIELD], null) + assertEquals("Template args severity does not match", templateArgs[Alert.SEVERITY_FIELD], alert.severity) + Assert.assertEquals( + "Template args bucketKeys do not match", + templateArgs[Alert.BUCKET_KEYS], alert.aggregationResultBucket?.bucketKeys?.joinToString(",") + ) + Assert.assertEquals( + "Template args parentBucketPath does not match", + templateArgs[Alert.PARENTS_BUCKET_PATH], alert.aggregationResultBucket?.parentBucketPath + ) + } + fun `test alert acknowledged`() { val ackAlert = randomAlert().copy(state = Alert.State.ACKNOWLEDGED) assertTrue("Alert is not acknowledged", ackAlert.isAcknowledged()) diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/model/WriteableTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/model/WriteableTests.kt index 0c23906ec..fda3bfbc2 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/model/WriteableTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/model/WriteableTests.kt @@ -28,19 +28,24 @@ package org.opensearch.alerting.model import org.opensearch.alerting.core.model.SearchInput import org.opensearch.alerting.model.action.Action +import org.opensearch.alerting.model.action.ActionExecutionPolicy import org.opensearch.alerting.model.action.Throttle import org.opensearch.alerting.model.destination.email.EmailAccount import org.opensearch.alerting.model.destination.email.EmailGroup import org.opensearch.alerting.randomAction +import org.opensearch.alerting.randomActionExecutionPolicy import org.opensearch.alerting.randomActionRunResult +import org.opensearch.alerting.randomBucketLevelMonitorRunResult +import org.opensearch.alerting.randomBucketLevelTrigger +import org.opensearch.alerting.randomBucketLevelTriggerRunResult import org.opensearch.alerting.randomEmailAccount import org.opensearch.alerting.randomEmailGroup import org.opensearch.alerting.randomInputRunResults -import org.opensearch.alerting.randomMonitor -import org.opensearch.alerting.randomMonitorRunResult +import org.opensearch.alerting.randomQueryLevelMonitor +import org.opensearch.alerting.randomQueryLevelMonitorRunResult +import org.opensearch.alerting.randomQueryLevelTrigger +import org.opensearch.alerting.randomQueryLevelTriggerRunResult import org.opensearch.alerting.randomThrottle -import org.opensearch.alerting.randomTrigger -import org.opensearch.alerting.randomTriggerRunResult import org.opensearch.alerting.randomUser import org.opensearch.alerting.randomUserEmpty import org.opensearch.common.io.stream.BytesStreamOutput @@ -96,22 +101,31 @@ class WriteableTests : OpenSearchTestCase() { assertEquals("Round tripping Action doesn't work", action, newAction) } - fun `test monitor as stream`() { - val monitor = randomMonitor().copy(inputs = listOf(SearchInput(emptyList(), SearchSourceBuilder()))) + fun `test query-level monitor as stream`() { + val monitor = randomQueryLevelMonitor().copy(inputs = listOf(SearchInput(emptyList(), SearchSourceBuilder()))) val out = BytesStreamOutput() monitor.writeTo(out) val sin = StreamInput.wrap(out.bytes().toBytesRef().bytes) val newMonitor = Monitor(sin) - assertEquals("Round tripping Monitor doesn't work", monitor, newMonitor) + assertEquals("Round tripping QueryLevelMonitor doesn't work", monitor, newMonitor) } - fun `test trigger as stream`() { - val trigger = randomTrigger() + fun `test query-level trigger as stream`() { + val trigger = randomQueryLevelTrigger() val out = BytesStreamOutput() trigger.writeTo(out) val sin = StreamInput.wrap(out.bytes().toBytesRef().bytes) - val newTrigger = Trigger(sin) - assertEquals("Round tripping Trigger doesn't work", trigger, newTrigger) + val newTrigger = QueryLevelTrigger.readFrom(sin) + assertEquals("Round tripping QueryLevelTrigger doesn't work", trigger, newTrigger) + } + + fun `test bucket-level trigger as stream`() { + val trigger = randomBucketLevelTrigger() + val out = BytesStreamOutput() + trigger.writeTo(out) + val sin = StreamInput.wrap(out.bytes().toBytesRef().bytes) + val newTrigger = BucketLevelTrigger.readFrom(sin) + assertEquals("Round tripping BucketLevelTrigger doesn't work", trigger, newTrigger) } fun `test actionrunresult as stream`() { @@ -123,12 +137,21 @@ class WriteableTests : OpenSearchTestCase() { assertEquals("Round tripping ActionRunResult doesn't work", actionRunResult, newActionRunResult) } - fun `test triggerrunresult as stream`() { - val runResult = randomTriggerRunResult() + fun `test query-level triggerrunresult as stream`() { + val runResult = randomQueryLevelTriggerRunResult() + val out = BytesStreamOutput() + runResult.writeTo(out) + val sin = StreamInput.wrap(out.bytes().toBytesRef().bytes) + val newRunResult = QueryLevelTriggerRunResult(sin) + assertEquals("Round tripping ActionRunResult doesn't work", runResult, newRunResult) + } + + fun `test bucket-level triggerrunresult as stream`() { + val runResult = randomBucketLevelTriggerRunResult() val out = BytesStreamOutput() runResult.writeTo(out) val sin = StreamInput.wrap(out.bytes().toBytesRef().bytes) - val newRunResult = TriggerRunResult(sin) + val newRunResult = BucketLevelTriggerRunResult(sin) assertEquals("Round tripping ActionRunResult doesn't work", runResult, newRunResult) } @@ -141,12 +164,21 @@ class WriteableTests : OpenSearchTestCase() { assertEquals("Round tripping InputRunResults doesn't work", runResult, newRunResult) } - fun `test monitorrunresult as stream`() { - val runResult = randomMonitorRunResult() + fun `test query-level monitorrunresult as stream`() { + val runResult = randomQueryLevelMonitorRunResult() val out = BytesStreamOutput() runResult.writeTo(out) val sin = StreamInput.wrap(out.bytes().toBytesRef().bytes) - val newRunResult = MonitorRunResult(sin) + val newRunResult = MonitorRunResult(sin) + assertEquals("Round tripping MonitorRunResult doesn't work", runResult, newRunResult) + } + + fun `test bucket-level monitorrunresult as stream`() { + val runResult = randomBucketLevelMonitorRunResult() + val out = BytesStreamOutput() + runResult.writeTo(out) + val sin = StreamInput.wrap(out.bytes().toBytesRef().bytes) + val newRunResult = MonitorRunResult(sin) assertEquals("Round tripping MonitorRunResult doesn't work", runResult, newRunResult) } @@ -194,4 +226,13 @@ class WriteableTests : OpenSearchTestCase() { val newEmailGroup = EmailGroup.readFrom(sin) assertEquals("Round tripping EmailGroup doesn't work", emailGroup, newEmailGroup) } + + fun `test action execution policy as stream`() { + val actionExecutionPolicy = randomActionExecutionPolicy() + val out = BytesStreamOutput() + actionExecutionPolicy.writeTo(out) + val sin = StreamInput.wrap(out.bytes().toBytesRef().bytes) + val newActionExecutionPolicy = ActionExecutionPolicy.readFrom(sin) + assertEquals("Round tripping ActionExecutionPolicy doesn't work", actionExecutionPolicy, newActionExecutionPolicy) + } } diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/model/XContentTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/model/XContentTests.kt index 228b86294..3268309d1 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/model/XContentTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/model/XContentTests.kt @@ -27,27 +27,37 @@ package org.opensearch.alerting.model import org.opensearch.alerting.builder +import org.opensearch.alerting.core.model.SearchInput import org.opensearch.alerting.elasticapi.string import org.opensearch.alerting.model.action.Action +import org.opensearch.alerting.model.action.ActionExecutionPolicy +import org.opensearch.alerting.model.action.PerExecutionActionScope import org.opensearch.alerting.model.action.Throttle import org.opensearch.alerting.model.destination.email.EmailAccount import org.opensearch.alerting.model.destination.email.EmailGroup import org.opensearch.alerting.parser import org.opensearch.alerting.randomAction +import org.opensearch.alerting.randomActionExecutionPolicy import org.opensearch.alerting.randomActionExecutionResult +import org.opensearch.alerting.randomActionWithPolicy import org.opensearch.alerting.randomAlert +import org.opensearch.alerting.randomBucketLevelMonitor +import org.opensearch.alerting.randomBucketLevelTrigger import org.opensearch.alerting.randomEmailAccount import org.opensearch.alerting.randomEmailGroup -import org.opensearch.alerting.randomMonitor -import org.opensearch.alerting.randomMonitorWithoutUser +import org.opensearch.alerting.randomQueryLevelMonitor +import org.opensearch.alerting.randomQueryLevelMonitorWithoutUser +import org.opensearch.alerting.randomQueryLevelTrigger import org.opensearch.alerting.randomThrottle -import org.opensearch.alerting.randomTrigger import org.opensearch.alerting.randomUser import org.opensearch.alerting.randomUserEmpty import org.opensearch.alerting.toJsonString import org.opensearch.common.xcontent.ToXContent import org.opensearch.commons.authuser.User +import org.opensearch.index.query.QueryBuilders +import org.opensearch.search.builder.SearchSourceBuilder import org.opensearch.test.OpenSearchTestCase +import java.time.temporal.ChronoUnit import kotlin.test.assertFailsWith class XContentTests : OpenSearchTestCase() { @@ -56,21 +66,21 @@ class XContentTests : OpenSearchTestCase() { val action = randomAction() val actionString = action.toXContent(builder(), ToXContent.EMPTY_PARAMS).string() val parsedAction = Action.parse(parser(actionString)) - assertEquals("Round tripping Monitor doesn't work", action, parsedAction) + assertEquals("Round tripping Action doesn't work", action, parsedAction) } fun `test action parsing with null subject template`() { val action = randomAction().copy(subjectTemplate = null) val actionString = action.toXContent(builder(), ToXContent.EMPTY_PARAMS).string() val parsedAction = Action.parse(parser(actionString)) - assertEquals("Round tripping Monitor doesn't work", action, parsedAction) + assertEquals("Round tripping Action doesn't work", action, parsedAction) } fun `test action parsing with null throttle`() { val action = randomAction().copy(throttle = null) val actionString = action.toXContent(builder(), ToXContent.EMPTY_PARAMS).string() val parsedAction = Action.parse(parser(actionString)) - assertEquals("Round tripping Monitor doesn't work", action, parsedAction) + assertEquals("Round tripping Action doesn't work", action, parsedAction) } fun `test action parsing with throttled enabled and null throttle`() { @@ -81,6 +91,18 @@ class XContentTests : OpenSearchTestCase() { } } + fun `test action with per execution scope does not support throttling`() { + try { + val action = randomActionWithPolicy().copy( + throttleEnabled = true, + throttle = Throttle(value = 5, unit = ChronoUnit.MINUTES), + actionExecutionPolicy = ActionExecutionPolicy(PerExecutionActionScope()) + ) + fail("Creating an action with per execution scope and throttle enabled did not fail.") + } catch (ignored: IllegalArgumentException) { + } + } + fun `test throttle parsing`() { val throttle = randomThrottle() val throttleString = throttle.toXContent(builder(), ToXContent.EMPTY_PARAMS).string() @@ -103,21 +125,30 @@ class XContentTests : OpenSearchTestCase() { assertFailsWith("Can only set positive throttle period") { Throttle.parse(parser(throttleString)) } } - fun `test monitor parsing`() { - val monitor = randomMonitor() + fun `test query-level monitor parsing`() { + val monitor = randomQueryLevelMonitor() val monitorString = monitor.toJsonString() val parsedMonitor = Monitor.parse(parser(monitorString)) - assertEquals("Round tripping Monitor doesn't work", monitor, parsedMonitor) + assertEquals("Round tripping QueryLevelMonitor doesn't work", monitor, parsedMonitor) + } + + fun `test query-level trigger parsing`() { + val trigger = randomQueryLevelTrigger() + + val triggerString = trigger.toXContent(builder(), ToXContent.EMPTY_PARAMS).string() + val parsedTrigger = Trigger.parse(parser(triggerString)) + + assertEquals("Round tripping QueryLevelTrigger doesn't work", trigger, parsedTrigger) } - fun `test trigger parsing`() { - val trigger = randomTrigger() + fun `test bucket-level trigger parsing`() { + val trigger = randomBucketLevelTrigger() val triggerString = trigger.toXContent(builder(), ToXContent.EMPTY_PARAMS).string() val parsedTrigger = Trigger.parse(parser(triggerString)) - assertEquals("Round tripping Trigger doesn't work", trigger, parsedTrigger) + assertEquals("Round tripping BucketLevelTrigger doesn't work", trigger, parsedTrigger) } fun `test alert parsing`() { @@ -162,8 +193,8 @@ class XContentTests : OpenSearchTestCase() { fun `test creating a monitor with duplicate trigger ids fails`() { try { - val repeatedTrigger = randomTrigger() - randomMonitor().copy(triggers = listOf(repeatedTrigger, repeatedTrigger)) + val repeatedTrigger = randomQueryLevelTrigger() + randomQueryLevelMonitor().copy(triggers = listOf(repeatedTrigger, repeatedTrigger)) fail("Creating a monitor with duplicate triggers did not fail.") } catch (ignored: IllegalArgumentException) { } @@ -186,12 +217,12 @@ class XContentTests : OpenSearchTestCase() { assertEquals(0, parsedUser.roles.size) } - fun `test monitor parsing without user`() { - val monitor = randomMonitorWithoutUser() + fun `test query-level monitor parsing without user`() { + val monitor = randomQueryLevelMonitorWithoutUser() val monitorString = monitor.toJsonString() val parsedMonitor = Monitor.parse(parser(monitorString)) - assertEquals("Round tripping Monitor doesn't work", monitor, parsedMonitor) + assertEquals("Round tripping QueryLevelMonitor doesn't work", monitor, parsedMonitor) assertNull(parsedMonitor.user) } @@ -210,4 +241,120 @@ class XContentTests : OpenSearchTestCase() { val parsedEmailGroup = EmailGroup.parse(parser(emailGroupString)) assertEquals("Round tripping EmailGroup doesn't work", emailGroup, parsedEmailGroup) } + + fun `test old monitor format parsing`() { + val monitorString = """ + { + "type": "monitor", + "schema_version": 3, + "name": "asdf", + "user": { + "name": "admin123", + "backend_roles": [], + "roles": [ + "all_access", + "security_manager" + ], + "custom_attribute_names": [], + "user_requested_tenant": null + }, + "enabled": true, + "enabled_time": 1613530078244, + "schedule": { + "period": { + "interval": 1, + "unit": "MINUTES" + } + }, + "inputs": [ + { + "search": { + "indices": [ + "test_index" + ], + "query": { + "size": 0, + "query": { + "bool": { + "filter": [ + { + "range": { + "order_date": { + "from": "{{period_end}}||-1h", + "to": "{{period_end}}", + "include_lower": true, + "include_upper": true, + "format": "epoch_millis", + "boost": 1.0 + } + } + } + ], + "adjust_pure_negative": true, + "boost": 1.0 + } + }, + "aggregations": {} + } + } + } + ], + "triggers": [ + { + "id": "e_sc0XcB98Q42rHjTh4K", + "name": "abc", + "severity": "1", + "condition": { + "script": { + "source": "ctx.results[0].hits.total.value > 100000", + "lang": "painless" + } + }, + "actions": [] + } + ], + "last_update_time": 1614121489719 + } + """.trimIndent() + val parsedMonitor = Monitor.parse(parser(monitorString)) + assertEquals("Incorrect monitor type", Monitor.MonitorType.QUERY_LEVEL_MONITOR, parsedMonitor.monitorType) + assertEquals("Incorrect trigger count", 1, parsedMonitor.triggers.size) + val trigger = parsedMonitor.triggers.first() + assertTrue("Incorrect trigger type", trigger is QueryLevelTrigger) + assertEquals("Incorrect name for parsed trigger", "abc", trigger.name) + } + + fun `test creating an query-level monitor with invalid trigger type fails`() { + try { + val bucketLevelTrigger = randomBucketLevelTrigger() + randomQueryLevelMonitor().copy(triggers = listOf(bucketLevelTrigger)) + fail("Creating a query-level monitor with bucket-level triggers did not fail.") + } catch (ignored: IllegalArgumentException) { + } + } + + fun `test creating an bucket-level monitor with invalid trigger type fails`() { + try { + val queryLevelTrigger = randomQueryLevelTrigger() + randomBucketLevelMonitor().copy(triggers = listOf(queryLevelTrigger)) + fail("Creating a bucket-level monitor with query-level triggers did not fail.") + } catch (ignored: IllegalArgumentException) { + } + } + + fun `test creating an bucket-level monitor with invalid input fails`() { + try { + val invalidInput = SearchInput(emptyList(), SearchSourceBuilder().query(QueryBuilders.matchAllQuery())) + randomBucketLevelMonitor().copy(inputs = listOf(invalidInput)) + fail("Creating an bucket-level monitor with an invalid input did not fail.") + } catch (ignored: IllegalArgumentException) { + } + } + + fun `test action execution policy`() { + val actionExecutionPolicy = randomActionExecutionPolicy() + val actionExecutionPolicyString = actionExecutionPolicy.toXContent(builder(), ToXContent.EMPTY_PARAMS).string() + val parsedActionExecutionPolicy = ActionExecutionPolicy.parse(parser(actionExecutionPolicyString)) + assertEquals("Round tripping ActionExecutionPolicy doesn't work", actionExecutionPolicy, parsedActionExecutionPolicy) + } } diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt b/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt index d17518e2e..b1a1cd916 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt @@ -32,6 +32,7 @@ import org.apache.http.nio.entity.NStringEntity import org.opensearch.alerting.ALERTING_BASE_URI import org.opensearch.alerting.ANOMALY_DETECTOR_INDEX import org.opensearch.alerting.AlertingRestTestCase +import org.opensearch.alerting.DESTINATION_BASE_URI import org.opensearch.alerting.LEGACY_OPENDISTRO_ALERTING_BASE_URI import org.opensearch.alerting.alerts.AlertIndices import org.opensearch.alerting.anomalyDetectorIndexMapping @@ -42,16 +43,20 @@ import org.opensearch.alerting.core.settings.ScheduledJobSettings import org.opensearch.alerting.makeRequest import org.opensearch.alerting.model.Alert import org.opensearch.alerting.model.Monitor -import org.opensearch.alerting.model.Trigger +import org.opensearch.alerting.model.QueryLevelTrigger +import org.opensearch.alerting.model.destination.Chime +import org.opensearch.alerting.model.destination.Destination import org.opensearch.alerting.randomADMonitor import org.opensearch.alerting.randomAction import org.opensearch.alerting.randomAlert import org.opensearch.alerting.randomAnomalyDetector import org.opensearch.alerting.randomAnomalyDetectorWithUser -import org.opensearch.alerting.randomMonitor +import org.opensearch.alerting.randomQueryLevelMonitor +import org.opensearch.alerting.randomQueryLevelTrigger import org.opensearch.alerting.randomThrottle -import org.opensearch.alerting.randomTrigger +import org.opensearch.alerting.randomUser import org.opensearch.alerting.settings.AlertingSettings +import org.opensearch.alerting.util.DestinationType import org.opensearch.client.ResponseException import org.opensearch.client.WarningFailureException import org.opensearch.common.bytes.BytesReference @@ -66,6 +71,7 @@ import org.opensearch.search.builder.SearchSourceBuilder import org.opensearch.test.OpenSearchTestCase import org.opensearch.test.junit.annotations.TestLogging import org.opensearch.test.rest.OpenSearchRestTestCase +import java.time.Instant import java.time.ZoneId import java.time.temporal.ChronoUnit @@ -102,7 +108,7 @@ class MonitorRestApiIT : AlertingRestTestCase() { @Throws(Exception::class) fun `test creating a monitor`() { - val monitor = randomMonitor() + val monitor = randomQueryLevelMonitor() val createResponse = client().makeRequest("POST", ALERTING_BASE_URI, emptyMap(), monitor.toHttpEntity()) @@ -116,7 +122,7 @@ class MonitorRestApiIT : AlertingRestTestCase() { } fun `test creating a monitor with legacy ODFE`() { - val monitor = randomMonitor() + val monitor = randomQueryLevelMonitor() val createResponse = client().makeRequest("POST", LEGACY_OPENDISTRO_ALERTING_BASE_URI, emptyMap(), monitor.toHttpEntity()) assertEquals("Create monitor failed", RestStatus.CREATED, createResponse.restStatus()) val responseBody = createResponse.asMap() @@ -161,7 +167,7 @@ class MonitorRestApiIT : AlertingRestTestCase() { fun `test creating a monitor with PUT fails`() { try { - val monitor = randomMonitor() + val monitor = randomQueryLevelMonitor() client().makeRequest("PUT", ALERTING_BASE_URI, emptyMap(), monitor.toHttpEntity()) fail("Expected 405 Method Not Allowed response") } catch (e: ResponseException) { @@ -172,7 +178,7 @@ class MonitorRestApiIT : AlertingRestTestCase() { fun `test creating a monitor with illegal index name`() { try { val si = SearchInput(listOf("_#*IllegalIndexCharacters"), SearchSourceBuilder().query(QueryBuilders.matchAllQuery())) - val monitor = randomMonitor() + val monitor = randomQueryLevelMonitor() client().makeRequest("POST", ALERTING_BASE_URI, emptyMap(), monitor.copy(inputs = listOf(si)).toHttpEntity()) } catch (e: ResponseException) { // When an index with invalid name is mentioned, instead of returning invalid_index_name_exception security plugin throws security_exception. @@ -319,7 +325,14 @@ class MonitorRestApiIT : AlertingRestTestCase() { fun `test updating conditions for a monitor`() { val monitor = createRandomMonitor() - val updatedTriggers = listOf(Trigger("foo", "1", Script("return true"), emptyList())) + val updatedTriggers = listOf( + QueryLevelTrigger( + name = "foo", + severity = "1", + condition = Script("return true"), + actions = emptyList() + ) + ) val updateResponse = client().makeRequest( "PUT", monitor.relativeUrl(), emptyMap(), monitor.copy(triggers = updatedTriggers).toHttpEntity() @@ -715,8 +728,8 @@ class MonitorRestApiIT : AlertingRestTestCase() { fun `test delete trigger moves alerts`() { client().updateSettings(ScheduledJobSettings.SWEEPER_ENABLED.key, true) putAlertMappings() - val trigger = randomTrigger() - val monitor = createMonitor(randomMonitor(triggers = listOf(trigger))) + val trigger = randomQueryLevelTrigger() + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(trigger))) val alert = createAlert(randomAlert(monitor).copy(triggerId = trigger.id, state = Alert.State.ACTIVE)) refreshIndex("*") val updatedMonitor = monitor.copy(triggers = emptyList()) @@ -740,9 +753,9 @@ class MonitorRestApiIT : AlertingRestTestCase() { fun `test delete trigger moves alerts only for deleted trigger`() { client().updateSettings(ScheduledJobSettings.SWEEPER_ENABLED.key, true) putAlertMappings() - val triggerToDelete = randomTrigger() - val triggerToKeep = randomTrigger() - val monitor = createMonitor(randomMonitor(triggers = listOf(triggerToDelete, triggerToKeep))) + val triggerToDelete = randomQueryLevelTrigger() + val triggerToKeep = randomQueryLevelTrigger() + val monitor = createMonitor(randomQueryLevelMonitor(triggers = listOf(triggerToDelete, triggerToKeep))) val alertKeep = createAlert(randomAlert(monitor).copy(triggerId = triggerToKeep.id, state = Alert.State.ACTIVE)) val alertDelete = createAlert(randomAlert(monitor).copy(triggerId = triggerToDelete.id, state = Alert.State.ACTIVE)) refreshIndex("*") @@ -878,7 +891,55 @@ class MonitorRestApiIT : AlertingRestTestCase() { private fun randomMonitorWithThrottle(value: Int, unit: ChronoUnit = ChronoUnit.MINUTES): Monitor { val throttle = randomThrottle(value, unit) val action = randomAction().copy(throttle = throttle) - val trigger = randomTrigger(actions = listOf(action)) - return randomMonitor(triggers = listOf(trigger)) + val trigger = randomQueryLevelTrigger(actions = listOf(action)) + return randomQueryLevelMonitor(triggers = listOf(trigger)) + } + + @Throws(Exception::class) + fun `test search monitors only`() { + + // 1. create monitor + val monitor = randomQueryLevelMonitor() + val createResponse = client().makeRequest("POST", ALERTING_BASE_URI, emptyMap(), monitor.toHttpEntity()) + assertEquals("Create monitor failed", RestStatus.CREATED, createResponse.restStatus()) + + // 2. create destination + val chime = Chime("http://abc.com") + val destination = Destination( + type = DestinationType.CHIME, + name = "test", + user = randomUser(), + lastUpdateTime = Instant.now(), + chime = chime, + slack = null, + customWebhook = null, + email = null + ) + val response = client().makeRequest( + "POST", + DESTINATION_BASE_URI, + emptyMap(), + destination.toHttpEntity() + ) + assertEquals("Unable to create a new destination", RestStatus.CREATED, response.restStatus()) + + // 3. search - must return only monitors. + val search = SearchSourceBuilder().query(QueryBuilders.matchAllQuery()).toString() + val searchResponse = client().makeRequest( + "GET", + "$ALERTING_BASE_URI/_search", + emptyMap(), + NStringEntity(search, ContentType.APPLICATION_JSON) + ) + assertEquals("Search monitor failed", RestStatus.OK, searchResponse.restStatus()) + val xcp = createParser(XContentType.JSON.xContent(), searchResponse.entity.content) + val hits = xcp.map()["hits"]!! as Map> + val numberDocsFound = hits["total"]?.get("value") + assertEquals("Destination objects are also returned by /_search.", 1, numberDocsFound) + + val searchHits = hits["hits"] as List + val hit = searchHits[0] as Map + val monitorHit = hit["_source"] as Map + assertEquals("Type is not monitor", monitorHit[Monitor.TYPE_FIELD], "monitor") } } diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/SecureMonitorRestApiIT.kt b/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/SecureMonitorRestApiIT.kt index 2a4b8244e..254659b65 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/SecureMonitorRestApiIT.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/SecureMonitorRestApiIT.kt @@ -24,9 +24,9 @@ import org.opensearch.alerting.makeRequest import org.opensearch.alerting.model.Alert import org.opensearch.alerting.randomAction import org.opensearch.alerting.randomAlert -import org.opensearch.alerting.randomMonitor +import org.opensearch.alerting.randomQueryLevelMonitor +import org.opensearch.alerting.randomQueryLevelTrigger import org.opensearch.alerting.randomTemplateScript -import org.opensearch.alerting.randomTrigger import org.opensearch.client.Response import org.opensearch.client.ResponseException import org.opensearch.client.RestClient @@ -72,7 +72,7 @@ class SecureMonitorRestApiIT : AlertingRestTestCase() { createUserRolesMapping("alerting_full_access", arrayOf(user)) try { // randomMonitor has a dummy user, api ignores the User passed as part of monitor, it picks user info from the logged-in user. - val monitor = randomMonitor().copy( + val monitor = randomQueryLevelMonitor().copy( inputs = listOf( SearchInput( indices = listOf("hr_data"), query = SearchSourceBuilder().query(QueryBuilders.matchAllQuery()) @@ -106,7 +106,7 @@ class SecureMonitorRestApiIT : AlertingRestTestCase() { createUserWithTestData(user, "hr_data", "hr_role", "HR") try { - val monitor = randomMonitor().copy( + val monitor = randomQueryLevelMonitor().copy( inputs = listOf( SearchInput( indices = listOf("hr_data"), query = SearchSourceBuilder().query(QueryBuilders.matchAllQuery()) @@ -129,7 +129,7 @@ class SecureMonitorRestApiIT : AlertingRestTestCase() { createUserWithTestData(user, "hr_data", "hr_role", "HR") createUserRolesMapping("alerting_full_access", arrayOf(user)) try { - val monitor = randomMonitor().copy( + val monitor = randomQueryLevelMonitor().copy( inputs = listOf( SearchInput( indices = listOf("not_hr_data"), query = SearchSourceBuilder().query(QueryBuilders.matchAllQuery()) @@ -150,14 +150,14 @@ class SecureMonitorRestApiIT : AlertingRestTestCase() { fun `test create monitor with disable filter by`() { disableFilterBy() - val monitor = randomMonitor() + val monitor = randomQueryLevelMonitor() val createResponse = client().makeRequest("POST", ALERTING_BASE_URI, emptyMap(), monitor.toHttpEntity()) assertEquals("Create monitor failed", RestStatus.CREATED, createResponse.restStatus()) } fun `test create monitor with enable filter by`() { enableFilterBy() - val monitor = randomMonitor() + val monitor = randomQueryLevelMonitor() if (securityEnabled()) { // when security is enabled. No errors, must succeed. @@ -185,7 +185,7 @@ class SecureMonitorRestApiIT : AlertingRestTestCase() { // Query Monitors related security tests fun `test update monitor with disable filter by`() { disableFilterBy() - val monitor = randomMonitor(enabled = true) + val monitor = randomQueryLevelMonitor(enabled = true) val createdMonitor = createMonitor(monitor = monitor) @@ -205,7 +205,7 @@ class SecureMonitorRestApiIT : AlertingRestTestCase() { // refer: `test create monitor with enable filter by` return } - val monitor = randomMonitor(enabled = true) + val monitor = randomQueryLevelMonitor(enabled = true) val createdMonitor = createMonitor(monitor = monitor) @@ -220,7 +220,7 @@ class SecureMonitorRestApiIT : AlertingRestTestCase() { fun `test delete monitor with disable filter by`() { disableFilterBy() - val monitor = randomMonitor(enabled = true) + val monitor = randomQueryLevelMonitor(enabled = true) val createdMonitor = createMonitor(monitor = monitor) @@ -254,7 +254,7 @@ class SecureMonitorRestApiIT : AlertingRestTestCase() { // refer: `test create monitor with enable filter by` return } - val monitor = randomMonitor(enabled = true) + val monitor = randomQueryLevelMonitor(enabled = true) val createdMonitor = createMonitor(monitor = monitor) @@ -463,7 +463,10 @@ class SecureMonitorRestApiIT : AlertingRestTestCase() { query = SearchSourceBuilder().query(QueryBuilders.matchAllQuery()) ) ) - val monitor = randomMonitor(triggers = listOf(randomTrigger(condition = ALWAYS_RUN, actions = listOf(action))), inputs = inputs) + val monitor = randomQueryLevelMonitor( + triggers = listOf(randomQueryLevelTrigger(condition = ALWAYS_RUN, actions = listOf(action))), + inputs = inputs + ) // Make sure the elevating the permissions fails execute. val adminUser = User("admin", listOf("admin"), listOf("all_access"), listOf()) diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/util/AggregationQueryRewriterTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/util/AggregationQueryRewriterTests.kt new file mode 100644 index 000000000..d56e6def7 --- /dev/null +++ b/alerting/src/test/kotlin/org/opensearch/alerting/util/AggregationQueryRewriterTests.kt @@ -0,0 +1,335 @@ +/* + * SPDX-License-Identifier: Apache-2.0 + * + * The OpenSearch Contributors require contributions made to + * this file be licensed under the Apache-2.0 license or a + * compatible open source license. + * + * Modifications Copyright OpenSearch Contributors. See + * GitHub history for details. + */ + +package org.opensearch.alerting.util + +import org.junit.Assert +import org.opensearch.action.search.SearchResponse +import org.opensearch.alerting.model.InputRunResults +import org.opensearch.alerting.model.Trigger +import org.opensearch.alerting.model.TriggerAfterKey +import org.opensearch.alerting.randomBucketLevelTrigger +import org.opensearch.alerting.randomBucketSelectorExtAggregationBuilder +import org.opensearch.alerting.randomQueryLevelTrigger +import org.opensearch.cluster.ClusterModule +import org.opensearch.common.CheckedFunction +import org.opensearch.common.ParseField +import org.opensearch.common.xcontent.NamedXContentRegistry +import org.opensearch.common.xcontent.XContentParser +import org.opensearch.common.xcontent.json.JsonXContent +import org.opensearch.search.aggregations.Aggregation +import org.opensearch.search.aggregations.AggregationBuilder +import org.opensearch.search.aggregations.bucket.composite.CompositeAggregationBuilder +import org.opensearch.search.aggregations.bucket.composite.ParsedComposite +import org.opensearch.search.aggregations.bucket.composite.TermsValuesSourceBuilder +import org.opensearch.search.aggregations.bucket.terms.TermsAggregationBuilder +import org.opensearch.search.builder.SearchSourceBuilder +import org.opensearch.test.OpenSearchTestCase +import java.io.IOException + +class AggregationQueryRewriterTests : OpenSearchTestCase() { + + fun `test RewriteQuery empty previous result`() { + val triggers: MutableList = mutableListOf() + for (i in 0 until 10) { + triggers.add(randomBucketLevelTrigger()) + } + val queryBuilder = SearchSourceBuilder() + val termAgg: AggregationBuilder = TermsAggregationBuilder("testPath").field("sports") + queryBuilder.aggregation(termAgg) + val prevResult = null + AggregationQueryRewriter.rewriteQuery(queryBuilder, prevResult, triggers) + Assert.assertEquals(queryBuilder.aggregations().pipelineAggregatorFactories.size, 10) + } + + fun `skip test RewriteQuery with non-empty previous result`() { + val triggers: MutableList = mutableListOf() + for (i in 0 until 10) { + triggers.add(randomBucketLevelTrigger()) + } + val queryBuilder = SearchSourceBuilder() + val termAgg: AggregationBuilder = CompositeAggregationBuilder( + "testPath", + listOf(TermsValuesSourceBuilder("k1"), TermsValuesSourceBuilder("k2")) + ) + queryBuilder.aggregation(termAgg) + val aggTriggersAfterKey = mutableMapOf() + for (trigger in triggers) { + aggTriggersAfterKey[trigger.id] = TriggerAfterKey(hashMapOf(Pair("k1", "v1"), Pair("k2", "v2")), false) + } + val prevResult = InputRunResults(emptyList(), null, aggTriggersAfterKey) + AggregationQueryRewriter.rewriteQuery(queryBuilder, prevResult, triggers) + Assert.assertEquals(queryBuilder.aggregations().pipelineAggregatorFactories.size, 10) + queryBuilder.aggregations().aggregatorFactories.forEach { + if (it.name.equals("testPath")) { +// val compAgg = it as CompositeAggregationBuilder + // TODO: This is calling forbidden API and causing build failures, need to find an alternative + // instead of trying to access private member variables +// val afterField = CompositeAggregationBuilder::class.java.getDeclaredField("after") +// afterField.isAccessible = true +// Assert.assertEquals(afterField.get(compAgg), hashMapOf(Pair("k1", "v1"), Pair("k2", "v2"))) + } + } + } + + fun `test RewriteQuery with non aggregation trigger`() { + val triggers: MutableList = mutableListOf() + for (i in 0 until 10) { + triggers.add(randomQueryLevelTrigger()) + } + val queryBuilder = SearchSourceBuilder() + val termAgg: AggregationBuilder = TermsAggregationBuilder("testPath").field("sports") + queryBuilder.aggregation(termAgg) + val prevResult = null + AggregationQueryRewriter.rewriteQuery(queryBuilder, prevResult, triggers) + Assert.assertEquals(queryBuilder.aggregations().pipelineAggregatorFactories.size, 0) + } + + fun `test after keys from search response`() { + val responseContent = """ + { + "took" : 97, + "timed_out" : false, + "_shards" : { + "total" : 3, + "successful" : 3, + "skipped" : 0, + "failed" : 0 + }, + "hits" : { + "total" : { + "value" : 20, + "relation" : "eq" + }, + "max_score" : null, + "hits" : [ ] + }, + "aggregations" : { + "composite#testPath" : { + "after_key" : { + "sport" : "Basketball" + }, + "buckets" : [ + { + "key" : { + "sport" : "Basketball" + }, + "doc_count" : 5 + } + ] + } + } + } + """.trimIndent() + + val aggTriggers: MutableList = mutableListOf(randomBucketLevelTrigger()) + val tradTriggers: MutableList = mutableListOf(randomQueryLevelTrigger()) + + val searchResponse = SearchResponse.fromXContent(createParser(JsonXContent.jsonXContent, responseContent)) + val afterKeys = AggregationQueryRewriter.getAfterKeysFromSearchResponse(searchResponse, aggTriggers, null) + Assert.assertEquals(afterKeys[aggTriggers[0].id]?.afterKey, hashMapOf(Pair("sport", "Basketball"))) + + val afterKeys2 = AggregationQueryRewriter.getAfterKeysFromSearchResponse(searchResponse, tradTriggers, null) + Assert.assertEquals(afterKeys2.size, 0) + } + + fun `test after keys from search responses for multiple bucket paths and different page counts`() { + val firstResponseContent = """ + { + "took" : 0, + "timed_out" : false, + "_shards" : { + "total" : 1, + "successful" : 1, + "skipped" : 0, + "failed" : 0 + }, + "hits" : { + "total" : { + "value" : 4675, + "relation" : "eq" + }, + "max_score" : null, + "hits" : [ ] + }, + "aggregations" : { + "composite2#smallerResults" : { + "after_key" : { + "category" : "Women's Shoes" + }, + "buckets" : [ + { + "key" : { + "category" : "Women's Shoes" + }, + "doc_count" : 1136 + } + ] + }, + "composite3#largerResults" : { + "after_key" : { + "user" : "abigail" + }, + "buckets" : [ + { + "key" : { + "user" : "abd" + }, + "doc_count" : 188 + }, + { + "key" : { + "user" : "abigail" + }, + "doc_count" : 128 + } + ] + } + } + } + """.trimIndent() + + val secondResponseContent = """ + { + "took" : 0, + "timed_out" : false, + "_shards" : { + "total" : 1, + "successful" : 1, + "skipped" : 0, + "failed" : 0 + }, + "hits" : { + "total" : { + "value" : 4675, + "relation" : "eq" + }, + "max_score" : null, + "hits" : [ ] + }, + "aggregations" : { + "composite2#smallerResults" : { + "buckets" : [ ] + }, + "composite3#largerResults" : { + "after_key" : { + "user" : "boris" + }, + "buckets" : [ + { + "key" : { + "user" : "betty" + }, + "doc_count" : 148 + }, + { + "key" : { + "user" : "boris" + }, + "doc_count" : 74 + } + ] + } + } + } + """.trimIndent() + + val thirdResponseContent = """ + { + "took" : 0, + "timed_out" : false, + "_shards" : { + "total" : 1, + "successful" : 1, + "skipped" : 0, + "failed" : 0 + }, + "hits" : { + "total" : { + "value" : 4675, + "relation" : "eq" + }, + "max_score" : null, + "hits" : [ ] + }, + "aggregations" : { + "composite2#smallerResults" : { + "buckets" : [ ] + }, + "composite3#largerResults" : { + "buckets" : [ ] + } + } + } + """.trimIndent() + + val bucketLevelTriggers: MutableList = mutableListOf( + randomBucketLevelTrigger(bucketSelector = randomBucketSelectorExtAggregationBuilder(parentBucketPath = "smallerResults")), + randomBucketLevelTrigger(bucketSelector = randomBucketSelectorExtAggregationBuilder(parentBucketPath = "largerResults")) + ) + + var searchResponse = SearchResponse.fromXContent(createParser(JsonXContent.jsonXContent, firstResponseContent)) + val afterKeys = AggregationQueryRewriter.getAfterKeysFromSearchResponse(searchResponse, bucketLevelTriggers, null) + assertEquals(hashMapOf(Pair("category", "Women's Shoes")), afterKeys[bucketLevelTriggers[0].id]?.afterKey) + assertEquals(false, afterKeys[bucketLevelTriggers[0].id]?.lastPage) + assertEquals(hashMapOf(Pair("user", "abigail")), afterKeys[bucketLevelTriggers[1].id]?.afterKey) + assertEquals(false, afterKeys[bucketLevelTriggers[1].id]?.lastPage) + + searchResponse = SearchResponse.fromXContent(createParser(JsonXContent.jsonXContent, secondResponseContent)) + val afterKeys2 = AggregationQueryRewriter.getAfterKeysFromSearchResponse(searchResponse, bucketLevelTriggers, afterKeys) + assertEquals(hashMapOf(Pair("category", "Women's Shoes")), afterKeys2[bucketLevelTriggers[0].id]?.afterKey) + assertEquals(true, afterKeys2[bucketLevelTriggers[0].id]?.lastPage) + assertEquals(hashMapOf(Pair("user", "boris")), afterKeys2[bucketLevelTriggers[1].id]?.afterKey) + assertEquals(false, afterKeys2[bucketLevelTriggers[1].id]?.lastPage) + + searchResponse = SearchResponse.fromXContent(createParser(JsonXContent.jsonXContent, thirdResponseContent)) + val afterKeys3 = AggregationQueryRewriter.getAfterKeysFromSearchResponse(searchResponse, bucketLevelTriggers, afterKeys2) + assertEquals(hashMapOf(Pair("category", "Women's Shoes")), afterKeys3[bucketLevelTriggers[0].id]?.afterKey) + assertEquals(true, afterKeys3[bucketLevelTriggers[0].id]?.lastPage) + assertEquals(hashMapOf(Pair("user", "boris")), afterKeys3[bucketLevelTriggers[1].id]?.afterKey) + assertEquals(true, afterKeys3[bucketLevelTriggers[1].id]?.lastPage) + } + + override fun xContentRegistry(): NamedXContentRegistry { + val entries = ClusterModule.getNamedXWriteables() + entries.add( + NamedXContentRegistry.Entry( + Aggregation::class.java, ParseField(CompositeAggregationBuilder.NAME), + CheckedFunction { parser: XContentParser? -> + ParsedComposite.fromXContent( + parser, "testPath" + ) + } + ) + ) + entries.add( + NamedXContentRegistry.Entry( + Aggregation::class.java, ParseField(CompositeAggregationBuilder.NAME + "2"), + CheckedFunction { parser: XContentParser? -> + ParsedComposite.fromXContent( + parser, "smallerResults" + ) + } + ) + ) + entries.add( + NamedXContentRegistry.Entry( + Aggregation::class.java, ParseField(CompositeAggregationBuilder.NAME + "3"), + CheckedFunction { parser: XContentParser? -> + ParsedComposite.fromXContent( + parser, "largerResults" + ) + } + ) + ) + return NamedXContentRegistry(entries) + } +} diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/util/AnomalyDetectionUtilsTests.kt b/alerting/src/test/kotlin/org/opensearch/alerting/util/AnomalyDetectionUtilsTests.kt index c51ad0332..15b41229d 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/util/AnomalyDetectionUtilsTests.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/util/AnomalyDetectionUtilsTests.kt @@ -29,7 +29,7 @@ package org.opensearch.alerting.util import org.opensearch.alerting.ANOMALY_RESULT_INDEX import org.opensearch.alerting.core.model.Input import org.opensearch.alerting.core.model.SearchInput -import org.opensearch.alerting.randomMonitor +import org.opensearch.alerting.randomQueryLevelMonitor import org.opensearch.common.io.stream.StreamOutput import org.opensearch.common.xcontent.ToXContent import org.opensearch.common.xcontent.XContentBuilder @@ -41,7 +41,7 @@ import org.opensearch.test.OpenSearchTestCase class AnomalyDetectionUtilsTests : OpenSearchTestCase() { fun `test is ad monitor`() { - val monitor = randomMonitor( + val monitor = randomQueryLevelMonitor( inputs = listOf( SearchInput( listOf(ANOMALY_RESULT_INDEX), @@ -54,14 +54,14 @@ class AnomalyDetectionUtilsTests : OpenSearchTestCase() { fun `test not ad monitor if monitor have no inputs`() { - val monitor = randomMonitor( + val monitor = randomQueryLevelMonitor( inputs = listOf() ) assertFalse(isADMonitor(monitor)) } fun `test not ad monitor if monitor input is not search input`() { - val monitor = randomMonitor( + val monitor = randomQueryLevelMonitor( inputs = listOf(object : Input { override fun name(): String { TODO("Not yet implemented") @@ -80,7 +80,7 @@ class AnomalyDetectionUtilsTests : OpenSearchTestCase() { } fun `test not ad monitor if monitor input has more than 1 indices`() { - val monitor = randomMonitor( + val monitor = randomQueryLevelMonitor( inputs = listOf( SearchInput( listOf(randomAlphaOfLength(5), randomAlphaOfLength(5)), @@ -92,7 +92,7 @@ class AnomalyDetectionUtilsTests : OpenSearchTestCase() { } fun `test not ad monitor if monitor input's index name is not AD result index`() { - val monitor = randomMonitor( + val monitor = randomQueryLevelMonitor( inputs = listOf(SearchInput(listOf(randomAlphaOfLength(5)), SearchSourceBuilder().query(QueryBuilders.matchAllQuery()))) ) assertFalse(isADMonitor(monitor)) diff --git a/build.gradle b/build.gradle index 2464ece65..c92b6430d 100644 --- a/build.gradle +++ b/build.gradle @@ -28,8 +28,10 @@ buildscript { apply from: 'build-tools/repositories.gradle' ext { - opensearch_version = System.getProperty("opensearch.version", "1.0.0") - common_utils_version = '1.0.0.0' + opensearch_version = System.getProperty("opensearch.version", "1.1.0-SNAPSHOT") + // 1.0.0 -> 1.0.0.0, and 1.0.0-SNAPSHOT -> 1.0.0.0-SNAPSHOT + opensearch_build = opensearch_version.replaceAll(/(\.\d)([^\d]*)$/, '$1.0$2') + common_utils_version = System.getProperty("common_utils.version", opensearch_build) kotlin_version = '1.3.72' } @@ -45,7 +47,6 @@ buildscript { } } - plugins { id 'nebula.ospackage' version "8.3.0" apply false id "com.dorongold.task-tree" version "1.5" @@ -55,11 +56,6 @@ apply plugin: 'base' apply plugin: 'jacoco' apply from: 'build-tools/merged-coverage.gradle' -ext { - opendistroVersion = "${version}" - isSnapshot = "true" == System.getProperty("build.snapshot", "true") -} - configurations { ktlint } @@ -88,12 +84,16 @@ task ktlintFormat(type: JavaExec, group: "formatting") { check.dependsOn ktlint +ext { + isSnapshot = "true" == System.getProperty("build.snapshot", "true") +} allprojects { group = "org.opensearch" - // Increment the final digit when there's a new plugin versions for the same opendistro version - // Reset the final digit to 0 when upgrading to a new opendistro version - version = "${opendistroVersion}.0" + version = "${opensearch_version}" - "-SNAPSHOT" + ".0" + if (isSnapshot) { + version += "-SNAPSHOT" + } apply from: "$rootDir/build-tools/repositories.gradle" diff --git a/core/src/main/resources/mappings/scheduled-jobs.json b/core/src/main/resources/mappings/scheduled-jobs.json index af013e01c..af3d10086 100644 --- a/core/src/main/resources/mappings/scheduled-jobs.json +++ b/core/src/main/resources/mappings/scheduled-jobs.json @@ -1,6 +1,6 @@ { "_meta" : { - "schema_version": 3 + "schema_version": 4 }, "properties": { "monitor": { @@ -18,6 +18,9 @@ } } }, + "monitor_type": { + "type": "keyword" + }, "user": { "properties": { "name": { @@ -115,6 +118,15 @@ } } }, + "group_by_fields": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + }, "triggers": { "type": "nested", "properties": { @@ -171,6 +183,64 @@ } } } + }, + "query_level_trigger": { + "properties": { + "name": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + }, + "min_time_between_executions": { + "type": "integer" + }, + "condition": { + "type": "object", + "enabled": false + }, + "actions": { + "type": "nested", + "properties": { + "name": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + }, + "destination_id": { + "type": "keyword" + }, + "subject_template": { + "type": "object", + "enabled": false + }, + "message_template": { + "type": "object", + "enabled": false + }, + "throttle_enabled": { + "type": "boolean" + }, + "throttle": { + "properties": { + "value": { + "type": "integer" + }, + "unit": { + "type": "keyword" + } + } + } + } + } + } } } }, diff --git a/docs/document-level-alerting-rfc.md b/docs/bucket-level-alerting-rfc.md similarity index 83% rename from docs/document-level-alerting-rfc.md rename to docs/bucket-level-alerting-rfc.md index e8e0d2246..2f845ac08 100644 --- a/docs/document-level-alerting-rfc.md +++ b/docs/bucket-level-alerting-rfc.md @@ -1,6 +1,6 @@ -# Document-Level Alerting RFC +# Bucket-Level Alerting RFC -The purpose of this request for comments (RFC) is to introduce our plans to enhance the Alerting plugin with document-level alerting functionality and collect feedback and discuss our plans with the community. This RFC is meant to cover the high-level functionality and does not go into implementation details and architecture. +The purpose of this request for comments (RFC) is to introduce our plans to enhance the Alerting plugin with bucket-level alerting functionality and collect feedback and discuss our plans with the community. This RFC is meant to cover the high-level functionality and does not go into implementation details and architecture. ## Problem Statement @@ -28,4 +28,4 @@ Similar to controlling the possible flood of alerts, the resulting actions/notif ## Providing Feedback -If you have comments or feedback on our plans for Document-Level Alerting, please comment on the [GitHub issue](https://github.com/opendistro-for-elasticsearch/alerting/issues/326) in this project to discuss. +If you have comments or feedback on our plans for Bucket-Level Alerting, please comment on the [GitHub issue](https://github.com/opensearch-project/alerting/issues/86) in this project to discuss. diff --git a/integtest.sh b/integtest.sh new file mode 100755 index 000000000..6d5aeb392 --- /dev/null +++ b/integtest.sh @@ -0,0 +1,77 @@ +#!/bin/bash + +set -e + +function usage() { + echo "" + echo "This script is used to run integration tests for plugin installed on a remote OpenSearch/Dashboards cluster." + echo "--------------------------------------------------------------------------" + echo "Usage: $0 [args]" + echo "" + echo "Required arguments:" + echo "None" + echo "" + echo "Optional arguments:" + echo -e "-b BIND_ADDRESS\t, defaults to localhost | 127.0.0.1, can be changed to any IP or domain name for the cluster location." + echo -e "-p BIND_PORT\t, defaults to 9200 or 5601 depends on OpenSearch or Dashboards, can be changed to any port for the cluster location." + echo -e "-s SECURITY_ENABLED\t(true | false), defaults to true. Specify the OpenSearch/Dashboards have security enabled or not." + echo -e "-c CREDENTIAL\t(usename:password), no defaults, effective when SECURITY_ENABLED=true." + echo -e "-h\tPrint this message." + echo "--------------------------------------------------------------------------" +} + +while getopts ":hb:p:s:c:" arg; do + case $arg in + h) + usage + exit 1 + ;; + b) + BIND_ADDRESS=$OPTARG + ;; + p) + BIND_PORT=$OPTARG + ;; + s) + SECURITY_ENABLED=$OPTARG + ;; + c) + CREDENTIAL=$OPTARG + ;; + :) + echo "-${OPTARG} requires an argument" + usage + exit 1 + ;; + ?) + echo "Invalid option: -${OPTARG}" + exit 1 + ;; + esac +done + + +if [ -z "$BIND_ADDRESS" ] +then + BIND_ADDRESS="localhost" +fi + +if [ -z "$BIND_PORT" ] +then + BIND_PORT="9200" +fi + +if [ -z "$SECURITY_ENABLED" ] +then + SECURITY_ENABLED="true" +fi + +if [ -z "$CREDENTIAL" ] +then + CREDENTIAL="admin:admin" + USERNAME=`echo $CREDENTIAL | awk -F ':' '{print $1}'` + PASSWORD=`echo $CREDENTIAL | awk -F ':' '{print $2}'` +fi + +./gradlew integTest -Dtests.rest.cluster="$BIND_ADDRESS:$BIND_PORT" -Dtests.cluster="$BIND_ADDRESS:$BIND_PORT" -Dtests.clustername="opensearch-integrationtest" -Dhttps=$SECURITY_ENABLED -Duser=$USERNAME -Dpassword=$PASSWORD --console=plain -Dsecurity=$SECURITY_ENABLED + diff --git a/worksheets/bucket_level_monitor_test.http b/worksheets/bucket_level_monitor_test.http new file mode 100644 index 000000000..a54d03f74 --- /dev/null +++ b/worksheets/bucket_level_monitor_test.http @@ -0,0 +1,104 @@ +POST localhost:9200/opensearch_dashboards_sample_data_ecommerce/_search +Content-Type: application/json + +{ + "size": 0, + "aggregations": { + "composite_agg": { + "composite": { + "sources": [ + { "category": { "terms": { "field": "category.keyword" } } } + ] + }, + "aggs": { + "avg_total_price": { + "avg": { "field": "taxful_total_price" } + } + } + } + } +} +### +POST localhost:9200/_opendistro/_alerting/monitors +Content-Type: application/json + +{ + "type": "monitor", + "monitor_type": "bucket_level_monitor", + "name": "test-monitor", + "enabled": false, + "schedule": { + "period": { + "interval": 1, + "unit": "MINUTES" + } + }, + "inputs": [{ + "search": { + "indices": ["opensearch_dashboards_sample_data_ecommerce"], + "query": { + "size": 0, + "query": { + "bool": { + "filter": { + "range": { + "order_date": { + "gte": "now-1d/d", + "lte": "now", + "format": "epoch_millis" + } + } + } + } + }, + "aggregations": { + "composite_agg": { + "composite": { + "sources": [ + { "category": { "terms": { "field": "category.keyword" } } } + ] + }, + "aggs": { + "avg_total_price": { + "avg": { "field": "taxful_total_price" } + } + } + } + } + } + } + }], + "triggers": [{ + "aggregation_trigger": { + "name": "test-trigger", + "severity": "1", + "condition": { + "parent_bucket_path": "composite_agg", + "buckets_path": { + "avg_price": "avg_total_price" + }, + "script": { + "source": "params.avg_price >= 80" + } + }, + "actions": [{ + "name": "test-action", + "destination_id": "ld7912sBlQ5JUWWFThoW", + "subject_template": { + "source": "TheSubject" + }, + "message_template": { + "source" : "Monitor {{ctx.monitor.name}} just entered alert status. Please investigate the issue.\n- Trigger: {{ctx.trigger.name}}\n- Severity: {{ctx.trigger.severity}}\n- Period start: {{ctx.periodStart}}\n- Period end: {{ctx.periodEnd}}\n- Deduped Alerts: \n{{#ctx.dedupedAlerts}}\n * {{id}} : {{bucket_keys}}\n{{/ctx.dedupedAlerts}}\n- New Alerts:\n{{#ctx.newAlerts}}\n * {{id}} : {{bucket_keys}}\n{{/ctx.newAlerts}}\n- Completed Alerts:\n{{#ctx.completedAlerts}}\n * {{id}} : {{bucket_keys}}\n{{/ctx.completedAlerts}}", + "lang" : "mustache" + }, + "throttle_enabled": true, + "throttle": { + "value": 5, + "unit": "MINUTES" + } + }] + } + }] +} +### +POST localhost:9200/_plugins/_alerting/monitors//_execute \ No newline at end of file diff --git a/worksheets/opensearch_dashboards_sample_data.http b/worksheets/opensearch_dashboards_sample_data.http new file mode 100644 index 000000000..777906e7b --- /dev/null +++ b/worksheets/opensearch_dashboards_sample_data.http @@ -0,0 +1,9 @@ +# This will ingest the opensearch dashboards sample ecommerce data into the cluster iff opensearch dashboards +# is running on 5601. If using local dev opensearch dashboards make sure to pass yarn start --no-base-path +POST localhost:5601/api/sample_data/ecommerce +Connection: keep-alive +Accept: */* +osd-xsrf: opensearch-dashboards +User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.102 Safari/537.36 +Content-Type: application/json +### \ No newline at end of file From e6b5d341a6cb2fe6484f2f723d99fe494dd259ad Mon Sep 17 00:00:00 2001 From: Mohammad Qureshi <47198598+qreshi@users.noreply.github.com> Date: Wed, 8 Sep 2021 11:33:54 -0700 Subject: [PATCH 03/11] Add release notes for 1.1.0.0 release (#166) (#167) Signed-off-by: Mohammad Qureshi --- ...ensearch-alerting.release-notes-1.1.0.0.md | 45 +++++++++++++++++++ 1 file changed, 45 insertions(+) create mode 100644 release-notes/opensearch-alerting.release-notes-1.1.0.0.md diff --git a/release-notes/opensearch-alerting.release-notes-1.1.0.0.md b/release-notes/opensearch-alerting.release-notes-1.1.0.0.md new file mode 100644 index 000000000..2ec826afd --- /dev/null +++ b/release-notes/opensearch-alerting.release-notes-1.1.0.0.md @@ -0,0 +1,45 @@ +## Version 1.1.0.0 2021-09-08 + +Compatible with OpenSearch 1.1.0 + +### Features + +* Add BucketSelector pipeline aggregation extension ([#144](https://github.com/opensearch-project/alerting/pull/144)) +* Add AggregationResultBucket ([#148](https://github.com/opensearch-project/alerting/pull/148)) +* Add ActionExecutionPolicy ([#149](https://github.com/opensearch-project/alerting/pull/149)) +* Refactor Monitor and Trigger to split into Query-Level and Bucket-Level Monitors ([#150](https://github.com/opensearch-project/alerting/pull/150)) +* Update InputService for Bucket-Level Alerting ([#152](https://github.com/opensearch-project/alerting/pull/152)) +* Update TriggerService for Bucket-Level Alerting ([#153](https://github.com/opensearch-project/alerting/pull/153)) +* Update AlertService for Bucket-Level Alerting ([#154](https://github.com/opensearch-project/alerting/pull/154)) +* Add worksheets to help with testing ([#151](https://github.com/opensearch-project/alerting/pull/151)) +* Update MonitorRunner for Bucket-Level Alerting ([#155](https://github.com/opensearch-project/alerting/pull/155)) +* Fix ktlint formatting issues ([#156](https://github.com/opensearch-project/alerting/pull/156)) +* Execute Actions on runTrigger exceptions for Bucket-Level Monitor ([#157](https://github.com/opensearch-project/alerting/pull/157)) +* Skip execution of Actions on ACKNOWLEDGED Alerts for Bucket-Level Monitors ([#158](https://github.com/opensearch-project/alerting/pull/158)) +* Return first page of input results in MonitorRunResult for Bucket-Level Monitor ([#159](https://github.com/opensearch-project/alerting/pull/159)) +* Add setting to limit per alert action executions and don't save Alerts for test Bucket-Level Monitors ([#161](https://github.com/opensearch-project/alerting/pull/161)) +* Resolve default for ActionExecutionPolicy at runtime ([#165](https://github.com/opensearch-project/alerting/pull/165)) + +### Bug Fixes + +* Removing All Usages of Action Get Method Calls and adding the listeners ([#130](https://github.com/opensearch-project/alerting/pull/130)) +* Fix bug in paginating multiple bucket paths for Bucket-Level Monitor ([#163](https://github.com/opensearch-project/alerting/pull/163)) +* Various bug fixes for Bucket-Level Alerting ([#164](https://github.com/opensearch-project/alerting/pull/164)) +* Return only monitors for /monitors/_search ([#162](https://github.com/opensearch-project/alerting/pull/162)) + +### Infrastructure + +* Add Integtest.sh for OpenSearch integtest setups ([#121](https://github.com/opensearch-project/alerting/pull/121)) +* Fix snapshot build and increment to 1.1.0 ([#142](https://github.com/opensearch-project/alerting/pull/142)) + +### Documentation + +* Update Bucket-Level Alerting RFC ([#145](https://github.com/opensearch-project/alerting/pull/145)) + +### Maintenance + +* Remove default assignee ([#127](https://github.com/opensearch-project/alerting/pull/127)) + +### Refactoring + +* Refactor MonitorRunner ([#143](https://github.com/opensearch-project/alerting/pull/143)) \ No newline at end of file From 05eb22ea3a56ea982599a27562999d3de6b6472e Mon Sep 17 00:00:00 2001 From: "Daniel Doubrovkine (dB.)" Date: Thu, 23 Sep 2021 16:29:19 -0400 Subject: [PATCH 04/11] Remove default integtest.sh. (#181) Signed-off-by: dblock --- integtest.sh | 77 ---------------------------------------------------- 1 file changed, 77 deletions(-) delete mode 100755 integtest.sh diff --git a/integtest.sh b/integtest.sh deleted file mode 100755 index 6d5aeb392..000000000 --- a/integtest.sh +++ /dev/null @@ -1,77 +0,0 @@ -#!/bin/bash - -set -e - -function usage() { - echo "" - echo "This script is used to run integration tests for plugin installed on a remote OpenSearch/Dashboards cluster." - echo "--------------------------------------------------------------------------" - echo "Usage: $0 [args]" - echo "" - echo "Required arguments:" - echo "None" - echo "" - echo "Optional arguments:" - echo -e "-b BIND_ADDRESS\t, defaults to localhost | 127.0.0.1, can be changed to any IP or domain name for the cluster location." - echo -e "-p BIND_PORT\t, defaults to 9200 or 5601 depends on OpenSearch or Dashboards, can be changed to any port for the cluster location." - echo -e "-s SECURITY_ENABLED\t(true | false), defaults to true. Specify the OpenSearch/Dashboards have security enabled or not." - echo -e "-c CREDENTIAL\t(usename:password), no defaults, effective when SECURITY_ENABLED=true." - echo -e "-h\tPrint this message." - echo "--------------------------------------------------------------------------" -} - -while getopts ":hb:p:s:c:" arg; do - case $arg in - h) - usage - exit 1 - ;; - b) - BIND_ADDRESS=$OPTARG - ;; - p) - BIND_PORT=$OPTARG - ;; - s) - SECURITY_ENABLED=$OPTARG - ;; - c) - CREDENTIAL=$OPTARG - ;; - :) - echo "-${OPTARG} requires an argument" - usage - exit 1 - ;; - ?) - echo "Invalid option: -${OPTARG}" - exit 1 - ;; - esac -done - - -if [ -z "$BIND_ADDRESS" ] -then - BIND_ADDRESS="localhost" -fi - -if [ -z "$BIND_PORT" ] -then - BIND_PORT="9200" -fi - -if [ -z "$SECURITY_ENABLED" ] -then - SECURITY_ENABLED="true" -fi - -if [ -z "$CREDENTIAL" ] -then - CREDENTIAL="admin:admin" - USERNAME=`echo $CREDENTIAL | awk -F ':' '{print $1}'` - PASSWORD=`echo $CREDENTIAL | awk -F ':' '{print $2}'` -fi - -./gradlew integTest -Dtests.rest.cluster="$BIND_ADDRESS:$BIND_PORT" -Dtests.cluster="$BIND_ADDRESS:$BIND_PORT" -Dtests.clustername="opensearch-integrationtest" -Dhttps=$SECURITY_ENABLED -Duser=$USERNAME -Dpassword=$PASSWORD --console=plain -Dsecurity=$SECURITY_ENABLED - From 3a8b8748e0dc01c3418f50143cdd651e66b78557 Mon Sep 17 00:00:00 2001 From: Sriram <59816283+skkosuri-amzn@users.noreply.github.com> Date: Fri, 1 Oct 2021 16:17:38 -0700 Subject: [PATCH 05/11] Add valid search filters. (#191) * Add valid search filters. * Added this fix to release notes --- .../resthandler/RestSearchMonitorAction.kt | 9 ++++++++- .../alerting/resthandler/MonitorRestApiIT.kt | 18 ++++++++++++++++++ ...pensearch-alerting.release-notes-1.1.0.0.md | 3 ++- 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/resthandler/RestSearchMonitorAction.kt b/alerting/src/main/kotlin/org/opensearch/alerting/resthandler/RestSearchMonitorAction.kt index d19cd6560..a7b167af9 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/resthandler/RestSearchMonitorAction.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/resthandler/RestSearchMonitorAction.kt @@ -31,6 +31,7 @@ import org.opensearch.action.search.SearchResponse import org.opensearch.alerting.AlertingPlugin import org.opensearch.alerting.action.SearchMonitorAction import org.opensearch.alerting.action.SearchMonitorRequest +import org.opensearch.alerting.alerts.AlertIndices.Companion.ALL_INDEX_PATTERN import org.opensearch.alerting.core.model.ScheduledJob import org.opensearch.alerting.core.model.ScheduledJob.Companion.SCHEDULED_JOBS_INDEX import org.opensearch.alerting.model.Monitor @@ -108,12 +109,18 @@ class RestSearchMonitorAction( log.debug("${request.method()} ${AlertingPlugin.MONITOR_BASE_URI}/_search") val index = request.param("index", SCHEDULED_JOBS_INDEX) + if (index != SCHEDULED_JOBS_INDEX && index != ALL_INDEX_PATTERN) { + throw IllegalArgumentException("Invalid index name.") + } + val searchSourceBuilder = SearchSourceBuilder() searchSourceBuilder.parseXContent(request.contentOrSourceParamParser()) searchSourceBuilder.fetchSource(context(request)) val queryBuilder = QueryBuilders.boolQuery().must(searchSourceBuilder.query()) - queryBuilder.filter(QueryBuilders.existsQuery(Monitor.MONITOR_TYPE)) + if (index == SCHEDULED_JOBS_INDEX) { + queryBuilder.filter(QueryBuilders.existsQuery(Monitor.MONITOR_TYPE)) + } searchSourceBuilder.query(queryBuilder) .seqNoAndPrimaryTerm(true) diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt b/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt index b1a1cd916..730b3b241 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt @@ -942,4 +942,22 @@ class MonitorRestApiIT : AlertingRestTestCase() { val monitorHit = hit["_source"] as Map assertEquals("Type is not monitor", monitorHit[Monitor.TYPE_FIELD], "monitor") } + + @Throws(Exception::class) + fun `test search monitor with alerting indices only`() { + // 1. search - must return error as invalid index is passed + val search = SearchSourceBuilder().query(QueryBuilders.matchAllQuery()).toString() + val params: MutableMap = HashMap() + params["index"] = "data-logs" + try { + client().makeRequest( + "GET", + "$ALERTING_BASE_URI/_search", + params, + NStringEntity(search, ContentType.APPLICATION_JSON) + ) + } catch (e: ResponseException) { + assertEquals("Unexpected status", RestStatus.BAD_REQUEST, e.response.restStatus()) + } + } } diff --git a/release-notes/opensearch-alerting.release-notes-1.1.0.0.md b/release-notes/opensearch-alerting.release-notes-1.1.0.0.md index 2ec826afd..ebe3395d9 100644 --- a/release-notes/opensearch-alerting.release-notes-1.1.0.0.md +++ b/release-notes/opensearch-alerting.release-notes-1.1.0.0.md @@ -26,6 +26,7 @@ Compatible with OpenSearch 1.1.0 * Fix bug in paginating multiple bucket paths for Bucket-Level Monitor ([#163](https://github.com/opensearch-project/alerting/pull/163)) * Various bug fixes for Bucket-Level Alerting ([#164](https://github.com/opensearch-project/alerting/pull/164)) * Return only monitors for /monitors/_search ([#162](https://github.com/opensearch-project/alerting/pull/162)) +* Add valid search filters ([#191](https://github.com/opensearch-project/alerting/pull/191)) ### Infrastructure @@ -42,4 +43,4 @@ Compatible with OpenSearch 1.1.0 ### Refactoring -* Refactor MonitorRunner ([#143](https://github.com/opensearch-project/alerting/pull/143)) \ No newline at end of file +* Refactor MonitorRunner ([#143](https://github.com/opensearch-project/alerting/pull/143)) From 0e9ae1fa1960c6fef97d10dba02f436dc6db1442 Mon Sep 17 00:00:00 2001 From: "Daniel Doubrovkine (dB.)" Date: Wed, 6 Oct 2021 14:28:48 -0400 Subject: [PATCH 06/11] Publish notification JARs checksums. (#197) Signed-off-by: dblock --- .github/workflows/push-notification-jar.yml | 44 ------------ notification/build.gradle | 20 ++++-- scripts/build.sh | 75 +++++++++++++++++++++ 3 files changed, 90 insertions(+), 49 deletions(-) delete mode 100644 .github/workflows/push-notification-jar.yml create mode 100755 scripts/build.sh diff --git a/.github/workflows/push-notification-jar.yml b/.github/workflows/push-notification-jar.yml deleted file mode 100644 index 3b76e5854..000000000 --- a/.github/workflows/push-notification-jar.yml +++ /dev/null @@ -1,44 +0,0 @@ -name: Upload Notification Jar to Maven - -on: - push: - tags: - - v* -jobs: - upload-notification-jar: - runs-on: [ubuntu-16.04] - name: Upload Notification Jar to Maven - steps: - - name: Checkout Repo - uses: actions/checkout@v2 - - - name: Configure AWS CLI - uses: aws-actions/configure-aws-credentials@v1 - with: - aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} - aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} - aws-region: us-east-1 - - - name: Setup Java - uses: actions/setup-java@v1 - with: - java-version: '14' - - - name: Upload Notification Jar to Maven - env: - passphrase: ${{ secrets.PASSPHRASE }} - run: | - cd .. - export JAVA14_HOME=$JAVA_HOME - aws s3 cp s3://opendistro-docs/github-actions/pgp-public-key . - aws s3 cp s3://opendistro-docs/github-actions/pgp-private-key . - - gpg --import pgp-public-key - gpg --allow-secret-key-import --import pgp-private-key - - mkdir /home/runner/.gradle - aws s3 cp s3://opendistro-docs/github-actions/gradle.properties /home/runner/.gradle/ - - cd alerting/notification - - ../gradlew publishShadowPublicationToSonatype-stagingRepository -Dcompiler.java=14 -Dbuild.snapshot=false -Djavax.net.ssl.trustStore=$JAVA_HOME/lib/security/cacerts diff --git a/notification/build.gradle b/notification/build.gradle index 5fb42ccaf..471565529 100644 --- a/notification/build.gradle +++ b/notification/build.gradle @@ -45,6 +45,11 @@ shadowJar { relocate 'org.apache.commons.logging', 'org.opensearch.notification.repackage.org.apache.commons.logging' relocate 'org.apache.commons.codec', 'org.opensearch.notification.repackage.org.apache.commons.codec' classifier = null + + doLast { + ant.checksum algorithm: 'md5', file: it.archivePath + ant.checksum algorithm: 'sha1', file: it.archivePath + } } task sourcesJar(type: Jar) { @@ -52,11 +57,21 @@ task sourcesJar(type: Jar) { from sourceSets.main.allJava } +sourcesJar.doLast { + ant.checksum algorithm: 'md5', file: it.archivePath + ant.checksum algorithm: 'sha1', file: it.archivePath +} + task javadocJar(type: Jar) { classifier = 'javadoc' from javadoc.destinationDir } +javadocJar.doLast { + ant.checksum algorithm: 'md5', file: it.archivePath + ant.checksum algorithm: 'sha1', file: it.archivePath +} + publishing { publications { shadow(MavenPublication) { @@ -108,9 +123,4 @@ publishing { // TODO - enabled debug logging for the time being, remove this eventually gradle.startParameter.setShowStacktrace(ShowStacktrace.ALWAYS) gradle.startParameter.setLogLevel(LogLevel.DEBUG) - - signing { - required { gradle.taskGraph.hasTask("publishShadowPublicationToSonatype-stagingRepository") } - sign publishing.publications.shadow - } } diff --git a/scripts/build.sh b/scripts/build.sh new file mode 100755 index 000000000..19bdef2bd --- /dev/null +++ b/scripts/build.sh @@ -0,0 +1,75 @@ +#!/bin/bash + +# SPDX-License-Identifier: Apache-2.0 +# +# The OpenSearch Contributors require contributions made to +# this file be licensed under the Apache-2.0 license or a +# compatible open source license. + +set -ex + +function usage() { + echo "Usage: $0 [args]" + echo "" + echo "Arguments:" + echo -e "-v VERSION\t[Required] OpenSearch version." + echo -e "-s SNAPSHOT\t[Optional] Build a snapshot, default is 'false'." + echo -e "-a ARCHITECTURE\t[Optional] Build architecture, ignored." + echo -e "-o OUTPUT\t[Optional] Output path, default is 'artifacts'." + echo -e "-h help" +} + +while getopts ":h:v:s:o:a:" arg; do + case $arg in + h) + usage + exit 1 + ;; + v) + VERSION=$OPTARG + ;; + s) + SNAPSHOT=$OPTARG + ;; + o) + OUTPUT=$OPTARG + ;; + a) + ARCHITECTURE=$OPTARG + ;; + :) + echo "Error: -${OPTARG} requires an argument" + usage + exit 1 + ;; + ?) + echo "Invalid option: -${arg}" + exit 1 + ;; + esac +done + +if [ -z "$VERSION" ]; then + echo "Error: You must specify the OpenSearch version" + usage + exit 1 +fi + +[[ "$SNAPSHOT" == "true" ]] && VERSION=$VERSION-SNAPSHOT +[ -z "$OUTPUT" ] && OUTPUT=artifacts + +mkdir -p $OUTPUT/plugins + +./gradlew assemble --no-daemon --refresh-dependencies -DskipTests=true -Dopensearch.version=$VERSION -Dbuild.snapshot=$SNAPSHOT -x ktlint + +zipPath=$(find . -path \*build/distributions/*.zip) +distributions="$(dirname "${zipPath}")" + +echo "COPY ${distributions}/*.zip" +cp ${distributions}/*.zip ./$OUTPUT/plugins + +./gradlew publishShadowPublicationToMavenLocal -Dopensearch.version=$VERSION -Dbuild.snapshot=$SNAPSHOT -x ktlint + +mkdir -p $OUTPUT/maven/org/opensearch +cp -r ./notification/build/libs $OUTPUT/maven/org/opensearch/notification + From ff7c1c99fb41fa521fa76c7263c1634d98b67ee5 Mon Sep 17 00:00:00 2001 From: "Daniel Doubrovkine (dB.)" Date: Wed, 6 Oct 2021 17:11:01 -0400 Subject: [PATCH 07/11] Also publish SHA 256 and 512 checksums. (#198) * Also publish SHA 256 and 512 checksums. Signed-off-by: dblock * Remove sonatype staging. Signed-off-by: dblock --- notification/build.gradle | 31 +++++++------------------------ 1 file changed, 7 insertions(+), 24 deletions(-) diff --git a/notification/build.gradle b/notification/build.gradle index 471565529..62c2acae7 100644 --- a/notification/build.gradle +++ b/notification/build.gradle @@ -45,11 +45,6 @@ shadowJar { relocate 'org.apache.commons.logging', 'org.opensearch.notification.repackage.org.apache.commons.logging' relocate 'org.apache.commons.codec', 'org.opensearch.notification.repackage.org.apache.commons.codec' classifier = null - - doLast { - ant.checksum algorithm: 'md5', file: it.archivePath - ant.checksum algorithm: 'sha1', file: it.archivePath - } } task sourcesJar(type: Jar) { @@ -57,19 +52,18 @@ task sourcesJar(type: Jar) { from sourceSets.main.allJava } -sourcesJar.doLast { - ant.checksum algorithm: 'md5', file: it.archivePath - ant.checksum algorithm: 'sha1', file: it.archivePath -} - task javadocJar(type: Jar) { classifier = 'javadoc' from javadoc.destinationDir } -javadocJar.doLast { - ant.checksum algorithm: 'md5', file: it.archivePath - ant.checksum algorithm: 'sha1', file: it.archivePath +tasks.withType(Jar) { task -> + task.doLast { + ant.checksum algorithm: 'md5', file: it.archivePath + ant.checksum algorithm: 'sha1', file: it.archivePath + ant.checksum algorithm: 'sha-256', file: it.archivePath, fileext: '.sha256' + ant.checksum algorithm: 'sha-512', file: it.archivePath, fileext: '.sha512' + } } publishing { @@ -109,17 +103,6 @@ publishing { } } - repositories { - maven { - name = "sonatype-staging" - url "https://aws.oss.sonatype.org/service/local/staging/deploy/maven2" - credentials { - username project.hasProperty('ossrhUsername') ? project.property('ossrhUsername') : '' - password project.hasProperty('ossrhPassword') ? project.property('ossrhPassword') : '' - } - } - } - // TODO - enabled debug logging for the time being, remove this eventually gradle.startParameter.setShowStacktrace(ShowStacktrace.ALWAYS) gradle.startParameter.setLogLevel(LogLevel.DEBUG) From 07073ad4fe61df8053c848a4afd6d6ddab6d5350 Mon Sep 17 00:00:00 2001 From: AWSHurneyt Date: Wed, 13 Oct 2021 16:22:59 -0700 Subject: [PATCH 08/11] Fixed a bug that was preventing the AcknowledgeAlerts API from acknowledging more than 10 alerts at once. Signed-off-by: Thomas Hurney --- .../alerting/transport/TransportAcknowledgeAlertAction.kt | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/alerting/src/main/kotlin/org/opensearch/alerting/transport/TransportAcknowledgeAlertAction.kt b/alerting/src/main/kotlin/org/opensearch/alerting/transport/TransportAcknowledgeAlertAction.kt index ae8653dbd..2780277a0 100644 --- a/alerting/src/main/kotlin/org/opensearch/alerting/transport/TransportAcknowledgeAlertAction.kt +++ b/alerting/src/main/kotlin/org/opensearch/alerting/transport/TransportAcknowledgeAlertAction.kt @@ -75,7 +75,13 @@ class TransportAcknowledgeAlertAction @Inject constructor( val searchRequest = SearchRequest() .indices(AlertIndices.ALERT_INDEX) .routing(request.monitorId) - .source(SearchSourceBuilder().query(queryBuilder).version(true).seqNoAndPrimaryTerm(true)) + .source( + SearchSourceBuilder() + .query(queryBuilder) + .version(true) + .seqNoAndPrimaryTerm(true) + .size(request.alertIds.size) + ) client.search( searchRequest, From 9235edd611d928a65576d8d61c502df660b0ab12 Mon Sep 17 00:00:00 2001 From: AWSHurneyt Date: Thu, 14 Oct 2021 11:30:47 -0700 Subject: [PATCH 09/11] Implemented integration tests to ensure fix for issue 203 is working as expected. Signed-off-by: Thomas Hurney --- .../alerting/resthandler/MonitorRestApiIT.kt | 123 ++++++++++++++++++ 1 file changed, 123 insertions(+) diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt b/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt index 730b3b241..1a51761b4 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt @@ -569,6 +569,129 @@ class MonitorRestApiIT : AlertingRestTestCase() { assertFalse("Alert in state ${activeAlert.state} found in failed list", failedResponseList.contains(activeAlert.id)) } + fun `test acknowledging more than 10 alerts at once`() { + // GIVEN + putAlertMappings() // Required as we do not have a create alert API. + val monitor = createRandomMonitor(refresh = true) + val alertsToAcknowledge = arrayOf( + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)) + ) + + // WHEN + val response = acknowledgeAlerts(monitor, *alertsToAcknowledge) + + // THEN + val responseMap = response.asMap() + val expectedAcknowledgedCount = alertsToAcknowledge.size + + val acknowledgedAlerts = responseMap["success"] as List + assertTrue("Expected $expectedAcknowledgedCount alerts to be acknowledged successfully.", acknowledgedAlerts.size == expectedAcknowledgedCount) + + val acknowledgedAlertsList = acknowledgedAlerts.toString() + alertsToAcknowledge.forEach { alert -> assertTrue("Alert with ID ${alert.id} not found in failed list.", acknowledgedAlertsList.contains(alert.id)) } + + val failedResponse = responseMap["failed"] as List + assertTrue("Expected 0 alerts to fail acknowledgment.", failedResponse.isEmpty()) + + val failedResponseList = failedResponse.toString() + alertsToAcknowledge.forEach { alert -> assertFalse("Alert with ID ${alert.id} found in failed list.", failedResponseList.contains(alert.id)) } + } + + fun `test acknowledging more than 10 alerts at once, including acknowledged alerts`() { + // GIVEN + putAlertMappings() // Required as we do not have a create alert API. + val monitor = createRandomMonitor(refresh = true) + val alertsGroup1 = arrayOf( + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)) + ) + acknowledgeAlerts(monitor, *alertsGroup1) // Acknowledging the first array of alerts. + + val alertsGroup2 = arrayOf( + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), + createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)) + ) + + val alertsToAcknowledge = arrayOf(*alertsGroup1, *alertsGroup2) // Creating an array of alerts that includes alerts that have been already acknowledged, and new alerts. + + // WHEN + val response = acknowledgeAlerts(monitor, *alertsToAcknowledge) + + // THEN + val responseMap = response.asMap() + val expectedAcknowledgedCount = alertsToAcknowledge.size - alertsGroup1.size + + val acknowledgedAlerts = responseMap["success"] as List + assertTrue("Expected $expectedAcknowledgedCount alerts to be acknowledged successfully.", acknowledgedAlerts.size == expectedAcknowledgedCount) + + val acknowledgedAlertsList = acknowledgedAlerts.toString() + alertsGroup2.forEach { alert -> assertTrue("Alert with ID ${alert.id} not found in failed list.", acknowledgedAlertsList.contains(alert.id)) } + alertsGroup1.forEach { alert -> assertFalse("Alert with ID ${alert.id} found in failed list.", acknowledgedAlertsList.contains(alert.id)) } + + val failedResponse = responseMap["failed"] as List + assertTrue("Expected ${alertsGroup1.size} alerts to fail acknowledgment.", failedResponse.size == alertsGroup1.size) + + val failedResponseList = failedResponse.toString() + alertsGroup1.forEach { alert -> assertTrue("Alert with ID ${alert.id} not found in failed list.", failedResponseList.contains(alert.id)) } + alertsGroup2.forEach { alert -> assertFalse("Alert with ID ${alert.id} found in failed list.", failedResponseList.contains(alert.id)) } + } + + @Throws(Exception::class) + fun `test acknowledging 0 alerts`() { + // GIVEN + putAlertMappings() // Required as we do not have a create alert API. + val monitor = createRandomMonitor(refresh = true) + val alertsToAcknowledge = arrayOf() + + // WHEN & THEN + try { + acknowledgeAlerts(monitor, *alertsToAcknowledge) + fail("Expected acknowledgeAlerts to throw an exception.") + } catch (e: ResponseException) { + assertEquals("Unexpected status", RestStatus.BAD_REQUEST, e.response.restStatus()) + } + } + fun `test get all alerts in all states`() { putAlertMappings() // Required as we do not have a create alert API. val monitor = createRandomMonitor(refresh = true) From e1250d72fd826c91abda30e887fd92854dcc29ba Mon Sep 17 00:00:00 2001 From: AWSHurneyt Date: Thu, 14 Oct 2021 14:46:23 -0700 Subject: [PATCH 10/11] Refactored integ tests based on PR feedback, and listed the bug fix in the release notes. Signed-off-by: Thomas Hurney --- .../alerting/resthandler/MonitorRestApiIT.kt | 57 +------------------ ...ensearch-alerting.release-notes-1.1.0.0.md | 1 + 2 files changed, 4 insertions(+), 54 deletions(-) diff --git a/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt b/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt index 1a51761b4..ccc23f894 100644 --- a/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt +++ b/alerting/src/test/kotlin/org/opensearch/alerting/resthandler/MonitorRestApiIT.kt @@ -573,23 +573,7 @@ class MonitorRestApiIT : AlertingRestTestCase() { // GIVEN putAlertMappings() // Required as we do not have a create alert API. val monitor = createRandomMonitor(refresh = true) - val alertsToAcknowledge = arrayOf( - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)) - ) + val alertsToAcknowledge = (1..15).map { createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)) }.toTypedArray() // WHEN val response = acknowledgeAlerts(monitor, *alertsToAcknowledge) @@ -606,51 +590,16 @@ class MonitorRestApiIT : AlertingRestTestCase() { val failedResponse = responseMap["failed"] as List assertTrue("Expected 0 alerts to fail acknowledgment.", failedResponse.isEmpty()) - - val failedResponseList = failedResponse.toString() - alertsToAcknowledge.forEach { alert -> assertFalse("Alert with ID ${alert.id} found in failed list.", failedResponseList.contains(alert.id)) } } fun `test acknowledging more than 10 alerts at once, including acknowledged alerts`() { // GIVEN putAlertMappings() // Required as we do not have a create alert API. val monitor = createRandomMonitor(refresh = true) - val alertsGroup1 = arrayOf( - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)) - ) + val alertsGroup1 = (1..15).map { createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)) }.toTypedArray() acknowledgeAlerts(monitor, *alertsGroup1) // Acknowledging the first array of alerts. - val alertsGroup2 = arrayOf( - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)), - createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)) - ) + val alertsGroup2 = (1..15).map { createAlert(randomAlert(monitor).copy(state = Alert.State.ACTIVE)) }.toTypedArray() val alertsToAcknowledge = arrayOf(*alertsGroup1, *alertsGroup2) // Creating an array of alerts that includes alerts that have been already acknowledged, and new alerts. diff --git a/release-notes/opensearch-alerting.release-notes-1.1.0.0.md b/release-notes/opensearch-alerting.release-notes-1.1.0.0.md index ebe3395d9..e673c5090 100644 --- a/release-notes/opensearch-alerting.release-notes-1.1.0.0.md +++ b/release-notes/opensearch-alerting.release-notes-1.1.0.0.md @@ -27,6 +27,7 @@ Compatible with OpenSearch 1.1.0 * Various bug fixes for Bucket-Level Alerting ([#164](https://github.com/opensearch-project/alerting/pull/164)) * Return only monitors for /monitors/_search ([#162](https://github.com/opensearch-project/alerting/pull/162)) * Add valid search filters ([#191](https://github.com/opensearch-project/alerting/pull/191)) +* Fixed a bug that was preventing the AcknowledgeAlerts API from acknowledging more than 10 alerts at once ([#205](https://github.com/opensearch-project/alerting/pull/205/)) ### Infrastructure From 01ce30ae33af23e99abdf6fa528ec04b8790b456 Mon Sep 17 00:00:00 2001 From: AWSHurneyt Date: Fri, 15 Oct 2021 09:55:43 -0700 Subject: [PATCH 11/11] Removing bug fixes from release notes. Currently discussing adding separate notes for this patch. Signed-off-by: Thomas Hurney --- release-notes/opensearch-alerting.release-notes-1.1.0.0.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/release-notes/opensearch-alerting.release-notes-1.1.0.0.md b/release-notes/opensearch-alerting.release-notes-1.1.0.0.md index e673c5090..59d83ea09 100644 --- a/release-notes/opensearch-alerting.release-notes-1.1.0.0.md +++ b/release-notes/opensearch-alerting.release-notes-1.1.0.0.md @@ -26,8 +26,6 @@ Compatible with OpenSearch 1.1.0 * Fix bug in paginating multiple bucket paths for Bucket-Level Monitor ([#163](https://github.com/opensearch-project/alerting/pull/163)) * Various bug fixes for Bucket-Level Alerting ([#164](https://github.com/opensearch-project/alerting/pull/164)) * Return only monitors for /monitors/_search ([#162](https://github.com/opensearch-project/alerting/pull/162)) -* Add valid search filters ([#191](https://github.com/opensearch-project/alerting/pull/191)) -* Fixed a bug that was preventing the AcknowledgeAlerts API from acknowledging more than 10 alerts at once ([#205](https://github.com/opensearch-project/alerting/pull/205/)) ### Infrastructure