[Security Solution] Test plan for rule type field diff algorithm
#193372
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dplumlee
added
release_note:skip
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
dplumlee
requested review from
jpdjere and
pborgonovi
and removed request for
xcrzx
dplumlee
added
v9.0.0
backport:prev-minor
This was referenced Sep 19, 2024
jpdjere
reviewed
Sep 26, 2024
| @@ -179,8 +217,42 @@ Examples: | |||
| | esql_query | esql_query | {query: "FROM query WHERE true", language: "esql"} | {query: "FROM query WHERE false", language: "esql"} | {query: "FROM query WHERE false", language: "esql"} | {query: "FROM query WHERE false", language: "esql"} | | |||
| ``` | |||
|
|
|||
| #### **Scenario: `ABB` - Rule field is rule `type`** | |||
There was a problem hiding this comment.
@dplumlee The same in this PR for possible and not possible cases:
Maybe @pborgonovi has an opinion on this: some of the scenarios are not possible in our system (see comment above), but technically we can create an outcome for them. What should be the way forward from a testing perspective? Should we write in the test plan that the scenarios are not possible? Should they have test cases anyways (even though the mock data that needs to be generated in order to create those scenarios is technically not possible to achieve in the normal flow of the application)?
There was a problem hiding this comment.
For edge scenarios that are not possible within the current flow of our application but can be tested using mock data, I recommend we still include them in our test plan.
We should include an explanation as comment of why these scenarios are not achievable in the normal flow, outlining any relevant technical limitations. This will help manage expectations and ensure everyone is aware of the system's current capabilities. Including these scenarios can also help future-proof the system and ensure it doesn’t break if they arise due to future changes. In the corresponding test cases, we should indicate that the tests depend on mock data.
Let me know if that makes sense.
2 tasks
jpdjere
approved these changes
Sep 30, 2024
There was a problem hiding this comment.
@dplumlee Update about the rule type changes: #193369 (comment)
I'm approving cause overall looks good, let's just add some additional explanation or note as I mentioned in the link above, and @pborgonovi mentioned.
|
Starting backport for target branches: 8.x https://github.com/elastic/kibana/actions/runs/11110399814 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Sep 30, 2024
…lastic#193372) ## Summary Related ticket: elastic#190482 Adds test plan for diff algorithm for `type` field diff algorithm implemented here: elastic#193369 ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) (cherry picked from commit fefa59f)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Sep 30, 2024
…iff algorithm (#193372) (#194459) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Test plan for rule `type` field diff algorithm (#193372)](#193372) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Davis Plumlee","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-09-30T16:37:29Z","message":"[Security Solution] Test plan for rule `type` field diff algorithm (#193372)\n\n## Summary\r\n\r\nRelated ticket: https://github.com/elastic/kibana/issues/190482\r\n\r\nAdds test plan for diff algorithm for `type` field diff algorithm\r\nimplemented here: https://github.com/elastic/kibana/pull/193369\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"fefa59f41206c534297813af2cb6f732c2c59aeb","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:prev-minor","v8.16.0"],"title":"[Security Solution] Test plan for rule `type` field diff algorithm","number":193372,"url":"https://github.com/elastic/kibana/pull/193372","mergeCommit":{"message":"[Security Solution] Test plan for rule `type` field diff algorithm (#193372)\n\n## Summary\r\n\r\nRelated ticket: https://github.com/elastic/kibana/issues/190482\r\n\r\nAdds test plan for diff algorithm for `type` field diff algorithm\r\nimplemented here: https://github.com/elastic/kibana/pull/193369\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"fefa59f41206c534297813af2cb6f732c2c59aeb"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193372","number":193372,"mergeCommit":{"message":"[Security Solution] Test plan for rule `type` field diff algorithm (#193372)\n\n## Summary\r\n\r\nRelated ticket: https://github.com/elastic/kibana/issues/190482\r\n\r\nAdds test plan for diff algorithm for `type` field diff algorithm\r\nimplemented here: https://github.com/elastic/kibana/pull/193369\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"fefa59f41206c534297813af2cb6f732c2c59aeb"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Davis Plumlee <[email protected]>
dplumlee
added a commit
that referenced
this pull request
Oct 8, 2024
…rithms (#193375) ## Summary Completes #190482 Switches rule `type` field to use the implemented diff algorithms assigned to them in #193369 Adds integration tests in accordance to #193372 for the `upgrade/_review` API endpoint for the rule `type` field diff algorithm. Also fixes some nested bracket misalignment that occurred in earlier PRs with some test files ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Oct 8, 2024
…rithms (elastic#193375) ## Summary Completes elastic#190482 Switches rule `type` field to use the implemented diff algorithms assigned to them in elastic#193369 Adds integration tests in accordance to elastic#193372 for the `upgrade/_review` API endpoint for the rule `type` field diff algorithm. Also fixes some nested bracket misalignment that occurred in earlier PRs with some test files ### Checklist Delete any items that are not applicable to this PR. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) (cherry picked from commit e119d83)
kibanamachine
added a commit
that referenced
this pull request
Oct 9, 2024
… field diff algorithms (#193375) (#195518) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Integration tests for rule `type` field diff algorithms (#193375)](#193375) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Davis Plumlee","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-08T22:46:33Z","message":"[Security Solution] Integration tests for rule `type` field diff algorithms (#193375)\n\n## Summary\r\n\r\nCompletes https://github.com/elastic/kibana/issues/190482\r\n\r\n\r\nSwitches rule `type` field to use the implemented diff algorithms\r\nassigned to them in https://github.com/elastic/kibana/pull/193369\r\n\r\n\r\nAdds integration tests in accordance to\r\nhttps://github.com//pull/193372 for the `upgrade/_review`\r\nAPI endpoint for the rule `type` field diff algorithm.\r\n\r\nAlso fixes some nested bracket misalignment that occurred in earlier PRs\r\nwith some test files\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"e119d83c26387e85e9fdf3cc5d5eeceeebb04edb","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:prev-minor","v8.16.0"],"title":"[Security Solution] Integration tests for rule `type` field diff algorithms","number":193375,"url":"https://github.com/elastic/kibana/pull/193375","mergeCommit":{"message":"[Security Solution] Integration tests for rule `type` field diff algorithms (#193375)\n\n## Summary\r\n\r\nCompletes https://github.com/elastic/kibana/issues/190482\r\n\r\n\r\nSwitches rule `type` field to use the implemented diff algorithms\r\nassigned to them in https://github.com/elastic/kibana/pull/193369\r\n\r\n\r\nAdds integration tests in accordance to\r\nhttps://github.com//pull/193372 for the `upgrade/_review`\r\nAPI endpoint for the rule `type` field diff algorithm.\r\n\r\nAlso fixes some nested bracket misalignment that occurred in earlier PRs\r\nwith some test files\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"e119d83c26387e85e9fdf3cc5d5eeceeebb04edb"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/193375","number":193375,"mergeCommit":{"message":"[Security Solution] Integration tests for rule `type` field diff algorithms (#193375)\n\n## Summary\r\n\r\nCompletes https://github.com/elastic/kibana/issues/190482\r\n\r\n\r\nSwitches rule `type` field to use the implemented diff algorithms\r\nassigned to them in https://github.com/elastic/kibana/pull/193369\r\n\r\n\r\nAdds integration tests in accordance to\r\nhttps://github.com//pull/193372 for the `upgrade/_review`\r\nAPI endpoint for the rule `type` field diff algorithm.\r\n\r\nAlso fixes some nested bracket misalignment that occurred in earlier PRs\r\nwith some test files\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [x] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n\r\n### For maintainers\r\n\r\n- [ ] This was checked for breaking API changes and was [labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"e119d83c26387e85e9fdf3cc5d5eeceeebb04edb"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Davis Plumlee <[email protected]> Co-authored-by: Elastic Machine <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Related ticket: #190482
Adds test plan for diff algorithm for
typefield diff algorithm implemented here: #193369For maintainers