[Detection Engine] FTR test to replicate undefined elasticsearch state#230148
[Detection Engine] FTR test to replicate undefined elasticsearch state#230148rylnd wants to merge 13 commits intoelastic:mainfrom
Conversation
This integration test is executed 100 times, with the single-document archive being loaded and unloaded before/after each test. If this test fails, it will eliminate the detection engine as a culprit.
This should allow us to more clearly trace what's happening via APM data.
As there might be useful info in here, and using the logging service is apparently equivalent to `> /dev/null`
Again, this should offer more visibility into what the tests are doing.
By specifying an ID in our archive doc, we should no longer get the duplication, but I expect instead to see the index request fail in the state produced by these tests.
|
🤖 Jobs for this PR can be triggered through checkboxes. 🚧
ℹ️ To trigger the CI, please tick the checkbox below 👇
|
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#8960[❌] x-pack/test/security_solution_api_integration/test_suites/detections_response/detection_engine/rule_execution_logic/general_logic/basic_license_essentials_tier/configs/ess.config.ts: 0/100 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#8961[❌] x-pack/test/security_solution_api_integration/test_suites/detections_response/detection_engine/rule_execution_logic/general_logic/basic_license_essentials_tier/configs/ess.config.ts: 0/1 tests passed. |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#8971[✅] x-pack/test/security_solution_api_integration/test_suites/detections_response/detection_engine/rule_execution_logic/general_logic/basic_license_essentials_tier/configs/ess.config.ts: 10/10 tests passed. |
|
The previous 10x run with an |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#8972[✅] x-pack/test/security_solution_api_integration/test_suites/detections_response/detection_engine/rule_execution_logic/general_logic/basic_license_essentials_tier/configs/ess.config.ts: 100/100 tests passed. |
Just making sure the tests now fail, again, as a sanity check.
|
The 100x run passed, as well, so it seems as though the issue is closely tied to the |
rylnd
changed the title
|
Welp, those 30x without |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#8976[✅] x-pack/test/security_solution_api_integration/test_suites/detections_response/detection_engine/rule_execution_logic/general_logic/basic_license_essentials_tier/configs/ess.config.ts: 30/30 tests passed. |
|
Closing as this work has now (mostly) been completed with #229457 for I suspect there may still be a case where we would now fail due to a conflict on |
4 participants
Summary
🚧
This is temporary branch. It is a rebased copy of #228850, excluding Robert's fixes. I'm creating a draft PR here in order to trigger a flaky run, to try and accomplish what failed (by succeeding the build) in this comment.