[Response Ops] Fix Event Log Flaky E2E Test#227559
Conversation
JiaweiWu
added
release_note:skip
|
Pinging @elastic/response-ops (Team:ResponseOps) |
|
Flaky test runner: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8604 |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#8604[❌] x-pack/platform/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 72/100 tests passed. |
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#8648[❌] x-pack/platform/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 72/100 tests passed. |
pmuellr
left a comment
pmuellr
left a comment
There was a problem hiding this comment.
Fixes look good to me, but there are still some flaky errors? Are these new?
| }); | ||
| }; | ||
|
|
||
| export const waitForRuleExecute = ({ |
There was a problem hiding this comment.
nit: I'm surprised we didn't already have a function for this, but didn't see one. Since it also takes action executions into account, perhaps adding that in the name? waitForRuleExecuteWithActions?
There was a problem hiding this comment.
yep you're right, I wrote this to wrap getEventLog just to make the test cleaner, however I am trying a different approach that asserts actions execute success so I am removing this.
|
Haha yea my fix didn't seem to work, trying a new approach 😄 |
Flaky Test Runner Stats🎉 All tests passed! - kibana-flaky-test-suite-runner#9133[✅] x-pack/platform/test/alerting_api_integration/spaces_only/tests/alerting/group1/config.ts: 100/100 tests passed. |
pmuellr
left a comment
pmuellr
left a comment
There was a problem hiding this comment.
LGTM, left a note about having to check on action event logs
| }); | ||
| } | ||
|
|
||
| const provider = shouldUseActionsProvider ? 'actions' : 'alerting'; |
There was a problem hiding this comment.
Is there a case for us having to wait for the action execution, just vs waiting on alert/execute-action? If we're only looking at alerting event logs, I don't think we should need to be looking at the action event logs.
Seems like the existing code should work, but is somewhat overkill, and will be a little slower since it has to wait for the connector to run.
There was a problem hiding this comment.
Yea unfortunately we have to check the actions event log because the alert/execute-action is now merely queuing the action. The previously flakiness was caused by this as actions are being queue before the previous actions are finishing, therefore never starting the second actions
|
@elasticmachine merge upstream |
|
merge conflict between base and head |
⏳ Build in-progress, with failures
Failed CI Steps
Test Failures
History
|
## Summary Resolves: elastic#209911 Fixes a flaky test where the `execute-action` events were not appearing. This PR changes the test procedure to manually trigger the rule runs instead of letting it run by itself with a low interval. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary Resolves: elastic#209911 Fixes a flaky test where the `execute-action` events were not appearing. This PR changes the test procedure to manually trigger the rule runs instead of letting it run by itself with a low interval. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
## Summary Resolves: elastic#209911 Fixes a flaky test where the `execute-action` events were not appearing. This PR changes the test procedure to manually trigger the rule runs instead of letting it run by itself with a low interval. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
6 participants
Summary
Resolves: #209911
Fixes a flaky test where the
execute-actionevents were not appearing. This PR changes the test procedure to manually trigger the rule runs instead of letting it run by itself with a low interval.Checklist