Skip to content

[Security Solution] [Attack discovery] Fixes a flakey jest test#191736

Merged
andrew-goldstein merged 3 commits intoelastic:mainfrom
andrew-goldstein:fix_flaky_test_issue_189757
Aug 29, 2024
Merged

[Security Solution] [Attack discovery] Fixes a flakey jest test#191736
andrew-goldstein merged 3 commits intoelastic:mainfrom
andrew-goldstein:fix_flaky_test_issue_189757

Conversation

@andrew-goldstein
Copy link
Contributor

@andrew-goldstein andrew-goldstein commented Aug 29, 2024 •
edited by kibanamachine
Loading

Fixes a flakey jest test that failed twice since it was introduced ~2 months ago in this pr, as tracked by #189757

CI and desk flake testing

To test the fix for fakyness, it ran 10,000 times in CI, and locally, with a 100% success rate. (The test was wrapped in a loop to execute 10,000 times.)

  • Tested in CI 10,000 times via following Buildkite build:

💚 Build Succeeded
Buildkite Build

The 10,000 passing test runs are included in the following (abbreviated) build log output:

Ran all test suites.
$ node scripts/jest --config x-pack/plugins/security_solution/public/attack_discovery/jest.config.js
actual full command is:
NODE_OPTIONS="--max-old-space-size=12288 --trace-warnings" node ./scripts/jest --config="x-pack/plugins/security_solution/public/attack_discovery/jest.config.js" --runInBand --coverage=false --passWithNoTests
 PASS  x-pack/plugins/security_solution/public/attack_discovery/pages/loading_callout/countdown/index.test.tsx (742.054 s)
# other passing tests...

Test Suites: 54 passed, 54 total
Tests:       10297 passed, 10297 total
Snapshots:   0 total
Time:        962.106 s
  • Tested locally 10,000 times, as illustrated by the following console output:
    ✓ renders the expected the timer text - iteration 9996 (43 ms)
    ✓ renders the expected the timer text - iteration 9997 (44 ms)
    ✓ renders the expected the timer text - iteration 9998 (48 ms)
    ✓ renders the expected the timer text - iteration 9999 (49 ms)
    ✓ renders the expected the timer text - iteration 10000 (46 ms)

Test Suites: 1 passed, 1 total
Tests:       10002 passed, 10002 total
Snapshots:   0 total
Time:        294.54 s
Ran all test suites matching /x-pack\/plugins\/security_solution\/public\/attack_discovery\/pages\/loading_callout\/countdown\/index.test.tsx/i.

@andrew-goldstein andrew-goldstein added release_note:skip Skip the PR/issue when compiling release notes Team:Security Generative AI Security Generative AI v8.16.0 v8.15.1 labels Aug 29, 2024
@andrew-goldstein andrew-goldstein self-assigned this Aug 29, 2024
@andrew-goldstein andrew-goldstein requested a review from a team as a code owner August 29, 2024 13:27
@andrew-goldstein andrew-goldstein force-pushed the fix_flaky_test_issue_189757 branch from 1a05947 to e8e7552 Compare August 29, 2024 17:32
@andrew-goldstein
Fixes a flakey test that failed twice since it was introduced ~2 mont…
4a716c7 
…hs ago in [this pr](elastic#186679), as tracked by elastic#189757
@andrew-goldstein
- run the test 10,000 times (to test locally, and on CI)
2f0ac57 
```
    ✓ renders the expected the timer text - iteration 9996 (43 ms)
    ✓ renders the expected the timer text - iteration 9997 (44 ms)
    ✓ renders the expected the timer text - iteration 9998 (48 ms)
    ✓ renders the expected the timer text - iteration 9999 (49 ms)
    ✓ renders the expected the timer text - iteration 10000 (46 ms)

Test Suites: 1 passed, 1 total
Tests:       10002 passed, 10002 total
Snapshots:   0 total
Time:        294.54 s
Ran all test suites matching /x-pack\/plugins\/security_solution\/public\/attack_discovery\/pages\/loading_callout\/countdown\/index.test.tsx/i.
```
@andrew-goldstein
- removes temp loop for testing 10,000 times on CI (and locally)
07104f4
@andrew-goldstein andrew-goldstein force-pushed the fix_flaky_test_issue_189757 branch from e8e7552 to 07104f4 Compare August 29, 2024 19:42
@kibana-ci
Copy link

kibana-ci commented Aug 29, 2024

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

  • 💚 Build #230901 succeeded e8e7552561ade474e7d34a2766789085f50f801d
  • 💛 Build #230794 was flaky 1a05947e14b9c39355b1bbbb8683b833eaa681b8

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @andrew-goldstein

@andrew-goldstein andrew-goldstein merged commit e0f8eb1 into elastic:main Aug 29, 2024
@andrew-goldstein andrew-goldstein deleted the fix_flaky_test_issue_189757 branch August 29, 2024 21:29
@andrew-goldstein andrew-goldstein mentioned this pull request Aug 29, 2024
Closed
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Aug 29, 2024
@andrew-goldstein
[Security Solution] [Attack discovery] Fixes a flakey jest test (elas…
2c72c9d 
…tic#191736)

Fixes a flakey jest test that failed twice since it was introduced ~2 months ago in [this pr](elastic#186679), as tracked by elastic#189757

### CI and desk flake testing

To test the fix for fakyness, it ran `10,000` times in CI, and locally, with a `100%` success rate. (The test was wrapped in a loop to execute 10,000 times.)

- Tested in CI 10,000 times via following Buildkite build:

:green_heart: Build Succeeded
[Buildkite Build](https://buildkite.com/elastic/kibana-pull-request/builds/230901)

The 10,000 passing test runs are included in the following (abbreviated) build log output:

```
Ran all test suites.
$ node scripts/jest --config x-pack/plugins/security_solution/public/attack_discovery/jest.config.js
actual full command is:
NODE_OPTIONS="--max-old-space-size=12288 --trace-warnings" node ./scripts/jest --config="x-pack/plugins/security_solution/public/attack_discovery/jest.config.js" --runInBand --coverage=false --passWithNoTests
 PASS  x-pack/plugins/security_solution/public/attack_discovery/pages/loading_callout/countdown/index.test.tsx (742.054 s)
# other passing tests...

Test Suites: 54 passed, 54 total
Tests:       10297 passed, 10297 total
Snapshots:   0 total
Time:        962.106 s
```

- Tested locally 10,000 times, as illustrated by the following console output:

```
    ✓ renders the expected the timer text - iteration 9996 (43 ms)
    ✓ renders the expected the timer text - iteration 9997 (44 ms)
    ✓ renders the expected the timer text - iteration 9998 (48 ms)
    ✓ renders the expected the timer text - iteration 9999 (49 ms)
    ✓ renders the expected the timer text - iteration 10000 (46 ms)

Test Suites: 1 passed, 1 total
Tests:       10002 passed, 10002 total
Snapshots:   0 total
Time:        294.54 s
Ran all test suites matching /x-pack\/plugins\/security_solution\/public\/attack_discovery\/pages\/loading_callout\/countdown\/index.test.tsx/i.
```

(cherry picked from commit e0f8eb1)
@kibanamachine kibanamachine mentioned this pull request Aug 29, 2024
Merged
@kibanamachine
Copy link
Contributor

kibanamachine commented Aug 29, 2024

💚 All backports created successfully

Status Branch Result
✅ 8.15

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Aug 29, 2024
@kibanamachine @andrew-goldstein
[8.15] [Security Solution] [Attack discovery] Fixes a flakey jest test (
e454743 
#191736) (#191774)

# Backport

This will backport the following commits from `main` to `8.15`:
- [[Security Solution] [Attack discovery] Fixes a flakey jest test
(#191736)](#191736)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Andrew
Macri","email":"andrew.macri@elastic.co"},"sourceCommit":{"committedDate":"2024-08-29T21:29:52Z","message":"[Security
Solution] [Attack discovery] Fixes a flakey jest test (#191736)\n\nFixes
a flakey jest test that failed twice since it was introduced ~2 months
ago in [this pr](#186679), as
tracked by https://github.com/elastic/kibana/issues/189757\r\n\r\n### CI
and desk flake testing\r\n\r\nTo test the fix for fakyness, it ran
`10,000` times in CI, and locally, with a `100%` success rate. (The test
was wrapped in a loop to execute 10,000 times.)\r\n\r\n- Tested in CI
10,000 times via following Buildkite build:\r\n\r\n:green_heart: Build
Succeeded\r\n[Buildkite
Build](https://buildkite.com/elastic/kibana-pull-request/builds/230901)\r\n\r\nThe
10,000 passing test runs are included in the following (abbreviated)
build log output:\r\n\r\n```\r\nRan all test suites.\r\n$ node
scripts/jest --config
x-pack/plugins/security_solution/public/attack_discovery/jest.config.js\r\nactual
full command is:\r\nNODE_OPTIONS=\"--max-old-space-size=12288
--trace-warnings\" node ./scripts/jest
--config=\"x-pack/plugins/security_solution/public/attack_discovery/jest.config.js\"
--runInBand --coverage=false --passWithNoTests\r\n PASS
x-pack/plugins/security_solution/public/attack_discovery/pages/loading_callout/countdown/index.test.tsx
(742.054 s)\r\n# other passing tests...\r\n\r\nTest Suites: 54 passed,
54 total\r\nTests: 10297 passed, 10297 total\r\nSnapshots: 0
total\r\nTime: 962.106 s\r\n```\r\n\r\n- Tested locally 10,000 times, as
illustrated by the following console output:\r\n\r\n```\r\n ✓ renders
the expected the timer text - iteration 9996 (43 ms)\r\n ✓ renders the
expected the timer text - iteration 9997 (44 ms)\r\n ✓ renders the
expected the timer text - iteration 9998 (48 ms)\r\n ✓ renders the
expected the timer text - iteration 9999 (49 ms)\r\n ✓ renders the
expected the timer text - iteration 10000 (46 ms)\r\n\r\nTest Suites: 1
passed, 1 total\r\nTests: 10002 passed, 10002 total\r\nSnapshots: 0
total\r\nTime: 294.54 s\r\nRan all test suites matching
/x-pack\\/plugins\\/security_solution\\/public\\/attack_discovery\\/pages\\/loading_callout\\/countdown\\/index.test.tsx/i.\r\n```","sha":"e0f8eb1b6bf0f609b7b2b0b6a34216745a3be29f","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Security
Generative AI","v8.16.0","v8.15.1"],"title":"[Security Solution] [Attack
discovery] Fixes a flakey jest
test","number":191736,"url":"https://github.com/elastic/kibana/pull/191736","mergeCommit":{"message":"[Security
Solution] [Attack discovery] Fixes a flakey jest test (#191736)\n\nFixes
a flakey jest test that failed twice since it was introduced ~2 months
ago in [this pr](#186679), as
tracked by https://github.com/elastic/kibana/issues/189757\r\n\r\n### CI
and desk flake testing\r\n\r\nTo test the fix for fakyness, it ran
`10,000` times in CI, and locally, with a `100%` success rate. (The test
was wrapped in a loop to execute 10,000 times.)\r\n\r\n- Tested in CI
10,000 times via following Buildkite build:\r\n\r\n:green_heart: Build
Succeeded\r\n[Buildkite
Build](https://buildkite.com/elastic/kibana-pull-request/builds/230901)\r\n\r\nThe
10,000 passing test runs are included in the following (abbreviated)
build log output:\r\n\r\n```\r\nRan all test suites.\r\n$ node
scripts/jest --config
x-pack/plugins/security_solution/public/attack_discovery/jest.config.js\r\nactual
full command is:\r\nNODE_OPTIONS=\"--max-old-space-size=12288
--trace-warnings\" node ./scripts/jest
--config=\"x-pack/plugins/security_solution/public/attack_discovery/jest.config.js\"
--runInBand --coverage=false --passWithNoTests\r\n PASS
x-pack/plugins/security_solution/public/attack_discovery/pages/loading_callout/countdown/index.test.tsx
(742.054 s)\r\n# other passing tests...\r\n\r\nTest Suites: 54 passed,
54 total\r\nTests: 10297 passed, 10297 total\r\nSnapshots: 0
total\r\nTime: 962.106 s\r\n```\r\n\r\n- Tested locally 10,000 times, as
illustrated by the following console output:\r\n\r\n```\r\n ✓ renders
the expected the timer text - iteration 9996 (43 ms)\r\n ✓ renders the
expected the timer text - iteration 9997 (44 ms)\r\n ✓ renders the
expected the timer text - iteration 9998 (48 ms)\r\n ✓ renders the
expected the timer text - iteration 9999 (49 ms)\r\n ✓ renders the
expected the timer text - iteration 10000 (46 ms)\r\n\r\nTest Suites: 1
passed, 1 total\r\nTests: 10002 passed, 10002 total\r\nSnapshots: 0
total\r\nTime: 294.54 s\r\nRan all test suites matching
/x-pack\\/plugins\\/security_solution\\/public\\/attack_discovery\\/pages\\/loading_callout\\/countdown\\/index.test.tsx/i.\r\n```","sha":"e0f8eb1b6bf0f609b7b2b0b6a34216745a3be29f"}},"sourceBranch":"main","suggestedTargetBranches":["8.15"],"targetPullRequestStates":[{"branch":"main","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/191736","number":191736,"mergeCommit":{"message":"[Security
Solution] [Attack discovery] Fixes a flakey jest test (#191736)\n\nFixes
a flakey jest test that failed twice since it was introduced ~2 months
ago in [this pr](#186679), as
tracked by https://github.com/elastic/kibana/issues/189757\r\n\r\n### CI
and desk flake testing\r\n\r\nTo test the fix for fakyness, it ran
`10,000` times in CI, and locally, with a `100%` success rate. (The test
was wrapped in a loop to execute 10,000 times.)\r\n\r\n- Tested in CI
10,000 times via following Buildkite build:\r\n\r\n:green_heart: Build
Succeeded\r\n[Buildkite
Build](https://buildkite.com/elastic/kibana-pull-request/builds/230901)\r\n\r\nThe
10,000 passing test runs are included in the following (abbreviated)
build log output:\r\n\r\n```\r\nRan all test suites.\r\n$ node
scripts/jest --config
x-pack/plugins/security_solution/public/attack_discovery/jest.config.js\r\nactual
full command is:\r\nNODE_OPTIONS=\"--max-old-space-size=12288
--trace-warnings\" node ./scripts/jest
--config=\"x-pack/plugins/security_solution/public/attack_discovery/jest.config.js\"
--runInBand --coverage=false --passWithNoTests\r\n PASS
x-pack/plugins/security_solution/public/attack_discovery/pages/loading_callout/countdown/index.test.tsx
(742.054 s)\r\n# other passing tests...\r\n\r\nTest Suites: 54 passed,
54 total\r\nTests: 10297 passed, 10297 total\r\nSnapshots: 0
total\r\nTime: 962.106 s\r\n```\r\n\r\n- Tested locally 10,000 times, as
illustrated by the following console output:\r\n\r\n```\r\n ✓ renders
the expected the timer text - iteration 9996 (43 ms)\r\n ✓ renders the
expected the timer text - iteration 9997 (44 ms)\r\n ✓ renders the
expected the timer text - iteration 9998 (48 ms)\r\n ✓ renders the
expected the timer text - iteration 9999 (49 ms)\r\n ✓ renders the
expected the timer text - iteration 10000 (46 ms)\r\n\r\nTest Suites: 1
passed, 1 total\r\nTests: 10002 passed, 10002 total\r\nSnapshots: 0
total\r\nTime: 294.54 s\r\nRan all test suites matching
/x-pack\\/plugins\\/security_solution\\/public\\/attack_discovery\\/pages\\/loading_callout\\/countdown\\/index.test.tsx/i.\r\n```","sha":"e0f8eb1b6bf0f609b7b2b0b6a34216745a3be29f"}},{"branch":"8.15","label":"v8.15.1","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Andrew Macri <andrew.macri@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@spong spong spong approved these changes

Assignees

@andrew-goldstein andrew-goldstein

Labels

release_note:skip Skip the PR/issue when compiling release notes Team:Security Generative AI Security Generative AI v8.15.1 v8.16.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@andrew-goldstein @kibana-ci @kibanamachine @spong