Skip to content

[ci] run scout tests as on merge non-blocking pipeline#210425

Merged
dmlemeshko merged 2 commits intoelastic:mainfrom
dmlemeshko:ci/run-scout-tests-on-merge
Feb 11, 2025
Merged

[ci] run scout tests as on merge non-blocking pipeline#210425
dmlemeshko merged 2 commits intoelastic:mainfrom
dmlemeshko:ci/run-scout-tests-on-merge

Conversation

@dmlemeshko
Copy link
Copy Markdown
Contributor

@dmlemeshko dmlemeshko commented Feb 10, 2025 •
edited by kibanamachine
Loading

Summary

To collect more stats (run time & flakiness) we would like to run Scout tests in "non-blocking" way for on-merge pipeline.

I basically copied script and configuration from https://github.com/elastic/kibana/blob/9dc67bfd8836e8d3a2f0480d997b11d9e7e9106f/.buildkite/pipelines/pull_request/scout_ui_tests.yml

@dmlemeshko dmlemeshko self-assigned this Feb 10, 2025
@dmlemeshko dmlemeshko added test:scout release_note:skip Skip the PR/issue when compiling release notes backport:prev-minor v9.1.0 labels Feb 10, 2025
@dmlemeshko dmlemeshko requested a review from pheyos February 10, 2025 17:00
@dmlemeshko dmlemeshko added the Team:QA Platform QA t// label Feb 10, 2025
@dmlemeshko dmlemeshko marked this pull request as ready for review February 10, 2025 17:00
@dmlemeshko dmlemeshko requested a review from a team as a code owner February 10, 2025 17:00
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Feb 10, 2025

Pinging @elastic/appex-qa (Team:QA)

pheyos
pheyos approved these changes Feb 10, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@pheyos pheyos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dmlemeshko dmlemeshko added v9.0.0 v8.19.0 backport:version Backport to applied version labels and removed backport:prev-minor labels Feb 10, 2025
@dmlemeshko dmlemeshko enabled auto-merge (squash) February 11, 2025 13:41
@dmlemeshko dmlemeshko merged commit 745cbb7 into elastic:main Feb 11, 2025
1 check passed
@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Feb 11, 2025

Starting backport for target branches: 8.x, 9.0

https://github.com/elastic/kibana/actions/runs/13265058368

@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Feb 11, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #106 / APM API tests fleet/input_only_package.spec.ts basic no archive APM package policy input only package when ingesting events using the scoped api key the events can be seen on the Service Inventory Page
  • [job] [logs] Jest Tests #20 / useReplaceCustomField calls the api when invoked with the correct parameters

Metrics [docs]

✅ unchanged

History

cc @dmlemeshko

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 11, 2025
@dmlemeshko
[ci] run scout tests as on merge non-blocking pipeline (elastic#210425)
87401c8 
## Summary

To collect more stats (run time & flakiness) we would like to run Scout
tests in "non-blocking" way for on-merge pipeline.

I basically copied script and configuration from
https://github.com/elastic/kibana/blob/9dc67bfd8836e8d3a2f0480d997b11d9e7e9106f/.buildkite/pipelines/pull_request/scout_ui_tests.yml

(cherry picked from commit 745cbb7)
@kibanamachine kibanamachine mentioned this pull request Feb 11, 2025
Merged
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 11, 2025
@dmlemeshko
[ci] run scout tests as on merge non-blocking pipeline (elastic#210425)
a58d2fd 
## Summary

To collect more stats (run time & flakiness) we would like to run Scout
tests in "non-blocking" way for on-merge pipeline.

I basically copied script and configuration from
https://github.com/elastic/kibana/blob/9dc67bfd8836e8d3a2f0480d997b11d9e7e9106f/.buildkite/pipelines/pull_request/scout_ui_tests.yml

(cherry picked from commit 745cbb7)
@kibanamachine kibanamachine mentioned this pull request Feb 11, 2025
Merged
@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Feb 11, 2025

💚 All backports created successfully

Status Branch Result
✅ 8.x
✅ 9.0

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Feb 11, 2025
@kibanamachine @dmlemeshko
[9.0] [ci] run scout tests as on merge non-blocking pipeline (#210425) (
fcbc906 
#210591)

# Backport

This will backport the following commits from `main` to `9.0`:
- [[ci] run scout tests as on merge non-blocking pipeline
(#210425)](#210425)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2025-02-11T14:21:20Z","message":"[ci]
run scout tests as on merge non-blocking pipeline (#210425)\n\n##
Summary\r\n\r\nTo collect more stats (run time & flakiness) we would
like to run Scout\r\ntests in \"non-blocking\" way for on-merge
pipeline.\r\n\r\nI basically copied script and configuration
from\r\nhttps://github.com/elastic/kibana/blob/9dc67bfd8836e8d3a2f0480d997b11d9e7e9106f/.buildkite/pipelines/pull_request/scout_ui_tests.yml","sha":"745cbb7d8db5cb1f8e7810d018f01c032ece072c","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:QA","release_note:skip","v9.0.0","backport:version","test:scout","v9.1.0","v8.19.0"],"title":"[ci]
run scout tests as on merge non-blocking
pipeline","number":210425,"url":"https://github.com/elastic/kibana/pull/210425","mergeCommit":{"message":"[ci]
run scout tests as on merge non-blocking pipeline (#210425)\n\n##
Summary\r\n\r\nTo collect more stats (run time & flakiness) we would
like to run Scout\r\ntests in \"non-blocking\" way for on-merge
pipeline.\r\n\r\nI basically copied script and configuration
from\r\nhttps://github.com/elastic/kibana/blob/9dc67bfd8836e8d3a2f0480d997b11d9e7e9106f/.buildkite/pipelines/pull_request/scout_ui_tests.yml","sha":"745cbb7d8db5cb1f8e7810d018f01c032ece072c"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210425","number":210425,"mergeCommit":{"message":"[ci]
run scout tests as on merge non-blocking pipeline (#210425)\n\n##
Summary\r\n\r\nTo collect more stats (run time & flakiness) we would
like to run Scout\r\ntests in \"non-blocking\" way for on-merge
pipeline.\r\n\r\nI basically copied script and configuration
from\r\nhttps://github.com/elastic/kibana/blob/9dc67bfd8836e8d3a2f0480d997b11d9e7e9106f/.buildkite/pipelines/pull_request/scout_ui_tests.yml","sha":"745cbb7d8db5cb1f8e7810d018f01c032ece072c"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
@dmlemeshko dmlemeshko mentioned this pull request Feb 11, 2025
Merged
kibanamachine added a commit that referenced this pull request Feb 11, 2025
@kibanamachine @dmlemeshko
[8.x] [ci] run scout tests as on merge non-blocking pipeline (#210425) (
0746c40 
#210590)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[ci] run scout tests as on merge non-blocking pipeline
(#210425)](#210425)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2025-02-11T14:21:20Z","message":"[ci]
run scout tests as on merge non-blocking pipeline (#210425)\n\n##
Summary\r\n\r\nTo collect more stats (run time & flakiness) we would
like to run Scout\r\ntests in \"non-blocking\" way for on-merge
pipeline.\r\n\r\nI basically copied script and configuration
from\r\nhttps://github.com/elastic/kibana/blob/9dc67bfd8836e8d3a2f0480d997b11d9e7e9106f/.buildkite/pipelines/pull_request/scout_ui_tests.yml","sha":"745cbb7d8db5cb1f8e7810d018f01c032ece072c","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:QA","release_note:skip","v9.0.0","backport:version","test:scout","v9.1.0","v8.19.0"],"title":"[ci]
run scout tests as on merge non-blocking
pipeline","number":210425,"url":"https://github.com/elastic/kibana/pull/210425","mergeCommit":{"message":"[ci]
run scout tests as on merge non-blocking pipeline (#210425)\n\n##
Summary\r\n\r\nTo collect more stats (run time & flakiness) we would
like to run Scout\r\ntests in \"non-blocking\" way for on-merge
pipeline.\r\n\r\nI basically copied script and configuration
from\r\nhttps://github.com/elastic/kibana/blob/9dc67bfd8836e8d3a2f0480d997b11d9e7e9106f/.buildkite/pipelines/pull_request/scout_ui_tests.yml","sha":"745cbb7d8db5cb1f8e7810d018f01c032ece072c"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210425","number":210425,"mergeCommit":{"message":"[ci]
run scout tests as on merge non-blocking pipeline (#210425)\n\n##
Summary\r\n\r\nTo collect more stats (run time & flakiness) we would
like to run Scout\r\ntests in \"non-blocking\" way for on-merge
pipeline.\r\n\r\nI basically copied script and configuration
from\r\nhttps://github.com/elastic/kibana/blob/9dc67bfd8836e8d3a2f0480d997b11d9e7e9106f/.buildkite/pipelines/pull_request/scout_ui_tests.yml","sha":"745cbb7d8db5cb1f8e7810d018f01c032ece072c"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
mistic pushed a commit that referenced this pull request Feb 11, 2025
@dmlemeshko
[ci] enable Scout reporter for on-merge-unsupported-ftrs (#210627)
48992ed 
## Summary

In #210425 we added Scout UI tests to be run for on-merge pipeline, but
test results are not ingested by default.

This PR enables Scout reporter for that pipeline
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 11, 2025
@dmlemeshko
[ci] enable Scout reporter for on-merge-unsupported-ftrs (elastic#210627
c1f38ea 
)

## Summary

In elastic#210425 we added Scout UI tests to be run for on-merge pipeline, but
test results are not ingested by default.

This PR enables Scout reporter for that pipeline

(cherry picked from commit 48992ed)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 11, 2025
@dmlemeshko
[ci] enable Scout reporter for on-merge-unsupported-ftrs (elastic#210627
1fc7bf8 
)

## Summary

In elastic#210425 we added Scout UI tests to be run for on-merge pipeline, but
test results are not ingested by default.

This PR enables Scout reporter for that pipeline

(cherry picked from commit 48992ed)
kibanamachine added a commit that referenced this pull request Feb 11, 2025
@kibanamachine @dmlemeshko
[9.0] [ci] enable Scout reporter for on-merge-unsupported-ftrs (#210627…
43c2190 
…) (#210673)

# Backport

This will backport the following commits from `main` to `9.0`:
- [[ci] enable Scout reporter for on-merge-unsupported-ftrs
(#210627)](#210627)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2025-02-11T18:01:54Z","message":"[ci]
enable Scout reporter for on-merge-unsupported-ftrs (#210627)\n\n##
Summary\r\n\r\nIn #210425 we added Scout UI tests to be run for on-merge
pipeline, but\r\ntest results are not ingested by default.\r\n\r\nThis
PR enables Scout reporter for that
pipeline","sha":"48992ed9f6b78201884b28e1aa77794ea1e61336","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:version","test:scout","v9.1.0","v8.19.0"],"title":"[ci]
enable Scout reporter for
on-merge-unsupported-ftrs","number":210627,"url":"https://github.com/elastic/kibana/pull/210627","mergeCommit":{"message":"[ci]
enable Scout reporter for on-merge-unsupported-ftrs (#210627)\n\n##
Summary\r\n\r\nIn #210425 we added Scout UI tests to be run for on-merge
pipeline, but\r\ntest results are not ingested by default.\r\n\r\nThis
PR enables Scout reporter for that
pipeline","sha":"48992ed9f6b78201884b28e1aa77794ea1e61336"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210627","number":210627,"mergeCommit":{"message":"[ci]
enable Scout reporter for on-merge-unsupported-ftrs (#210627)\n\n##
Summary\r\n\r\nIn #210425 we added Scout UI tests to be run for on-merge
pipeline, but\r\ntest results are not ingested by default.\r\n\r\nThis
PR enables Scout reporter for that
pipeline","sha":"48992ed9f6b78201884b28e1aa77794ea1e61336"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
kibanamachine added a commit that referenced this pull request Feb 11, 2025
@kibanamachine @dmlemeshko
[8.x] [ci] enable Scout reporter for on-merge-unsupported-ftrs (#210627…
180f5d9 
…) (#210672)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[ci] enable Scout reporter for on-merge-unsupported-ftrs
(#210627)](#210627)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Dzmitry
Lemechko","email":"dzmitry.lemechko@elastic.co"},"sourceCommit":{"committedDate":"2025-02-11T18:01:54Z","message":"[ci]
enable Scout reporter for on-merge-unsupported-ftrs (#210627)\n\n##
Summary\r\n\r\nIn #210425 we added Scout UI tests to be run for on-merge
pipeline, but\r\ntest results are not ingested by default.\r\n\r\nThis
PR enables Scout reporter for that
pipeline","sha":"48992ed9f6b78201884b28e1aa77794ea1e61336","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","backport:version","test:scout","v9.1.0","v8.19.0"],"title":"[ci]
enable Scout reporter for
on-merge-unsupported-ftrs","number":210627,"url":"https://github.com/elastic/kibana/pull/210627","mergeCommit":{"message":"[ci]
enable Scout reporter for on-merge-unsupported-ftrs (#210627)\n\n##
Summary\r\n\r\nIn #210425 we added Scout UI tests to be run for on-merge
pipeline, but\r\ntest results are not ingested by default.\r\n\r\nThis
PR enables Scout reporter for that
pipeline","sha":"48992ed9f6b78201884b28e1aa77794ea1e61336"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/210627","number":210627,"mergeCommit":{"message":"[ci]
enable Scout reporter for on-merge-unsupported-ftrs (#210627)\n\n##
Summary\r\n\r\nIn #210425 we added Scout UI tests to be run for on-merge
pipeline, but\r\ntest results are not ingested by default.\r\n\r\nThis
PR enables Scout reporter for that
pipeline","sha":"48992ed9f6b78201884b28e1aa77794ea1e61336"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Dzmitry Lemechko <dzmitry.lemechko@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pheyos pheyos pheyos approved these changes

@Ikuni17 Ikuni17 Ikuni17 approved these changes

Assignees

@dmlemeshko dmlemeshko

Labels

backport:version Backport to applied version labels release_note:skip Skip the PR/issue when compiling release notes Team:QA Platform QA t// test:scout v8.19.0 v9.0.0 v9.1.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@dmlemeshko @elasticmachine @kibanamachine @pheyos @Ikuni17