Skip to content

[FTR] KbnClientSavedObjects improvements #149582

Merged
pgayvallet merged 8 commits intoelastic:mainfrom
pgayvallet:kbn-148412-follow-up-ftr-bulk-delete
Jan 30, 2023
Merged

[FTR] KbnClientSavedObjects improvements #149582
pgayvallet merged 8 commits intoelastic:mainfrom
pgayvallet:kbn-148412-follow-up-ftr-bulk-delete

Conversation

@pgayvallet
Copy link
Contributor

@pgayvallet pgayvallet commented Jan 26, 2023

Summary

Follow-up of #149188

  • Use the bulkDelete API for KbnClientSavedObjects.bulkDelete
  • Create a dedicated /_clean endpoint for KbnClientSavedObjects.clean and KbnClientSavedObjects.cleanStandardList

@pgayvallet pgayvallet added Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// release_note:skip Skip the PR/issue when compiling release notes Feature:Functional Testing v8.7.0 labels Jan 26, 2023
@pgayvallet pgayvallet marked this pull request as ready for review January 27, 2023 09:33
@pgayvallet pgayvallet requested a review from a team as a code owner January 27, 2023 09:33
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-core (Team:Core)

@pgayvallet pgayvallet requested a review from a team January 27, 2023 09:34
Copy link
Member

@afharo afharo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

import { FtrProviderContext } from '../services';
import { createData, createTestSpaces, deleteData, deleteTestSpaces } from './test_utils';

// eslint-disable-next-line import/no-default-export
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we may want to edit .eslintrc to avoid these exceptions:

'x-pack/test/*/{tests,test_suites,apis,apps}/**/*',

];

const newOptions = { types, space: options?.space };
const newOptions = { types: STANDARD_LIST_TYPES, space: options?.space };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this! It makes it so much easier to track what the test server is doing 😄

Copy link
Contributor

@TinaHeiligers TinaHeiligers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

@pgayvallet
Copy link
Contributor Author

@elasticmachine merge upstream

@pgayvallet pgayvallet enabled auto-merge (squash) January 30, 2023 14:54
@kibana-ci
Copy link

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@pgayvallet pgayvallet merged commit e70fcea into elastic:main Jan 30, 2023
@kibanamachine kibanamachine added the backport:skip This PR does not require backporting label Jan 30, 2023
nreese added a commit that referenced this pull request Jan 30, 2023
…dashboard/group4/dashboard_empty·ts (#149889)

Fixes #149256

Flaky test runner
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1824

Fix in #149582. `await
kibanaServer.importExport.unload(kbnDirectory);` is getting the `error
ECONNRESET`. Underlining cause determined to be @pgayvallet "Yeah, so
apparently it's related to the fact that
kibanaServer.importExport.unload calls bulkDelete with the summary from
the import file, which is not a valid SO. Not sure why it doesn't
constantly crashes on master though, as my changes in
#149582 makes it quite obvious"
nreese added a commit that referenced this pull request Jan 31, 2023
…earch_sessions_integration/tests/apps/dashboard/async_search (#149977)

Fixes #103043

Flaky test runner
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1844

Failure in saved object client bulk delete. Resolved by
#149582
```
[00:00:11]       │ debg Cleaning all saved objects { space: undefined }
[00:00:11]       │ info [o.e.c.m.MetadataMappingService] [ftr] [.kibana_8.7.0_001/0Buw8xPkRQi22GGDb-S18g] update_mapping [_doc]
[00:00:11]       │ info deleting batch of 104 objects
[00:00:12]       │ proc [kibana] {"log.level":"info","@timestamp":"2023-01-24T21:52:20.336Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"93951b44b2dfb2b8890fed26a1ce2fdd\"}"}
[00:00:13]       │ERROR [DELETE http://elastic:changeme@localhost:5620/api/saved_objects/epm-packages-assets/f104eb71-d6fe-5506-a359-2770af4e2746?force=true] request failed (attempt=1/5): read ECONNRESET
[00:00:13]       │ info Taking screenshot "/var/lib/buildkite-agent/builds/kb-n2-4-spot-24a3e2a91102e68e/elastic/kibana-on-merge/kibana/x-pack/test/functional/screenshots/failure/Dashboard before all hook in Dashboard-33b135acce0aa45337f463568919cb7166a26f0546efcb90d3a789017d5338f2.png"
[00:00:13]       │ info Current URL is: data:,
[00:00:13]       │ info Saving page source to: /var/lib/buildkite-agent/builds/kb-n2-4-spot-24a3e2a91102e68e/elastic/kibana-on-merge/kibana/x-pack/test/functional/failure_debug/html/Dashboard before all hook in Dashboard-33b135acce0aa45337f463568919cb7166a26f0546efcb90d3a789017d5338f2.html
[00:00:13]       └- ✖ fail: Dashboard "before all" hook in "Dashboard"
[00:00:13]       │      Error: 400 resp: ''
[00:00:13]       │       at createFailError (dev_cli_errors.ts:27:24)
[00:00:13]       │       at kbn_client_saved_objects.ts:284:32
[00:00:13]       │       at processTicksAndRejections (node:internal/process/task_queues:95:5)
[00:00:13]       │       at kbn_client_saved_objects.ts:88:50
[00:00:13]       │ 
[00:00:13]       │ 
```
kqualters-elastic pushed a commit to kqualters-elastic/kibana that referenced this pull request Feb 6, 2023
## Summary

Follow-up of elastic#149188


- Use the bulkDelete API for `KbnClientSavedObjects.bulkDelete`
- Create a dedicated `/_clean` endpoint for
`KbnClientSavedObjects.clean` and
`KbnClientSavedObjects.cleanStandardList`

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
kqualters-elastic pushed a commit to kqualters-elastic/kibana that referenced this pull request Feb 6, 2023
…dashboard/group4/dashboard_empty·ts (elastic#149889)

Fixes elastic#149256

Flaky test runner
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1824

Fix in elastic#149582. `await
kibanaServer.importExport.unload(kbnDirectory);` is getting the `error
ECONNRESET`. Underlining cause determined to be @pgayvallet "Yeah, so
apparently it's related to the fact that
kibanaServer.importExport.unload calls bulkDelete with the summary from
the import file, which is not a valid SO. Not sure why it doesn't
constantly crashes on master though, as my changes in
elastic#149582 makes it quite obvious"
kqualters-elastic pushed a commit to kqualters-elastic/kibana that referenced this pull request Feb 6, 2023
…earch_sessions_integration/tests/apps/dashboard/async_search (elastic#149977)

Fixes elastic#103043

Flaky test runner
https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1844

Failure in saved object client bulk delete. Resolved by
elastic#149582
```
[00:00:11]       │ debg Cleaning all saved objects { space: undefined }
[00:00:11]       │ info [o.e.c.m.MetadataMappingService] [ftr] [.kibana_8.7.0_001/0Buw8xPkRQi22GGDb-S18g] update_mapping [_doc]
[00:00:11]       │ info deleting batch of 104 objects
[00:00:12]       │ proc [kibana] {"log.level":"info","@timestamp":"2023-01-24T21:52:20.336Z","log":{"logger":"elastic-apm-node"},"ecs":{"version":"1.6.0"},"message":"Sending error to Elastic APM: {\"id\":\"93951b44b2dfb2b8890fed26a1ce2fdd\"}"}
[00:00:13]       │ERROR [DELETE http://elastic:changeme@localhost:5620/api/saved_objects/epm-packages-assets/f104eb71-d6fe-5506-a359-2770af4e2746?force=true] request failed (attempt=1/5): read ECONNRESET
[00:00:13]       │ info Taking screenshot "/var/lib/buildkite-agent/builds/kb-n2-4-spot-24a3e2a91102e68e/elastic/kibana-on-merge/kibana/x-pack/test/functional/screenshots/failure/Dashboard before all hook in Dashboard-33b135acce0aa45337f463568919cb7166a26f0546efcb90d3a789017d5338f2.png"
[00:00:13]       │ info Current URL is: data:,
[00:00:13]       │ info Saving page source to: /var/lib/buildkite-agent/builds/kb-n2-4-spot-24a3e2a91102e68e/elastic/kibana-on-merge/kibana/x-pack/test/functional/failure_debug/html/Dashboard before all hook in Dashboard-33b135acce0aa45337f463568919cb7166a26f0546efcb90d3a789017d5338f2.html
[00:00:13]       └- ✖ fail: Dashboard "before all" hook in "Dashboard"
[00:00:13]       │      Error: 400 resp: ''
[00:00:13]       │       at createFailError (dev_cli_errors.ts:27:24)
[00:00:13]       │       at kbn_client_saved_objects.ts:284:32
[00:00:13]       │       at processTicksAndRejections (node:internal/process/task_queues:95:5)
[00:00:13]       │       at kbn_client_saved_objects.ts:88:50
[00:00:13]       │ 
[00:00:13]       │ 
```
jeramysoucy added a commit that referenced this pull request Feb 9, 2023
…50492)

closes #86545
closes #149544
closes #149547
closes #149563 

Fixed by PR #149582

- [Flaky test runner on
x-pack/test/spaces_api_integration/security_and_spaces](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1880)
- [Flaky test runner on
x-pack/test/saved_object_api_integration/security_and_spaces](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1888)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Feb 9, 2023
…astic#150492)

closes elastic#86545
closes elastic#149544
closes elastic#149547
closes elastic#149563

Fixed by PR elastic#149582

- [Flaky test runner on
x-pack/test/spaces_api_integration/security_and_spaces](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1880)
- [Flaky test runner on
x-pack/test/saved_object_api_integration/security_and_spaces](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1888)

(cherry picked from commit c0c023a)
kibanamachine added a commit that referenced this pull request Feb 9, 2023
…ts (#150492) (#150724)

# Backport

This will backport the following commits from `main` to `8.7`:
- [[Flaky Test] Unskips 'single-namespace types' copy to space tests
(#150492)](#150492)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Jeramy
Soucy","email":"jeramy.soucy@elastic.co"},"sourceCommit":{"committedDate":"2023-02-09T15:00:02Z","message":"[Flaky
Test] Unskips 'single-namespace types' copy to space tests
(#150492)\n\ncloses #86545\r\ncloses #149544\r\ncloses #149547\r\ncloses
#149563 \r\n\r\nFixed by PR #149582\r\n\r\n- [Flaky test runner
on\r\nx-pack/test/spaces_api_integration/security_and_spaces](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1880)\r\n-
[Flaky test runner
on\r\nx-pack/test/saved_object_api_integration/security_and_spaces](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1888)","sha":"c0c023aaa6bf977a8a7a357998d612f04ea2f024","branchLabelMapping":{"^v8.8.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Team:Security","release_note:skip","test-failure-flaky","v8.7.0","v8.8.0"],"number":150492,"url":"https://github.com/elastic/kibana/pull/150492","mergeCommit":{"message":"[Flaky
Test] Unskips 'single-namespace types' copy to space tests
(#150492)\n\ncloses #86545\r\ncloses #149544\r\ncloses #149547\r\ncloses
#149563 \r\n\r\nFixed by PR #149582\r\n\r\n- [Flaky test runner
on\r\nx-pack/test/spaces_api_integration/security_and_spaces](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1880)\r\n-
[Flaky test runner
on\r\nx-pack/test/saved_object_api_integration/security_and_spaces](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1888)","sha":"c0c023aaa6bf977a8a7a357998d612f04ea2f024"}},"sourceBranch":"main","suggestedTargetBranches":["8.7"],"targetPullRequestStates":[{"branch":"8.7","label":"v8.7.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.8.0","labelRegex":"^v8.8.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/150492","number":150492,"mergeCommit":{"message":"[Flaky
Test] Unskips 'single-namespace types' copy to space tests
(#150492)\n\ncloses #86545\r\ncloses #149544\r\ncloses #149547\r\ncloses
#149563 \r\n\r\nFixed by PR #149582\r\n\r\n- [Flaky test runner
on\r\nx-pack/test/spaces_api_integration/security_and_spaces](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1880)\r\n-
[Flaky test runner
on\r\nx-pack/test/saved_object_api_integration/security_and_spaces](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/1888)","sha":"c0c023aaa6bf977a8a7a357998d612f04ea2f024"}}]}]
BACKPORT-->

Co-authored-by: Jeramy Soucy <jeramy.soucy@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting Feature:Functional Testing release_note:skip Skip the PR/issue when compiling release notes Team:Core Platform Core services: plugins, logging, config, saved objects, http, ES client, i18n, etc t// v8.7.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants