[Spaces] Moved tests to agnostic setup by elena-shostak · Pull Request #200606 · elastic/kibana

elena-shostak · 2024-11-18T16:17:11Z

Summary

Moved space tests to deployment agnostic setup.

Checklist

Unit or functional tests were updated or added to match the most common scenarios

Closes: #194584

elena-shostak · 2024-11-18T16:17:19Z

/ci

elena-shostak · 2024-11-18T16:43:45Z

/ci

elena-shostak · 2024-11-18T17:46:29Z

/ci

elena-shostak · 2024-11-18T19:18:48Z

/ci

elena-shostak · 2024-11-18T20:07:24Z

/ci

elena-shostak · 2024-11-18T22:54:58Z

/ci

elena-shostak · 2024-11-19T07:59:42Z

/ci

elena-shostak · 2024-11-19T08:23:04Z

/ci

elena-shostak · 2024-11-19T12:06:54Z

/ci

elena-shostak · 2024-11-19T12:37:46Z

/ci

elena-shostak · 2024-11-19T15:01:45Z

/ci

kibanamachine · 2024-12-27T13:29:10Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7634

[❌] x-pack/test/spaces_api_integration/deployment_agnostic/stateful.config_trial.ts: 0/10 tests passed.

see run history

kibanamachine · 2024-12-27T17:36:41Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7635

[❌] x-pack/test/spaces_api_integration/deployment_agnostic/stateful.config_trial.ts: 0/1 tests passed.
[❌] x-pack/test/spaces_api_integration/deployment_agnostic/stateful.copy_to_space.config_basic.ts: 0/1 tests passed.
[❌] x-pack/test/spaces_api_integration/deployment_agnostic/stateful.copy_to_space.config_trial.ts: 0/1 tests passed.
[❌] x-pack/test/spaces_api_integration/deployment_agnostic/spaces_only/config.ts: 0/1 tests passed.
[❌] x-pack/test/spaces_api_integration/security_and_spaces/config_basic.ts: 0/1 tests passed.
[❌] x-pack/test/spaces_api_integration/security_and_spaces/config_trial.ts: 0/1 tests passed.

see run history

elena-shostak · 2024-12-27T17:56:29Z

/ci

elena-shostak · 2024-12-27T20:41:35Z

/сi

elena-shostak · 2025-01-03T14:01:15Z

x-pack/test/spaces_api_integration/deployment_agnostic/services/role_scoped_supertest.ts

Important
We have a shared test suite for basic/trial licenses, which includes tests for both authorized and unauthorized users. Decoupling these tests is a complex and time-consuming task that requires careful consideration to avoid ending up with a significant amount of duplicated test code for different licenses.

Our custom service for RoleScopedSupertest takes it into account license and user passed/not passed to adjust needed supertest accordingly.

elena-shostak · 2025-01-03T14:04:18Z

x-pack/test/spaces_api_integration/common/fixtures/kbn_archiver/default_space.json

Note
sharedtype changed to event-annotation-group SO
isolatedtype changed to url SO

I failed to change sharedtype to index-pattern SO type since I was getting a lot of missing references errors, probably it was due to the fact that index-pattern was migrated to the multiple namespace from 8.0.0

Interesting. I wonder if they were just missing some information. There is an index pattern in the data file, and it contains this:

"migrationVersion": { "index-pattern": "8.0.0" },

elena-shostak · 2025-01-03T15:05:19Z

src/dev/eslint/types.eslint.config.template.cjs

agent.auth(this.user.username!, this.user.password!);

Needed to suppress the eslint error for this line in our custom test service, the promise is awaited in the tests itself and adding a catch block breaks some of the test cases

eslint-disable line gets removed because this rule runs dynamically

x-pack/test/spaces_api_integration/common/services/test_data_loader.ts

elena-shostak · 2025-01-07T13:55:44Z

x-pack/test/spaces_api_integration/common/services/test_data_loader.ts

Note
Decided to not add complexity to the custom roleScopedSupertest to accommodate this use case, so made it in place.

elena-shostak · 2025-01-07T13:58:29Z

x-pack/test/spaces_api_integration/deployment_agnostic/security_and_spaces/apis/get_all.ts

Note
There are no machine_learning_admin, machine_learning_user and monitoring_user roles in serverless.
We could rather move it to it's own test only for stateful or define custom roles, would like to have suggestions on this one 🙂

…-fix'

elena-shostak · 2025-01-21T10:45:11Z

@jeramysoucy won't this.tags(['skipMKI']); work for us?

Yes, that should work. Are we using this? I couldn't find it in the test suite.

Added in 23bc295

wayneseymour

One question...

wayneseymour · 2025-01-21T11:42:54Z

..._api_integration/deployment_agnostic/security_and_spaces/apis/copy_to_space/copy_to_space.ts

  } = copyToSpaceTestSuiteFactory(context);

  describe('copy to spaces', () => {
+    this.tags('skipMKI');


Hello there, what's the reason for these 4 skip on mki tags?
I ask as this PR is about being agnostic but it looks liek 4 suites are not agnostic.
Is there an issue on this already?

This test suite uses custom roles and custom roles cannot be provisioned for MKI in the FTR yet.
That was agreed that we would disable it for now

Ok, so can we have a link to the issue for this just above this line?
If not, I'm afraid it will never be followed-up and remain skipped.

We've way too many test gaps already :(

I will link the issue in the comment

Added in 2bd07ce

@wayneseymour is there anything else? I'm waiting only for your approval 🙂

jeramysoucy

LGTM! Thanks for the updates!

wayneseymour

thanks!

elena-shostak · 2025-01-23T12:06:10Z

@elasticmachine merge upstream

elasticmachine · 2025-01-23T14:02:51Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: dc6a47c

Failed CI Steps

Jest Tests #19

Test Failures

[job] [logs] Jest Tests #19 / useGetCaseConnectors shows a toast error message when an error occurs in the response

Metrics [docs]

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`@kbn/test-suites-xpack`	743	727	-16

Total ESLint disabled count

id	before	after	diff
`@kbn/test-suites-xpack`	768	752	-16

History

💚 Build #268852 succeeded 2bd07ce
💔 Build #268786 failed 2064f15
💔 Build #268763 failed 12a11e9
💚 Build #268203 succeeded 5e7de03da7ff45d65821d4778adfdaf4fee1d5d1
💛 Build #264772 was flaky e8c8bfc2455d26aa6c89b8c6bf97380ed124e92c
💔 Build #264710 failed 4f21b9ecad9f82bb3aed86e4fac032fac004b5c4

## Summary Moved space tests to deployment agnostic setup. ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios __Closes: https://github.com/elastic/kibana/issues/194584__ --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

elena-shostak changed the title ~~Moved tests to agnostic setup~~ [Spaces] Moved tests to agnostic setup Nov 19, 2024

elena-shostak added Team:Security Platform Security: Auth, Users, Roles, Spaces, Audit Logging, etc t// Feature:Security/Spaces Platform Security - Spaces feature FTR release_note:skip Skip the PR/issue when compiling release notes labels Nov 19, 2024

elena-shostak force-pushed the space-deployment-agnostic branch from 8fedf64 to f6def7f Compare November 19, 2024 14:11

azasypkin self-requested a review November 19, 2024 16:02

elena-shostak force-pushed the space-deployment-agnostic branch 2 times, most recently from 2df08a0 to 1464f66 Compare December 27, 2024 12:55

elena-shostak force-pushed the space-deployment-agnostic branch 2 times, most recently from 06a8199 to e82dcfe Compare January 3, 2025 11:21

elena-shostak commented Jan 3, 2025

View reviewed changes

azasypkin reviewed Jan 6, 2025

View reviewed changes

x-pack/test/spaces_api_integration/common/services/test_data_loader.ts Outdated Show resolved Hide resolved

elena-shostak commented Jan 7, 2025

View reviewed changes

elena-shostak and others added 12 commits January 21, 2025 11:28

Fixes

f089b02

Lint fix

008ed23

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

0e6c69c

…-fix'

Fixup

6e362b3

Serverless config

c7c91eb

Fixes

9cd8a5d

Fixes

8206419

Fixes

f289e05

Service fix

670e1d1

Fixes

703f1e5

Fixes

094aaee

Added skipInMKI tag and dropped unnecessary builtin roles from check

23bc295

elena-shostak force-pushed the space-deployment-agnostic branch from 5e7de03 to 23bc295 Compare January 21, 2025 10:43

copy_to_space serverless config

12a11e9

wayneseymour suggested changes Jan 21, 2025

View reviewed changes

elena-shostak added 2 commits January 21, 2025 13:21

Fix

2064f15

Fixes

2bd07ce

jeramysoucy approved these changes Jan 21, 2025

View reviewed changes

wayneseymour self-requested a review January 23, 2025 12:04

wayneseymour approved these changes Jan 23, 2025

View reviewed changes

Merge branch 'main' into space-deployment-agnostic

dc6a47c

elena-shostak merged commit ac5366a into elastic:main Jan 23, 2025
8 checks passed

kibanamachine added the v9.0.0 label Jan 23, 2025

csr mentioned this pull request Apr 15, 2025

[8.x] [FTR] unify custom role name with Scout (#217882) #218260

Merged

Conversation

elena-shostak commented Nov 18, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Checklist

Uh oh!

elena-shostak commented Nov 18, 2024

Uh oh!

elena-shostak commented Nov 18, 2024

Uh oh!

elena-shostak commented Nov 18, 2024

Uh oh!

elena-shostak commented Nov 18, 2024

Uh oh!

elena-shostak commented Nov 18, 2024

Uh oh!

elena-shostak commented Nov 18, 2024

Uh oh!

elena-shostak commented Nov 19, 2024

Uh oh!

elena-shostak commented Nov 19, 2024

Uh oh!

elena-shostak commented Nov 19, 2024

Uh oh!

elena-shostak commented Nov 19, 2024

Uh oh!

elena-shostak commented Nov 19, 2024

Uh oh!

kibanamachine commented Dec 27, 2024

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7634

Uh oh!

kibanamachine commented Dec 27, 2024

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#7635

Uh oh!

elena-shostak commented Dec 27, 2024

Uh oh!

elena-shostak commented Dec 27, 2024

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elena-shostak Jan 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elena-shostak Jan 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elena-shostak commented Jan 21, 2025

Uh oh!

wayneseymour left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeramysoucy left a comment

Choose a reason for hiding this comment

Uh oh!

wayneseymour left a comment

Choose a reason for hiding this comment

Uh oh!

elena-shostak commented Nov 18, 2024 •

edited

Loading

elena-shostak Jan 3, 2025 •

edited

Loading

elena-shostak Jan 7, 2025 •

edited

Loading