Skip to content

[Defend Workflows] Enable FTRs for serverless using tags#169946

Merged
gergoabraham merged 23 commits intoelastic:mainfrom
gergoabraham:serverless-tags-for-security-solution-ftrs
Nov 1, 2023
Merged

[Defend Workflows] Enable FTRs for serverless using tags#169946
gergoabraham merged 23 commits intoelastic:mainfrom
gergoabraham:serverless-tags-for-security-solution-ftrs

Conversation

@gergoabraham
Copy link
Contributor

@gergoabraham gergoabraham commented Oct 26, 2023

Summary

This PR enables most of our FTR tests in x-pack/test/security_solution_endpoint/ to run against both ESS and Serverless, using the following tags:

  • @ess
  • @serverless
  • @brokenInServerless
  • @skipInServerless
  • @skipInEss

The ones that are not enabled are the ones that use different users/roles.

Usage

describe('This is the suite.', function() {
    // a custom function wraps the `this.tags()` function to provide type safety
    targetTags(this, ['@ess', '@serverless', '@skipInServerless']);
})

Note
Only describe() blocks can be tagged.

@gergoabraham gergoabraham added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting Team:Defend Workflows “EDR Workflows” sub-team of Security Solution labels Oct 26, 2023
@gergoabraham gergoabraham self-assigned this Oct 26, 2023
@gergoabraham gergoabraham force-pushed the serverless-tags-for-security-solution-ftrs branch from c8a5b8f to 6a2fd4c Compare October 30, 2023 14:40
@gergoabraham gergoabraham marked this pull request as ready for review October 30, 2023 14:43
@gergoabraham gergoabraham requested review from a team as code owners October 30, 2023 14:43
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

@gergoabraham
Copy link
Contributor Author

@elasticmachine merge upstream

@jbudz jbudz requested a review from a team October 31, 2023 12:43
Copy link
Contributor

@jbudz jbudz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I added appex-qa as an optional reviewer for the discussion on #169946 (comment).

I know we're running tests with a filter on the skip mki tag, but I believe it's only for certain suites. Want to make @elastic/appex-qa aware of the skipInServerless tag too.

@kibana-ci
Copy link

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @gergoabraham

@gergoabraham gergoabraham merged commit bc90e56 into elastic:main Nov 1, 2023
@gergoabraham gergoabraham deleted the serverless-tags-for-security-solution-ftrs branch November 1, 2023 13:31
delanni pushed a commit to delanni/kibana that referenced this pull request Nov 6, 2023
)

## Summary

This PR enables **most of** our FTR tests in
`x-pack/test/security_solution_endpoint/` to run against both ESS and
Serverless, using the following tags:
- `@ess`
- `@serverless`
- `@brokenInServerless`
- `@skipInServerless`
- `@skipInEss`

The ones that are not enabled are the ones that use different
users/roles.


### Usage
```ts
describe('This is the suite.', function() {
    // a custom function wraps the `this.tags()` function to provide type safety
    targetTags(this, ['@ess', '@serverless', '@skipInServerless']);
})
```
> **Note**
> Only `describe()` blocks can be tagged.

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
gergoabraham added a commit that referenced this pull request Nov 9, 2023
…70418)

## Summary

With a similar solution as #169946, almost all of our FTR API tests
(`x-pack/test/security_solution_endpoint_api_int/`) can now run against
serverless, using `suiteTags` under the hood.

## Usage
```ts
describe('This is the suite.', function() {
    // a custom function wraps the `this.tags()` function to provide type safety
    targetTags(this, ['@ess', '@serverless', '@skipInServerless']);
})
```

> **Note**
> Only `describe()` blocks can be tagged.

## Changes:

- serverless config is added for endpoint FTR API tests:
`x-pack/test/security_solution_endpoint_api_int/serverless.config.ts`
- roles are created only when running against ESS. when running against
serverless, a subset of users/roles already exist (see [this
list](https://github.com/elastic/kibana/pull/170418/files#diff-5aaeaeaedad4321151d5388437084b27f271a6254e95f0352c9a10c3126eddc8R54))
- tests that use roles that doesn't exist on serverless
(`artifact_read_privileges`, `hunter`) are therefore skipped against
serverless
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Defend Workflows “EDR Workflows” sub-team of Security Solution v8.12.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants