Setup E2E against Serverless ES, Kibana, Fleet server standalone and Elastic endpoint agent in VM by patrykkopycinski · Pull Request #167720 · elastic/kibana

patrykkopycinski · 2023-09-30T16:36:12Z

Summary

Run Defend Workflows Cypress E2E against Serverless stack, similar to #165415

…cypress

…cypress # Conflicts: # x-pack/test/security_solution_cypress/cypress/support/old_headless.ts # x-pack/test_serverless/functional/test_suites/security/cypress/cypress.config.ts # yarn.lock

…cypress # Conflicts: # package.json # packages/kbn-management/settings/components/field_row/__stories__/common.tsx # packages/kbn-management/settings/components/field_row/description/deprecation.test.tsx # x-pack/plugins/fleet/package.json # x-pack/plugins/threat_intelligence/package.json # x-pack/test_serverless/functional/test_suites/observability/cypress/package.json # x-pack/test_serverless/functional/test_suites/security/cypress/package.json # yarn.lock

…-ref HEAD~1..HEAD --fix'

tomsonpl

:elasticheart:

x-pack/plugins/security_solution/public/management/cypress/e2e/endpoint_list/endpoints.cy.ts

paul-tavares

Left some comments, but ok to merge 👍

...y_solution/public/management/cypress/e2e/response_actions/response_console_mocked_data.cy.ts

paul-tavares · 2023-10-02T13:44:41Z

x-pack/plugins/security_solution/scripts/endpoint/common/stack_services.ts

-    return u.toString();
-  })();
-
+export const waitForKibana = async (kbnClient: KbnClient): Promise<void> => {


can you edit the @param above in the jsdocs to match the new param name

FYI: I seem to being able to use kbnClient for some reason when I initially added this method, but don't recall what that was now. Did you test this code here to ensure it works? (ex. ensuring that the pRetry() below is executes the callback at least twice)

yes, I did, the previous solution was failing due to the nodeFetch missing cert for SSL endpoints

x-pack/plugins/security_solution/scripts/endpoint/common/stack_services.ts

…ykkopycinski/kibana into feat/dw-serverless-endpoint-cypress

maximpn

@patrykkopycinski thank you for making it possible to run Fleet server standalone in Servelress 🙏

I'm good with the changes, left some minor comments.

maximpn · 2023-10-03T19:21:34Z

packages/kbn-es/src/utils/get_localhost_real_ip.ts

+
+export const getLocalhostRealIp = (): string | undefined => {
+  for (const netInterfaceList of Object.values(networkInterfaces())) {
+    if (netInterfaceList) {


nit: this could be simplified for improved readability

Suggested change

if (netInterfaceList) {

const address = netInterfaceList?.find(

(networkInterface) =>

networkInterface.family === 'IPv4' &&

networkInterface.internal === false &&

networkInterface.address

)?.address;

if (address) {

return address;

}

maximpn · 2023-10-03T19:26:28Z

packages/kbn-es/src/utils/docker.ts

+  const realLocalhostIp = getLocalhostRealIp();
+
+  if (realLocalhostIp) {
+    value.push('-p', `${realLocalhostIp}:${port}:${port}`);


I'm curious why it was necessary to add real address while it wasn't necessary in #165415?

because in previous case all the containers are running on the Docker and in this case we need to be able to reach the ES docker container from the VM running via Vagrant on the host machine

maximpn · 2023-10-03T19:32:31Z

x-pack/plugins/security_solution/scripts/run_cypress/parallel.ts


      const getEsPort = <T>(): T | number => {
+        if (isOpen) {
+          return 9220;


nit: Avoid hardcoded values. Is there a chance the port changes and this file must be updated as well?

nope, 9220 is the default port we are using in our FTR tests

maximpn · 2023-10-03T19:35:07Z

...lugins/security_solution/public/management/cypress/e2e/artifacts/artifacts_mocked_data.cy.ts

 };

-describe('Artifacts pages', { tags: '@ess' }, () => {
+describe('Artifacts pages', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {


Is the test really broken in Serverless or you wanna just skip it for now? @skipInServerless tag could be a better choice, see here

to me, it seems both tags are doing the same, maybe we could simplify and just have one tag for both use cases?

maximpn · 2023-10-03T19:35:10Z

...lugins/security_solution/public/management/cypress/e2e/automated_response_actions/form.cy.ts

 import { login, ROLE } from '../../tasks/login';

-describe('Form', { tags: '@ess' }, () => {
+describe('Form', { tags: ['@ess', '@serverless', '@brokenInServerless'] }, () => {


Is the test really broken in Serverless or you wanna just skip it for now? @skipInServerless tag could be a better choice, see here

jbudz

Small comment, otherwise kbn-test, kbn-es, and ftr_configs.yml LGTM

jbudz · 2023-10-04T00:07:06Z

packages/kbn-es/src/utils/docker.ts


 export interface ServerlessOptions extends EsClusterExecOptions, BaseOptions {
+  /** Publish ES docker container on additional host IP */
+  host?: string;


Can we add this to the cli --help to https://github.com/elastic/kibana/blob/main/packages/kbn-es/src/cli_commands/serverless.ts#L37?

dmlemeshko · 2023-10-04T08:18:05Z

packages/kbn-es/src/utils/docker.ts

+  if ((options as ServerlessOptions).host) {
+    value.push('-p', `${(options as ServerlessOptions).host}:${port}:${port}`);
+  }


Just curious if this change needed for Cypress tests or as a general capability to improve developer UX?
If we are keeping the function name resolvePort as is, could you update the description to have few words about host usage and return value.

this change is needed to improve dev UX, mostly in case we want to set elastic-agent on a dedicated VM outside of the Docker network

@dmlemeshko updated description, please let me know if that works

dmlemeshko

kbn-test changes LGTM

kibana-ci · 2023-10-04T09:53:45Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💔 Build #165166 failed d471bee
💔 Build #165152 failed be6b0e5
💚 Build #164863 succeeded e24a8b3
💚 Build #164711 succeeded ec4aaa7
💔 Build #164675 failed 7adac99
💔 Build #164254 failed 9a62a62

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @patrykkopycinski

patrykkopycinski and others added 30 commits July 22, 2023 15:10

Bump cypress@12.17.2

58871f7

fix

fa16f43

test

cc22a07

fix

62001bd

Merge branch 'main' into chore/renovate-cypress

0eb2ccc

Merge branch 'main' into chore/renovate-cypress

30be411

Merge branch 'main' into chore/renovate-cypress

421188e

Merge branch 'main' into chore/renovate-cypress

d6105f8

Merge branch 'main' into chore/renovate-cypress

97f8211

Merge branch 'main' into chore/renovate-cypress

c73a905

Merge branch 'main' of github.com:elastic/kibana into chore/renovate-…

72439fb

…cypress

bump

b331b7f

test

dfbaf91

Merge branch 'main' of github.com:elastic/kibana into chore/renovate-…

a7af0a8

…cypress # Conflicts: # x-pack/test/security_solution_cypress/cypress/support/old_headless.ts # x-pack/test_serverless/functional/test_suites/security/cypress/cypress.config.ts # yarn.lock

fix

0e5d548

fix

902e647

fix

5c1c5be

Merge branch 'main' into chore/renovate-cypress

1e34539

Merge branch 'main' into chore/renovate-cypress

59b921f

Update cypress_ci.config.ts

797b5e6

[CI] Auto-commit changed files from 'node scripts/precommit_hook.js -…

1a1167e

…-ref HEAD~1..HEAD --fix'

Update index.ts

a21cb1a

Merge branch 'main' into chore/renovate-cypress

d0dc156

Merge branch 'main' into chore/renovate-cypress

d009b64

Merge branch 'main' into chore/renovate-cypress

70af4d9

fix

170fe9e

changing the APM browser to electron

2dac722

Merge branch 'main' into chore/renovate-cypress

955b554

skipping open timeline test

9ad98d6

patrykkopycinski requested review from a team as code owners October 2, 2023 07:20

patrykkopycinski requested review from gergoabraham and pzl October 2, 2023 07:20

patrykkopycinski self-assigned this Oct 2, 2023

patrykkopycinski added v8.11.0 release_note:skip Skip the PR/issue when compiling release notes labels Oct 2, 2023

tomsonpl approved these changes Oct 2, 2023

View reviewed changes

kevinlog reviewed Oct 2, 2023

View reviewed changes

x-pack/plugins/security_solution/public/management/cypress/e2e/endpoint_list/endpoints.cy.ts Show resolved Hide resolved

paul-tavares approved these changes Oct 2, 2023

View reviewed changes

test

7adac99

kevinlog approved these changes Oct 2, 2023

View reviewed changes

patrykkopycinski added 4 commits October 2, 2023 22:38

add tests

bea13b4

Merge branch 'main' into feat/dw-serverless-endpoint-cypress

ec4aaa7

update jsdoc

fa0292c

Merge branch 'feat/dw-serverless-endpoint-cypress' of github.com:patr…

e24a8b3

…ykkopycinski/kibana into feat/dw-serverless-endpoint-cypress

maximpn approved these changes Oct 3, 2023

View reviewed changes

PR comments

be6b0e5

patrykkopycinski requested a review from a team as a code owner October 3, 2023 23:51

jbudz approved these changes Oct 4, 2023

View reviewed changes

PR comments

d471bee

dmlemeshko reviewed Oct 4, 2023

View reviewed changes

MadameSheema approved these changes Oct 4, 2023

View reviewed changes

update docs

fe33b5b

dmlemeshko self-requested a review October 4, 2023 08:54

dmlemeshko approved these changes Oct 4, 2023

View reviewed changes

patrykkopycinski merged commit 5fb9a38 into elastic:main Oct 4, 2023

kibanamachine added the backport:skip This PR does not require backporting label Oct 4, 2023

patrykkopycinski deleted the feat/dw-serverless-endpoint-cypress branch October 4, 2023 09:56

-    if (netInterfaceList) {
+const address = netInterfaceList?.find(
+  (networkInterface) =>
+    networkInterface.family === 'IPv4' &&
+    networkInterface.internal === false &&
+    networkInterface.address
+)?.address;
+if (address) {
+  return address;
+}

Conversation

patrykkopycinski commented Sep 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

tomsonpl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

paul-tavares left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

maximpn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jbudz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmlemeshko Oct 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmlemeshko left a comment

Choose a reason for hiding this comment

Uh oh!

kibana-ci commented Oct 4, 2023

💚 Build Succeeded

Metrics [docs]

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

patrykkopycinski commented Sep 30, 2023 •

edited

Loading

dmlemeshko Oct 4, 2023 •

edited

Loading