Skip to content
This repository was archived by the owner on Sep 17, 2024. It is now read-only.

feat: bootstrap fleet-server for the deployment of regular elastic-agents (backport #1078)#1104

Closed
mergify[bot] wants to merge 5 commits into7.xfrom
mergify/bp/7.x/pr-1078
Closed

feat: bootstrap fleet-server for the deployment of regular elastic-agents (backport #1078)#1104
mergify[bot] wants to merge 5 commits into7.xfrom
mergify/bp/7.x/pr-1078

Conversation

@mergify
Copy link
Copy Markdown
Contributor

@mergify mergify Bot commented Apr 27, 2021

This is an automatic backport of pull request #1078 done by Mergify.

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.io/

@mdelapenya @adam-stokes
feat: bootstrap fleet-server for the deployment of regular elastic-ag…
f67e1ec 
…ents (#1078)

* chore: provide a fleet-server base image based on centos/debian with systemd

* WIP

* fix: remove duplicated fields after merge conflicts

* fix: update method call after merge conflicts

* chore: extract service name calculation to a method

* chore: extract container name calculation to a method

* chore: refactor get container name method

* chore: refactor method even more

* chore: use installer state to retrieve container name

* chore: use installer when calculating service name

* fix: adapt service names for fleet server

* chore: enrich log when creating an installer

* fix: use fleet server host when creating fleet config

* fix: use https when connecting to fleet-server

It's creating its own self-signed certs

* feat: bootstrap a fleet server before a regular agent is deployed to fleet

It will define the server host to be used when enrolling agents

* fix: use fleet policy for agents, not the server one

* fix: get different installers for fleet-server and agents

* fix: use the old step for deploying regular agents

* chore: rename variable with consistent name

* chore: rename fleet-server scenario

* fix: use proper container name for standalone mode

* chore: save two variables

* chore: rename standalone scenario for bootstrapping fleet-server

* chore: rename bootstrap methods

* chore: encapsulate bootstrap fleet-server logic

* Update fleet.go

* chore: remove Fleet Server CI parallel execution

* chore: remove feature file for fleet-server

* chore: boostrap fleet server only once

We want to have it bootstrapped for the entire test suite, not for each scenario

* fix: an agent was needed when adding integrations

Co-authored-by: Adam Stokes <51892+adam-stokes@users.noreply.github.com>
(cherry picked from commit 193337a)
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Apr 27, 2021 •
edited
Loading

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview

Expand to view the summary

Build stats

  • Build Cause: Started by user Manuel de la Peña

  • Start Time: 2021-05-04T08:28:41.893+0000

  • Duration: 41 min 6 sec

  • Commit: 83a7629

Test stats 🧪

Test Results
Failed 3
Passed 149
Skipped 0
Total 152

Trends 🧪

Image of Build Times

Image of Tests

Test errors 3

Expand to view the tests failures

Initializing / End-To-End Tests / ubuntu-18.04_fleet_agent_endpoint_integration / Adding the Endpoint Integration to an Agent makes the host to show in Security App – Agent Endpoint Integration
    Expand to view the error details

     Step the agent is listed in Fleet as "online"

  • no stacktrace
Initializing / End-To-End Tests / ubuntu-18.04_fleet_backend_processes / Deploying the centos agent – Backend Processes
    Expand to view the error details

     Step the "elastic-agent" process is in the "started" state on the host

  • no stacktrace
Initializing / End-To-End Tests / ubuntu-18.04_fleet_fleet_mode_agent / Deploying the centos agent – Fleet Mode Agent
    Expand to view the error details

     Step the "elastic-agent" process is in the "started" state on the host

  • no stacktrace

Steps errors 3

Expand to view the steps failures

Run functional tests for fleet:fleet_mode_agent && ~@nightly && ~debian
  • Took 12 min 1 sec . View more details on here
  • Description: .ci/scripts/functional-test.sh "fleet" "fleet_mode_agent && ~@nightly && ~debian" "7.x-SNAPSHOT" "7.x-SNAPSHOT"
Run functional tests for fleet:agent_endpoint_integration && ~@nightly && ~debian
  • Took 12 min 43 sec . View more details on here
  • Description: .ci/scripts/functional-test.sh "fleet" "agent_endpoint_integration && ~@nightly && ~debian" "7.x-SNAPSHOT" "7.x-SNAPSHOT"
Run functional tests for fleet:backend_processes && ~@nightly && ~debian
  • Took 14 min 0 sec . View more details on here
  • Description: .ci/scripts/functional-test.sh "fleet" "backend_processes && ~@nightly && ~debian" "7.x-SNAPSHOT" "7.x-SNAPSHOT"

Log output

Expand to view the last 100 lines of log output 

[2021-05-04T09:05:18.059Z] github.com/cucumber/godog/internal/builder
[2021-05-04T09:05:18.059Z] github.com/gofrs/uuid
[2021-05-04T09:05:18.059Z] github.com/gogo/protobuf/proto
[2021-05-04T09:05:18.059Z] github.com/hashicorp/go-immutable-radix
[2021-05-04T09:05:18.321Z] github.com/hashicorp/go-memdb
[2021-05-04T09:05:18.897Z] github.com/gogo/protobuf/io
[2021-05-04T09:05:19.159Z] github.com/cucumber/messages-go/v10
[2021-05-04T09:05:19.734Z] github.com/cucumber/godog/formatters
[2021-05-04T09:05:19.734Z] github.com/cucumber/godog/internal/tags
[2021-05-04T09:05:19.734Z] github.com/cucumber/gherkin-go/v11
[2021-05-04T09:05:19.734Z] github.com/cucumber/godog/internal/models
[2021-05-04T09:05:19.996Z] github.com/cucumber/godog/internal/storage
[2021-05-04T09:05:19.996Z] github.com/cucumber/godog/internal/formatters
[2021-05-04T09:05:20.571Z] github.com/cucumber/godog/internal/parser
[2021-05-04T09:05:20.571Z] github.com/cucumber/godog
[2021-05-04T09:05:20.571Z] github.com/cucumber/godog/cmd/godog/internal
[2021-05-04T09:05:20.571Z] github.com/cucumber/godog/cmd/godog
[2021-05-04T09:05:21.523Z] cd _suites/helm && \
[2021-05-04T09:05:21.523Z] OP_LOG_LEVEL=DEBUG \
[2021-05-04T09:05:21.523Z] OP_LOG_INCLUDE_TIMESTAMP=TRUE \
[2021-05-04T09:05:21.523Z] TIMEOUT_FACTOR=5 \
[2021-05-04T09:05:21.523Z] STACK_VERSION=7.x-SNAPSHOT \
[2021-05-04T09:05:21.523Z] DEVELOPER_MODE=false \
[2021-05-04T09:05:21.524Z] ELASTIC_APM_SERVICE_NAME="E2E Tests" \
[2021-05-04T09:05:21.524Z] ELASTIC_APM_CENTRAL_CONFIG="false" \
[2021-05-04T09:05:21.524Z] ELASTIC_APM_GLOBAL_LABELS=build_pr=true,build_id=8 \
[2021-05-04T09:05:21.524Z] ELASTIC_APM_SERVICE_VERSION="7.x-SNAPSHOT" \
[2021-05-04T09:05:21.524Z] ELASTIC_APM_ENVIRONMENT="ci" \
[2021-05-04T09:05:21.524Z] ELASTIC_APM_SECRET_TOKEN="********" \
[2021-05-04T09:05:21.524Z] ELASTIC_APM_SERVER_URL="********" \
[2021-05-04T09:05:21.524Z] godog --format=junit:/var/lib/jenkins/workspace/PR-1104-8-367ca918-5f6a-421b-b681-044d049d7305/src/github.com/elastic/e2e-testing/outputs/TEST-helm.xml --tags "apm-server && ~@nightly && ~skip"
[2021-05-04T09:05:36.459Z] time="2021-05-04T09:05:35Z" level=info msg="Current execution will be instrumented 🛠" apm-environment=ci
[2021-05-04T09:05:36.459Z] time="2021-05-04T09:05:35Z" level=debug msg="The Elastic artifacts API is available" elapsedTime=115.001328ms retries=1 statusEndpoint="https://artifacts-api.elastic.co/v1/versions/7.x-SNAPSHOT/?x-elastic-no-kpi=true"
[2021-05-04T09:05:36.459Z] time="2021-05-04T09:05:35Z" level=debug msg="Latest version for current version obtained" alias=7.x-SNAPSHOT version=7.14.0-SNAPSHOT
[2021-05-04T09:05:36.459Z] Creating network "helm_default" with the default driver
[2021-05-04T09:05:36.722Z] Creating helm_elasticsearch_1 ... 
[2021-05-04T09:05:40.029Z] 
Creating helm_elasticsearch_1 ... done
time="2021-05-04T09:05:39Z" level=debug msg="Docker compose executed." cmd="[up -d]" composeFilePaths="[/var/lib/jenkins/workspace/PR-1104-8-367ca918-5f6a-421b-b681-044d049d7305/.op/compose/profiles/helm/docker-compose.yml]" env="map[stackVersion:7.14.0-SNAPSHOT]" profile=helm
[2021-05-04T09:05:50.053Z] {"level":"debug","time":"2021-05-04T09:05:49Z","message":"sent request with 0 transactions, 1 span, 0 errors, 0 metricsets"}
[2021-05-04T09:06:08.199Z] {"level":"debug","time":"2021-05-04T09:06:05Z","message":"gathering metrics"}
[2021-05-04T09:06:16.378Z] {"level":"debug","time":"2021-05-04T09:06:15Z","message":"sent request with 0 transactions, 0 spans, 0 errors, 1 metricset"}
[2021-05-04T09:06:31.298Z] time="2021-05-04T09:06:31Z" level=info msg="Cluster created" cluster=helm-charts-test-suite k8sVersion=1.18.2 output=
[2021-05-04T09:06:31.560Z] time="2021-05-04T09:06:31Z" level=debug msg="elastic Helm charts added" name=elastic output="\"elastic\" has been added to your repositories" url="https://helm.elastic.co"
[2021-05-04T09:06:32.134Z] time="2021-05-04T09:06:31Z" level=info msg="Rancher Local Path Provisioner and local-path storage class for Elasticsearch volumes installed" chart=elasticsearch
[2021-05-04T09:06:32.134Z] time="2021-05-04T09:06:31Z" level=debug msg="Applying workaround to use Rancher's local-path storage class for Elasticsearch volumes"
[2021-05-04T09:06:35.440Z] {"level":"debug","time":"2021-05-04T09:06:35Z","message":"gathering metrics"}
[2021-05-04T09:06:42.028Z] {"level":"debug","time":"2021-05-04T09:06:41Z","message":"sent request with 0 transactions, 5 spans, 0 errors, 1 metricset"}
[2021-05-04T09:07:08.627Z] {"level":"debug","time":"2021-05-04T09:07:05Z","message":"gathering metrics"}
[2021-05-04T09:07:16.778Z] {"level":"debug","time":"2021-05-04T09:07:15Z","message":"sent request with 0 transactions, 0 spans, 0 errors, 1 metricset"}
[2021-05-04T09:07:38.759Z] {"level":"debug","time":"2021-05-04T09:07:35Z","message":"gathering metrics"}
[2021-05-04T09:07:45.356Z] {"level":"debug","time":"2021-05-04T09:07:45Z","message":"sent request with 0 transactions, 0 spans, 0 errors, 1 metricset"}
[2021-05-04T09:08:07.335Z] {"level":"debug","time":"2021-05-04T09:08:05Z","message":"gathering metrics"}
[2021-05-04T09:08:15.487Z] {"level":"debug","time":"2021-05-04T09:08:15Z","message":"sent request with 0 transactions, 0 spans, 0 errors, 1 metricset"}
[2021-05-04T09:08:37.503Z] {"level":"debug","time":"2021-05-04T09:08:35Z","message":"gathering metrics"}
[2021-05-04T09:08:45.668Z] {"level":"debug","time":"2021-05-04T09:08:45Z","message":"sent request with 0 transactions, 0 spans, 0 errors, 1 metricset"}
[2021-05-04T09:09:07.656Z] {"level":"debug","time":"2021-05-04T09:09:05Z","message":"gathering metrics"}
[2021-05-04T09:09:15.810Z] {"level":"debug","time":"2021-05-04T09:09:15Z","message":"sent request with 0 transactions, 0 spans, 0 errors, 1 metricset"}
[2021-05-04T09:09:21.108Z] time="2021-05-04T09:09:20Z" level=info msg="Chart installed" chart=elastic/elasticsearch name=elasticsearch output="NAME: elasticsearch\nLAST DEPLOYED: Tue May  4 09:06:32 2021\nNAMESPACE: default\nSTATUS: deployed\nREVISION: 1\nNOTES:\n1. Watch all cluster members come up.\n  $ kubectl get pods --namespace=default -l app=elasticsearch-master -w\n2. Test cluster health using Helm test.\n  $ helm test elasticsearch" version=7.11.2
[2021-05-04T09:09:21.109Z] {"level":"debug","time":"2021-05-04T09:09:20Z","message":"sent request with 1 transaction, 4 spans, 0 errors, 0 metricsets"}
[2021-05-04T09:09:21.109Z] time="2021-05-04T09:09:20Z" level=debug msg="Cluster is running" output=helm-charts-test-suite
[2021-05-04T09:09:21.684Z] time="2021-05-04T09:09:21Z" level=info msg="Chart installed" chart=elastic/apm-server name=apm-server output="NAME: apm-server\nLAST DEPLOYED: Tue May  4 09:09:21 2021\nNAMESPACE: default\nSTATUS: deployed\nREVISION: 1\nTEST SUITE: None\nNOTES:\n1. Watch all containers come up.\n  $ kubectl get pods --namespace=default -l app=apm-server-apm-server -w" version=7.11.2
[2021-05-04T09:09:22.261Z] time="2021-05-04T09:09:22Z" level=warning msg="Endpoints not present yet" elapsedTime=83.458439ms resource=endpoints retry=1 selector="app=apm-server,release=apm-server"
[2021-05-04T09:09:22.836Z] time="2021-05-04T09:09:22Z" level=warning msg="Endpoints not present yet" elapsedTime=724.851382ms resource=endpoints retry=2 selector="app=apm-server,release=apm-server"
[2021-05-04T09:09:24.240Z] time="2021-05-04T09:09:24Z" level=warning msg="Endpoints not present yet" elapsedTime=2.249272182s resource=endpoints retry=3 selector="app=apm-server,release=apm-server"
[2021-05-04T09:09:26.791Z] time="2021-05-04T09:09:26Z" level=warning msg="Endpoints not present yet" elapsedTime=4.659924016s resource=endpoints retry=4 selector="app=apm-server,release=apm-server"
[2021-05-04T09:09:31.004Z] time="2021-05-04T09:09:30Z" level=warning msg="Endpoints not present yet" elapsedTime=8.49411262s resource=endpoints retry=5 selector="app=apm-server,release=apm-server"
[2021-05-04T09:09:31.004Z] {"level":"debug","time":"2021-05-04T09:09:30Z","message":"sent request with 0 transactions, 25 spans, 0 errors, 0 metricsets"}
[2021-05-04T09:09:35.221Z] {"level":"debug","time":"2021-05-04T09:09:35Z","message":"gathering metrics"}
[2021-05-04T09:09:35.221Z] time="2021-05-04T09:09:35Z" level=info msg="Endpoints found" elapsedTime=13.203950589s resource=endpoints retry=6 selector="app=apm-server,release=apm-server"
[2021-05-04T09:09:35.482Z] time="2021-05-04T09:09:35Z" level=debug msg="A ConfigMap resource contains the apm-server.yml key" name=apm-server output="\"apm-server:\n  host: \"0.0.0.0:8200\"\n\nqueue: {}\n\noutput.elasticsearch:\n  hosts: [\"http://elasticsearch-master:9200\"]\n  ## If you have security enabled- you'll need to add the credentials\n  ## as environment variables\n  # username: \"${ELASTICSEARCH_USERNAME}\"\n  # password: \"${ELASTICSEARCH_PASSWORD}\"\n  ## If SSL is enabled\n  # protocol: https\n  # ssl.certificate_authorities:\n  #  - /usr/share/apm-server/config/certs/elastic-ca.pem\n\""
[2021-05-04T09:09:35.482Z] time="2021-05-04T09:09:35Z" level=debug msg="A ServiceAccount resource manages K8S RBAC" name=apm-server output="\"'apm-server-7.11.2'\""
[2021-05-04T09:09:35.482Z] time="2021-05-04T09:09:35Z" level=debug msg="A ClusterRole resource manages K8S RBAC" name=apm-server output="\"'apm-server-7.11.2'\""
[2021-05-04T09:09:35.482Z] time="2021-05-04T09:09:35Z" level=debug msg="A ClusterRoleBinding resource manages K8S RBAC" name=apm-server output="\"'apm-server-7.11.2'\""
[2021-05-04T09:09:35.744Z] time="2021-05-04T09:09:35Z" level=debug msg="Resourcelimits is applied" constraint=limits name=apm-server output="'map[cpu:1 memory:512Mi]'"
[2021-05-04T09:09:35.744Z] time="2021-05-04T09:09:35Z" level=debug msg="Resourcerequests is applied" constraint=requests name=apm-server output="'map[cpu:100m memory:100Mi]'"
[2021-05-04T09:09:35.744Z] time="2021-05-04T09:09:35Z" level=debug msg="The strategy can be used for resource during updates" name=apm-server-apm-server resource=Deployment strategy=RollingUpdate
[2021-05-04T09:09:36.320Z] time="2021-05-04T09:09:36Z" level=debug msg="Chart deleted" chart=apm-server output="release \"apm-server\" uninstalled"
[2021-05-04T09:09:36.597Z] {"level":"debug","time":"2021-05-04T09:09:36Z","message":"sent request with 1 transaction, 19 spans, 0 errors, 10 metricsets"}
[2021-05-04T09:09:38.521Z] time="2021-05-04T09:09:38Z" level=debug msg="Cluster destroyed" cluster=helm-charts-test-suite output=
[2021-05-04T09:09:39.093Z] Stopping helm_elasticsearch_1 ... 
[2021-05-04T09:09:40.482Z] 
Stopping helm_elasticsearch_1 ... done
Removing helm_elasticsearch_1 ... 
[2021-05-04T09:09:40.482Z] 
Removing helm_elasticsearch_1 ... done
Removing network helm_default
[2021-05-04T09:09:40.744Z] time="2021-05-04T09:09:40Z" level=debug msg="Docker compose executed." cmd="[down --remove-orphans]" composeFilePaths="[/var/lib/jenkins/workspace/PR-1104-8-367ca918-5f6a-421b-b681-044d049d7305/.op/compose/profiles/helm/docker-compose.yml]" env="map[stackVersion:7.14.0-SNAPSHOT]" profile=helm
[2021-05-04T09:09:41.008Z] {"level":"debug","time":"2021-05-04T09:09:40Z","message":"sent request with 1 transaction, 3 spans, 0 errors, 0 metricsets"}
[2021-05-04T09:09:41.113Z] [INFO] Stopping Filebeat Docker container
[2021-05-04T09:09:41.410Z] + docker exec -t 2484a7caba72a00e4fd0f318fb338b8f7411c7630404ecdaed72e2ba89db0fb9 chmod -R ugo+rw /output
[2021-05-04T09:09:41.672Z] + docker stop --time 30 2484a7caba72a00e4fd0f318fb338b8f7411c7630404ecdaed72e2ba89db0fb9
[2021-05-04T09:09:42.249Z] 2484a7caba72a00e4fd0f318fb338b8f7411c7630404ecdaed72e2ba89db0fb9
[2021-05-04T09:09:42.270Z] Archiving artifacts
[2021-05-04T09:09:43.419Z] Recording test results
[2021-05-04T09:09:44.012Z] [Checks API] No suitable checks publisher found.
[2021-05-04T09:09:44.034Z] Archiving artifacts
[2021-05-04T09:09:44.078Z] Running in /var/lib/jenkins/workspace/PR-1104-8-367ca918-5f6a-421b-b681-044d049d7305/src/github.com/elastic/e2e-testing
[2021-05-04T09:09:44.395Z] + go clean -modcache
[2021-05-04T09:09:46.167Z] Stage "Release" skipped due to earlier failure(s)
[2021-05-04T09:09:47.097Z] Running on Jenkins in /var/lib/jenkins/workspace/2e-tests_e2e-testing-mbp_PR-1104
[2021-05-04T09:09:47.142Z] [INFO] getVaultSecret: Getting secrets
[2021-05-04T09:09:47.182Z] Masking supported pattern matches of $VAULT_ADDR or $VAULT_ROLE_ID or $VAULT_SECRET_ID
[2021-05-04T09:09:47.854Z] + chmod 755 generate-build-data.sh
[2021-05-04T09:09:47.854Z] + ./generate-build-data.sh https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-1104/ https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-1104/runs/8 FAILURE 2465694
[2021-05-04T09:09:47.854Z] INFO: curl https://beats-ci.elastic.co/blue/rest/organizations/jenkins/pipelines/e2e-tests/e2e-testing-mbp/PR-1104/runs/8/steps/?limit=10000 -o steps-info.json

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Test stats 🧪

Test Results
Failed 3
Passed 149
Skipped 0
Total 152

Genuine test errors 3

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Initializing / End-To-End Tests / ubuntu-18.04_fleet_agent_endpoint_integration / Adding the Endpoint Integration to an Agent makes the host to show in Security App – Agent Endpoint Integration
  • Name: Initializing / End-To-End Tests / ubuntu-18.04_fleet_backend_processes / Deploying the centos agent – Backend Processes
  • Name: Initializing / End-To-End Tests / ubuntu-18.04_fleet_fleet_mode_agent / Deploying the centos agent – Fleet Mode Agent

mdelapenya
mdelapenya reviewed May 3, 2021
View reviewed changes
Comment thread internal/kibana/fleet.go Outdated
@mdelapenya
Copy link
Copy Markdown
Contributor

mdelapenya commented May 4, 2021

Superseded by #1126

@mdelapenya mdelapenya closed this May 4, 2021
@mergify mergify Bot deleted the mergify/bp/7.x/pr-1078 branch May 4, 2021 14:25
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

2 more reviewers

@adam-stokes adam-stokes adam-stokes left review comments

@mdelapenya mdelapenya mdelapenya left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

@mdelapenya mdelapenya

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@elasticmachine @mdelapenya @adam-stokes