Skip to content

[Fleet] [Cloud Security Posture] Add CloudFormation agent install method#155045

Merged
opauloh merged 43 commits intoelastic:mainfrom
opauloh:vuln/add-cloudformation-deploy-method
Apr 26, 2023
Merged

[Fleet] [Cloud Security Posture] Add CloudFormation agent install method#155045
opauloh merged 43 commits intoelastic:mainfrom
opauloh:vuln/add-cloudformation-deploy-method

Conversation

@opauloh
Copy link
Copy Markdown
Contributor

@opauloh opauloh commented Apr 17, 2023 •
edited
Loading

Summary

It solves: #153067

This PR introduces the new CloudFormation method to install the elastic agent.

Changes Summary

Fleet

  • Added cloud_formation_template_url to the AgentPolicy schema.
  • Added CloudFormation to the Platform selector (Fleet Managed only).
  • Added the onChangeAgentPolicy method to the fleet UI Extension of type package-policy-replace-define-step
  • Updated Platform selector to auto select CloudFormation tab when it's available

Cloud Security Posture

  • Updated the Cloud Security Posture integration to automatically set the cloud_formation_template_url in the PackagePolicy for the Vulnerability Management integration, it fetches the CloudFormation template from the @elastic/integrations repository

Screenshots

Add Agent Flyout when CloudFormation is provided

image

The Launch CloudFormation button opens the Launch CloudFormation Stack

image

opauloh added 19 commits April 10, 2023 10:13
@opauloh
Merge branch 'main' into vuln/add-cloudformation-deploy-method
811827c
@opauloh
WIP: Cloud Formation
076d6d8
@opauloh
Merge branch 'main' into vuln/add-cloudformation-deploy-method
54b7a07
@opauloh
wip: cloud formation instructions
d1aabec
@opauloh
Merge branch 'main' into vuln/add-cloudformation-deploy-method
3fdd5a1
@opauloh
adding vuln mgmt mocks
e82be93
@opauloh
add vuln mgmt cloud formation utils
fc2c9c4
@opauloh
add use cloud formation hook to policy template form
30825a8
@opauloh
add test for vuln mgmt
735c026
@opauloh
add cloud formation types
22be93f
@opauloh
Add CloudFormation in agent policy
4369ef6
@opauloh
add CloudFormation to agent management
cddaea8
@opauloh
add method to update AgentPolicy on integration
735f504
@opauloh
update AgentPolicy saved object
78bf72d
@opauloh
update AgentPolicy Schema
605ef26
@opauloh
Refactor: allow cloud formation to be more generic
37f18f9
@opauloh
refactor: allow cloud formation to be more generic
06a9956
@opauloh
CloudFormation agent enrolment instructions
912efc6
@opauloh
update text
52800fb
@opauloh opauloh added Team:Cloud Security Cloud Security team related v8.8.0 labels Apr 17, 2023
@opauloh opauloh requested review from a team as code owners April 17, 2023 13:57
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Apr 17, 2023

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

@botelastic botelastic bot added the Team:Fleet Team label for Observability Data Collection Fleet team label Apr 17, 2023
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Apr 17, 2023

Pinging @elastic/fleet (Team:Fleet)

@opauloh opauloh added release_note:skip Skip the PR/issue when compiling release notes release_note:feature Makes this part of the condensed release notes and removed release_note:skip Skip the PR/issue when compiling release notes labels Apr 17, 2023
@kfirpeled kfirpeled requested a review from jloleysens April 20, 2023 20:04
kpollich
kpollich requested changes Apr 21, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@kpollich kpollich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR here. The Fleet team wasn't aware of these incoming changes as far as I know, so please forgive me as I try to get some context here.

Is there any documentation or writing around why we only present CloudFormation here compared to other orchestration tools? Fleet hasn't necessarily coalesced on a single cloud platform or orchestration tool anywhere else in the docs/UI (yet), so it does seem a little jarring to present an AWS product in our UI like this.

Also, this is limited via a UI extension, right? So only CSP policies will ever display the CF interface in the policy selector, correct?

Few minor changes suggested in the diff.

@opauloh opauloh linked an issue Apr 21, 2023 that may be closed by this pull request
[Cloud Security] [BUG] [CNVM] Cloudformation deployment doesn't appear when not installed to a newly created agent policy #155443
Closed
2 tasks
@opauloh opauloh requested a review from kpollich April 21, 2023 21:26
@opauloh
Copy link
Copy Markdown
Contributor Author

opauloh commented Apr 21, 2023 •
edited
Loading

Thanks for reviewing it @kpollich

Also, this is limited via a UI extension, right? So only CSP policies will ever display the CF interface in the policy selector, correct?

After chatting with @kfirpeled, we realized it's better not to add CloudFormation in the AgentPolicy like initially suggested because CSP is the only integration that will trigger that, so I reverted those changes. Instead, I added a utility that checks for cloud_formation_template_url as config in the Package Policy.

Is there any documentation or writing around why we only present CloudFormation here compared to other orchestration tools? Fleet hasn't necessarily coalesced on a single cloud platform or orchestration tool anywhere else in the docs/UI (yet), so it does seem a little jarring to present an AWS product in our UI like this.

There will be public documentation for deploying Elastic Agent with CloudFormation. Still, for now, it will be related to the CSP integration that supports it, so that is one more reason why it was better to remove it from the Agent Policy as a generally available feature.

Few minor changes suggested in the diff.

Thanks for those! I addressed the suggestions and would appreciate it if you could review them again.

kpollich
kpollich requested changes Apr 25, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@kpollich kpollich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more React specific things to clean up in the UI here. Thanks for addressing the previous comments!

@opauloh opauloh requested a review from kpollich April 25, 2023 05:08
kpollich
kpollich approved these changes Apr 25, 2023
View reviewed changes
Copy link
Copy Markdown
Member

@kpollich kpollich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🚀 - thanks for making those changes! Appreciate the contributions 🙏

@opauloh opauloh enabled auto-merge (squash) April 25, 2023 20:35
@kibana-ci
Copy link
Copy Markdown

kibana-ci commented Apr 26, 2023 •
edited
Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
fleet 805 807 +2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
cloudSecurityPosture 192.3KB 193.2KB +912.0B
fleet 957.4KB 959.7KB +2.3KB
total +3.2KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
fleet 127.4KB 128.1KB +633.0B
Unknown metric groups

ESLint disabled line counts

id before after diff
cloudSecurityPosture 11 12 +1
enterpriseSearch 17 19 +2
securitySolution 399 402 +3
total +6

Total ESLint disabled count

id before after diff
cloudSecurityPosture 12 13 +1
enterpriseSearch 18 20 +2
securitySolution 479 482 +3
total +6

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@opauloh opauloh merged commit 0dad0c5 into elastic:main Apr 26, 2023
@kibanamachine kibanamachine added the backport:skip This PR does not require backporting label Apr 26, 2023
jloleysens added a commit to jloleysens/kibana that referenced this pull request Apr 26, 2023
@jloleysens
Merge branch 'main' into compatible-schema-change-check
ab5ff42 
* main: (1294 commits)
  [SecuritySolution] Refactor security packages (elastic#155365)
  [Discover] Show "Temporary" badge for ad-hoc data views in Alerts flyout (elastic#155717)
  [RAM] Conditional actions feedback on pr review (elastic#155804)
  [Files] Adds bulk delete method (elastic#155628)
  [Lens] Use proper way to generate absolute short URL (elastic#155512)
  [Guided onboarding] Use Kibana features to grant access (elastic#155065)
  [Index Management] Fix duped mock (elastic#155844)
  [Lens] Enhance visualization modifier popup with layer palette (elastic#155280)
  Fix flaky combobox tests on role management screen (elastic#155711)
  [Infrastructure UI] Create InventoryViewsService and InventoryViewsClient (elastic#155126)
  [Fleet] always create agent upload write indices (elastic#155729)
  [Fleet] [Cloud Security Posture] Add CloudFormation agent install method (elastic#155045)
  Add tech preview label for search applications (elastic#155649)
  [ML] AIOps: Stabilize flaky functional tests. (elastic#155710)
  [ES UI Shared] Migrate JsonEditor to monaco (elastic#155610)
  [Security Solution] Fixes security_solution storybooks always rendering in a flyout (elastic#155814)
  [Synthetics] Make error popover disappear `onMouseLeave` of metric item card (elastic#155800)
  Remove Exploratory View components from Observability (elastic#155629)
  [Discover] Remove redundant "Filter was added" toast (elastic#155645)
  [RAM][Security Solution][Alerts] Support the ability to trigger a rule action per alert generated (elastic#153611) (elastic#155384)
  ...
@orestisfl orestisfl mentioned this pull request May 15, 2023
Closed
@nastasha-solomon nastasha-solomon mentioned this pull request May 21, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kpollich kpollich kpollich approved these changes

@jloleysens jloleysens jloleysens approved these changes

@kfirpeled kfirpeled kfirpeled approved these changes

@Omolola-Akinleye Omolola-Akinleye Awaiting requested review from Omolola-Akinleye

Assignees

No one assigned

Labels

backport:skip This PR does not require backporting ci:cloud-deploy Create or update a Cloud deployment release_note:feature Makes this part of the condensed release notes Team:Cloud Security Cloud Security team related Team:Fleet Team label for Observability Data Collection Fleet team v8.8.0

Projects

None yet

Milestone

No milestone

7 participants

@opauloh @elasticmachine @kfirpeled @kibana-ci @kpollich @jloleysens @kibanamachine