Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Fleet] Chunk asset installation during package install #189045

Merged
merged 2 commits into from
Jul 25, 2024
Merged

[Fleet] Chunk asset installation during package install #189045

merged 2 commits into from
Jul 25, 2024

Conversation

xcrzx
Copy link
Contributor

@xcrzx xcrzx commented Jul 24, 2024

Resolves: #189043

Summary

This PR limits the number of saved objects installed in a single request. During saved object installation a lot of auxiliary objects are created in memory. Chunking allows for the garbage collection of memory objects that are not needed for response.

Memory consumption before

Screenshot 2024-07-12 at 12 44 58

After
Screenshot 2024-07-18 at 11 53 30

@xcrzx xcrzx added technical debt Improvement of the software architecture and operational architecture release_note:skip Skip the PR/issue when compiling release notes Team:Fleet Team label for Observability Data Collection Fleet team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules v8.16.0 labels Jul 24, 2024
@xcrzx xcrzx self-assigned this Jul 24, 2024
@xcrzx xcrzx requested a review from a team as a code owner July 24, 2024 10:01
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

juliaElastic
juliaElastic approved these changes Jul 24, 2024
View reviewed changes
Copy link
Contributor

@juliaElastic juliaElastic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@kibana-ci
Copy link
Collaborator

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] Jest Integration Tests #3 / telemetry tasks detection-rules should send task metrics

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @xcrzx

@xcrzx xcrzx merged commit b7e66eb into elastic:main Jul 25, 2024
28 checks passed
@kibanamachine kibanamachine added the backport:skip This commit does not require backporting label Jul 25, 2024
@xcrzx xcrzx deleted the memory-optimization-3 branch July 25, 2024 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@juliaElastic juliaElastic juliaElastic approved these changes

Assignees

@xcrzx xcrzx

Labels
backport:skip This commit does not require backporting Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team:Fleet Team label for Observability Data Collection Fleet team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. technical debt Improvement of the software architecture and operational architecture v8.16.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Fleet] Split asset installation into chunks to avoid high memory pressure
5 participants
@xcrzx @elasticmachine @kibana-ci @juliaElastic @kibanamachine