[Entity Store] Document 9.4.2 log extraction tuning knobs#6677
Draft
uri-weisman wants to merge 1 commit into
Draft
[Entity Store] Document 9.4.2 log extraction tuning knobs#6677uri-weisman wants to merge 1 commit into
uri-weisman wants to merge 1 commit into
Conversation
Adds troubleshooting entries for the per-window volume cap introduced in 9.4.2 (maxLogsPerWindow, maxLogsPerWindowCapBehavior) and version-gates the maxLogsPerPage default to reflect the 9.4.3 value bump. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
Contributor
🔍 Preview links for changed docs |
Contributor
Vale Linting ResultsSummary: 1 warning found
|
| File | Line | Rule | Message |
|---|---|---|---|
| solutions/security/advanced-entity-analytics/entity-store.md | 234 | Elastic.Spelling | 'cpu' is a possible misspelling. |
The Vale linter checks documentation changes against the Elastic Docs style guide.
To use Vale locally or report issues, refer to Elastic style guide for Vale.
Contributor
Author
|
@natasha-moore-elastic we would also like to add a recommendation: Note for CCS users: Entity Analytics ingests logs from every remote cluster. To avoid unnecessary load, please disable EA on any remote cluster where it isn't actively used |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Docs review summary
Focus areas
- Style and clarity: Two grammar and spelling issues found (CPU capitalization, spelling error, subject-verb agreement).
- Jargon: No jargon issues detected. Technical terms are appropriately explained in context.
- Frontmatter and applies_to: Frontmatter is correct. The
applies_toversion gating for the new 9.4.2 parameters follows repository conventions correctly. - Content type fit: Changes fit the troubleshooting section well. The new parameters are clearly documented with use-case guidance and defaults.
- Parent issue satisfaction: Not applicable (no linked parent issue).
Notes
- The Vale linter already flagged the CPU capitalization issue on line 234, which is included in the inline comments.
- The new 9.4.2 tuning parameters (
maxLogsPerWindowandmaxLogsPerWindowCapBehavior) are well-structured and provide clear operational guidance. - The version gating using
applies_toblocks and inline{applies_to}syntax is correct and consistent with the rest of the page.
Generated by Docs review agent for issue #6677 · ● 645.3K
| Use `maxLogsPerWindow` to cap the total number of raw log documents processed in a single extraction run, across all slices in the window. | ||
|
|
||
| * Lower it if a single task run can still saturate {{es}} CPU even after lowering `maxLogsPerPage`. This is the most effective lever for protecting a cluster from CPU overload, because it bounds the work a single extraction task can do. | ||
| * Increase in case cluster cpu is not overloaded and can handle the amount of proccessed logs. |
Contributor
There was a problem hiding this comment.
Spelling: "cpu" should be capitalized as "CPU". Also "proccessed" should be "processed".
Suggested change
| * Increase in case cluster cpu is not overloaded and can handle the amount of proccessed logs. | |
| * Increase in case cluster CPU is not overloaded and can handle the amount of processed logs. |
| Use `maxLogsPerWindowCapBehavior` to control what happens when `maxLogsPerWindow` is reached during a run. | ||
|
|
||
| * `drop` — the next run advances past the uncapped logs (cursor jumps to the end of the window). Logs above the cap are skipped permanently. Use this to keep the cluster healthy in exchange for coverage gaps when ingest exceeds the cap. | ||
| * `defer` — the next run resumes from where the cap fired and processes the remaining logs. Use this to preserve full coverage at the cost of falling behind real time when logs exceeds the cap. |
Contributor
There was a problem hiding this comment.
Grammar: Subject-verb agreement issue. "logs" (plural) should pair with "exceed" (plural verb), not "exceeds".
Suggested change
| * `defer` — the next run resumes from where the cap fired and processes the remaining logs. Use this to preserve full coverage at the cost of falling behind real time when logs exceeds the cap. | |
| * `defer` — the next run resumes from where the cap fired and processes the remaining logs. Use this to preserve full coverage at the cost of falling behind real time when logs exceed the cap. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Updates the Entity Store troubleshooting section in
solutions/security/advanced-entity-analytics/entity-store.mdto cover the log extraction tunables added in 9.4.2 (elastic/kibana#269482, elastic/kibana#270180) and refreshes themaxLogsPerPagedefault to reflect the 9.4.2 value bump (elastic/kibana#270617).What's new in the section
maxLogsPerWindow— per-task-run total-volume cap (new in 9.4.2). Documented as the most effective lever for protecting a cluster from sudden ingest spikes, because it bounds the work a single extraction task can do regardless of how many slices the window is split into.maxLogsPerWindowCapBehavior— new in 9.4.2. Documents bothdrop(default, cluster-health-first) anddefer(coverage-first) modes and the coverage-vs-freshness trade-off between them.maxLogsPerPagedefault — version-gated to show40000for 9.4.0–9.4.1 and50000for 9.4.2Why now
We've seen multiple SDHs (#1675, #9945, EOG, plus internal Infosec) where Entity Store v2 extraction overwhelmed hot data nodes after 9.4.0. The 9.4.2 knobs are the supported mitigation path; this page is where Support and customers look first.
applies_to notes
serverless: plannedto match the rest of the troubleshoot section. Once the Serverless rollout date is known, update accordingly.