Skip to content

[Aggs] Fix single percentile case on index with many docs#115214

Merged
dej611 merged 3 commits intoelastic:masterfrom
dej611:fix/115209
Oct 19, 2021
Merged

[Aggs] Fix single percentile case on index with many docs#115214
dej611 merged 3 commits intoelastic:masterfrom
dej611:fix/115209

Conversation

@dej611
Copy link
Contributor

@dej611 dej611 commented Oct 15, 2021 •
edited
Loading

Summary

Fixes #115209

To reproduce this bug it is required to have many documents (> 1M documents should work - use the makelogs in case) and create a Lens visualization with the Percentile function. It may be required to refresh a couple of times to see the error, as Elasticsearch, in some circumstances, returns an empty object.

This is a similar issue as the one fixed in #113216, but affecting single percentile.
Tests have been provided.

Checklist

Delete any items that are not applicable to this PR.

Risk Matrix

Delete this section if it is not applicable to this PR.

Before closing this PR, invite QA, stakeholders, and other developers to identify risks that should be tested prior to the change/feature release.

When forming the risk matrix, consider some of the following examples and how they may potentially impact the change:

Risk Probability Severity Mitigation/Notes
Multiple Spaces—unexpected behavior in non-default Kibana Space. Low High Integration tests will verify that all features are still supported in non-default Kibana Space and when user switches between spaces.
Multiple nodes—Elasticsearch polling might have race conditions when multiple Kibana nodes are polling for the same tasks. High Low Tasks are idempotent, so executing them multiple times will not result in logical error, but will degrade performance. To test for this case we add plenty of unit tests around this logic and document manual testing procedure.
Code should gracefully handle cases when feature X or plugin Y are disabled. Medium High Unit tests will verify that any feature flag or plugin combination still results in our service operational.
See more potential risk examples

For maintainers

@dej611 dej611 added Feature:Aggregations Aggregation infrastructure (AggConfig, esaggs, ...) release_note:fix v8.0.0 Team:AppServicesSv auto-backport Deprecated - use backport:version if exact versions are needed v7.16.0 v7.15.2 labels Oct 15, 2021
@dej611 dej611 marked this pull request as ready for review October 18, 2021 08:11
@dej611 dej611 requested a review from a team as a code owner October 18, 2021 08:11
@dej611
Copy link
Contributor Author

dej611 commented Oct 18, 2021

@elasticmachine merge upstream

@shahzad31
Copy link
Contributor

shahzad31 commented Oct 18, 2021

It fixes the issue i was facing with eden cluster in exploratory view.

@dej611
Copy link
Contributor Author

dej611 commented Oct 18, 2021

@elasticmachine merge upstream

@kibanamachine
Copy link
Contributor

kibanamachine commented Oct 18, 2021

💚 Build Succeeded

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
data 465.6KB 465.6KB +51.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

jloleysens
jloleysens approved these changes Oct 19, 2021
View reviewed changes
Copy link
Contributor

@jloleysens jloleysens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code LGTM. Would you mind adding steps to reproduce to the PR description?

@dej611
Copy link
Contributor Author

dej611 commented Oct 19, 2021

Updated description with some more reproduction steps.

@dej611 dej611 merged commit 44d0150 into elastic:master Oct 19, 2021
kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Oct 19, 2021
@dej611 @kibanamachine
🐛 Fix single percentile case when ES is returning no buckets (elastic…
1789a2c 
…#115214)

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@kibanamachine kibanamachine mentioned this pull request Oct 19, 2021
Merged
kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Oct 19, 2021
@dej611 @kibanamachine
🐛 Fix single percentile case when ES is returning no buckets (elastic…
09b4d6d 
…#115214)

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@kibanamachine kibanamachine mentioned this pull request Oct 19, 2021
Merged
@kibanamachine
Copy link
Contributor

kibanamachine commented Oct 19, 2021

💚 Backport successful

Status Branch Result
✅ 7.x
✅ 7.15

The backport PRs will be merged automatically after passing CI.

kibanamachine added a commit that referenced this pull request Oct 19, 2021
@kibanamachine @dej611
🐛 Fix single percentile case when ES is returning no buckets (#115214) (
528841c 
#115567)

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
jloleysens added a commit to jloleysens/kibana that referenced this pull request Oct 19, 2021
@jloleysens
Merge branch 'master' of github.com:elastic/kibana into reporting/add…
2781812 
…-link-to-kibana-app

* 'master' of github.com:elastic/kibana: (30 commits)
  Fix potential error from undefined (elastic#115562)
  [App Search, Crawler] Fix validation step panel padding/whitespace (elastic#115542)
  [Cases][Connectors] ServiceNow ITOM: MVP (elastic#114125)
  Change default session idle timeout to 8 hours. (elastic#115565)
  Upgrade EUI to v39.1.1 (elastic#114732)
  [App Search] Wired up organic results on Curation Suggestions view (elastic#114717)
  [i18n] remove i18n html extractor (elastic#115004)
  [Logs/Metrics UI] Add deprecated field configuration to Deprecations API (elastic#115103)
  [Transform] Add alerting rules management to Transform UI (elastic#115363)
  Update UI links to Fleet and Agent docs (elastic#115295)
  [ML] Adding ability to change data view in advanced job wizard (elastic#115191)
  Change deleteByNamespace to include legacy URL aliases (elastic#115459)
  [Unified Integrations] Remove and cleanup add data views (elastic#115424)
  [Discover] Show ignored field values (elastic#115040)
  [ML] Stop reading the ml.max_open_jobs node attribute (elastic#115524)
  [Discover] Improve doc viewer code in Discover (elastic#114759)
  [Security Solutions] Adds security detection rule actions as importable and exportable (elastic#115243)
  [Security Solution] [Platform] Migrate legacy actions whenever user interacts with the rule (elastic#115101)
  [Fleet] Add telemetry for integration cards (elastic#115413)
  🐛 Fix single percentile case when ES is returning no buckets (elastic#115214)
  ...

# Conflicts:
#	x-pack/plugins/reporting/public/management/__snapshots__/report_listing.test.tsx.snap
kibanamachine added a commit that referenced this pull request Oct 19, 2021
@kibanamachine @dej611
🐛 Fix single percentile case when ES is returning no buckets (#115214) (
055ab5b 
#115568)

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: Marco Liberati <dej611@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jloleysens jloleysens jloleysens approved these changes

Assignees

No one assigned

Labels

auto-backport Deprecated - use backport:version if exact versions are needed Feature:Aggregations Aggregation infrastructure (AggConfig, esaggs, ...) release_note:fix v7.15.2 v7.16.0 v8.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Aggs] Single percentile error on index with many docs

4 participants

@dej611 @shahzad31 @kibanamachine @jloleysens