Skip to content

[Cascade] Handle grouping on fields with unset values#260033

Merged
eokoneyo merged 3 commits intoelastic:mainfrom
eokoneyo:chore/resolve-260026
Mar 31, 2026
Merged

[Cascade] Handle grouping on fields with unset values#260033
eokoneyo merged 3 commits intoelastic:mainfrom
eokoneyo:chore/resolve-260026

Conversation

@eokoneyo
Copy link
Copy Markdown
Contributor

@eokoneyo eokoneyo commented Mar 27, 2026

Summary

Closes #260026

Any record with an unset field is represented as "(null)", whilst constructing queries for the cascade experience when a value that matches this string is encountered, it receives a special treatment;

  • For column operations on the leaf node; an encounter with a column with the value of "(null)", will result in a query where the we search for a column that is null, this is reproducible with the sample data that's available by default in Kibana, from the index kibana_sample_data_logs, running a STATS query on the column memory we encounter some records where the fields value is not set in this scenario, we'd use a query similar to the one below;
FROM kibana_sample_data_logs
  | INLINE STATS count = COUNT(*), avg_bytes = AVG(bytes), p95 = PERCENTILE(memory, 95), median_ram_size = MEDIAN(machine.ram) BY memory
  | WHERE memory IS NULL
  • For filtering operations, to filter in we apply a where clause that asserts the columns is null like so; WHERE column IS NULL, and to filter out we assert that we want all columns that aren't null like so WHERE column IS NOT NULL.

P.S. It should be noted we don't apply this conditions to the CATEGORIZE function.

@eokoneyo eokoneyo self-assigned this Mar 27, 2026
@eokoneyo eokoneyo added release_note:skip Skip the PR/issue when compiling release notes Team:DataDiscovery Discover, search (data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. t// labels Mar 27, 2026
@eokoneyo eokoneyo changed the title ensure the first cascade group has at least one value [Cascade] Ensure theres at least one row with a value for the selected cascade group Mar 27, 2026
@eokoneyo eokoneyo added the backport:skip This PR does not require backporting label Mar 30, 2026
@eokoneyo eokoneyo force-pushed the chore/resolve-260026 branch from 74ec839 to b1963ce Compare March 30, 2026 14:50
@eokoneyo eokoneyo changed the title [Cascade] Ensure theres at least one row with a value for the selected cascade group [Cascade] Handle grouping on fields with unset values Mar 30, 2026
@eokoneyo
Copy link
Copy Markdown
Contributor Author

/ci

@eokoneyo
Copy link
Copy Markdown
Contributor Author

@elasticmachine merge upstream

@eokoneyo eokoneyo marked this pull request as ready for review March 30, 2026 16:55
@eokoneyo eokoneyo requested review from a team as code owners March 30, 2026 16:55
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

@eokoneyo eokoneyo force-pushed the chore/resolve-260026 branch from 9d78ca8 to 9ae41e6 Compare March 30, 2026 18:27
Copy link
Copy Markdown
Contributor

@davismcphee davismcphee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as expected, thanks!

@eokoneyo
Copy link
Copy Markdown
Contributor Author

@elasticmachine merge upstream

@elasticmachine
Copy link
Copy Markdown
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
discover 1.6MB 1.6MB +352.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
discover 27.5KB 27.6KB +19.0B
Unknown metric groups

API count

id before after diff
@kbn/esql-utils 206 207 +1

History

cc @eokoneyo

Copy link
Copy Markdown
Contributor

@stratoula stratoula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@eokoneyo eokoneyo merged commit dd30fa2 into elastic:main Mar 31, 2026
18 checks passed
@eokoneyo eokoneyo deleted the chore/resolve-260026 branch March 31, 2026 07:38
mbondyra added a commit to mbondyra/kibana that referenced this pull request Mar 31, 2026
…e_for_children6

* commit '3402744f63ca1196e97b11ffac4e7f7efab240df': (80 commits)
  [PerUserAuth] Add EARS auth type for Connectors V2 (elastic#253695)
  Fix `@elastic/eui/require-aria-label-for-modals` lint violations across `@elastic/kibana-core` files (elastic#259757)
  [Entity Analytics][Leads generation][4] Add API routes, LeadDataClient, and async generation (elastic#257046)
  [Agent Builder] Agent-centric UX redesign (elastic#258005)
  fix query streams failing test (elastic#260277)
  [Lens as code] Add list layout to the new API (elastic#259967)
  [FTR] Add warning comments to deployment-agnostic FTR base configs (elastic#260018)
  [Discover][Logs profile] Fix missing search highlights (elastic#260056)
  Plugin system: safe deletion (elastic#259038)
  [Infra] Fix Hosts filter options to match selected schema (elastic#259825)
  Manual Entity Resolution and flyout representation (elastic#260162)
  [Cascade] Handle grouping on fields with unset values (elastic#260033)
  [Fleet] generate OTel config for integration packages with otelcol inputs (elastic#259968)
  [Search] Switch over to V2 index management details (elastic#259866)
  [inference] increase timeout for ES inference calls (elastic#260382)
  [ES|QL] Enable subqueries (elastic#257455)
  [ES|QL] Change Point order free options (elastic#260282)
  [Auth] Added authentication strategy for UIAM OAuth (elastic#256182)
  [Security Solution] Add "alerts_candidate_count" rule execution metric (elastic#259917)
  [api-docs] 2026-03-31 Daily api_docs build (elastic#260380)
  ...
jeramysoucy pushed a commit to jeramysoucy/kibana that referenced this pull request Apr 1, 2026
## Summary

Closes elastic#260026

Any record with an unset field is represented as "(null)", whilst
constructing queries for the cascade experience when a value that
matches this string is encountered, it receives a special treatment;

- For column operations on the leaf node; an encounter with a column
with the value of "(null)", will result in a query where the we search
for a column that is null, this is reproducible with the sample data
that's available by default in Kibana, from the index
**kibana_sample_data_logs**, running a STATS query on the column
`memory` we encounter some records where the fields value is not set in
this scenario, we'd use a query similar to the one below;

```txt
FROM kibana_sample_data_logs
  | INLINE STATS count = COUNT(*), avg_bytes = AVG(bytes), p95 = PERCENTILE(memory, 95), median_ram_size = MEDIAN(machine.ram) BY memory
  | WHERE memory IS NULL
```
- For filtering operations, to filter in we apply a where clause that
asserts the columns is null like so; `WHERE column IS NULL`, and to
filter out we assert that we want all columns that aren't null like so
`WHERE column IS NOT NULL`.


P.S. It should be noted we don't apply this conditions to the CATEGORIZE
function.

<!--

### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- [ ] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...

-->

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
paulinashakirova pushed a commit to paulinashakirova/kibana that referenced this pull request Apr 2, 2026
## Summary

Closes elastic#260026

Any record with an unset field is represented as "(null)", whilst
constructing queries for the cascade experience when a value that
matches this string is encountered, it receives a special treatment;

- For column operations on the leaf node; an encounter with a column
with the value of "(null)", will result in a query where the we search
for a column that is null, this is reproducible with the sample data
that's available by default in Kibana, from the index
**kibana_sample_data_logs**, running a STATS query on the column
`memory` we encounter some records where the fields value is not set in
this scenario, we'd use a query similar to the one below;

```txt
FROM kibana_sample_data_logs
  | INLINE STATS count = COUNT(*), avg_bytes = AVG(bytes), p95 = PERCENTILE(memory, 95), median_ram_size = MEDIAN(machine.ram) BY memory
  | WHERE memory IS NULL
```
- For filtering operations, to filter in we apply a where clause that
asserts the columns is null like so; `WHERE column IS NULL`, and to
filter out we assert that we want all columns that aren't null like so
`WHERE column IS NOT NULL`.


P.S. It should be noted we don't apply this conditions to the CATEGORIZE
function.

<!--

### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This was checked for breaking HTTP API changes, and any breaking
changes have been approved by the breaking-change committee. The
`release_note:breaking` label should be applied in these situations.
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
- [ ] Review the [backport
guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing)
and apply applicable `backport:*` labels.

### Identify risks

Does this PR introduce any risks? For example, consider risks like hard
to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified
risk. Invite stakeholders and evaluate how to proceed before merging.

- [ ] [See some risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx)
- [ ] ...

-->

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:DataDiscovery Discover, search (data plugin and KQL), data views, saved searches. For ES|QL, use Team:ES|QL. t// v9.4.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Discover][GroupBy] Table blank when count is 1

5 participants