[Investigations][Bug] - Fix alert page filtering by checking for empty dataView

[michaelolo24]

Summary

This PR fixes an issue with the alert page filtering when the below configuration is enabled:

When enabled, the config looks to make sure that searches are only done against index patterns that are mapped to the given dataView. This configuration is utilized in a utility function here

kibana/src/platform/packages/shared/kbn-es-query/src/es_query/from_filters.ts

Line 86 in 5aa463e

return !ignoreFilterIfFieldNotInIndex || filterMatchesIndex(filter, indexPattern);

that is used across security solution for querying.

When introducing the code to migrate to our new dataView picker here in the following PR #225726, a check was done to only apply the new DataView when it was provided. To fix a separate issue regarding flashing of the alerts page when loading, a default empty data view was introduced to fix the initial flash with the following initial dataView and was introduced with this pr: #225675.

In short, the empty dataView object was always defined, which only caused problems when this setting was turned on, because the field being filtered on was being checked against an empty dataView. It's not reproducible otherwise. And the impact area is limited to just SecuritySolution

There were no tests to account for this configuration, but those were added here to prevent this from occurring again.

The necessary checks are added in this PR

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/security-threat-hunting-investigations (Team:Threat Hunting:Investigations)

[PhilippeOberti]

Thanks for fixing this! I left a couple of minor non-blocker comments.

If I understand correctly, this was introduced in 9.1 but kind of fixed/hidden by my alerts page refactor PR, as in this one we don't display anything until the dataView is fully ready... That's why we couldn't reproduce it on main, correct?

[PhilippeOberti]

Any reason for using title here which is deprecated? Why not using getIndexPattern()?
Also maybe even better, we could use this hasMatchedIndices function?

[michaelolo24]

I'll change it, but the definition for getIndexPattern() is

  getIndexPattern = () => this.title;

😅. And I'm not using matchedIndices as the check is primarily against whether or not the dataView is empty (our INIT_DV in useDataView.ts) vs whether there are any matching indices in the env. I would prefer to pass down the newDataViewPickerEnabled check, but the surface area for that change is larger for a patch fix.

[PhilippeOberti]

I'll change it, but the definition for getIndexPattern() is getIndexPattern = () => this.title;

Yup yup I know, it's just if someone wants to remove the export of this.title then it's one less place to change 😉
Thanks for updating it!

[kqualters-elastic]

desk tested that this fixed the bug on main, lgtm 👍

[kqualters-elastic]

If I understand correctly, this was introduced in 9.1 but kind of fixed/hidden by my alerts page refactor PR, as in this one we don't display anything until the dataView is fully ready... That's why we couldn't reproduce it on main, correct?

I am able to reproduce on main, you need to enable the advanced setting: courier:ignoreFilterIfFieldNotInIndex: true to see it though.

[michaelolo24]

If I understand correctly, this was introduced in 9.1 but kind of fixed/hidden by my alerts page refactor #222457, as in this one we don't display anything until the dataView is fully ready... That's why we couldn't reproduce it on main, correct?

@PhilippeOberti no not really. This one actually happens because by default we don't check if the field is in a given dataView when searching, if you follow the code path here:

kibana/src/platform/packages/shared/kbn-es-query/src/es_query/from_filters.ts

Line 86 in 1a838aa

return !ignoreFilterIfFieldNotInIndex || filterMatchesIndex(filter, indexPattern);

specifically filterMatchesIndex you'll see that the failure happens because the dataView that it's checking against doesn't actually have the fields since the useDataView always returns a dataView now..even though it's empty

[kibanamachine]

Starting backport for target branches: 8.19, 9.1

https://github.com/elastic/kibana/actions/runs/17768603604

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 40eba20

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	10.5MB	10.5MB	+62.0B

Saved Objects .kibana field count

Every field in each saved object type adds overhead to Elasticsearch. Kibana needs to keep the total field count below Elasticsearch's default limit of 1000 fields. Only specify field mappings for the fields you wish to search on or query. See https://www.elastic.co/guide/en/kibana/master/saved-objects-service.html#_mappings

id	before	after	diff
_data_stream_timestamp	1	-	-1
_doc_count	1	-	-1
_ignored_source	1	-	-1
_index_mode	1	-	-1
_inference_fields	1	-	-1
_tier	1	-	-1
apm-custom-dashboards	5	-	-5
apm-server-schema	2	-	-2
apm-service-group	5	-	-5
application_usage_daily	2	-	-2
config	2	-	-2
config-global	2	-	-2
coreMigrationVersion	1	-	-1
created_at	1	-	-1
created_by	1	-	-1
entity-definition	9	-	-9
entity-discovery-api-key	2	-	-2
event_loop_delays_daily	2	-	-2
favorites	4	-	-4
file	11	-	-11
file-upload-usage-collection-telemetry	3	-	-3
fileShare	5	-	-5
infra-custom-dashboards	4	-	-4
infrastructure-monitoring-log-view	2	-	-2
intercept_trigger_record	5	-	-5
legacy-url-alias	7	-	-7
managed	1	-	-1
ml-job	6	-	-6
ml-module	13	-	-13
ml-trained-model	7	-	-7
monitoring-telemetry	2	-	-2
namespace	1	-	-1
namespaces	1	-	-1
observability-onboarding-state	2	-	-2
originId	1	-	-1
product-doc-install-status	7	-	-7
references	4	-	-4
sample-data-telemetry	3	-	-3
security-ai-prompt	8	-	-8
slo	11	-	-11
space	5	-	-5
synthetics-monitor	34	-	-34
synthetics-monitor-multi-space	34	-	-34
tag	4	-	-4
type	1	-	-1
typeMigrationVersion	1	-	-1
ui-metric	2	-	-2
updated_at	1	-	-1
updated_by	1	-	-1
upgrade-assistant-ml-upgrade-operation	3	-	-3
upgrade-assistant-reindex-operation	3	-	-3
uptime-synthetics-api-key	2	-	-2
url	5	-	-5
usage-counters	2	-	-2
total			-246

History

• 💛 Build #339604 was flaky 64eb1f5
• 💔 Build #339588 failed 4d712c0

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.19
✅	9.1

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation