You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The Log AI Insight attachment doesn't appear when using ES|QL mode in Discover. In KQL/Lucene mode, the standard _search API automatically includes _id and _index metadata on every hit. ES|QL, however, returns tabular results where metadata fields are opt-in via the METADATA clause. The Log AI Insight component relies on _id and _index to identify documents, so it silently returns null when they're missing effectively making the feature unavailable in ES|QL mode.
Solution
Auto-inject METADATA _id, _index into ES|QL queries before execution in Discover's fetchEsql, like KQL mode. Uses @elastic/esql AST utilities (copied approach Security uses for _id injection in detection rules into kbn-esql-utils) to safely modify the query, with KEEP command compatibiity to prevent the injected fields from being dropped. Falls back to the original query on failure.
The reason will be displayed to describe this comment to others. Learn more.
This PR still has test failures, and unless something has changed, our rule in Discover has been to not silently modify the main query before running it. @stratoula wdyt?
[job][logs] src/platform/test/functional/apps/discover/esql/config.ts / discover/esql discover esql view errors should show error messages for syntax errors in query
[job][logs] FTR Configs #20 / discover/esql discover esql view errors should show error messages for syntax errors in query
[job][logs] FTR Configs #30 / discover/esql discover esql view errors should show error messages for syntax errors in query
[job][logs] FTR Configs #100 / discover/esql discover esql view errors should show error messages for syntax errors in query
[job][logs] FTR Configs #109 / discover/esql discover esql view errors should show error messages for syntax errors in query
[job][logs] src/platform/test/functional/apps/discover/esql/config.ts / discover/esql discover esql view errors should show error messages for syntax errors in query
[job][logs] FTR Configs #20 / discover/esql discover esql view errors should show error messages for syntax errors in query
[job][logs] FTR Configs #109 / discover/esql discover esql view errors should show error messages for syntax errors in query
[job][logs] FTR Configs #30 / discover/esql discover esql view errors should show error messages for syntax errors in query
[job][logs] FTR Configs #100 / discover/esql discover esql view errors should show error messages for syntax errors in query
Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.
This PR still has test failures, and unless something has changed, our rule in Discover has been to not silently modify the main query before running it. @stratoula wdyt?
Yes I agree, this PR comes out of surprise. We don't want to do "magic" things in discover. This hasn't changed. We were discussing with ES the possibility to have a Setting for this but this hasn't prioritized. This is also tricky technically, it adds complexity that we don't want.
I suggest to bring the problem you are trying to solve in one discover sync. Then we can discuss alternatives and also check with ES.
@stratoula@davismcphee Understandable and sorry for the noise. I've opened a workaround PR using available fields instead of fetching the fresh doc with the id #258595. This is actually sufficient for our needs. It would be nice-to-have, to have the id and index available.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
neptunian
added
release_note:fix
backport:version
davismcphee
left a comment
Summary
Problem
The Log AI Insight attachment doesn't appear when using ES|QL mode in Discover. In KQL/Lucene mode, the standard
_searchAPI automatically includes_idand_indexmetadata on every hit. ES|QL, however, returns tabular results where metadata fields are opt-in via theMETADATAclause. The Log AI Insight component relies on_idand_indexto identify documents, so it silently returnsnullwhen they're missing effectively making the feature unavailable in ES|QL mode.Solution
Auto-inject
METADATA _id, _indexinto ES|QL queries before execution in Discover'sfetchEsql, like KQL mode. Uses@elastic/esqlAST utilities (copied approach Security uses for_idinjection in detection rules intokbn-esql-utils) to safely modify the query, withKEEPcommand compatibiity to prevent the injected fields from being dropped. Falls back to the original query on failure.