[9.3] [Obs AI] Replace get_data_sources with get_index_info tool (#248234)#249117
Closed
sorenlouv wants to merge 3 commits intoelastic:9.3from
Closed
[9.3] [Obs AI] Replace get_data_sources with get_index_info tool (#248234)#249117sorenlouv wants to merge 3 commits intoelastic:9.3from
get_data_sources with get_index_info tool (#248234)#249117sorenlouv wants to merge 3 commits intoelastic:9.3from
Conversation
…ic#248234) Closes elastic/obs-ai-team#455 This PR introduces the `get_index_info` tool which replaces `get_data_sources` and adds field discovery capabilities. This is similar to the `get_dataset_info` tool we have for Obs AI Assistant. ### What it does The tool has three operations: **`get_index_info({ operation: "get-overview" })`** Returns the same data sources as `get_data_sources` (APM indices, logs, metrics, alerts) plus a list of curated observability fields that exist in the cluster. Each field includes a `schema` indicator (`ecs`, `otel`, or `internal`). **`get_index_info({ operation: "list-fields", index, start?, end?, kqlFilter?, intent? })`** Returns fields with actual data. If the LLM specifies an `intent` and there are >100 fields, we filter them using a model to just the relevant ones. **`get_index_info({ operation: "get-field-values", index, fields })`** Returns field values: - Distinct values for keyword fields - Min/max ranges for numeric and date fields ### Purpose of tool The LLM needs to know what fields exist in the user's cluster before building queries. Without this, it guesses field names which leads to invalid filters and confusing errors. This is especially important because customers can use different schemas (ECS vs OTel). ## Manual testing Run the synthtrace scenario to populate observability indices with test data: ```bash node scripts/synthtrace \ src/platform/packages/shared/kbn-synthtrace/src/scenarios/agent_builder/tools/get_index_info/curated_fields.ts \ --from "now-15m" --to "now" --clean --workers=1 ``` ### Execute tool to get overview ``` POST kbn:///api/agent_builder/tools/_execute { "tool_id": "observability.get_index_info", "tool_params": { "operation": "get-overview" } } ``` ### Execute tool to get field values ``` POST kbn:///api/agent_builder/tools/_execute { "tool_id": "observability.get_index_info", "tool_params": { "operation": "get-field-values", "index": "metrics-*", "fields": "host.name" } } ``` (cherry picked from commit b6be8eb) # Conflicts: # src/platform/packages/shared/kbn-synthtrace/src/scenarios/agent_builder/index.ts # x-pack/solutions/observability/plugins/observability_agent_builder/server/tools/get_data_sources/tool.ts # x-pack/solutions/observability/plugins/observability_agent_builder/server/tools/get_services/tool.ts # x-pack/solutions/observability/plugins/observability_agent_builder/server/tools/index.ts # x-pack/solutions/observability/test/api_integration_deployment_agnostic/apis/observability_agent_builder/index.ts # x-pack/solutions/observability/test/api_integration_deployment_agnostic/apis/observability_agent_builder/tools/get_data_sources.spec.ts
botelastic
bot
added
the
Team:obs-presentation
Contributor
|
Pinging @elastic/obs-presentation-team (Team:obs-presentation) |
Contributor
💚 Build Succeeded
Metrics [docs]
History
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto9.3:get_data_sourceswithget_index_infotool (#248234)Questions ?
Please refer to the Backport tool documentation