[Agent builder] Builtin Product documentation tool by stephmilovic · Pull Request #242598 · elastic/kibana

stephmilovic · 2025-11-11T20:13:01Z

Summary

Adds a product documentation tool to Agent Builder that retrieves Elastic product documentation using the LlmTasks plugin. The tool uses the conditional availability pattern and is only available when product documentation is installed.

Changes

Added platform.core.product_documentation builtin tool to agent_builder_platform that searches Elastic product docs
Tool is registered in plugin setup() with conditional availability using the availability configuration pattern
Availability is checked at runtime via llmTasks.retrieveDocumentationAvailable() using the inference endpoint from the connector configuration
Tool dynamically extracts inferenceId from the connector config when using an inference connector, falling back to the default ELSER endpoint (defaultInferenceEndpoints.ELSER) for other connector types
Tool accesses llmTasks via closure through coreSetup.getStartServices() instead of through service dependencies

Technical Details

Added llmTasks as optional dependency to agent_builder_platform plugin (kibana.jsonc and PluginStartDependencies)
Tool registration moved to registerTools() in plugin setup() phase, following the same pattern as other built-in tools
Uses availability configuration with cacheMode: 'space' to conditionally show/hide the tool based on product documentation availability
llmTasks is accessed via closure in the tool definition, not passed through ToolHandlerContext or internal service layers
Availability handler uses coreSetup.getStartServices() to access llmTasks at runtime
Defaults to defaultInferenceEndpoints.ELSER, TODO to update to dynamic inferenceId

Considerations

Tool requires product documentation to be installed via the product-doc-base plugin
Tool availability is checked per-space and cached for performance
No Kibana restart required - tool appears/disappears dynamically based on availability

… into product_docs_ab_tool

elasticmachine · 2025-11-12T21:14:23Z

Pinging @elastic/security-solution (Team: SecuritySolution)

pgayvallet

The tool implementation looks good. I have concerns on how the tool is being registered and how we propagate the required llmTasks dependency to it (more details on slack)

… into product_docs_ab_tool

stephmilovic · 2025-11-17T16:29:49Z

@elasticmachine merge upstream

stephmilovic · 2025-11-17T16:31:52Z

+        // Try to get inferenceId from the connector, fall back to default ELSER endpoint
+        let inferenceId = defaultInferenceEndpoints.ELSER;
+        if (connector.type === InferenceConnectorType.Inference && connector.config?.inferenceId) {
+          inferenceId = connector.config.inferenceId;
+        }


@pgayvallet not sure this is the best way to make the inference endpoint configurable, but not sure another way at the moment. Do you think we should go forward with getting it off the connector, should I just default it to ELSER or do you see another way forward?

I don't have a better idea. Now that we're getting "unified" through AB, I imagine we should have a platform level setting for the embedding model we use for our semantic_text fields at some point. Until we do...

One question - is that inferenceId on the inference connector config something users can already configure to specify an embedding model to use for a specific connector? if so, I'm fine using it. Otherwise I would just KISS and default to using ELSER until we figure out how to do it better.

I think inferenceId is only on inference connector type:

elastic-llm: name: Elastic LLM actionTypeId: .inference exposeConfig: true config: provider: 'elastic' taskType: 'chat_completion' inferenceId: '.rainbow-sprinkles-elastic' providerConfig: model_id: 'rainbow-sprinkles'

And considering we're expecting .elser-2-elasticsearch or .multilingual-e5-small-elasticsearch, that doesn't seem right. I'll hardcode to ELSER for now and leave a comment

pgayvallet

LGTM

pgayvallet · 2025-11-18T13:45:08Z

+        // Try to get inferenceId from the connector, fall back to default ELSER endpoint
+        let inferenceId = defaultInferenceEndpoints.ELSER;
+        if (connector.type === InferenceConnectorType.Inference && connector.config?.inferenceId) {
+          inferenceId = connector.config.inferenceId;
+        }


I don't have a better idea. Now that we're getting "unified" through AB, I imagine we should have a platform level setting for the embedding model we use for our semantic_text fields at some point. Until we do...

One question - is that inferenceId on the inference connector config something users can already configure to specify an embedding model to use for a specific connector? if so, I'm fine using it. Otherwise I would just KISS and default to using ELSER until we figure out how to do it better.

stephmilovic · 2025-11-24T15:06:35Z

@elasticmachine merge upstream

elasticmachine · 2025-11-24T15:06:38Z

merge conflict between base and head

elasticmachine · 2025-11-30T20:45:28Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 3282c73

Failed CI Steps

Scout Test Run Builder

Test Failures

[job] [logs] Scout Test Run Builder / checkbox
[job] [logs] Scout Test Run Builder / serverless-security - EUI testing wrapper: EuiCheckBox - checkbox
[job] [logs] Scout Test Run Builder / serverless-security - EUI testing wrapper: EuiComboBox - with multiple selections (pills)
[job] [logs] Scout Test Run Builder / with multiple selections (pills)

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`onechat`	465.4KB	465.5KB	+49.0B

History

💚 Build #362363 succeeded 414df60
💚 Build #361983 succeeded e71f2dd
💚 Build #361519 succeeded 24b046d
💚 Build #360761 succeeded caa6709
💔 Build #360743 failed 99b53ec
💔 Build #360586 failed 779e84c

…ocumentation (#245749) ## Summary This PR adds a new `Documentation` section to the `GenAI Settings` page for managing the installation of documentation assets like the [Elastic Documentation](https://www.elastic.co/docs/reference/kibana/configuration-reference/ai-assistant-settings) used by the [Product Docs Platform Tool](#242598) and the Security Labs content used by the `security.security_labs_search` tool Design issue: elastic/ai-enhancements#77 <p align="center"> <img width="700" src="https://github.com/user-attachments/assets/46d7805e-5ef1-4213-8ec9-25875a4f039d" /> </p> > [!NOTE] > Security Labs content is not yet served up by the product docs CDN, so it is currently a disabled placeholder until I work #244946 next. We can hide this item for now if that is preferred. ### Implementation notes * The `product_docs` API's are called directly from the client instead of plumbing a new AB API. In support of this, the `llm_product_doc` privilege was added to the `ONECHAT_FEATURE_ID`. If this is not desired, we can remove this addition and plumb a dedicated API. * This UI section should be conditionally visible based on if the Agent Builder experience is enabled/feature is available. ~I need to confirm, but I believe this is coming in #244532 This functionality has been added. * Client hooks for managing product docs were added to `/ai_infra/product_doc_base` instead of `gen_ai_settings`. I originally had them in the setting public code, but figured they made more sense alongside the product docs. Happy to change is there is preference here. * As previously discussed, we're only supporting installing ELSER embeddings at the moment. ~We'll probably want to update the Platform Docs tool do instruct the model to do query re-writing to english in support of this.~ This is done in: #245259 * This uses the same product docs API as the O11y/Security Assistants, so there is no compatibility issues when switching the AB experience on/off or using the old assistants. * RBAC support provided such that documentation management actions are disabled unless the user has `agentBuilder['all']` Kibana feature privileges. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [X] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) _PR developed with Cursor + Opus 4.5_ --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…ocumentation (elastic#245749) ## Summary This PR adds a new `Documentation` section to the `GenAI Settings` page for managing the installation of documentation assets like the [Elastic Documentation](https://www.elastic.co/docs/reference/kibana/configuration-reference/ai-assistant-settings) used by the [Product Docs Platform Tool](elastic#242598) and the Security Labs content used by the `security.security_labs_search` tool Design issue: elastic/ai-enhancements#77 <p align="center"> <img width="700" src="https://github.com/user-attachments/assets/46d7805e-5ef1-4213-8ec9-25875a4f039d" /> </p> > [!NOTE] > Security Labs content is not yet served up by the product docs CDN, so it is currently a disabled placeholder until I work elastic#244946 next. We can hide this item for now if that is preferred. ### Implementation notes * The `product_docs` API's are called directly from the client instead of plumbing a new AB API. In support of this, the `llm_product_doc` privilege was added to the `ONECHAT_FEATURE_ID`. If this is not desired, we can remove this addition and plumb a dedicated API. * This UI section should be conditionally visible based on if the Agent Builder experience is enabled/feature is available. ~I need to confirm, but I believe this is coming in elastic#244532 This functionality has been added. * Client hooks for managing product docs were added to `/ai_infra/product_doc_base` instead of `gen_ai_settings`. I originally had them in the setting public code, but figured they made more sense alongside the product docs. Happy to change is there is preference here. * As previously discussed, we're only supporting installing ELSER embeddings at the moment. ~We'll probably want to update the Platform Docs tool do instruct the model to do query re-writing to english in support of this.~ This is done in: elastic#245259 * This uses the same product docs API as the O11y/Security Assistants, so there is no compatibility issues when switching the AB experience on/off or using the old assistants. * RBAC support provided such that documentation management actions are disabled unless the user has `agentBuilder['all']` Kibana feature privileges. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [X] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) _PR developed with Cursor + Opus 4.5_ --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

stephmilovic added 4 commits November 11, 2025 11:53

wip

0e45c52

fix tool description and schema

461686c

rm

5996a99

fixes

779e84c

stephmilovic added release_note:enhancement backport:skip This PR does not require backporting Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v9.3.0 labels Nov 11, 2025

stephmilovic and others added 7 commits November 12, 2025 07:29

Merge branch 'main' into product_docs_ab_tool

771f19e

Changes from node scripts/lint_ts_projects --fix

8252602

Merge branch 'product_docs_ab_tool' of github.com:stephmilovic/kibana…

f2ee19c

… into product_docs_ab_tool

Merge branch 'main' into product_docs_ab_tool

b1d415c

shorter description

15f004b

product documentation registration only when available

67e00dd

update tool services tests

99b53ec

stephmilovic marked this pull request as ready for review November 12, 2025 21:14

stephmilovic requested a review from a team as a code owner November 12, 2025 21:14

stephmilovic and others added 2 commits November 12, 2025 14:56

promises for types

b356eef

Changes from node scripts/eslint_all_files --no-cache --fix

caa6709

pgayvallet reviewed Nov 13, 2025

View reviewed changes

stephmilovic and others added 8 commits November 14, 2025 09:39

Merge branch 'main' into product_docs_ab_tool

ccf5860

refactor

85f22ee

more fixing

6be7a9f

Changes from node scripts/lint_ts_projects --fix

2856fb6

rm async

789b159

Merge branch 'product_docs_ab_tool' of github.com:stephmilovic/kibana…

6f4bea1

… into product_docs_ab_tool

revert plugin

5afd16c

dynamic inference id

24b046d

Merge branch 'main' into product_docs_ab_tool

e71f2dd

stephmilovic commented Nov 17, 2025

View reviewed changes

pgayvallet approved these changes Nov 18, 2025

View reviewed changes

stephmilovic added 2 commits November 18, 2025 08:22

rm connector inferenceId code

7382505

Merge branch 'main' into product_docs_ab_tool

414df60

stephmilovic and others added 2 commits November 25, 2025 11:49

Merge remote-tracking branch 'upstream/main' into product_docs_ab_tool

0911d2a

Changes from node scripts/regenerate_moon_projects.js --update

63b605a

stephmilovic mentioned this pull request Nov 25, 2025

Security attachments in Agent Builder #243574

Merged

stephmilovic and others added 6 commits November 30, 2025 12:24

Merge remote-tracking branch 'upstream/main' into product_docs_ab_tool

d9a976f

move to new plugin

bad8112

productDocumentationTool

c52a35e

Changes from node scripts/lint_ts_projects --fix

5eaae5c

Changes from node scripts/regenerate_moon_projects.js --update

6409546

Changes from node scripts/eslint_all_files --no-cache --fix

3282c73

stephmilovic merged commit 36d7faa into elastic:main Dec 2, 2025
12 checks passed

NicholasPeretti pushed a commit to NicholasPeretti/kibana that referenced this pull request Dec 2, 2025

[Agent builder] Builtin Product documentation tool (elastic#242598)

7cb4470

JordanSh pushed a commit to JordanSh/kibana that referenced this pull request Dec 9, 2025

[Agent builder] Builtin Product documentation tool (elastic#242598)

ecf2e79

spong mentioned this pull request Dec 10, 2025

[Agent Builder] Adds GenAI Settings UI for managing installation of documentation #245749

Merged

4 tasks

Conversation

stephmilovic commented Nov 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Technical Details

Considerations

Uh oh!

elasticmachine commented Nov 12, 2025

Uh oh!

pgayvallet left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stephmilovic commented Nov 17, 2025

Uh oh!

stephmilovic Nov 17, 2025

Choose a reason for hiding this comment

Uh oh!

pgayvallet Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

stephmilovic Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

pgayvallet left a comment

Choose a reason for hiding this comment

Uh oh!

pgayvallet Nov 18, 2025

Choose a reason for hiding this comment

Uh oh!

stephmilovic commented Nov 24, 2025

Uh oh!

elasticmachine commented Nov 24, 2025

Uh oh!

elasticmachine commented Nov 30, 2025

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Async chunks

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

stephmilovic commented Nov 11, 2025 •

edited

Loading