update knowledge base installation flow by arturoliduena · Pull Request #214133 · elastic/kibana

arturoliduena · 2025-03-12T11:45:32Z

Summary

Closes #209791
Closes #204378

Improve Knowledge Base Installation and Inference Endpoint Management

Enhancements

The "Install Knowledge Base" button is only shown when there is no inference endpoint. If the inference endpoint creation fails, an error notification is displayed.
When an inference endpoint exists and is ready, no buttons or messages are shown.
If the installation was in progress and succeeded, a success message is displayed.
If an inference endpoint is present, users can inspect the state of the model deployment to diagnose issues.
When an inference endpoint exists but has an error, users can retry the installation.

Inspection & Troubleshooting

Users can inspect the inference endpoint's deployment status, including:

Model is deploying
No ML node available
Insufficient memory available

My.Movie.1.-.1.mp4

Suggested Improvements

Display autoscaling information for better resource insights.
Show ML node details to help users understand the infrastructure.
Allow users to install, deploy, and stop the model directly from the inspection popup.

These changes enhance visibility, troubleshooting, and control over inference endpoint deployment. 🚀

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

Identify risks

Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss.

Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging.

See some risk examples
...

elasticmachine · 2025-03-12T11:45:36Z

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

neptunian · 2025-03-12T12:06:06Z

I tried testing the component and nearly all the unit tests were failing https://github.com/elastic/kibana/blob/main/x-pack/platform/packages/shared/kbn-ai-assistant/src/chat/welcome_message_knowledge_base.test.tsx . We should update/add tests to track every possible state this component could be in as it's very prone to regressions because there are so many states and we are trying to map ML stats and the inference endpoint state to the concept of an installation state and a general health state.

neptunian · 2025-03-12T12:20:50Z

Previously, after adding an AI connector, the knowledge base would start installing automatically. Now, I get a message it needs to be installed. After closer inspection looking at the /setup request it is actually installing but we are not showing the installation message, so the user will try to install again while its installing.

neptunian · 2025-03-12T12:28:28Z

x-pack/platform/packages/shared/kbn-ai-assistant/src/chat/knowledge_base_callout.tsx

- * 2.0; you may not use this file except in compliance with the Elastic License
- * 2.0.
- */
-


was this file not in use? I wonder why it stopped being used. It has the messaging "Improve your experience by setting up the knowledge base." which seems like what we should show users who haven't yet installed it instead of "You knowledge base hasn't been setup".

CC @isaclfreire

it wasn’t being used. that's why I deleted it, should we keep it in case we want to use it in the future @isaclfreire?

neptunian · 2025-03-12T12:35:03Z

Why are we now showing the "inspect" while a user is installing?

clintandrewhall · 2025-03-12T13:37:56Z

@arturoliduena Just a heads up, and certainly not a blocker to this PR: the "error state" is currently being unified between @kbn/elastic-assistant and @kbn/ai-assistant. It's being reviewed and revised at the moment. It's next up for us to build as an independent component package, after Knowledge Base install status.

A lot of this feedback is useful to @isaclfreire and @julianrosado, as it's an early indication of whether the new design is going to cover the edge cases.

That said, I'd happily work with you to revise this PR to adopt the new error state design! Let me know where this stands in your current workload. Regardless, I'll be in touch to make sure the new component works as intended.

clintandrewhall · 2025-03-12T13:42:43Z

the knowledge base would start installing automatically

This would be inconsistent with the unified design. @kbn/elastic-assistant doesn't install it automatically, either. The new unified flow prompts for the KB install, presumably for a reason.

@isaclfreire @julianrosado

sorenlouv · 2025-03-12T14:06:56Z

the knowledge base would start installing automatically

This would be inconsistent with the unified design. @kbn/elastic-assistant doesn't install it automatically, either. The new unified flow prompts for the KB install, presumably for a reason.

@isaclfreire @julianrosado

Agreee - I don't think we should install the KB automatically. Even after selecting a model.

arturoliduena · 2025-03-12T15:28:44Z

Why are we now showing the "inspect" while a user is installing?
fixed, inspect button display when there is a inference endpoint and a status. @neptunian - thanks for catching this.

neptunian · 2025-03-12T16:59:17Z

x-pack/platform/packages/shared/kbn-ai-assistant/src/chat/welcome_message_knowledge_base.tsx

+    const MAX_ATTEMPTS = 5;
+
+    try {
+      // install


why was this code moved from the hook?

The code was moved from the hook because the logic belongs to the Installation KB component. There aren’t any other components handling KB installation, so keeping it here makes it more self-contained and easier to manage.

The component is more difficult to read with all the installation state here. I think having it in the hook was a lot neater for this amount of state complexity. I don't think the fact it's used by one component means we cannot have it there.

neptunian · 2025-03-12T17:03:53Z

Agreee - I don't think we should install the KB automatically. Even after selecting a model.

Also agree. Though, In that case I think we need better messaging. Right now it almost seems like we're displaying a message when something is amiss where it appears there used to be CTA messaging https://github.com/elastic/kibana/pull/214133/files#diff-e7c52625ed5cb6328bc801e2cfd5c7a96a02c5679526e0ca1458b359f4a32618L100

neptunian · 2025-03-18T20:34:13Z

x-pack/platform/packages/shared/kbn-ai-assistant/src/chat/welcome_message_knowledge_base.tsx

-  const prevIsInstalling = usePrevious(knowledgeBase.isInstalling);
+  const { notifications, ml } = useKibana().services;
+
+  const isInstalling = !!knowledgeBase.status.value?.endpoint && !knowledgeBase.status.value?.ready;


Wouldn't this mean if, for eg, deployment_stats?.allocation_status?.allocation_count became 0, we'd be showing instaling message again? That's what i'm seeing.

isInstalling used to be part of the install process: https://github.com/elastic/kibana/blob/main/x-pack/platform/packages/shared/kbn-ai-assistant/src/hooks/use_knowledge_base.tsx#L41

We discussed this in our meeting and this is intentional. The idea is to keep the user in the "installing" process with an "inspect" if the node is being created after being scaled down. I do find this UX to be confusing as the user already "installed the knowledge base" at this point and we're showing the same message as when they first install it plus an inspect link. So a user would probably see this often depending how often auto scaling occurs? Fine with it as an improvement from the current functionality of showing the installation button. CC @isaclfreire

Could we change the message to:

Knowledge base has been installed. Waiting for the model to be deployed on an available ML node.

or alternatively omitting anything about installing the KB and simply stating:

Waiting for the model to be deployed on an available ML node.

That way we clearly indicate what's the holdup, thereby also helping them diagnosing the problem (no available ML nodes)

clintandrewhall · 2025-03-20T19:21:21Z

FYI-- I've opened #215207 which will undoubtedly impact this PR, and vice versa. I'd definitely want this team's expertise in making sure these changes are compatible with the conversations here, but also if there are any changes I can make.

Keep in mind: unified generic components for Knowledge Base installation and error handling are up next.

x-pack/platform/packages/shared/kbn-ai-assistant/src/hooks/use_knowledge_base.tsx

x-pack/platform/packages/shared/kbn-ai-assistant/src/chat/welcome_message_knowledge_base.tsx

arturoliduena · 2025-03-24T11:26:36Z

FYI-- I've opened #215207 which will undoubtedly impact this PR, and vice versa. I'd definitely want this team's expertise in making sure these changes are compatible with the conversations here, but also if there are any changes I can make.

Keep in mind: unified generic components for Knowledge Base installation and error handling are up next.

Thanks, @clintandrewhall for the heads up! I’ve reviewed the changes on #215207 to make sure we’re aligned. It seems there isn’t much overlap, so I’ll keep an eye on it and provide feedback as needed.

We’re currently figuring out how to show the progress of the KB installation, so it might be best to merge this PR first and handle the UX changes in a separate PR where both (mine and yours PR) changes were apply.

arturoliduena · 2025-03-24T12:32:26Z

Hi @isaclfreire, I’d like to get your input on improving the UX for the KB installation process. There are a few areas in the setup and reconfiguration readiness deployment that are unclear, and I’d appreciate your insights. I’ll walk you through the current process:

Setting up genAI configuration:
When a user adds a connector from the AI assistant, the KB installation happens automatically. However, there have been discussions about potentially discontinuing this behavior.

When an AI Connector already exists:
The “Install KB” button is displayed. The installation proceeds in the background with a few steps: creating an inference endpoint, installing, and deploying the Elser model (if needed).

Installation Progress:
3.1 sucessfull flow:
it shows a spinner and "Setting up Knowledge base" message

the deployment status is show within the inspect button

3.2 error
If the installation fails, an error reason is shown with relevant details.

Installation Success Message:
Upon successful installation, a confirmation message is displayed.

After Setup:
Once the setup is complete, when the user returns to the AI Assistant, no message is displayed.

Deployment Status:
If the deployment status changes and the KB becomes unavailable, we display the status and reason. The “Install” button is not shown again because it’s not an installation error.

I’d love to hear your thoughts on how we can improve this flow and make the experience clearer for the user.

sorenlouv · 2025-03-24T14:03:35Z

When a user adds a connector from the AI assistant, the KB installation happens automatically. However, there have been discussions about potentially discontinuing this behavior.

With https://github.com/elastic/obs-ai-assistant-team/issues/161 we will have to ask users to pick a model before installing the knowledge base. So I don't believe we will be able to install the KB automatically, even if we wanted to

kibanamachine · 2025-04-09T09:44:55Z

💔 All backports failed

Status	Branch	Result
❌	8.x	Backport failed because of merge conflicts

Manual backport

To create the backport manually run:

node scripts/backport --pr 214133

Questions ?

Please refer to the Backport tool documentation

kibanamachine · 2025-04-11T09:48:54Z

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 214133 locally

arturoliduena · 2025-04-14T08:48:08Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

## Summary Closes [elastic#209791](elastic#209791) Closes [elastic#204378](elastic#204378) ### Improve Knowledge Base Installation and Inference Endpoint Management #### Enhancements - The **"Install Knowledge Base"** button is only shown when there is no inference endpoint. If the inference endpoint creation fails, an error notification is displayed. - When an inference endpoint **exists and is ready**, no buttons or messages are shown. - If the installation was in progress and **succeeded**, a success message is displayed. - If an inference endpoint is present, users can **inspect the state of the model deployment** to diagnose issues. - When an inference endpoint exists but has an **error**, users can retry the installation. #### Inspection & Troubleshooting Users can inspect the inference endpoint's deployment status, including: - **Model is deploying** - **No ML node available** - **Insufficient memory available** https://github.com/user-attachments/assets/03a91120-173c-4dc9-b6f2-59a8c8c8defb ### Suggested Improvements - Display **autoscaling information** for better resource insights. - Show **ML node details** to help users understand the infrastructure. - Allow users to **install, deploy, and stop the model** directly from the inspection popup. These changes enhance visibility, troubleshooting, and control over inference endpoint deployment. 🚀 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... (cherry picked from commit 4109420) # Conflicts: # x-pack/platform/plugins/shared/observability_ai_assistant/server/service/inference_endpoint.ts # x-pack/platform/plugins/shared/observability_ai_assistant/server/service/knowledge_base_service/index.ts

kibanamachine · 2025-04-14T09:56:45Z