[Obs AI Assistant] Anonymization support

[dgieselaar]

Re-submit of #216352 as it has merge conflicts and we don't have write permissions for Sandra's remote.

To test, add the following to your kibana.yml:

uiSettings:
  overrides:
    "observability:aiAssistantAnonymizationRules":
      - id: "ner"
        type: "ner"
        enabled: true
      - id: "beach"
        type: "regex"
        enabled: true
        pattern: "sandy"

[Copilot]

Pull Request Overview

Adds end-to-end anonymization support—including redaction, deanonymization, new types, UI settings, and chat‐level highlighting—to the Observability AI Assistant plugin.

• Introduces hash‐based redaction/unredaction utilities and service methods
• Defines new types (DetectedEntity, Unredaction, AnonymizationRule) and runtime schema updates
• Adds UI settings for anonymization rules and integrates entity highlighting in chat components

Reviewed Changes

Copilot reviewed 32 out of 32 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
server/service/anonymization/deanonymize_text.ts	New service to replace hashes with original values
server/service/anonymization/chunk_text.ts	Utility to split text into inference‐sized chunks
server/routes/runtime_types.ts	Added unredactions runtime type to message schema
public/plugin.tsx	Refactored config retrieval for scopeFromConfig
public/index.ts	Export anonymization utilities and new settings key
public/components/message_panel/message_text.tsx	Render highlighted/anonymized content when provided
common/utils/anonymization/redaction.ts	Core redaction and unhashing functions
common/types.ts	New interfaces for DetectedEntity, Unredaction, etc.
common/ui_settings/settings_keys.ts	Added key for anonymization rules
common/index.ts	Exported AnonymizationRule and settings key
private/.../server/plugin.ts	Excluded anonymization rules from core settings reg.
private/.../config.ts	Formatted exposeToBrowser object
private/.../settings_page.test.tsx	Reformatted test config object
private/.../common/ui_settings.ts	Defined UI setting for anonymization rules
packages/shared/kbn-ai-assistant/src/utils/builders.ts	Added unredactions to builder types
packages/shared/kbn-ai-assistant/src/chat/chat_timeline.tsx	Integrated UI logic for anonymization highlighting
packages/shared/kbn-ai-assistant/src/chat/chat_item_content_inline_prompt_editor.tsx	Passed anonymized content into MessageText
packages/shared/kbn-ai-assistant/src/chat/chat_item.tsx	Added text extraction from ReactNode for clipboard
packages/shared/ai-infra/inference-common/src/chat_complete/events.ts	Defined ChatCompletionUnredactedMessageEvent

Comments suppressed due to low confidence (3)

server/service/anonymization/deanonymize_text.ts:14

• The doc comment mentions placeholders with braces (e.g., "{hash123}"), but HASH_REGEX only matches raw 40-char hex strings. Update the doc or regex to stay consistent.

* Takes text containing hash placeholders like "{hash123}" and replaces

server/routes/runtime_types.ts:56

• unredactions is marked as required, but it should be optional to match the Message interface. Consider wrapping it in t.partial or using t.union([t.array(unredactionRt), t.undefined]).

unredactions: t.array(unredactionRt),

packages/shared/ai-infra/inference-common/src/chat_complete/events.ts:38

• The unredactions payload here includes only entity, class_name, and hash, but client highlighting relies on positional data (start_pos, end_pos). Consider adding those fields for consistency.

export interface ChatCompletionUnredactedMessageEvent<

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: d07e8c8
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-223351-d07e8c8dcdb1

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
observabilityAIAssistant	106	107	+1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
observabilityAIAssistant	427	443	+16

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
observabilityAIAssistantApp	173.8KB	174.8KB	+1013.0B
searchAssistant	100.1KB	101.1KB	+1007.0B
total			+2.0KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
observabilityAIAssistant	27	30	+3

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
observabilityAIAssistant	40.5KB	41.0KB	+485.0B

Unknown metric groups

API count

id	before	after	diff
observabilityAIAssistant	429	447	+18

History

[kibanamachine]

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15580615122

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.19	Backport failed because of merge conflicts You might need to backport the following PRs to 8.19: - [Obs AI Assistant] Add Elastic Managed LLM tour callout (#220782)

Manual backport

To create the backport manually run:

node scripts/backport --pr 223351

Questions ?

Please refer to the Backport tool documentation

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 223351 locally
cc: @dgieselaar

[kibanamachine]

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 223351 locally
cc: @dgieselaar

[neptunian]

💚 All backports created successfully

Status	Branch	Result
✅	8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation