Skip to content

[Obs AI Assistant] Anonymization support#216352

Closed
neptunian wants to merge 87 commits intoelastic:mainfrom
neptunian:poc-ner-anonymization
Closed

[Obs AI Assistant] Anonymization support#216352
neptunian wants to merge 87 commits intoelastic:mainfrom
neptunian:poc-ner-anonymization

Conversation

@neptunian
Copy link
Copy Markdown
Contributor

@neptunian neptunian commented Mar 28, 2025 •
edited
Loading

Summary

Implements anonymization support using NER (named entity recognition) model to detect entities (person, location, org) in combination with regex (url, email address, ip) before sending to LLM and deanonymizing response.

@neptunian neptunian changed the title [Obs AI Assistant][POC]anonyimization of direct user chat input [Obs AI Assistant][POC] anonyimization of entities communicating with LLM Apr 1, 2025
@neptunian neptunian changed the title [Obs AI Assistant][POC] anonyimization of entities communicating with LLM [Obs AI Assistant][POC] entity detection and anonymization Apr 1, 2025
@neptunian neptunian force-pushed the poc-ner-anonymization branch from 16d83c4 to de9c5e6 Compare April 15, 2025 17:45
@neptunian neptunian changed the title [Obs AI Assistant][POC] entity detection and anonymization [Obs AI Assistant] entity detection and anonymization Apr 18, 2025
neptunian added 19 commits April 22, 2025 16:01
@neptunian neptunian force-pushed the poc-ner-anonymization branch from 54ab13e to 20d2e5f Compare April 22, 2025 20:02
@neptunian neptunian marked this pull request as ready for review April 22, 2025 20:05
@neptunian neptunian requested review from a team as code owners April 22, 2025 20:05
@neptunian neptunian requested a review from a team as a code owner June 6, 2025 22:30
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jun 6, 2025 •
edited
Loading

💛 Build succeeded, but was flaky

  • Buildkite Build
  • Commit: 977d68b
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-216352-977d68bdff92

Failed CI Steps

Test Failures

  • [job] [logs] x-pack/test/alerting_api_integration/security_and_spaces/group1/config.ts / alerting api integration security and spaces enabled Alerts - Group 1 alerts backfill rule runs ad hoc backfill task should run all execution sets of a scheduled backfill and correctly generate alerts

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
observabilityAIAssistant 101 102 +1

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
observabilityAIAssistant 408 424 +16

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
observabilityAIAssistantApp 171.7KB 172.7KB +1007.0B
searchAssistant 98.0KB 99.0KB +1001.0B
total +2.0KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
observabilityAIAssistant 27 30 +3

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
observabilityAIAssistant 37.1KB 37.6KB +485.0B
Unknown metric groups

API count

id before after diff
observabilityAIAssistant 410 428 +18

History

qn895
qn895 reviewed Jun 9, 2025
View reviewed changes
x-pack/platform/plugins/shared/observability_ai_assistant/server/service/anonymization/index.ts
withInferenceSpan('infer_ner', async () => {
let response;
try {
response = await this.esClient.asCurrentUser.ml.inferTrainedModel({
Copy link
Copy Markdown
Member

@qn895 qn895 Jun 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if this model is not available? Should there be a quick check if model exists before these tasks are called?

Copy link
Copy Markdown
Contributor

@dgieselaar dgieselaar Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume inferTrainedModel fails if the model does not exist, which is what we want here?

qn895
qn895 reviewed Jun 9, 2025
View reviewed changes
x-pack/platform/plugins/shared/observability_ai_assistant/server/service/anonymization/index.ts
)
);

const results = await Promise.all(tasks);
Copy link
Copy Markdown
Member

@qn895 qn895 Jun 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if one of the the tasks failed but the rest succeeds? Do we want to reject it completely like here? If not, maybe we can use something like allSettled here.

Copy link
Copy Markdown
Contributor

@dgieselaar dgieselaar Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah we don't want to send any non-anonymized data to the LLM if it partly fails, so failing entirely is correct here.

@dgieselaar
Copy link
Copy Markdown
Contributor

dgieselaar commented Jun 10, 2025

@elasticmachine merge upstream

@dgieselaar dgieselaar mentioned this pull request Jun 11, 2025
Merged
dgieselaar added a commit that referenced this pull request Jun 11, 2025
@dgieselaar @neptunian
[Obs AI Assistant] Anonymization support (#223351)
71ec37a 
Re-submit of #216352 as it has
merge conflicts and we don't have write permissions for Sandra's remote.

To test, add the following to your kibana.yml:

```
uiSettings:
  overrides:
    "observability:aiAssistantAnonymizationRules":
      - id: "ner"
        type: "ner"
        enabled: true
      - id: "beach"
        type: "regex"
        enabled: true
        pattern: "sandy"
```

---------

Co-authored-by: Sandra Gonzales <sandra.gonzales@elastic.co>
Co-authored-by: Sandra G <neptunian@users.noreply.github.com>
@dgieselaar
Copy link
Copy Markdown
Contributor

dgieselaar commented Jun 11, 2025

Merged via #223351

@dgieselaar dgieselaar closed this Jun 11, 2025
nickpeihl pushed a commit to nickpeihl/kibana that referenced this pull request Jun 12, 2025
@dgieselaar @neptunian @nickpeihl
[Obs AI Assistant] Anonymization support (elastic#223351)
61b5d8d 
Re-submit of elastic#216352 as it has
merge conflicts and we don't have write permissions for Sandra's remote.

To test, add the following to your kibana.yml:

```
uiSettings:
  overrides:
    "observability:aiAssistantAnonymizationRules":
      - id: "ner"
        type: "ner"
        enabled: true
      - id: "beach"
        type: "regex"
        enabled: true
        pattern: "sandy"
```

---------

Co-authored-by: Sandra Gonzales <sandra.gonzales@elastic.co>
Co-authored-by: Sandra G <neptunian@users.noreply.github.com>
iblancof pushed a commit to iblancof/kibana that referenced this pull request Jun 16, 2025
@dgieselaar @neptunian @iblancof
[Obs AI Assistant] Anonymization support (elastic#223351)
f10fc69 
Re-submit of elastic#216352 as it has
merge conflicts and we don't have write permissions for Sandra's remote.

To test, add the following to your kibana.yml:

```
uiSettings:
  overrides:
    "observability:aiAssistantAnonymizationRules":
      - id: "ner"
        type: "ner"
        enabled: true
      - id: "beach"
        type: "regex"
        enabled: true
        pattern: "sandy"
```

---------

Co-authored-by: Sandra Gonzales <sandra.gonzales@elastic.co>
Co-authored-by: Sandra G <neptunian@users.noreply.github.com>
neptunian pushed a commit to neptunian/kibana that referenced this pull request Jun 16, 2025
@dgieselaar
[Obs AI Assistant] Anonymization support (elastic#223351)
8c33a8b 
Re-submit of elastic#216352 as it has
merge conflicts and we don't have write permissions for Sandra's remote.

To test, add the following to your kibana.yml:

```
uiSettings:
  overrides:
    "observability:aiAssistantAnonymizationRules":
      - id: "ner"
        type: "ner"
        enabled: true
      - id: "beach"
        type: "regex"
        enabled: true
        pattern: "sandy"
```

---------

Co-authored-by: Sandra Gonzales <sandra.gonzales@elastic.co>
Co-authored-by: Sandra G <neptunian@users.noreply.github.com>
(cherry picked from commit 71ec37a)

# Conflicts:
#	x-pack/platform/packages/shared/kbn-ai-assistant/src/chat/chat_timeline.tsx
#	x-pack/platform/plugins/shared/observability_ai_assistant/server/service/client/index.ts
neptunian added a commit that referenced this pull request Jun 16, 2025
@neptunian @dgieselaar @kibanamachine
[8.19] [Obs AI Assistant] Anonymization support (#223351) (#224100)
aac6c9e 
# Backport

This will backport the following commits from `main` to `8.19`:
- [[Obs AI Assistant] Anonymization support
(#223351)](#223351)

<!--- Backport version: 10.0.1 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Dario
Gieselaar","email":"dario.gieselaar@elastic.co"},"sourceCommit":{"committedDate":"2025-06-11T09:00:40Z","message":"[Obs
AI Assistant] Anonymization support (#223351)\n\nRe-submit of
#216352 as it has\nmerge conflicts
and we don't have write permissions for Sandra's remote.\n\nTo test, add
the following to your kibana.yml:\n\n```\nuiSettings:\n overrides:\n
\"observability:aiAssistantAnonymizationRules\":\n - id: \"ner\"\n type:
\"ner\"\n enabled: true\n - id: \"beach\"\n type: \"regex\"\n enabled:
true\n pattern: \"sandy\"\n```\n\n---------\n\nCo-authored-by: Sandra
Gonzales <sandra.gonzales@elastic.co>\nCo-authored-by: Sandra G
<neptunian@users.noreply.github.com>","sha":"71ec37a2a688ca5b0791658e7cc7d0634e7139a9","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:enhancement","backport
missing","Team:Obs AI
Assistant","ci:project-deploy-observability","backport:version","v9.1.0","v8.19.0"],"title":"[Obs
AI Assistant] Anonymization
support","number":223351,"url":"https://github.com/elastic/kibana/pull/223351","mergeCommit":{"message":"[Obs
AI Assistant] Anonymization support (#223351)\n\nRe-submit of
#216352 as it has\nmerge conflicts
and we don't have write permissions for Sandra's remote.\n\nTo test, add
the following to your kibana.yml:\n\n```\nuiSettings:\n overrides:\n
\"observability:aiAssistantAnonymizationRules\":\n - id: \"ner\"\n type:
\"ner\"\n enabled: true\n - id: \"beach\"\n type: \"regex\"\n enabled:
true\n pattern: \"sandy\"\n```\n\n---------\n\nCo-authored-by: Sandra
Gonzales <sandra.gonzales@elastic.co>\nCo-authored-by: Sandra G
<neptunian@users.noreply.github.com>","sha":"71ec37a2a688ca5b0791658e7cc7d0634e7139a9"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/223351","number":223351,"mergeCommit":{"message":"[Obs
AI Assistant] Anonymization support (#223351)\n\nRe-submit of
#216352 as it has\nmerge conflicts
and we don't have write permissions for Sandra's remote.\n\nTo test, add
the following to your kibana.yml:\n\n```\nuiSettings:\n overrides:\n
\"observability:aiAssistantAnonymizationRules\":\n - id: \"ner\"\n type:
\"ner\"\n enabled: true\n - id: \"beach\"\n type: \"regex\"\n enabled:
true\n pattern: \"sandy\"\n```\n\n---------\n\nCo-authored-by: Sandra
Gonzales <sandra.gonzales@elastic.co>\nCo-authored-by: Sandra G
<neptunian@users.noreply.github.com>","sha":"71ec37a2a688ca5b0791658e7cc7d0634e7139a9"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: Dario Gieselaar <dario.gieselaar@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
@SrdjanLL SrdjanLL mentioned this pull request Jun 19, 2025
Merged
4 tasks
SrdjanLL added a commit that referenced this pull request Jun 25, 2025
@SrdjanLL
[Obs AI Assistant] Add custom plugin for highlighting user and assist…
05b4fdd 
…ant responses when they have redacted entities (#224605)

Closes: elastic/obs-ai-team#264

## Summary

Add custom plugin for highlighting user and assistant responses when
they have redacted entities:

- Custom plugin for `EuiMarkdownFormat` that highlights all content
wrapped inside anonymized node. The parsing plugin follows the approach
from [EUI
docs](https://eui.elastic.co/docs/components/editors-and-syntax/markdown/plugins/#putting-it-all-together-a-simple-chart-plugin)
and is used as follows:
   ```
!{anonymized{"entityClass":"<entity class>", "content": "<content"}}
    ```  
- Highlighted content is currently rendered as `EuiCode`, but it can be
more sophisticated (i.e highlight differently depending on the entity
class).
- Allows using the same highlighting logic for messages from both `user`
and `assistant` roles.
- Currently **skipping highlighting inside the code blocks** - may
require customising the default plugins further.

Manually tested: 
- Function calling seems to work as expected.
- Search results with PII are highlighted in the table.
- Custom regex matches are highlighted correctly.

### Testing instructions:
1. Used setup from #216352 to set
up NER model locally.
2. Added to `kibana.dev.yml`:
```
uiSettings:
  overrides:
    'observability:aiAssistantAnonymizationRules': |
      [
        {
          "type": "regex",
          "pattern": "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[A-Za-z]{2,}",
          "enabled": true,
          "entityClass": "EMAIL"
        },
        {
          "type": "regex",
          "pattern": "https?://[^\\s]+",
          "enabled": true,
          "entityClass": "URL"
        },
        {
          "type": "regex",
          "pattern": "\\b(?:\\d{1,3}\\.){3}\\d{1,3}\\b",
          "enabled": true,
          "entityClass": "IP"
        },
        {
          "type": "ner",
          "enabled": true
        }
      ]
```
3. Used the assistant (see screenshots below)

### Screenshots
<img width="689" alt="image"
src="https://github.com/user-attachments/assets/d1f9bd57-7e76-43dc-88a6-d0be5fb15092"
/>

<img width="689" alt="image"
src="https://github.com/user-attachments/assets/ed63626b-b32d-45f8-9cf4-c575320d0dfc"
/>


<img width="689" alt="image"
src="https://github.com/user-attachments/assets/0b591158-9186-406a-aab3-e3be538216dc"
/>



### Checklist

Check the PR satisfies following conditions. 

Reviewers should verify this PR satisfies this list as well.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
SrdjanLL added a commit to SrdjanLL/kibana that referenced this pull request Jun 25, 2025
@SrdjanLL
[Obs AI Assistant] Add custom plugin for highlighting user and assist…
dc1d597 
…ant responses when they have redacted entities (elastic#224605)

Closes: elastic/obs-ai-team#264

## Summary

Add custom plugin for highlighting user and assistant responses when
they have redacted entities:

- Custom plugin for `EuiMarkdownFormat` that highlights all content
wrapped inside anonymized node. The parsing plugin follows the approach
from [EUI
docs](https://eui.elastic.co/docs/components/editors-and-syntax/markdown/plugins/#putting-it-all-together-a-simple-chart-plugin)
and is used as follows:
   ```
!{anonymized{"entityClass":"<entity class>", "content": "<content"}}
    ```
- Highlighted content is currently rendered as `EuiCode`, but it can be
more sophisticated (i.e highlight differently depending on the entity
class).
- Allows using the same highlighting logic for messages from both `user`
and `assistant` roles.
- Currently **skipping highlighting inside the code blocks** - may
require customising the default plugins further.

Manually tested:
- Function calling seems to work as expected.
- Search results with PII are highlighted in the table.
- Custom regex matches are highlighted correctly.

### Testing instructions:
1. Used setup from elastic#216352 to set
up NER model locally.
2. Added to `kibana.dev.yml`:
```
uiSettings:
  overrides:
    'observability:aiAssistantAnonymizationRules': |
      [
        {
          "type": "regex",
          "pattern": "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[A-Za-z]{2,}",
          "enabled": true,
          "entityClass": "EMAIL"
        },
        {
          "type": "regex",
          "pattern": "https?://[^\\s]+",
          "enabled": true,
          "entityClass": "URL"
        },
        {
          "type": "regex",
          "pattern": "\\b(?:\\d{1,3}\\.){3}\\d{1,3}\\b",
          "enabled": true,
          "entityClass": "IP"
        },
        {
          "type": "ner",
          "enabled": true
        }
      ]
```
3. Used the assistant (see screenshots below)

### Screenshots
<img width="689" alt="image"
src="https://github.com/user-attachments/assets/d1f9bd57-7e76-43dc-88a6-d0be5fb15092"
/>

<img width="689" alt="image"
src="https://github.com/user-attachments/assets/ed63626b-b32d-45f8-9cf4-c575320d0dfc"
/>

<img width="689" alt="image"
src="https://github.com/user-attachments/assets/0b591158-9186-406a-aab3-e3be538216dc"
/>

### Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [x]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] The PR description includes the appropriate Release Notes section,
and the correct `release_note:*` label is applied per the
[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit 05b4fdd)

# Conflicts:
#	x-pack/platform/packages/shared/kbn-ai-assistant/src/chat/chat_timeline.tsx
SrdjanLL added a commit that referenced this pull request Jun 25, 2025
@SrdjanLL
[8.19] [Obs AI Assistant] Add custom plugin for highlighting user and…
aed5559 
… assistant responses when they have redacted entities (#224605) (#225305)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[Obs AI Assistant] Add custom plugin for highlighting user and
assistant responses when they have redacted entities
(#224605)](#224605)

<!--- Backport version: 10.0.1 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Srdjan
Lulic","email":"srdjan.lulic@elastic.co"},"sourceCommit":{"committedDate":"2025-06-25T13:41:19Z","message":"[Obs
AI Assistant] Add custom plugin for highlighting user and assistant
responses when they have redacted entities (#224605)\n\nCloses:
https://github.com/elastic/obs-ai-assistant-team/issues/264\n\n##
Summary\n\nAdd custom plugin for highlighting user and assistant
responses when\nthey have redacted entities:\n\n- Custom plugin for
`EuiMarkdownFormat` that highlights all content\nwrapped inside
anonymized node. The parsing plugin follows the approach\nfrom
[EUI\ndocs](https://eui.elastic.co/docs/components/editors-and-syntax/markdown/plugins/#putting-it-all-together-a-simple-chart-plugin)\nand
is used as follows:\n ```\n!{anonymized{\"entityClass\":\"<entity
class>\", \"content\": \"<content\"}}\n ``` \n- Highlighted content is
currently rendered as `EuiCode`, but it can be\nmore sophisticated (i.e
highlight differently depending on the entity\nclass).\n- Allows using
the same highlighting logic for messages from both `user`\nand
`assistant` roles.\n- Currently **skipping highlighting inside the code
blocks** - may\nrequire customising the default plugins
further.\n\nManually tested: \n- Function calling seems to work as
expected.\n- Search results with PII are highlighted in the table.\n-
Custom regex matches are highlighted correctly.\n\n### Testing
instructions:\n1. Used setup from
#216352 to set\nup NER model
locally.\n2. Added to `kibana.dev.yml`:\n```\nuiSettings:\n overrides:\n
'observability:aiAssistantAnonymizationRules': |\n [\n {\n \"type\":
\"regex\",\n \"pattern\":
\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\\\.[A-Za-z]{2,}\",\n \"enabled\":
true,\n \"entityClass\": \"EMAIL\"\n },\n {\n \"type\": \"regex\",\n
\"pattern\": \"https?://[^\\\\s]+\",\n \"enabled\": true,\n
\"entityClass\": \"URL\"\n },\n {\n \"type\": \"regex\",\n \"pattern\":
\"\\\\b(?:\\\\d{1,3}\\\\.){3}\\\\d{1,3}\\\\b\",\n \"enabled\": true,\n
\"entityClass\": \"IP\"\n },\n {\n \"type\": \"ner\",\n \"enabled\":
true\n }\n ]\n```\n3. Used the assistant (see screenshots below)\n\n###
Screenshots\n<img width=\"689\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/d1f9bd57-7e76-43dc-88a6-d0be5fb15092\"\n/>\n\n<img
width=\"689\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/ed63626b-b32d-45f8-9cf4-c575320d0dfc\"\n/>\n\n\n<img
width=\"689\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/0b591158-9186-406a-aab3-e3be538216dc\"\n/>\n\n\n\n###
Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers
should verify this PR satisfies this list as well.\n\n- [x] Any text
added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials\n- [x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [ ] The PR
description includes the appropriate Release Notes section,\nand the
correct `release_note:*` label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"05b4fdd854e62b761b480a2d64b76ba4cd13511e","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:Obs
AI
Assistant","ci:project-deploy-observability","backport:version","v9.1.0","v8.19.0"],"title":"[Obs
AI Assistant] Add custom plugin for highlighting user and assistant
responses when they have redacted
entities","number":224605,"url":"https://github.com/elastic/kibana/pull/224605","mergeCommit":{"message":"[Obs
AI Assistant] Add custom plugin for highlighting user and assistant
responses when they have redacted entities (#224605)\n\nCloses:
https://github.com/elastic/obs-ai-assistant-team/issues/264\n\n##
Summary\n\nAdd custom plugin for highlighting user and assistant
responses when\nthey have redacted entities:\n\n- Custom plugin for
`EuiMarkdownFormat` that highlights all content\nwrapped inside
anonymized node. The parsing plugin follows the approach\nfrom
[EUI\ndocs](https://eui.elastic.co/docs/components/editors-and-syntax/markdown/plugins/#putting-it-all-together-a-simple-chart-plugin)\nand
is used as follows:\n ```\n!{anonymized{\"entityClass\":\"<entity
class>\", \"content\": \"<content\"}}\n ``` \n- Highlighted content is
currently rendered as `EuiCode`, but it can be\nmore sophisticated (i.e
highlight differently depending on the entity\nclass).\n- Allows using
the same highlighting logic for messages from both `user`\nand
`assistant` roles.\n- Currently **skipping highlighting inside the code
blocks** - may\nrequire customising the default plugins
further.\n\nManually tested: \n- Function calling seems to work as
expected.\n- Search results with PII are highlighted in the table.\n-
Custom regex matches are highlighted correctly.\n\n### Testing
instructions:\n1. Used setup from
#216352 to set\nup NER model
locally.\n2. Added to `kibana.dev.yml`:\n```\nuiSettings:\n overrides:\n
'observability:aiAssistantAnonymizationRules': |\n [\n {\n \"type\":
\"regex\",\n \"pattern\":
\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\\\.[A-Za-z]{2,}\",\n \"enabled\":
true,\n \"entityClass\": \"EMAIL\"\n },\n {\n \"type\": \"regex\",\n
\"pattern\": \"https?://[^\\\\s]+\",\n \"enabled\": true,\n
\"entityClass\": \"URL\"\n },\n {\n \"type\": \"regex\",\n \"pattern\":
\"\\\\b(?:\\\\d{1,3}\\\\.){3}\\\\d{1,3}\\\\b\",\n \"enabled\": true,\n
\"entityClass\": \"IP\"\n },\n {\n \"type\": \"ner\",\n \"enabled\":
true\n }\n ]\n```\n3. Used the assistant (see screenshots below)\n\n###
Screenshots\n<img width=\"689\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/d1f9bd57-7e76-43dc-88a6-d0be5fb15092\"\n/>\n\n<img
width=\"689\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/ed63626b-b32d-45f8-9cf4-c575320d0dfc\"\n/>\n\n\n<img
width=\"689\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/0b591158-9186-406a-aab3-e3be538216dc\"\n/>\n\n\n\n###
Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers
should verify this PR satisfies this list as well.\n\n- [x] Any text
added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials\n- [x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [ ] The PR
description includes the appropriate Release Notes section,\nand the
correct `release_note:*` label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"05b4fdd854e62b761b480a2d64b76ba4cd13511e"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/224605","number":224605,"mergeCommit":{"message":"[Obs
AI Assistant] Add custom plugin for highlighting user and assistant
responses when they have redacted entities (#224605)\n\nCloses:
https://github.com/elastic/obs-ai-assistant-team/issues/264\n\n##
Summary\n\nAdd custom plugin for highlighting user and assistant
responses when\nthey have redacted entities:\n\n- Custom plugin for
`EuiMarkdownFormat` that highlights all content\nwrapped inside
anonymized node. The parsing plugin follows the approach\nfrom
[EUI\ndocs](https://eui.elastic.co/docs/components/editors-and-syntax/markdown/plugins/#putting-it-all-together-a-simple-chart-plugin)\nand
is used as follows:\n ```\n!{anonymized{\"entityClass\":\"<entity
class>\", \"content\": \"<content\"}}\n ``` \n- Highlighted content is
currently rendered as `EuiCode`, but it can be\nmore sophisticated (i.e
highlight differently depending on the entity\nclass).\n- Allows using
the same highlighting logic for messages from both `user`\nand
`assistant` roles.\n- Currently **skipping highlighting inside the code
blocks** - may\nrequire customising the default plugins
further.\n\nManually tested: \n- Function calling seems to work as
expected.\n- Search results with PII are highlighted in the table.\n-
Custom regex matches are highlighted correctly.\n\n### Testing
instructions:\n1. Used setup from
#216352 to set\nup NER model
locally.\n2. Added to `kibana.dev.yml`:\n```\nuiSettings:\n overrides:\n
'observability:aiAssistantAnonymizationRules': |\n [\n {\n \"type\":
\"regex\",\n \"pattern\":
\"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\\\.[A-Za-z]{2,}\",\n \"enabled\":
true,\n \"entityClass\": \"EMAIL\"\n },\n {\n \"type\": \"regex\",\n
\"pattern\": \"https?://[^\\\\s]+\",\n \"enabled\": true,\n
\"entityClass\": \"URL\"\n },\n {\n \"type\": \"regex\",\n \"pattern\":
\"\\\\b(?:\\\\d{1,3}\\\\.){3}\\\\d{1,3}\\\\b\",\n \"enabled\": true,\n
\"entityClass\": \"IP\"\n },\n {\n \"type\": \"ner\",\n \"enabled\":
true\n }\n ]\n```\n3. Used the assistant (see screenshots below)\n\n###
Screenshots\n<img width=\"689\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/d1f9bd57-7e76-43dc-88a6-d0be5fb15092\"\n/>\n\n<img
width=\"689\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/ed63626b-b32d-45f8-9cf4-c575320d0dfc\"\n/>\n\n\n<img
width=\"689\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/0b591158-9186-406a-aab3-e3be538216dc\"\n/>\n\n\n\n###
Checklist\n\nCheck the PR satisfies following conditions. \n\nReviewers
should verify this PR satisfies this list as well.\n\n- [x] Any text
added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[x]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials\n- [x] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common scenarios\n- [ ] The PR
description includes the appropriate Release Notes section,\nand the
correct `release_note:*` label is applied per
the\n[guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)","sha":"05b4fdd854e62b761b480a2d64b76ba4cd13511e"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@qn895 qn895 qn895 left review comments

@pgayvallet pgayvallet pgayvallet approved these changes

@csr csr csr approved these changes

@jeramysoucy jeramysoucy jeramysoucy approved these changes

+1 more reviewer

@dgieselaar dgieselaar dgieselaar left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

backport:skip This PR does not require backporting ci:project-deploy-observability Create an Observability project release_note:feature Makes this part of the condensed release notes Team:Obs AI Assistant Observability AI Assistant

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@neptunian @elasticmachine @spong @jeramysoucy @kharsalan @dgieselaar @prodsecmachine @pgayvallet @csr @qn895