[Discover][Logs profile] Fix missing search highlights#260056
Merged
tfcmarques merged 6 commits intoelastic:mainfrom Mar 31, 2026
Merged
[Discover][Logs profile] Fix missing search highlights#260056tfcmarques merged 6 commits intoelastic:mainfrom
tfcmarques merged 6 commits intoelastic:mainfrom
Conversation
tfcmarques
added
Feature:Discover
Contributor
|
Pinging @elastic/obs-exploration-team (Team:obs-exploration) |
Contributor
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Public APIs missing comments
Async chunks
History
cc @tfcmarques |
davismcphee
reviewed
Mar 30, 2026
Contributor
davismcphee
left a comment
davismcphee
left a comment
There was a problem hiding this comment.
Thanks for looking at this! I don't remember the full context for some of these changes originally, so just looking to better understand.
Nothing stood out to me specifically in the code changes, but it's always a bit risky working with HTML directly. I think it's worth getting AI review on this one to see if it catches anything.
| * Escapes HTML in a string while preserving field-format highlight <mark> tags. | ||
| * Used for values already processed by formatFieldValue / getHighlightHtml (e.g. resource badges). | ||
| */ | ||
| export function escapeAndPreserveHighlightTags(value: string): string { |
Contributor
There was a problem hiding this comment.
I'm a bit confused about the purpose of this function. Values run through an HTML formatter with formatFieldValue should already be HTML escaped since that's it's purpose. Can you help me understand why it's needed?
Contributor
Author
There was a problem hiding this comment.
escapeAndPreserveHighlightTags was introduced in
#253210 as a defense measure since the values are rendered via dangerouslySetInnerHTML. My PR didn't introduce it, only updated it to handle multiple highlights (it was previously limited to two mark tags). I don't think any logic changed since then, so I kept that fix wherever it was needed still.
We can work on removing these temporary fixes once #259286 is completed.
Contributor
There was a problem hiding this comment.
Thanks for explaining. I'm still not sure this function actually does anything tbh, but it doesn't seem to introduce new risk, so I'll leave it up to you folks to decide.
| /** | ||
| * Merges ES highlight snippets into a field value, producing safe HTML with <mark> tags. | ||
| * Replicates the logic of getHighlightHtml from @kbn/field-formats-plugin, which packages | ||
| * cannot import directly. |
Contributor
There was a problem hiding this comment.
Do we need to duplicate the field formats logic here? The components which use this should already have access to the field formats plugin. Can we not just use formatFieldValue directly?
Contributor
Author
There was a problem hiding this comment.
We tried using formatFieldValue directly (I left a quick summary about this in the PR description). The issue is that formatFieldValue needs a DataViewField from dataView.fields.getByName(field) to look up highlights via field.name. For OTel documents, getMessageFieldWithFallbacks can resolve to body.text, which may not exist in the data view. When getByName returns undefined, the string formatter's htmlConvert can't look up hit.highlight[field.name] and silently skips highlighting.
Additionally, this component applies custom log-level coloring via getHighlightedMessage (wrapping ERROR, WARN, etc. in colored spans), which needs to run on the value after highlight processing. formatFieldValue would give the same output format, but the undefined field issue blocks it.
The duplication is minimal (just the core split/join logic from getHighlightHtml) and explicitly documented as a workaround for the package/plugin boundary. Happy to add a TODO to remove these duplicates, linking to the React-based field formatters work.
Contributor
There was a problem hiding this comment.
Thanks for clarifying. I think the fact we need to do this points to an issue around the OTel docs logic, but it doesn't seem to add new risk, so I won't block on it. But for clarity, you'll still need to provide a field when React field formatters land, so you'll probably need to fix the OTel docs logic soon regardless.
davismcphee
added
reviewer:coderabbit
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (9)
💤 Files with no reviewable changes (2)
📝 WalkthroughWalkthroughThe pull request refactors highlight tag handling across the Discover package. The original ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Tools execution failed with the following error: Failed to run tools: 13 INTERNAL: Received RST_STREAM with code 2 (Internal server error) Comment |
iblancof
approved these changes
Mar 30, 2026
Contributor
iblancof
left a comment
iblancof
left a comment
There was a problem hiding this comment.
I’m not a big fan of the duplicated logics we seem to be adding, but as I understand it, this is meant to be a temporary solution until the issues in [Discover][FieldFormatters] Migrate html field formatters to react gets done (which seem to be already in progress) are addressed.
Contributor
Author
|
Added a Also gave the PR link to a fresh context Cursor Agent, no risks raised regarding these changes 🙇🏼 |
Contributor
|
@tfcmarques I see this is not meant to be backported ( The PR that introduced this new logic was backported: |
tfcmarques
added
backport:version
davismcphee
approved these changes
Mar 30, 2026
Contributor
davismcphee
left a comment
davismcphee
left a comment
There was a problem hiding this comment.
I'm iffy on the changes overall since I'm not sure they get to the root of the issue even leaving aside React field formatters, but they don't seem to introduce new risk afaict, so no need to block on my end 👍
| * Escapes HTML in a string while preserving field-format highlight <mark> tags. | ||
| * Used for values already processed by formatFieldValue / getHighlightHtml (e.g. resource badges). | ||
| */ | ||
| export function escapeAndPreserveHighlightTags(value: string): string { |
Contributor
There was a problem hiding this comment.
Thanks for explaining. I'm still not sure this function actually does anything tbh, but it doesn't seem to introduce new risk, so I'll leave it up to you folks to decide.
| /** | ||
| * Merges ES highlight snippets into a field value, producing safe HTML with <mark> tags. | ||
| * Replicates the logic of getHighlightHtml from @kbn/field-formats-plugin, which packages | ||
| * cannot import directly. |
Contributor
There was a problem hiding this comment.
Thanks for clarifying. I think the fact we need to do this points to an issue around the OTel docs logic, but it doesn't seem to add new risk, so I won't block on it. But for clarity, you'll still need to provide a field when React field formatters land, so you'll probably need to fix the OTel docs logic soon regardless.
Contributor
|
Starting backport for target branches: 9.3 https://github.com/elastic/kibana/actions/runs/23788311731 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Mar 31, 2026
## Summary Fixes search result highlighting not working in the Logs profile summary column and content breakdown. > [!IMPORTANT] > This should be seen as a temporary fix, while the team works on React-based field formatters Closes elastic#259407 ### Root cause The message value was read from `row.flattened`, which doesn't contain ES highlights. The highlight snippets can be found in `row.raw.highlight[field]` with `@kibana-highlighted-field@` tags, but were never passed to the rendering pipeline, after recent changes to the implementation. ### Approaches considered We explored different ways to get the highlighted value into the message rendering. First approach was to reuse `getLogDocumentOverview`, which had been removed to solve an issue with OTel documents, but it called `formatFieldValue` internally. That receives the full ES hit, the field formatter would convert the ES tags to `<mark>` HTML tags automatically, so that we could then keep our current logic working. The idea was to get the field name from `getMessageFieldWithFallbacks(row.flattened)` (which handles OTel fallback) and the already highlighted value from `getMessageFieldWithFallbacks(documentOverview)`. However, this approach didn't work for two reasons: - There's a field resolution priority mismatch, where `getMessageFieldWithFallbacks` checks OTel fields first (`body.text` before `message`), while `getFieldValueWithFallback` inside `getLogDocumentOverview` checks ECS first (`message` before `body.text`). If a doc has both, the field name and value could come from different sources. - `formatFieldValue` passes the ECS field name to `dataView.fields.getByName()`, which can return `undefined` for OTel docs without aliases. When that happens, the field formatter can't look up `hit.highlight[field.name]` and silently skips highlighting entirely. In testing, highlights didn't appear. The final approach reads `row.raw.highlight[field]` directly using the correctly-resolved field name from `getMessageFieldWithFallbacks(row.flattened)`, bypassing the field formatter altogether. A new `getHighlightedFieldValue` utility replicates the logic of `getHighlightHtml` from `@kbn/field-formats-plugin` (which packages can't import directly), merging all ES highlight snippets into the escaped field value — handling multi-valued fields and multiple highlight regions within a single value. `escapeAndPreserveHighlightTags` was simplified back to only handle HTML `<mark>` tags (used by resource badges that receive pre-formatted values from `formatFieldValue`). Also, we fixed a pre-existing undetected bug where multiple highlight regions were silently dropped due to a `markTags.length === 2` condition that only allowed single keyword highlighting. ### Changes - **`content.tsx` / `summary_column.tsx` / `content_breakdown.tsx`** - Read the full ES highlight array from `row.raw.highlight[field]` (not just `[0]`) and pass it with the plain field value to `getHighlightedFieldValue`, which merges all snippets and produces XSS-safe HTML. - **`highlight_utils.ts`** (renamed from `escape_preserve_highlight_tags.ts`) - Added `getHighlightedFieldValue(fieldValue, highlights)`, directly replicating `getHighlightHtml` from `@kbn/field-formats-plugin`, iterating over all ES highlight snippets, stripping ES tags to find matching text, and replacing occurrences in the escaped field value with `<mark>` elements. Handles multi-valued fields and multiple highlight regions. - Simplified `escapeAndPreserveHighlightTags` to only handle HTML `<mark>` tags (its sole remaining use case in `cell_actions_popover.tsx` for resource badges). ### Demo https://github.com/user-attachments/assets/7aa67eb4-f2ff-4874-bd3b-f8734f2c2e00 (cherry picked from commit 7346059)
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Mar 31, 2026
#260397) # Backport This will backport the following commits from `main` to `9.3`: - [[Discover][Logs profile] Fix missing search highlights (#260056)](#260056) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Tiago Marques","email":"tiago.marques@elastic.co"},"sourceCommit":{"committedDate":"2026-03-31T08:38:04Z","message":"[Discover][Logs profile] Fix missing search highlights (#260056)\n\n## Summary\n\nFixes search result highlighting not working in the Logs profile summary\ncolumn and content breakdown.\n\n> [!IMPORTANT] \n> This should be seen as a temporary fix, while the team works on\nReact-based field formatters\n\nCloses #259407\n\n### Root cause\n\nThe message value was read from `row.flattened`, which doesn't contain\nES highlights. The highlight snippets can be found in\n`row.raw.highlight[field]` with `@kibana-highlighted-field@` tags, but\nwere never passed to the rendering pipeline, after recent changes to the\nimplementation.\n\n### Approaches considered\n\nWe explored different ways to get the highlighted value into the message\nrendering.\n\nFirst approach was to reuse `getLogDocumentOverview`, which had been\nremoved to solve an issue with OTel documents, but it called\n`formatFieldValue` internally. That receives the full ES hit, the field\nformatter would convert the ES tags to `<mark>` HTML tags automatically,\nso that we could then keep our current logic working. The idea was to\nget the field name from `getMessageFieldWithFallbacks(row.flattened)`\n(which handles OTel fallback) and the already highlighted value from\n`getMessageFieldWithFallbacks(documentOverview)`.\n\nHowever, this approach didn't work for two reasons:\n- There's a field resolution priority mismatch, where\n`getMessageFieldWithFallbacks` checks OTel fields first (`body.text`\nbefore `message`), while `getFieldValueWithFallback` inside\n`getLogDocumentOverview` checks ECS first (`message` before\n`body.text`). If a doc has both, the field name and value could come\nfrom different sources.\n\n- `formatFieldValue` passes the ECS field name to\n`dataView.fields.getByName()`, which can return `undefined` for OTel\ndocs without aliases. When that happens, the field formatter can't look\nup `hit.highlight[field.name]` and silently skips highlighting entirely.\nIn testing, highlights didn't appear.\n\nThe final approach reads `row.raw.highlight[field]` directly using the\ncorrectly-resolved field name from\n`getMessageFieldWithFallbacks(row.flattened)`, bypassing the field\nformatter altogether. A new `getHighlightedFieldValue` utility\nreplicates the logic of `getHighlightHtml` from\n`@kbn/field-formats-plugin` (which packages can't import directly),\nmerging all ES highlight snippets into the escaped field value —\nhandling multi-valued fields and multiple highlight regions within a\nsingle value. `escapeAndPreserveHighlightTags` was simplified back to\nonly handle HTML `<mark>` tags (used by resource badges that receive\npre-formatted values from `formatFieldValue`). Also, we fixed a\npre-existing undetected bug where multiple highlight regions were\nsilently dropped due to a `markTags.length === 2` condition that only\nallowed single keyword highlighting.\n\n### Changes\n\n- **`content.tsx` / `summary_column.tsx` / `content_breakdown.tsx`**\n- Read the full ES highlight array from `row.raw.highlight[field]` (not\njust `[0]`) and pass it with the plain field value to\n`getHighlightedFieldValue`, which merges all snippets and produces\nXSS-safe HTML.\n\n- **`highlight_utils.ts`** (renamed from\n`escape_preserve_highlight_tags.ts`)\n- Added `getHighlightedFieldValue(fieldValue, highlights)`, directly\nreplicating `getHighlightHtml` from `@kbn/field-formats-plugin`,\niterating over all ES highlight snippets, stripping ES tags to find\nmatching text, and replacing occurrences in the escaped field value with\n`<mark>` elements. Handles multi-valued fields and multiple highlight\nregions.\n- Simplified `escapeAndPreserveHighlightTags` to only handle HTML\n`<mark>` tags (its sole remaining use case in `cell_actions_popover.tsx`\nfor resource badges).\n\n### Demo\n\n\nhttps://github.com/user-attachments/assets/7aa67eb4-f2ff-4874-bd3b-f8734f2c2e00","sha":"7346059bf4f80300165ad809f27ebecc6142f7f1","branchLabelMapping":{"^v9.4.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["Feature:Discover","release_note:skip","backport:version","v9.3.0","Team:obs-exploration","v9.4.0","v9.3.1","v9.3.2","reviewer:coderabbit","v9.3.3","reviewer:macroscope"],"title":"[Discover][Logs profile] Fix missing search highlights","number":260056,"url":"https://github.com/elastic/kibana/pull/260056","mergeCommit":{"message":"[Discover][Logs profile] Fix missing search highlights (#260056)\n\n## Summary\n\nFixes search result highlighting not working in the Logs profile summary\ncolumn and content breakdown.\n\n> [!IMPORTANT] \n> This should be seen as a temporary fix, while the team works on\nReact-based field formatters\n\nCloses #259407\n\n### Root cause\n\nThe message value was read from `row.flattened`, which doesn't contain\nES highlights. The highlight snippets can be found in\n`row.raw.highlight[field]` with `@kibana-highlighted-field@` tags, but\nwere never passed to the rendering pipeline, after recent changes to the\nimplementation.\n\n### Approaches considered\n\nWe explored different ways to get the highlighted value into the message\nrendering.\n\nFirst approach was to reuse `getLogDocumentOverview`, which had been\nremoved to solve an issue with OTel documents, but it called\n`formatFieldValue` internally. That receives the full ES hit, the field\nformatter would convert the ES tags to `<mark>` HTML tags automatically,\nso that we could then keep our current logic working. The idea was to\nget the field name from `getMessageFieldWithFallbacks(row.flattened)`\n(which handles OTel fallback) and the already highlighted value from\n`getMessageFieldWithFallbacks(documentOverview)`.\n\nHowever, this approach didn't work for two reasons:\n- There's a field resolution priority mismatch, where\n`getMessageFieldWithFallbacks` checks OTel fields first (`body.text`\nbefore `message`), while `getFieldValueWithFallback` inside\n`getLogDocumentOverview` checks ECS first (`message` before\n`body.text`). If a doc has both, the field name and value could come\nfrom different sources.\n\n- `formatFieldValue` passes the ECS field name to\n`dataView.fields.getByName()`, which can return `undefined` for OTel\ndocs without aliases. When that happens, the field formatter can't look\nup `hit.highlight[field.name]` and silently skips highlighting entirely.\nIn testing, highlights didn't appear.\n\nThe final approach reads `row.raw.highlight[field]` directly using the\ncorrectly-resolved field name from\n`getMessageFieldWithFallbacks(row.flattened)`, bypassing the field\nformatter altogether. A new `getHighlightedFieldValue` utility\nreplicates the logic of `getHighlightHtml` from\n`@kbn/field-formats-plugin` (which packages can't import directly),\nmerging all ES highlight snippets into the escaped field value —\nhandling multi-valued fields and multiple highlight regions within a\nsingle value. `escapeAndPreserveHighlightTags` was simplified back to\nonly handle HTML `<mark>` tags (used by resource badges that receive\npre-formatted values from `formatFieldValue`). Also, we fixed a\npre-existing undetected bug where multiple highlight regions were\nsilently dropped due to a `markTags.length === 2` condition that only\nallowed single keyword highlighting.\n\n### Changes\n\n- **`content.tsx` / `summary_column.tsx` / `content_breakdown.tsx`**\n- Read the full ES highlight array from `row.raw.highlight[field]` (not\njust `[0]`) and pass it with the plain field value to\n`getHighlightedFieldValue`, which merges all snippets and produces\nXSS-safe HTML.\n\n- **`highlight_utils.ts`** (renamed from\n`escape_preserve_highlight_tags.ts`)\n- Added `getHighlightedFieldValue(fieldValue, highlights)`, directly\nreplicating `getHighlightHtml` from `@kbn/field-formats-plugin`,\niterating over all ES highlight snippets, stripping ES tags to find\nmatching text, and replacing occurrences in the escaped field value with\n`<mark>` elements. Handles multi-valued fields and multiple highlight\nregions.\n- Simplified `escapeAndPreserveHighlightTags` to only handle HTML\n`<mark>` tags (its sole remaining use case in `cell_actions_popover.tsx`\nfor resource badges).\n\n### Demo\n\n\nhttps://github.com/user-attachments/assets/7aa67eb4-f2ff-4874-bd3b-f8734f2c2e00","sha":"7346059bf4f80300165ad809f27ebecc6142f7f1"}},"sourceBranch":"main","suggestedTargetBranches":["9.3"],"targetPullRequestStates":[{"branch":"9.3","label":"v9.3.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.4.0","branchLabelMappingKey":"^v9.4.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/260056","number":260056,"mergeCommit":{"message":"[Discover][Logs profile] Fix missing search highlights (#260056)\n\n## Summary\n\nFixes search result highlighting not working in the Logs profile summary\ncolumn and content breakdown.\n\n> [!IMPORTANT] \n> This should be seen as a temporary fix, while the team works on\nReact-based field formatters\n\nCloses #259407\n\n### Root cause\n\nThe message value was read from `row.flattened`, which doesn't contain\nES highlights. The highlight snippets can be found in\n`row.raw.highlight[field]` with `@kibana-highlighted-field@` tags, but\nwere never passed to the rendering pipeline, after recent changes to the\nimplementation.\n\n### Approaches considered\n\nWe explored different ways to get the highlighted value into the message\nrendering.\n\nFirst approach was to reuse `getLogDocumentOverview`, which had been\nremoved to solve an issue with OTel documents, but it called\n`formatFieldValue` internally. That receives the full ES hit, the field\nformatter would convert the ES tags to `<mark>` HTML tags automatically,\nso that we could then keep our current logic working. The idea was to\nget the field name from `getMessageFieldWithFallbacks(row.flattened)`\n(which handles OTel fallback) and the already highlighted value from\n`getMessageFieldWithFallbacks(documentOverview)`.\n\nHowever, this approach didn't work for two reasons:\n- There's a field resolution priority mismatch, where\n`getMessageFieldWithFallbacks` checks OTel fields first (`body.text`\nbefore `message`), while `getFieldValueWithFallback` inside\n`getLogDocumentOverview` checks ECS first (`message` before\n`body.text`). If a doc has both, the field name and value could come\nfrom different sources.\n\n- `formatFieldValue` passes the ECS field name to\n`dataView.fields.getByName()`, which can return `undefined` for OTel\ndocs without aliases. When that happens, the field formatter can't look\nup `hit.highlight[field.name]` and silently skips highlighting entirely.\nIn testing, highlights didn't appear.\n\nThe final approach reads `row.raw.highlight[field]` directly using the\ncorrectly-resolved field name from\n`getMessageFieldWithFallbacks(row.flattened)`, bypassing the field\nformatter altogether. A new `getHighlightedFieldValue` utility\nreplicates the logic of `getHighlightHtml` from\n`@kbn/field-formats-plugin` (which packages can't import directly),\nmerging all ES highlight snippets into the escaped field value —\nhandling multi-valued fields and multiple highlight regions within a\nsingle value. `escapeAndPreserveHighlightTags` was simplified back to\nonly handle HTML `<mark>` tags (used by resource badges that receive\npre-formatted values from `formatFieldValue`). Also, we fixed a\npre-existing undetected bug where multiple highlight regions were\nsilently dropped due to a `markTags.length === 2` condition that only\nallowed single keyword highlighting.\n\n### Changes\n\n- **`content.tsx` / `summary_column.tsx` / `content_breakdown.tsx`**\n- Read the full ES highlight array from `row.raw.highlight[field]` (not\njust `[0]`) and pass it with the plain field value to\n`getHighlightedFieldValue`, which merges all snippets and produces\nXSS-safe HTML.\n\n- **`highlight_utils.ts`** (renamed from\n`escape_preserve_highlight_tags.ts`)\n- Added `getHighlightedFieldValue(fieldValue, highlights)`, directly\nreplicating `getHighlightHtml` from `@kbn/field-formats-plugin`,\niterating over all ES highlight snippets, stripping ES tags to find\nmatching text, and replacing occurrences in the escaped field value with\n`<mark>` elements. Handles multi-valued fields and multiple highlight\nregions.\n- Simplified `escapeAndPreserveHighlightTags` to only handle HTML\n`<mark>` tags (its sole remaining use case in `cell_actions_popover.tsx`\nfor resource badges).\n\n### Demo\n\n\nhttps://github.com/user-attachments/assets/7aa67eb4-f2ff-4874-bd3b-f8734f2c2e00","sha":"7346059bf4f80300165ad809f27ebecc6142f7f1"}}]}] BACKPORT--> Co-authored-by: Tiago Marques <tiago.marques@elastic.co>
mbondyra
added a commit
to mbondyra/kibana
that referenced
this pull request
Mar 31, 2026
…e_for_children6 * commit '3402744f63ca1196e97b11ffac4e7f7efab240df': (80 commits) [PerUserAuth] Add EARS auth type for Connectors V2 (elastic#253695) Fix `@elastic/eui/require-aria-label-for-modals` lint violations across `@elastic/kibana-core` files (elastic#259757) [Entity Analytics][Leads generation][4] Add API routes, LeadDataClient, and async generation (elastic#257046) [Agent Builder] Agent-centric UX redesign (elastic#258005) fix query streams failing test (elastic#260277) [Lens as code] Add list layout to the new API (elastic#259967) [FTR] Add warning comments to deployment-agnostic FTR base configs (elastic#260018) [Discover][Logs profile] Fix missing search highlights (elastic#260056) Plugin system: safe deletion (elastic#259038) [Infra] Fix Hosts filter options to match selected schema (elastic#259825) Manual Entity Resolution and flyout representation (elastic#260162) [Cascade] Handle grouping on fields with unset values (elastic#260033) [Fleet] generate OTel config for integration packages with otelcol inputs (elastic#259968) [Search] Switch over to V2 index management details (elastic#259866) [inference] increase timeout for ES inference calls (elastic#260382) [ES|QL] Enable subqueries (elastic#257455) [ES|QL] Change Point order free options (elastic#260282) [Auth] Added authentication strategy for UIAM OAuth (elastic#256182) [Security Solution] Add "alerts_candidate_count" rule execution metric (elastic#259917) [api-docs] 2026-03-31 Daily api_docs build (elastic#260380) ...
jeramysoucy
pushed a commit
to jeramysoucy/kibana
that referenced
this pull request
Apr 1, 2026
## Summary Fixes search result highlighting not working in the Logs profile summary column and content breakdown. > [!IMPORTANT] > This should be seen as a temporary fix, while the team works on React-based field formatters Closes elastic#259407 ### Root cause The message value was read from `row.flattened`, which doesn't contain ES highlights. The highlight snippets can be found in `row.raw.highlight[field]` with `@kibana-highlighted-field@` tags, but were never passed to the rendering pipeline, after recent changes to the implementation. ### Approaches considered We explored different ways to get the highlighted value into the message rendering. First approach was to reuse `getLogDocumentOverview`, which had been removed to solve an issue with OTel documents, but it called `formatFieldValue` internally. That receives the full ES hit, the field formatter would convert the ES tags to `<mark>` HTML tags automatically, so that we could then keep our current logic working. The idea was to get the field name from `getMessageFieldWithFallbacks(row.flattened)` (which handles OTel fallback) and the already highlighted value from `getMessageFieldWithFallbacks(documentOverview)`. However, this approach didn't work for two reasons: - There's a field resolution priority mismatch, where `getMessageFieldWithFallbacks` checks OTel fields first (`body.text` before `message`), while `getFieldValueWithFallback` inside `getLogDocumentOverview` checks ECS first (`message` before `body.text`). If a doc has both, the field name and value could come from different sources. - `formatFieldValue` passes the ECS field name to `dataView.fields.getByName()`, which can return `undefined` for OTel docs without aliases. When that happens, the field formatter can't look up `hit.highlight[field.name]` and silently skips highlighting entirely. In testing, highlights didn't appear. The final approach reads `row.raw.highlight[field]` directly using the correctly-resolved field name from `getMessageFieldWithFallbacks(row.flattened)`, bypassing the field formatter altogether. A new `getHighlightedFieldValue` utility replicates the logic of `getHighlightHtml` from `@kbn/field-formats-plugin` (which packages can't import directly), merging all ES highlight snippets into the escaped field value — handling multi-valued fields and multiple highlight regions within a single value. `escapeAndPreserveHighlightTags` was simplified back to only handle HTML `<mark>` tags (used by resource badges that receive pre-formatted values from `formatFieldValue`). Also, we fixed a pre-existing undetected bug where multiple highlight regions were silently dropped due to a `markTags.length === 2` condition that only allowed single keyword highlighting. ### Changes - **`content.tsx` / `summary_column.tsx` / `content_breakdown.tsx`** - Read the full ES highlight array from `row.raw.highlight[field]` (not just `[0]`) and pass it with the plain field value to `getHighlightedFieldValue`, which merges all snippets and produces XSS-safe HTML. - **`highlight_utils.ts`** (renamed from `escape_preserve_highlight_tags.ts`) - Added `getHighlightedFieldValue(fieldValue, highlights)`, directly replicating `getHighlightHtml` from `@kbn/field-formats-plugin`, iterating over all ES highlight snippets, stripping ES tags to find matching text, and replacing occurrences in the escaped field value with `<mark>` elements. Handles multi-valued fields and multiple highlight regions. - Simplified `escapeAndPreserveHighlightTags` to only handle HTML `<mark>` tags (its sole remaining use case in `cell_actions_popover.tsx` for resource badges). ### Demo https://github.com/user-attachments/assets/7aa67eb4-f2ff-4874-bd3b-f8734f2c2e00
paulinashakirova
pushed a commit
to paulinashakirova/kibana
that referenced
this pull request
Apr 2, 2026
## Summary Fixes search result highlighting not working in the Logs profile summary column and content breakdown. > [!IMPORTANT] > This should be seen as a temporary fix, while the team works on React-based field formatters Closes elastic#259407 ### Root cause The message value was read from `row.flattened`, which doesn't contain ES highlights. The highlight snippets can be found in `row.raw.highlight[field]` with `@kibana-highlighted-field@` tags, but were never passed to the rendering pipeline, after recent changes to the implementation. ### Approaches considered We explored different ways to get the highlighted value into the message rendering. First approach was to reuse `getLogDocumentOverview`, which had been removed to solve an issue with OTel documents, but it called `formatFieldValue` internally. That receives the full ES hit, the field formatter would convert the ES tags to `<mark>` HTML tags automatically, so that we could then keep our current logic working. The idea was to get the field name from `getMessageFieldWithFallbacks(row.flattened)` (which handles OTel fallback) and the already highlighted value from `getMessageFieldWithFallbacks(documentOverview)`. However, this approach didn't work for two reasons: - There's a field resolution priority mismatch, where `getMessageFieldWithFallbacks` checks OTel fields first (`body.text` before `message`), while `getFieldValueWithFallback` inside `getLogDocumentOverview` checks ECS first (`message` before `body.text`). If a doc has both, the field name and value could come from different sources. - `formatFieldValue` passes the ECS field name to `dataView.fields.getByName()`, which can return `undefined` for OTel docs without aliases. When that happens, the field formatter can't look up `hit.highlight[field.name]` and silently skips highlighting entirely. In testing, highlights didn't appear. The final approach reads `row.raw.highlight[field]` directly using the correctly-resolved field name from `getMessageFieldWithFallbacks(row.flattened)`, bypassing the field formatter altogether. A new `getHighlightedFieldValue` utility replicates the logic of `getHighlightHtml` from `@kbn/field-formats-plugin` (which packages can't import directly), merging all ES highlight snippets into the escaped field value — handling multi-valued fields and multiple highlight regions within a single value. `escapeAndPreserveHighlightTags` was simplified back to only handle HTML `<mark>` tags (used by resource badges that receive pre-formatted values from `formatFieldValue`). Also, we fixed a pre-existing undetected bug where multiple highlight regions were silently dropped due to a `markTags.length === 2` condition that only allowed single keyword highlighting. ### Changes - **`content.tsx` / `summary_column.tsx` / `content_breakdown.tsx`** - Read the full ES highlight array from `row.raw.highlight[field]` (not just `[0]`) and pass it with the plain field value to `getHighlightedFieldValue`, which merges all snippets and produces XSS-safe HTML. - **`highlight_utils.ts`** (renamed from `escape_preserve_highlight_tags.ts`) - Added `getHighlightedFieldValue(fieldValue, highlights)`, directly replicating `getHighlightHtml` from `@kbn/field-formats-plugin`, iterating over all ES highlight snippets, stripping ES tags to find matching text, and replacing occurrences in the escaped field value with `<mark>` elements. Handles multi-valued fields and multiple highlight regions. - Simplified `escapeAndPreserveHighlightTags` to only handle HTML `<mark>` tags (its sole remaining use case in `cell_actions_popover.tsx` for resource badges). ### Demo https://github.com/user-attachments/assets/7aa67eb4-f2ff-4874-bd3b-f8734f2c2e00
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes search result highlighting not working in the Logs profile summary column and content breakdown.
Important
This should be seen as a temporary fix, while the team works on React-based field formatters
Closes #259407
Root cause
The message value was read from
row.flattened, which doesn't contain ES highlights. The highlight snippets can be found inrow.raw.highlight[field]with@kibana-highlighted-field@tags, but were never passed to the rendering pipeline, after recent changes to the implementation.Approaches considered
We explored different ways to get the highlighted value into the message rendering.
First approach was to reuse
getLogDocumentOverview, which had been removed to solve an issue with OTel documents, but it calledformatFieldValueinternally. That receives the full ES hit, the field formatter would convert the ES tags to<mark>HTML tags automatically, so that we could then keep our current logic working. The idea was to get the field name fromgetMessageFieldWithFallbacks(row.flattened)(which handles OTel fallback) and the already highlighted value fromgetMessageFieldWithFallbacks(documentOverview).However, this approach didn't work for two reasons:
There's a field resolution priority mismatch, where
getMessageFieldWithFallbackschecks OTel fields first (body.textbeforemessage), whilegetFieldValueWithFallbackinsidegetLogDocumentOverviewchecks ECS first (messagebeforebody.text). If a doc has both, the field name and value could come from different sources.formatFieldValuepasses the ECS field name todataView.fields.getByName(), which can returnundefinedfor OTel docs without aliases. When that happens, the field formatter can't look uphit.highlight[field.name]and silently skips highlighting entirely. In testing, highlights didn't appear.The final approach reads
row.raw.highlight[field]directly using the correctly-resolved field name fromgetMessageFieldWithFallbacks(row.flattened), bypassing the field formatter altogether. A newgetHighlightedFieldValueutility replicates the logic ofgetHighlightHtmlfrom@kbn/field-formats-plugin(which packages can't import directly), merging all ES highlight snippets into the escaped field value — handling multi-valued fields and multiple highlight regions within a single value.escapeAndPreserveHighlightTagswas simplified back to only handle HTML<mark>tags (used by resource badges that receive pre-formatted values fromformatFieldValue). Also, we fixed a pre-existing undetected bug where multiple highlight regions were silently dropped due to amarkTags.length === 2condition that only allowed single keyword highlighting.Changes
content.tsx/summary_column.tsx/content_breakdown.tsxrow.raw.highlight[field](not just[0]) and pass it with the plain field value togetHighlightedFieldValue, which merges all snippets and produces XSS-safe HTML.highlight_utils.ts(renamed fromescape_preserve_highlight_tags.ts)getHighlightedFieldValue(fieldValue, highlights), directly replicatinggetHighlightHtmlfrom@kbn/field-formats-plugin, iterating over all ES highlight snippets, stripping ES tags to find matching text, and replacing occurrences in the escaped field value with<mark>elements. Handles multi-valued fields and multiple highlight regions.escapeAndPreserveHighlightTagsto only handle HTML<mark>tags (its sole remaining use case incell_actions_popover.tsxfor resource badges).Demo
Screen.Recording.2026-03-27.at.17.16.26.mov