[KQL] Moves KQL related code to a separate plugin by stratoula · Pull Request #246674 · elastic/kibana

stratoula · 2025-12-17T08:16:23Z

Summary

Moves KQL to a separate plugin in order to be reused more easily and not create circular dependencies. Specifically this moves:

KQL autocomplete server side code
QueryStringInput ui component

The circular dependencies problem is caused because:

unified search is using the ES|QL editor (esql plugin)
we want to give kql autocomplete capabilities to the esql editor autocomplete + the ES|QL editor visor (smart search) needs to use the KQL search bar

The above were not possible as the unified search plugin references the esql plugin and to achieve the above we will need to import to esql plugin the unified search (causing circular dependencies)

Testing

As a code owner I am testing that if my applications are using KQL and/or unified search that everything works as expected

Checklist

Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The release_note:breaking label should be applied in these situations.
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines

github-actions · 2025-12-17T11:38:36Z

🔍 Preview links for changed docs

docs/extend/plugin-list.md

sabarasaba

kibana management changes (transform plugin) lgtm

maxcold

cloud_security_posture changes lgtm

kdelemme

actionable-obs code changes lgtm

mbondyra · 2026-01-13T12:55:58Z

 # Time in milliseconds to wait for autocomplete suggestions from Elasticsearch.
 # This value must be a whole number greater than zero. Defaults to 1000ms
-#unifiedSearch.autocomplete.valueSuggestions.timeout: 1000
+unifiedSearch.autocomplete.valueSuggestions.timeout: 1000


this is not a needed change, since 1000ms is a default, right?

yes Marta, nice catch. I was testing the settings and forgot to comment out again. Done!

use reusable useMemoCss

mbondyra

Nice, 337 files modified 😅
I reviewed the code and tested - let's wait for the bundle to pass, hopefully moving the useMemoCss didn't change anything for the worse. Approved!

elasticmachine · 2026-01-14T09:20:57Z

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #120 / Endpoint plugin @ess @serverless @skipInServerlessMKI test metadata apis list endpoints GET route "before all" hook for "should return one entry for each host with default paging"

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`kql`	-	40	+40
`unifiedSearch`	610	578	-32
total			+8

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/cloud-security-posture`	204	205	+1
`@kbn/lens-common`	1347	1348	+1
`kql`	-	75	+75
`observability`	629	630	+1
`unifiedSearch`	120	56	-64
total			+14

Any counts in public APIs

Total count of every any typed public API. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats any for more detailed information.

id	before	after	diff
`kql`	-	2	+2
`unifiedSearch`	2	0	-2
total			-0

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`apm`	2.9MB	2.9MB	+77.0B
`eventAnnotationListing`	207.0KB	207.0KB	+2.0B
`fleet`	2.1MB	2.1MB	-23.0B
`graph`	372.7KB	372.6KB	-35.0B
`kql`	-	38.2KB	+38.2KB
`lens`	1.9MB	1.9MB	-12.0B
`ml`	5.6MB	5.6MB	-20.0B
`monitoring`	633.6KB	633.6KB	-20.0B
`observability`	1.8MB	1.8MB	-54.0B
`securitySolution`	10.8MB	10.8MB	-20.0B
`stackAlerts`	69.1KB	69.0KB	-15.0B
`synthetics`	1.0MB	1.0MB	-5.0B
`transform`	624.6KB	624.6KB	-3.0B
`unifiedSearch`	401.5KB	361.1KB	-40.5KB
`uptime`	490.1KB	490.1KB	-30.0B
`visDefaultEditor`	81.8KB	81.8KB	-15.0B
`visTypeTimeseries`	440.7KB	440.7KB	-8.0B
total			-2.4KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`kql`	-	8	+8
`unifiedSearch`	20	11	-9
total			-1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`eventAnnotationListing`	11.8KB	11.7KB	-30.0B
`fleet`	180.7KB	180.7KB	-10.0B
`graph`	8.8KB	8.8KB	-20.0B
`inputControlVis`	6.8KB	6.8KB	+12.0B
`kql`	-	13.8KB	+13.8KB
`lens`	63.9KB	64.0KB	+132.0B
`maps`	41.2KB	41.2KB	-10.0B
`ml`	86.7KB	86.7KB	+20.0B
`monitoring`	26.5KB	26.5KB	-10.0B
`observability`	96.4KB	96.3KB	-10.0B
`securitySolution`	171.8KB	171.8KB	+12.0B
`unifiedSearch`	23.7KB	17.8KB	-5.8KB
`visTypeTimeseries`	19.4KB	19.4KB	-30.0B
`visualizations`	34.1KB	34.1KB	+10.0B
total			+8.0KB

Unknown metric groups

API count

id	before	after	diff
`@kbn/cloud-security-posture`	216	217	+1
`@kbn/lens-common`	1491	1492	+1
`kql`	-	80	+80
`observability`	637	638	+1
`unifiedSearch`	158	91	-67
total			+16

async chunk count

id	before	after	diff
`kql`	-	3	+3
`unifiedSearch`	4	3	-1
total			+2

ESLint disabled line counts

id	before	after	diff
`kql`	-	4	+4
`unifiedSearch`	6	3	-3
total			+1

References to deprecated APIs

id	before	after	diff
`kql`	-	10	+10
`unifiedSearch`	11	2	-9
total			+1

Total ESLint disabled count

id	before	after	diff
`kql`	-	4	+4
`unifiedSearch`	6	3	-3
total			+1

## Summary Part of elastic#247265 Moves KQL to a separate plugin in order to be reused more easily and not create circular dependencies. Specifically this moves: - KQL autocomplete server side code - QueryStringInput ui component The circular dependencies problem is caused because: - unified search is using the ES|QL editor (esql plugin) - we want to give kql autocomplete capabilities to the esql editor autocomplete + the ES|QL editor visor (smart search) needs to use the KQL search bar The above were not possible as the unified search plugin references the esql plugin and to achieve the above we will need to import to esql plugin the unified search (causing circular dependencies) ### Testing As a code owner I am testing that if my applications are using KQL and/or unified search that everything works as expected ### Checklist - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Rudolf Meijering <skaapgif@gmail.com> Co-authored-by: mbondyra <marta.bondyra@elastic.co>

#250044) It closes #249869 Related to: #246674 ## Summary - Fixes KQL autocomplete not working in the custom threshold rule creation/editing form - The `kql` service is now injected directly into the rule expression component via a factory function - This ensures KQL autocomplete works regardless of which plugin opens the rule form (e.g., infra, triggers_actions_ui, stack_alerts) — consumers no longer need to have `kql` in their plugin dependencies - Moves `kql` from optional to required plugins in the observability plugin manifest ## Technical details ### Problem The custom threshold rule can be opened from multiple plugins (observability, infra, triggers_actions_ui). Previously, the rule expression component tried to get the `kql` service from the Kibana context, but this failed when the rule form was opened from plugins that don't have `kql` in their dependencies. Additionally, rule types are registered during the **setup phase**, but `KqlPluginStart` is only available during the **start phase**. ### Solution The `kql` service is now directly injected into the custom threshold rule expression component using a factory function. This approach: 1. **Ensures consistency**: The `kql` service always comes from the observability plugin (which owns the rule type), not from the consumer plugin 2. **Avoids breaking other plugins**: Plugins like infra, triggers_actions_ui, or stack_alerts don't need to add `kql` to their dependencies just to use the custom threshold rule 3. **Handles the setup/start lifecycle**: Uses `dynamic()` with `getStartServices()` to defer fetching the `kql` service until the component is actually rendered **E2E test:** - Added new Scout test to verify KQL autocomplete works in custom threshold rule creation Before: https://github.com/user-attachments/assets/150c22f0-f619-4f5f-bae6-0986ea319b65 After: https://github.com/user-attachments/assets/1cb9fa30-ac16-4ac3-b960-3d87d1b0f313 --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

## Summary It looks like [this conversation](#246674 (comment)) had a misunderstanding: it focused on the `unifiedSearch` -> `kql` change, but it forgot the pre-existing rename from `data.autocomplete` to `unifiedSearch.autocomplete`. We've seen some customers hitting this old path, so this PR brings it back. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

## Summary It looks like [this conversation](elastic#246674 (comment)) had a misunderstanding: it focused on the `unifiedSearch` -> `kql` change, but it forgot the pre-existing rename from `data.autocomplete` to `unifiedSearch.autocomplete`. We've seen some customers hitting this old path, so this PR brings it back. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…269868) ## Summary This PR addresses elastic/docs-content#6511. The page [General settings in Kibana](https://www.elastic.co/docs/reference/kibana/configuration-reference/general-settings) lists \`data.autocomplete.valueSuggestions.terminateAfter\` and \`data.autocomplete.valueSuggestions.timeout\` without any deprecation indicator, making them appear as valid current settings. These settings were deprecated in Kibana 8.3.0 when the autocomplete logic was moved from the \`data\` plugin to the \`unified_search\` plugin (now \`kql\`) via [#129977](#129977). The canonical settings since 8.3.0 are \`unifiedSearch.autocomplete.valueSuggestions.terminateAfter\` and \`unifiedSearch.autocomplete.valueSuggestions.timeout\`. The backward-compatible deprecation handler (which remaps the old keys to the new ones with a warning) was accidentally removed in [#246674](#246674) and restored in [#268872](#268872) (backported to v9.4.2 and v9.5.0). During the window between 9.4.0 and 9.4.1, customers using \`data.autocomplete.*\` in \`kibana.yml\` experienced startup failures. All three generations of the settings (\`kibana.*\`, \`data.*\`, \`unifiedSearch.*\`) are explicitly allowlisted in ECH and work for users on ECH deployments. ## Changes - **\`docs/reference/configuration-reference/general-settings.yml\`**: - Marks \`data.autocomplete.valueSuggestions.terminateAfter\` and \`data.autocomplete.valueSuggestions.timeout\` as \`deprecated 8.3\` on stack, with \`deprecation_details\` pointing to the \`unifiedSearch.*\` replacements. - Adds an inline note on both deprecated settings warning that using them in Kibana 9.4.0–9.4.1 causes a startup failure, and advising users to migrate or upgrade. - Adds \`unifiedSearch.autocomplete.valueSuggestions.timeout\` and \`unifiedSearch.autocomplete.valueSuggestions.terminateAfter\` as \`stack: ga 8.3\` entries so the canonical settings are properly documented. ## Resolves Closes elastic/docs-content#6511 --- > **AI-assisted draft** — created with Claude Sonnet 4.6 in Cursor. > Review all generated content for factual accuracy before merging. --------- Co-authored-by: Cursor <cursoragent@cursor.com>

…lastic#269868) ## Summary This PR addresses elastic/docs-content#6511. The page [General settings in Kibana](https://www.elastic.co/docs/reference/kibana/configuration-reference/general-settings) lists \`data.autocomplete.valueSuggestions.terminateAfter\` and \`data.autocomplete.valueSuggestions.timeout\` without any deprecation indicator, making them appear as valid current settings. These settings were deprecated in Kibana 8.3.0 when the autocomplete logic was moved from the \`data\` plugin to the \`unified_search\` plugin (now \`kql\`) via [elastic#129977](elastic#129977). The canonical settings since 8.3.0 are \`unifiedSearch.autocomplete.valueSuggestions.terminateAfter\` and \`unifiedSearch.autocomplete.valueSuggestions.timeout\`. The backward-compatible deprecation handler (which remaps the old keys to the new ones with a warning) was accidentally removed in [elastic#246674](elastic#246674) and restored in [elastic#268872](elastic#268872) (backported to v9.4.2 and v9.5.0). During the window between 9.4.0 and 9.4.1, customers using \`data.autocomplete.*\` in \`kibana.yml\` experienced startup failures. All three generations of the settings (\`kibana.*\`, \`data.*\`, \`unifiedSearch.*\`) are explicitly allowlisted in ECH and work for users on ECH deployments. ## Changes - **\`docs/reference/configuration-reference/general-settings.yml\`**: - Marks \`data.autocomplete.valueSuggestions.terminateAfter\` and \`data.autocomplete.valueSuggestions.timeout\` as \`deprecated 8.3\` on stack, with \`deprecation_details\` pointing to the \`unifiedSearch.*\` replacements. - Adds an inline note on both deprecated settings warning that using them in Kibana 9.4.0–9.4.1 causes a startup failure, and advising users to migrate or upgrade. - Adds \`unifiedSearch.autocomplete.valueSuggestions.timeout\` and \`unifiedSearch.autocomplete.valueSuggestions.terminateAfter\` as \`stack: ga 8.3\` entries so the canonical settings are properly documented. ## Resolves Closes elastic/docs-content#6511 --- > **AI-assisted draft** — created with Claude Sonnet 4.6 in Cursor. > Review all generated content for factual accuracy before merging. --------- Co-authored-by: Cursor <cursoragent@cursor.com>

stratoula added 3 commits December 17, 2025 09:15

[KQL] Create different plugin

597b7b0

Remove package

43029ce

Create new plugin

dfa176f

stratoula force-pushed the extract-kql-separate-package branch from aa2f23b to dfa176f Compare December 17, 2025 10:43

kibanamachine added 5 commits December 17, 2025 11:10

Changes from node scripts/eslint_all_files --no-cache --fix

cc04e76

Changes from node scripts/lint_ts_projects --fix

d5e4e4c

Changes from node scripts/build_plugin_list_docs

a9c4107

Changes from node scripts/generate codeowners

af81bbe

Changes from node scripts/regenerate_moon_projects.js --update

04af071

stratoula and others added 20 commits December 18, 2025 08:45

Merge branch 'main' into extract-kql-separate-package

c52ad04

Fixes the paths

008596b

Some fixes

76db41d

Chaos

6a3e881

Merge branch 'main' into extract-kql-separate-package

84ce800

Cleanup

452847e

Changes from node scripts/lint_ts_projects --fix

a9ef094

Changes from node scripts/regenerate_moon_projects.js --update

0fb32ba

Small fix

1417aad

Server related fixes

5cab524

Changes from node scripts/regenerate_moon_projects.js --update

8fb0fbe

More path fixes

400fc7c

Changes from node scripts/lint_ts_projects --fix

7dac424

Changes from node scripts/regenerate_moon_projects.js --update

863bfe0

Fixes

8e302f5

Fixes

b1115b0

Fixes

864cb56

Changes from node scripts/lint_ts_projects --fix

ef7f73b

Changes from node scripts/regenerate_moon_projects.js --update

1647563

Merge branch 'main' into extract-kql-separate-package

daa39cf

sabarasaba approved these changes Jan 12, 2026

View reviewed changes

maxcold approved these changes Jan 12, 2026

View reviewed changes

Merge branch 'main' into extract-kql-separate-package

cfb013c

kdelemme approved these changes Jan 12, 2026

View reviewed changes

Merge branch 'main' into extract-kql-separate-package

8b52c81

mbondyra reviewed Jan 13, 2026

View reviewed changes

mbondyra and others added 3 commits January 13, 2026 14:07

use reusable useMemoCss

25530b0

Merge with main and resolve conflicts

f179969

Merge pull request #46 from mbondyra/pr/246674

fc0ec98

use reusable useMemoCss

mbondyra reviewed Jan 13, 2026

View reviewed changes

mbondyra approved these changes Jan 13, 2026

View reviewed changes

stratoula enabled auto-merge (squash) January 13, 2026 13:32

kibanamachine and others added 4 commits January 13, 2026 13:37

Changes from node scripts/lint_ts_projects --fix

525bc95

Changes from node scripts/regenerate_moon_projects.js --update

66a1bf9

Merge branch 'main' into extract-kql-separate-package

c714cdb

Merge with main and resolve conflicts

9af6281

elastic deleted a comment from elasticmachine Jan 14, 2026

stratoula merged commit 4c5f5f1 into elastic:main Jan 14, 2026
15 checks passed

This was referenced Jan 21, 2026

[Observability] Fix KQL autocomplete in custom threshold rule creation #249894

Closed

[Observability] Fix KQL autocomplete in custom threshold rule creation #250044

Merged

afharo mentioned this pull request May 12, 2026

[KQL] Bring back config deprecations #268872

Merged

florent-leborgne mentioned this pull request May 19, 2026

docs: mark data.autocomplete.* settings as deprecated since 8.3.0 #269868

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[KQL] Moves KQL related code to a separate plugin#246674

[KQL] Moves KQL related code to a separate plugin#246674
stratoula merged 80 commits into
elastic:mainfrom
stratoula:extract-kql-separate-package

stratoula commented Dec 17, 2025 •

edited

Loading

Uh oh!

github-actions Bot commented Dec 17, 2025 •

edited

Loading

Uh oh!

sabarasaba left a comment

Uh oh!

maxcold left a comment

Uh oh!

kdelemme left a comment

Uh oh!

mbondyra Jan 13, 2026

Uh oh!

stratoula Jan 13, 2026

Uh oh!

mbondyra left a comment

Uh oh!

Uh oh!

elasticmachine commented Jan 14, 2026

API count

async chunk count

ESLint disabled line counts

References to deprecated APIs

Total ESLint disabled count

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

Conversation

stratoula commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Checklist

Uh oh!

github-actions Bot commented Dec 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔍 Preview links for changed docs

Uh oh!

sabarasaba left a comment

Choose a reason for hiding this comment

Uh oh!

maxcold left a comment

Choose a reason for hiding this comment

Uh oh!

kdelemme left a comment

Choose a reason for hiding this comment

Uh oh!

mbondyra Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

stratoula Jan 13, 2026

Choose a reason for hiding this comment

Uh oh!

mbondyra left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

elasticmachine commented Jan 14, 2026

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Any counts in public APIs

Async chunks

Public APIs missing exports

Page load bundle

API count

async chunk count

ESLint disabled line counts

References to deprecated APIs

Total ESLint disabled count

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

stratoula commented Dec 17, 2025 •

edited

Loading

github-actions Bot commented Dec 17, 2025 •

edited

Loading