Skip to content

[ES|QL] Editor extensions registry - recommended queries#221474

Merged
stratoula merged 23 commits intoelastic:mainfrom
stratoula:esql-registry-v2
Jun 4, 2025
Merged

[ES|QL] Editor extensions registry - recommended queries#221474
stratoula merged 23 commits intoelastic:mainfrom
stratoula:esql-registry-v2

Conversation

@stratoula
Copy link
Contributor

@stratoula stratoula commented May 26, 2025

Summary

Closes #200026

Scope

This PR is allowing the solutions teams to register extensions to the editor. The first extension is the recommended queries. I prefer to add extra extensions in follow up PRs mostly to keep it clean and also because I want to see the usage first.

These extensions should not appear in the classic mode. They should appear only on the solution that they are being registered for.

Some examples below:

  • Only static suggestions, for index without solution extensions
image
  • Additional suggestions, for index with solution extensions
image

Some more technical notes

  • I am creating a server side registry in the esql plugin. I considered to create a plugin from scratch but I decided against it as at this point is only esql related extensions and we are trying to keep our plugins number as low as possible. It can get moved to another plugin if we ever decide in favor of it.
  • Each solution can register in their plugins their extensions. I have added a Readme which I hope it helps but this is how it should look
const esqlExtensionsRegistry = esql.getExtensionsRegistry();
        esqlExtensionsRegistry.setRecommendedQueries(
          [
            {
              name: 'Logs count by log level',
              query: 'from logs* | STATS count(*) by log_level',
            },
            {
              name: 'Apache logs counts',
              query: 'from logs-apache_error | STATS count(*)',
            },
            {
              name: 'Another index, not logs',
              query: 'from movies | STATS count(*)',
            },
          ],
            'oblt'
        );
  • I preferred a server side registry for multiple reasons:
    • In the case we want to make it more advanced in the future (more checks, LLMs etc) it make more sense to have a server side solution
    • It is more performant to do some checks such as (does this source exist?) etc
    • It allows me to create a route to get these extensions. I am using this route to get the extensions to the editor but in the future we could use the same route to display these extensions in a flyout or wherever the solutions teams want

Note

This PR is not registering anything at the editor but just sets up the architecture. The registration of recommended queries should happen from the solutions teams in their plugins.

Checklist

@stratoula stratoula changed the title Creation of the plugin [ES|QL] Editor extensions registry May 26, 2025
* @returns A RegExp object.
*/
function createPatternRegex(pattern: string): RegExp {
const escapedPattern = pattern.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to test with the code scanner

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: Tested! Code scanner passes ✅

@stratoula stratoula changed the title [ES|QL] Editor extensions registry [ES|QL] Editor extensions registry - recommended queries Jun 2, 2025
@stratoula stratoula added Feature:ES|QL ES|QL related features in Kibana Team:ESQL ES|QL related features in Kibana t// backport:version Backport to applied version labels v9.1.0 labels Jun 3, 2025
Copy link
Contributor

@sddonne sddonne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks amazing!

@elasticmachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

  • [job] [logs] FTR Configs #68 / Rules Management - Rule Import & Export APIs @ess @serverless @serverlessQA import_rules importing rules with an index should set the response content types to be expected

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
esql 230 232 +2
stackAlerts 284 286 +2
total +4

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
@kbn/esql-types 31 39 +8
@kbn/esql-validation-autocomplete 164 166 +2
@kbn/monaco 183 185 +2
esql 11 13 +2
total +14

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
esql 247.4KB 247.9KB +550.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id before after diff
esql 1 2 +1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
esql 8.1KB 8.7KB +613.0B
kbnUiSharedDeps-srcJs 3.7MB 3.7MB +651.0B
total +1.2KB
Unknown metric groups

API count

id before after diff
@kbn/esql-types 32 40 +8
@kbn/esql-validation-autocomplete 185 187 +2
@kbn/monaco 183 185 +2
esql 32 34 +2
total +14

History

@stratoula stratoula merged commit 0bba089 into elastic:main Jun 4, 2025
10 checks passed
@kibanamachine
Copy link
Contributor

Starting backport for target branches: 8.19

https://github.com/elastic/kibana/actions/runs/15436729493

kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Jun 4, 2025
)

## Summary

Closes elastic#200026

### Scope

This PR is allowing the solutions teams to register extensions to the
editor. The first extension is **the recommended queries**. I prefer to
add extra extensions in follow up PRs mostly to keep it clean and also
because I want to see the usage first.

These extensions should not appear in the classic mode. They should
appear only on the solution that they are being registered for.

Some examples below:

- Only static suggestions, for index without solution extensions

<img width="913" alt="image"
src="https://github.com/user-attachments/assets/7fad3b42-3e03-43a6-91e6-6166337536ca"
/>

- Additional suggestions, for index with solution extensions

<img width="760" alt="image"
src="https://github.com/user-attachments/assets/0988c9f4-8986-4d45-a064-a536c71a690e"
/>

### Some more technical notes

- I am creating a server side registry in the esql plugin. I considered
to create a plugin from scratch but I decided against it as at this
point is only esql related extensions and we are trying to keep our
plugins number as low as possible. It can get moved to another plugin if
we ever decide in favor of it.
- Each solution can register in their plugins their extensions. I have
added a Readme which I hope it helps but this is how it should look

```
const esqlExtensionsRegistry = esql.getExtensionsRegistry();
        esqlExtensionsRegistry.setRecommendedQueries(
          [
            {
              name: 'Logs count by log level',
              query: 'from logs* | STATS count(*) by log_level',
            },
            {
              name: 'Apache logs counts',
              query: 'from logs-apache_error | STATS count(*)',
            },
            {
              name: 'Another index, not logs',
              query: 'from movies | STATS count(*)',
            },
          ],
            'oblt'
        );
```

- I preferred a server side registry for multiple reasons:
- In the case we want to make it more advanced in the future (more
checks, LLMs etc) it make more sense to have a server side solution
- It is more performant to do some checks such as (does this source
exist?) etc
- It allows me to create a route to get these extensions. I am using
this route to get the extensions to the editor but in the future we
could use the same route to display these extensions in a flyout or
wherever the solutions teams want

### Note
This PR is not registering anything at the editor but just sets up the
architecture. The registration of recommended queries should happen from
the solutions teams in their plugins.

### Checklist

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 0bba089)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.19

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Jun 4, 2025
) (#222509)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[ES|QL] Editor extensions registry - recommended queries
(#221474)](#221474)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Stratoula
Kalafateli","email":"efstratia.kalafateli@elastic.co"},"sourceCommit":{"committedDate":"2025-06-04T07:52:22Z","message":"[ES|QL]
Editor extensions registry - recommended queries (#221474)\n\n##
Summary\n\nCloses https://github.com/elastic/kibana/issues/200026\n\n###
Scope\n\nThis PR is allowing the solutions teams to register extensions
to the\neditor. The first extension is **the recommended queries**. I
prefer to\nadd extra extensions in follow up PRs mostly to keep it clean
and also\nbecause I want to see the usage first.\n\nThese extensions
should not appear in the classic mode. They should\nappear only on the
solution that they are being registered for.\n\nSome examples
below:\n\n- Only static suggestions, for index without solution
extensions\n\n<img width=\"913\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/7fad3b42-3e03-43a6-91e6-6166337536ca\"\n/>\n\n-
Additional suggestions, for index with solution extensions\n\n<img
width=\"760\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/0988c9f4-8986-4d45-a064-a536c71a690e\"\n/>\n\n###
Some more technical notes\n\n- I am creating a server side registry in
the esql plugin. I considered\nto create a plugin from scratch but I
decided against it as at this\npoint is only esql related extensions and
we are trying to keep our\nplugins number as low as possible. It can get
moved to another plugin if\nwe ever decide in favor of it.\n- Each
solution can register in their plugins their extensions. I have\nadded a
Readme which I hope it helps but this is how it should
look\n\n```\nconst esqlExtensionsRegistry =
esql.getExtensionsRegistry();\n
esqlExtensionsRegistry.setRecommendedQueries(\n [\n {\n name: 'Logs
count by log level',\n query: 'from logs* | STATS count(*) by
log_level',\n },\n {\n name: 'Apache logs counts',\n query: 'from
logs-apache_error | STATS count(*)',\n },\n {\n name: 'Another index,
not logs',\n query: 'from movies | STATS count(*)',\n },\n ],\n 'oblt'\n
);\n```\n\n- I preferred a server side registry for multiple reasons:\n-
In the case we want to make it more advanced in the future
(more\nchecks, LLMs etc) it make more sense to have a server side
solution\n- It is more performant to do some checks such as (does this
source\nexist?) etc\n- It allows me to create a route to get these
extensions. I am using\nthis route to get the extensions to the editor
but in the future we\ncould use the same route to display these
extensions in a flyout or\nwherever the solutions teams want\n\n\n###
Note\nThis PR is not registering anything at the editor but just sets up
the\narchitecture. The registration of recommended queries should happen
from\nthe solutions teams in their plugins.\n\n### Checklist\n\n- [ ]
Any text added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[
]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials\n- [ ] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"0bba089f62ad98539fcf316fde96b551a95f8aca","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Feature:ES|QL","Team:ESQL","backport:version","v9.1.0","v8.19.0"],"title":"[ES|QL]
Editor extensions registry - recommended
queries","number":221474,"url":"https://github.com/elastic/kibana/pull/221474","mergeCommit":{"message":"[ES|QL]
Editor extensions registry - recommended queries (#221474)\n\n##
Summary\n\nCloses https://github.com/elastic/kibana/issues/200026\n\n###
Scope\n\nThis PR is allowing the solutions teams to register extensions
to the\neditor. The first extension is **the recommended queries**. I
prefer to\nadd extra extensions in follow up PRs mostly to keep it clean
and also\nbecause I want to see the usage first.\n\nThese extensions
should not appear in the classic mode. They should\nappear only on the
solution that they are being registered for.\n\nSome examples
below:\n\n- Only static suggestions, for index without solution
extensions\n\n<img width=\"913\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/7fad3b42-3e03-43a6-91e6-6166337536ca\"\n/>\n\n-
Additional suggestions, for index with solution extensions\n\n<img
width=\"760\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/0988c9f4-8986-4d45-a064-a536c71a690e\"\n/>\n\n###
Some more technical notes\n\n- I am creating a server side registry in
the esql plugin. I considered\nto create a plugin from scratch but I
decided against it as at this\npoint is only esql related extensions and
we are trying to keep our\nplugins number as low as possible. It can get
moved to another plugin if\nwe ever decide in favor of it.\n- Each
solution can register in their plugins their extensions. I have\nadded a
Readme which I hope it helps but this is how it should
look\n\n```\nconst esqlExtensionsRegistry =
esql.getExtensionsRegistry();\n
esqlExtensionsRegistry.setRecommendedQueries(\n [\n {\n name: 'Logs
count by log level',\n query: 'from logs* | STATS count(*) by
log_level',\n },\n {\n name: 'Apache logs counts',\n query: 'from
logs-apache_error | STATS count(*)',\n },\n {\n name: 'Another index,
not logs',\n query: 'from movies | STATS count(*)',\n },\n ],\n 'oblt'\n
);\n```\n\n- I preferred a server side registry for multiple reasons:\n-
In the case we want to make it more advanced in the future
(more\nchecks, LLMs etc) it make more sense to have a server side
solution\n- It is more performant to do some checks such as (does this
source\nexist?) etc\n- It allows me to create a route to get these
extensions. I am using\nthis route to get the extensions to the editor
but in the future we\ncould use the same route to display these
extensions in a flyout or\nwherever the solutions teams want\n\n\n###
Note\nThis PR is not registering anything at the editor but just sets up
the\narchitecture. The registration of recommended queries should happen
from\nthe solutions teams in their plugins.\n\n### Checklist\n\n- [ ]
Any text added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[
]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials\n- [ ] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"0bba089f62ad98539fcf316fde96b551a95f8aca"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/221474","number":221474,"mergeCommit":{"message":"[ES|QL]
Editor extensions registry - recommended queries (#221474)\n\n##
Summary\n\nCloses https://github.com/elastic/kibana/issues/200026\n\n###
Scope\n\nThis PR is allowing the solutions teams to register extensions
to the\neditor. The first extension is **the recommended queries**. I
prefer to\nadd extra extensions in follow up PRs mostly to keep it clean
and also\nbecause I want to see the usage first.\n\nThese extensions
should not appear in the classic mode. They should\nappear only on the
solution that they are being registered for.\n\nSome examples
below:\n\n- Only static suggestions, for index without solution
extensions\n\n<img width=\"913\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/7fad3b42-3e03-43a6-91e6-6166337536ca\"\n/>\n\n-
Additional suggestions, for index with solution extensions\n\n<img
width=\"760\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/0988c9f4-8986-4d45-a064-a536c71a690e\"\n/>\n\n###
Some more technical notes\n\n- I am creating a server side registry in
the esql plugin. I considered\nto create a plugin from scratch but I
decided against it as at this\npoint is only esql related extensions and
we are trying to keep our\nplugins number as low as possible. It can get
moved to another plugin if\nwe ever decide in favor of it.\n- Each
solution can register in their plugins their extensions. I have\nadded a
Readme which I hope it helps but this is how it should
look\n\n```\nconst esqlExtensionsRegistry =
esql.getExtensionsRegistry();\n
esqlExtensionsRegistry.setRecommendedQueries(\n [\n {\n name: 'Logs
count by log level',\n query: 'from logs* | STATS count(*) by
log_level',\n },\n {\n name: 'Apache logs counts',\n query: 'from
logs-apache_error | STATS count(*)',\n },\n {\n name: 'Another index,
not logs',\n query: 'from movies | STATS count(*)',\n },\n ],\n 'oblt'\n
);\n```\n\n- I preferred a server side registry for multiple reasons:\n-
In the case we want to make it more advanced in the future
(more\nchecks, LLMs etc) it make more sense to have a server side
solution\n- It is more performant to do some checks such as (does this
source\nexist?) etc\n- It allows me to create a route to get these
extensions. I am using\nthis route to get the extensions to the editor
but in the future we\ncould use the same route to display these
extensions in a flyout or\nwherever the solutions teams want\n\n\n###
Note\nThis PR is not registering anything at the editor but just sets up
the\narchitecture. The registration of recommended queries should happen
from\nthe solutions teams in their plugins.\n\n### Checklist\n\n- [ ]
Any text added follows [EUI's
writing\nguidelines](https://elastic.github.io/eui/#/guidelines/writing),
uses\nsentence case text and includes
[i18n\nsupport](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)\n-
[
]\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\nwas
added for features that require explanation or tutorials\n- [ ] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"0bba089f62ad98539fcf316fde96b551a95f8aca"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

---------

Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>
zacharyparikh pushed a commit to zacharyparikh/kibana that referenced this pull request Jun 4, 2025
)

## Summary

Closes elastic#200026

### Scope

This PR is allowing the solutions teams to register extensions to the
editor. The first extension is **the recommended queries**. I prefer to
add extra extensions in follow up PRs mostly to keep it clean and also
because I want to see the usage first.

These extensions should not appear in the classic mode. They should
appear only on the solution that they are being registered for.

Some examples below:

- Only static suggestions, for index without solution extensions

<img width="913" alt="image"
src="https://github.com/user-attachments/assets/7fad3b42-3e03-43a6-91e6-6166337536ca"
/>

- Additional suggestions, for index with solution extensions

<img width="760" alt="image"
src="https://github.com/user-attachments/assets/0988c9f4-8986-4d45-a064-a536c71a690e"
/>

### Some more technical notes

- I am creating a server side registry in the esql plugin. I considered
to create a plugin from scratch but I decided against it as at this
point is only esql related extensions and we are trying to keep our
plugins number as low as possible. It can get moved to another plugin if
we ever decide in favor of it.
- Each solution can register in their plugins their extensions. I have
added a Readme which I hope it helps but this is how it should look

```
const esqlExtensionsRegistry = esql.getExtensionsRegistry();
        esqlExtensionsRegistry.setRecommendedQueries(
          [
            {
              name: 'Logs count by log level',
              query: 'from logs* | STATS count(*) by log_level',
            },
            {
              name: 'Apache logs counts',
              query: 'from logs-apache_error | STATS count(*)',
            },
            {
              name: 'Another index, not logs',
              query: 'from movies | STATS count(*)',
            },
          ],
            'oblt'
        );
```

- I preferred a server side registry for multiple reasons:
- In the case we want to make it more advanced in the future (more
checks, LLMs etc) it make more sense to have a server side solution
- It is more performant to do some checks such as (does this source
exist?) etc
- It allows me to create a route to get these extensions. I am using
this route to get the extensions to the editor but in the future we
could use the same route to display these extensions in a flyout or
wherever the solutions teams want


### Note
This PR is not registering anything at the editor but just sets up the
architecture. The registration of recommended queries should happen from
the solutions teams in their plugins.

### Checklist

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
stratoula added a commit that referenced this pull request Jun 12, 2025
…p menu (#223362)

## Summary

In #221474 we introduced the
mechanism to register queries per solution at the editor.

This PR displays these queries in the unified search menu too. In the
following screenshot the 2 first queries are the oblt solution
registered queries. I also have a max-height just to be sure that this
list doesnt get too long now that the solutions can register queries.

<img width="439" alt="image"
src="https://github.com/user-attachments/assets/8ca48b8a-1880-42f3-944f-dd1fb012661f"
/>

### Note

This change won't display anything in the popover as no solution has
registered any queries yet. This PR sets the mechanism though on when
they will do so.

### How to test
If you want to test it the fastest way is to go to the esql plugin
(server side) and add (change the examples as you wish):

```
this.extensionsRegistry.setRecommendedQueries(
      [
        {
          name: 'Logs count by log level',
          query: 'from logs* | STATS count(*) by log_level',
        },
        {
          name: 'Apache logs counts',
          query: 'from logs-apache_error | STATS count(*)',
        },
        {
          name: 'Another index, not logs',
          query: 'from movies | STATS count(*)',
        },
      ],
      'oblt'
    );

```
Then go to Discover (solutions mode) and type `from logs*` and hit the
query. Open the popover and check the 2 first recommendations are been
suggested.

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
nickpeihl pushed a commit to nickpeihl/kibana that referenced this pull request Jun 12, 2025
)

## Summary

Closes elastic#200026

### Scope

This PR is allowing the solutions teams to register extensions to the
editor. The first extension is **the recommended queries**. I prefer to
add extra extensions in follow up PRs mostly to keep it clean and also
because I want to see the usage first.

These extensions should not appear in the classic mode. They should
appear only on the solution that they are being registered for.

Some examples below:

- Only static suggestions, for index without solution extensions

<img width="913" alt="image"
src="https://github.com/user-attachments/assets/7fad3b42-3e03-43a6-91e6-6166337536ca"
/>

- Additional suggestions, for index with solution extensions

<img width="760" alt="image"
src="https://github.com/user-attachments/assets/0988c9f4-8986-4d45-a064-a536c71a690e"
/>

### Some more technical notes

- I am creating a server side registry in the esql plugin. I considered
to create a plugin from scratch but I decided against it as at this
point is only esql related extensions and we are trying to keep our
plugins number as low as possible. It can get moved to another plugin if
we ever decide in favor of it.
- Each solution can register in their plugins their extensions. I have
added a Readme which I hope it helps but this is how it should look

```
const esqlExtensionsRegistry = esql.getExtensionsRegistry();
        esqlExtensionsRegistry.setRecommendedQueries(
          [
            {
              name: 'Logs count by log level',
              query: 'from logs* | STATS count(*) by log_level',
            },
            {
              name: 'Apache logs counts',
              query: 'from logs-apache_error | STATS count(*)',
            },
            {
              name: 'Another index, not logs',
              query: 'from movies | STATS count(*)',
            },
          ],
            'oblt'
        );
```

- I preferred a server side registry for multiple reasons:
- In the case we want to make it more advanced in the future (more
checks, LLMs etc) it make more sense to have a server side solution
- It is more performant to do some checks such as (does this source
exist?) etc
- It allows me to create a route to get these extensions. I am using
this route to get the extensions to the editor but in the future we
could use the same route to display these extensions in a flyout or
wherever the solutions teams want


### Note
This PR is not registering anything at the editor but just sets up the
architecture. The registration of recommended queries should happen from
the solutions teams in their plugins.

### Checklist

- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
stratoula added a commit that referenced this pull request Jun 13, 2025
…p menu (#223602)

## Summary

Reopening of #223362

In #221474 we introduced the
mechanism to register queries per solution at the editor.

This PR displays these queries in the unified search menu too. In the
following screenshot the 2 first queries are the oblt solution
registered queries. I also have a max-height just to be sure that this
list doesnt get too long now that the solutions can register queries.

<img width="439" alt="image"
src="https://github.com/user-attachments/assets/8ca48b8a-1880-42f3-944f-dd1fb012661f"
/>

### Note

This change won't display anything in the popover as no solution has
registered any queries yet. This PR sets the mechanism though on when
they will do so.

### How to test
If you want to test it the fastest way is to go to the esql plugin
(server side) and add (change the examples as you wish):

```
this.extensionsRegistry.setRecommendedQueries(
      [
        {
          name: 'Logs count by log level',
          query: 'from logs* | STATS count(*) by log_level',
        },
        {
          name: 'Apache logs counts',
          query: 'from logs-apache_error | STATS count(*)',
        },
        {
          name: 'Another index, not logs',
          query: 'from movies | STATS count(*)',
        },
      ],
      'oblt'
    );

```
Then go to Discover (solutions mode) and type `from logs*` and hit the
query. Open the popover and check the 2 first recommendations are been
suggested.

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios





<!--ONMERGE {"backportTargets":["8.19"]} ONMERGE-->

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
iblancof pushed a commit to iblancof/kibana that referenced this pull request Jun 16, 2025
…p menu (elastic#223362)

## Summary

In elastic#221474 we introduced the
mechanism to register queries per solution at the editor.

This PR displays these queries in the unified search menu too. In the
following screenshot the 2 first queries are the oblt solution
registered queries. I also have a max-height just to be sure that this
list doesnt get too long now that the solutions can register queries.

<img width="439" alt="image"
src="https://github.com/user-attachments/assets/8ca48b8a-1880-42f3-944f-dd1fb012661f"
/>

### Note

This change won't display anything in the popover as no solution has
registered any queries yet. This PR sets the mechanism though on when
they will do so.

### How to test
If you want to test it the fastest way is to go to the esql plugin
(server side) and add (change the examples as you wish):

```
this.extensionsRegistry.setRecommendedQueries(
      [
        {
          name: 'Logs count by log level',
          query: 'from logs* | STATS count(*) by log_level',
        },
        {
          name: 'Apache logs counts',
          query: 'from logs-apache_error | STATS count(*)',
        },
        {
          name: 'Another index, not logs',
          query: 'from movies | STATS count(*)',
        },
      ],
      'oblt'
    );

```
Then go to Discover (solutions mode) and type `from logs*` and hit the
query. Open the popover and check the 2 first recommendations are been
suggested.

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
iblancof pushed a commit to iblancof/kibana that referenced this pull request Jun 16, 2025
…p menu (elastic#223602)

## Summary

Reopening of elastic#223362

In elastic#221474 we introduced the
mechanism to register queries per solution at the editor.

This PR displays these queries in the unified search menu too. In the
following screenshot the 2 first queries are the oblt solution
registered queries. I also have a max-height just to be sure that this
list doesnt get too long now that the solutions can register queries.

<img width="439" alt="image"
src="https://github.com/user-attachments/assets/8ca48b8a-1880-42f3-944f-dd1fb012661f"
/>

### Note

This change won't display anything in the popover as no solution has
registered any queries yet. This PR sets the mechanism though on when
they will do so.

### How to test
If you want to test it the fastest way is to go to the esql plugin
(server side) and add (change the examples as you wish):

```
this.extensionsRegistry.setRecommendedQueries(
      [
        {
          name: 'Logs count by log level',
          query: 'from logs* | STATS count(*) by log_level',
        },
        {
          name: 'Apache logs counts',
          query: 'from logs-apache_error | STATS count(*)',
        },
        {
          name: 'Another index, not logs',
          query: 'from movies | STATS count(*)',
        },
      ],
      'oblt'
    );

```
Then go to Discover (solutions mode) and type `from logs*` and hit the
query. Open the popover and check the 2 first recommendations are been
suggested.

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios





<!--ONMERGE {"backportTargets":["8.19"]} ONMERGE-->

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
stratoula added a commit to stratoula/kibana that referenced this pull request Jun 17, 2025
…p menu (elastic#223602)

## Summary

Reopening of elastic#223362

In elastic#221474 we introduced the
mechanism to register queries per solution at the editor.

This PR displays these queries in the unified search menu too. In the
following screenshot the 2 first queries are the oblt solution
registered queries. I also have a max-height just to be sure that this
list doesnt get too long now that the solutions can register queries.

<img width="439" alt="image"
src="https://github.com/user-attachments/assets/8ca48b8a-1880-42f3-944f-dd1fb012661f"
/>

### Note

This change won't display anything in the popover as no solution has
registered any queries yet. This PR sets the mechanism though on when
they will do so.

### How to test
If you want to test it the fastest way is to go to the esql plugin
(server side) and add (change the examples as you wish):

```
this.extensionsRegistry.setRecommendedQueries(
      [
        {
          name: 'Logs count by log level',
          query: 'from logs* | STATS count(*) by log_level',
        },
        {
          name: 'Apache logs counts',
          query: 'from logs-apache_error | STATS count(*)',
        },
        {
          name: 'Another index, not logs',
          query: 'from movies | STATS count(*)',
        },
      ],
      'oblt'
    );

```
Then go to Discover (solutions mode) and type `from logs*` and hit the
query. Open the popover and check the 2 first recommendations are been
suggested.

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

<!--ONMERGE {"backportTargets":["8.19"]} ONMERGE-->

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 563abe2)

# Conflicts:
#	src/platform/plugins/shared/unified_search/public/search_bar/search_bar.test.tsx
stratoula added a commit that referenced this pull request Jun 17, 2025
…the help menu (#223602) (#224174)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[Unified search] Display the solutions recommended queries in the
help menu (#223602)](#223602)

<!--- Backport version: 10.0.0 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Stratoula
Kalafateli","email":"efstratia.kalafateli@elastic.co"},"sourceCommit":{"committedDate":"2025-06-13T04:22:45Z","message":"[Unified
search] Display the solutions recommended queries in the help menu
(#223602)\n\n## Summary\n\nReopening of
https://github.com/elastic/kibana/pull/223362\n\nIn
#221474 we introduced
the\nmechanism to register queries per solution at the editor.\n\nThis
PR displays these queries in the unified search menu too. In
the\nfollowing screenshot the 2 first queries are the oblt
solution\nregistered queries. I also have a max-height just to be sure
that this\nlist doesnt get too long now that the solutions can register
queries.\n\n<img width=\"439\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/8ca48b8a-1880-42f3-944f-dd1fb012661f\"\n/>\n\n###
Note\n\nThis change won't display anything in the popover as no solution
has\nregistered any queries yet. This PR sets the mechanism though on
when\nthey will do so.\n\n### How to test\nIf you want to test it the
fastest way is to go to the esql plugin\n(server side) and add (change
the examples as you
wish):\n\n```\nthis.extensionsRegistry.setRecommendedQueries(\n [\n {\n
name: 'Logs count by log level',\n query: 'from logs* | STATS count(*)
by log_level',\n },\n {\n name: 'Apache logs counts',\n query: 'from
logs-apache_error | STATS count(*)',\n },\n {\n name: 'Another index,
not logs',\n query: 'from movies | STATS count(*)',\n },\n ],\n 'oblt'\n
);\n\n```\nThen go to Discover (solutions mode) and type `from logs*`
and hit the\nquery. Open the popover and check the 2 first
recommendations are been\nsuggested.\n\n### Checklist\n\n- [ ] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios\n\n\n\n\n\n\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"563abe2cac151f59162639658ba4a3017d15cfb1","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport
missing","Feature:ES|QL","Team:ESQL","backport:version","v9.1.0","v8.19.0"],"title":"[Unified
search] Display the solutions recommended queries in the help
menu","number":223602,"url":"https://github.com/elastic/kibana/pull/223602","mergeCommit":{"message":"[Unified
search] Display the solutions recommended queries in the help menu
(#223602)\n\n## Summary\n\nReopening of
https://github.com/elastic/kibana/pull/223362\n\nIn
#221474 we introduced
the\nmechanism to register queries per solution at the editor.\n\nThis
PR displays these queries in the unified search menu too. In
the\nfollowing screenshot the 2 first queries are the oblt
solution\nregistered queries. I also have a max-height just to be sure
that this\nlist doesnt get too long now that the solutions can register
queries.\n\n<img width=\"439\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/8ca48b8a-1880-42f3-944f-dd1fb012661f\"\n/>\n\n###
Note\n\nThis change won't display anything in the popover as no solution
has\nregistered any queries yet. This PR sets the mechanism though on
when\nthey will do so.\n\n### How to test\nIf you want to test it the
fastest way is to go to the esql plugin\n(server side) and add (change
the examples as you
wish):\n\n```\nthis.extensionsRegistry.setRecommendedQueries(\n [\n {\n
name: 'Logs count by log level',\n query: 'from logs* | STATS count(*)
by log_level',\n },\n {\n name: 'Apache logs counts',\n query: 'from
logs-apache_error | STATS count(*)',\n },\n {\n name: 'Another index,
not logs',\n query: 'from movies | STATS count(*)',\n },\n ],\n 'oblt'\n
);\n\n```\nThen go to Discover (solutions mode) and type `from logs*`
and hit the\nquery. Open the popover and check the 2 first
recommendations are been\nsuggested.\n\n### Checklist\n\n- [ ] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios\n\n\n\n\n\n\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"563abe2cac151f59162639658ba4a3017d15cfb1"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/223602","number":223602,"mergeCommit":{"message":"[Unified
search] Display the solutions recommended queries in the help menu
(#223602)\n\n## Summary\n\nReopening of
https://github.com/elastic/kibana/pull/223362\n\nIn
#221474 we introduced
the\nmechanism to register queries per solution at the editor.\n\nThis
PR displays these queries in the unified search menu too. In
the\nfollowing screenshot the 2 first queries are the oblt
solution\nregistered queries. I also have a max-height just to be sure
that this\nlist doesnt get too long now that the solutions can register
queries.\n\n<img width=\"439\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/8ca48b8a-1880-42f3-944f-dd1fb012661f\"\n/>\n\n###
Note\n\nThis change won't display anything in the popover as no solution
has\nregistered any queries yet. This PR sets the mechanism though on
when\nthey will do so.\n\n### How to test\nIf you want to test it the
fastest way is to go to the esql plugin\n(server side) and add (change
the examples as you
wish):\n\n```\nthis.extensionsRegistry.setRecommendedQueries(\n [\n {\n
name: 'Logs count by log level',\n query: 'from logs* | STATS count(*)
by log_level',\n },\n {\n name: 'Apache logs counts',\n query: 'from
logs-apache_error | STATS count(*)',\n },\n {\n name: 'Another index,
not logs',\n query: 'from movies | STATS count(*)',\n },\n ],\n 'oblt'\n
);\n\n```\nThen go to Discover (solutions mode) and type `from logs*`
and hit the\nquery. Open the popover and check the 2 first
recommendations are been\nsuggested.\n\n### Checklist\n\n- [ ] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios\n\n\n\n\n\n\n\n---------\n\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"563abe2cac151f59162639658ba4a3017d15cfb1"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
stratoula added a commit that referenced this pull request Jun 23, 2025
## Summary

Adds the ability to register recommended fields per datasource. It
follows the logic [of the recommended queries]
(#221474). 

These fields will get prioritized in the editor. So the fields
prioritization goes like:

- recommended fields from the registry first
- ecs schema second
- everything else

### How to register them

- Use the esql server side registry as described in the aforementioned
PR
- Register the fields like that

```
    this.extensionsRegistry.setRecommendedFields(
      [
        {
          name: 'log_level',
          pattern: 'logs*', // This field is relevant for any index starting with 'logs...'
        },
        {
          name: 'host.ip.keyword',
          pattern: 'logs-apache_error', // This field is specific to 'logs-apache_error'
        },
        {
          name: 'http.request.method',
          pattern: 'logs*',
        },
      ],
      'oblt'
    );
```

You will see that the fields are getting prioritized in the editor

<img width="730" alt="image"
src="https://github.com/user-attachments/assets/c4941579-c04f-4c5f-875d-369355420a3d"
/>


As a bonus: The good thing with the recommended fields is that we are
prioritizing fields that exist in the datasource. So this means that a
solution team can register fields per datasource and not afraid if these
fields do not exist at the instance


### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
stratoula added a commit to stratoula/kibana that referenced this pull request Jun 23, 2025
## Summary

Adds the ability to register recommended fields per datasource. It
follows the logic [of the recommended queries]
(elastic#221474).

These fields will get prioritized in the editor. So the fields
prioritization goes like:

- recommended fields from the registry first
- ecs schema second
- everything else

### How to register them

- Use the esql server side registry as described in the aforementioned
PR
- Register the fields like that

```
    this.extensionsRegistry.setRecommendedFields(
      [
        {
          name: 'log_level',
          pattern: 'logs*', // This field is relevant for any index starting with 'logs...'
        },
        {
          name: 'host.ip.keyword',
          pattern: 'logs-apache_error', // This field is specific to 'logs-apache_error'
        },
        {
          name: 'http.request.method',
          pattern: 'logs*',
        },
      ],
      'oblt'
    );
```

You will see that the fields are getting prioritized in the editor

<img width="730" alt="image"
src="https://github.com/user-attachments/assets/c4941579-c04f-4c5f-875d-369355420a3d"
/>

As a bonus: The good thing with the recommended fields is that we are
prioritizing fields that exist in the datasource. So this means that a
solution team can register fields per datasource and not afraid if these
fields do not exist at the instance

### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios

(cherry picked from commit 642edd2)

# Conflicts:
#	src/platform/plugins/shared/esql/server/extensions_registry/index.test.ts
#	src/platform/plugins/shared/esql/server/extensions_registry/index.ts
stratoula added a commit that referenced this pull request Jun 23, 2025
…224816)

# Backport

This will backport the following commits from `main` to `8.19`:
- [[ES|QL] Prioritize recommended fields in the editor
(#224359)](#224359)

<!--- Backport version: 10.0.1 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Stratoula
Kalafateli","email":"efstratia.kalafateli@elastic.co"},"sourceCommit":{"committedDate":"2025-06-23T08:59:08Z","message":"[ES|QL]
Prioritize recommended fields in the editor (#224359)\n\n##
Summary\n\nAdds the ability to register recommended fields per
datasource. It\nfollows the logic [of the recommended
queries]\n(#221474). \n\nThese
fields will get prioritized in the editor. So the fields\nprioritization
goes like:\n\n- recommended fields from the registry first\n- ecs schema
second\n- everything else\n\n### How to register them\n\n- Use the esql
server side registry as described in the aforementioned\nPR\n- Register
the fields like that\n\n```\n
this.extensionsRegistry.setRecommendedFields(\n [\n {\n name:
'log_level',\n pattern: 'logs*', // This field is relevant for any index
starting with 'logs...'\n },\n {\n name: 'host.ip.keyword',\n pattern:
'logs-apache_error', // This field is specific to 'logs-apache_error'\n
},\n {\n name: 'http.request.method',\n pattern: 'logs*',\n },\n ],\n
'oblt'\n );\n```\n\nYou will see that the fields are getting prioritized
in the editor\n\n<img width=\"730\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/c4941579-c04f-4c5f-875d-369355420a3d\"\n/>\n\n\nAs
a bonus: The good thing with the recommended fields is that we
are\nprioritizing fields that exist in the datasource. So this means
that a\nsolution team can register fields per datasource and not afraid
if these\nfields do not exist at the instance\n\n\n### Checklist\n\n- [
] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios","sha":"642edd2f435d12a2f8ca7f337d8691645763e441","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Feature:ES|QL","Team:ESQL","backport:version","v9.1.0","v8.19.0"],"title":"[ES|QL]
Prioritize recommended fields in the
editor","number":224359,"url":"https://github.com/elastic/kibana/pull/224359","mergeCommit":{"message":"[ES|QL]
Prioritize recommended fields in the editor (#224359)\n\n##
Summary\n\nAdds the ability to register recommended fields per
datasource. It\nfollows the logic [of the recommended
queries]\n(#221474). \n\nThese
fields will get prioritized in the editor. So the fields\nprioritization
goes like:\n\n- recommended fields from the registry first\n- ecs schema
second\n- everything else\n\n### How to register them\n\n- Use the esql
server side registry as described in the aforementioned\nPR\n- Register
the fields like that\n\n```\n
this.extensionsRegistry.setRecommendedFields(\n [\n {\n name:
'log_level',\n pattern: 'logs*', // This field is relevant for any index
starting with 'logs...'\n },\n {\n name: 'host.ip.keyword',\n pattern:
'logs-apache_error', // This field is specific to 'logs-apache_error'\n
},\n {\n name: 'http.request.method',\n pattern: 'logs*',\n },\n ],\n
'oblt'\n );\n```\n\nYou will see that the fields are getting prioritized
in the editor\n\n<img width=\"730\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/c4941579-c04f-4c5f-875d-369355420a3d\"\n/>\n\n\nAs
a bonus: The good thing with the recommended fields is that we
are\nprioritizing fields that exist in the datasource. So this means
that a\nsolution team can register fields per datasource and not afraid
if these\nfields do not exist at the instance\n\n\n### Checklist\n\n- [
] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios","sha":"642edd2f435d12a2f8ca7f337d8691645763e441"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/224359","number":224359,"mergeCommit":{"message":"[ES|QL]
Prioritize recommended fields in the editor (#224359)\n\n##
Summary\n\nAdds the ability to register recommended fields per
datasource. It\nfollows the logic [of the recommended
queries]\n(#221474). \n\nThese
fields will get prioritized in the editor. So the fields\nprioritization
goes like:\n\n- recommended fields from the registry first\n- ecs schema
second\n- everything else\n\n### How to register them\n\n- Use the esql
server side registry as described in the aforementioned\nPR\n- Register
the fields like that\n\n```\n
this.extensionsRegistry.setRecommendedFields(\n [\n {\n name:
'log_level',\n pattern: 'logs*', // This field is relevant for any index
starting with 'logs...'\n },\n {\n name: 'host.ip.keyword',\n pattern:
'logs-apache_error', // This field is specific to 'logs-apache_error'\n
},\n {\n name: 'http.request.method',\n pattern: 'logs*',\n },\n ],\n
'oblt'\n );\n```\n\nYou will see that the fields are getting prioritized
in the editor\n\n<img width=\"730\"
alt=\"image\"\nsrc=\"https://github.com/user-attachments/assets/c4941579-c04f-4c5f-875d-369355420a3d\"\n/>\n\n\nAs
a bonus: The good thing with the recommended fields is that we
are\nprioritizing fields that exist in the datasource. So this means
that a\nsolution team can register fields per datasource and not afraid
if these\nfields do not exist at the instance\n\n\n### Checklist\n\n- [
] [Unit or
functional\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\nwere
updated or added to match the most common
scenarios","sha":"642edd2f435d12a2f8ca7f337d8691645763e441"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->
gbamparop added a commit that referenced this pull request Jun 23, 2025
## Summary
This PRs builds on top of the [extension point added to the E|QL
editor](#221474) by registering
recommended ES|QL queries from the Observability plugin.

It registers two recommended queries for logs and two for metrics:
- Kubernetes pods sorted by memory usage
- Kubernetes pods sorted by CPU usage
- Logs with "error" or "warn" messages
- Error occurrences by host name

### Demo - Solution navigation enabled

https://github.com/user-attachments/assets/ca06b89b-5574-4775-8beb-157fa1eaa97a

### Demo - Solution navigation disabled
Only the default recommended ES|QL queries are enabled in this mode.


https://github.com/user-attachments/assets/52043076-6467-44fa-a847-937d3e723afc

## Testing instructions
There is an environment deployed with `/oblt-deploy` that has logs and
metrics (check last month in Discover), you can find the credentials in
the issue posted by the bot in the issue timeline below

For local testing:
- Ingest logs and metrics. This can be done by running a Synthtrace
scenario, e.g. `node scripts/synthtrace.js logs_traces_hosts`
- Enable the Observability solution navigation
- Start writing an ES|QL query targeting indices that match the `logs-*`
or `metrics-*` patterns

## Open questions
- Should the `KQL` command be used in the queries?

Closes #224027

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
kibanamachine added a commit to kibanamachine/kibana that referenced this pull request Jun 23, 2025
## Summary
This PRs builds on top of the [extension point added to the E|QL
editor](elastic#221474) by registering
recommended ES|QL queries from the Observability plugin.

It registers two recommended queries for logs and two for metrics:
- Kubernetes pods sorted by memory usage
- Kubernetes pods sorted by CPU usage
- Logs with "error" or "warn" messages
- Error occurrences by host name

### Demo - Solution navigation enabled

https://github.com/user-attachments/assets/ca06b89b-5574-4775-8beb-157fa1eaa97a

### Demo - Solution navigation disabled
Only the default recommended ES|QL queries are enabled in this mode.

https://github.com/user-attachments/assets/52043076-6467-44fa-a847-937d3e723afc

## Testing instructions
There is an environment deployed with `/oblt-deploy` that has logs and
metrics (check last month in Discover), you can find the credentials in
the issue posted by the bot in the issue timeline below

For local testing:
- Ingest logs and metrics. This can be done by running a Synthtrace
scenario, e.g. `node scripts/synthtrace.js logs_traces_hosts`
- Enable the Observability solution navigation
- Start writing an ES|QL query targeting indices that match the `logs-*`
or `metrics-*` patterns

## Open questions
- Should the `KQL` command be used in the queries?

Closes elastic#224027

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
(cherry picked from commit 3a23553)
kibanamachine added a commit that referenced this pull request Jun 23, 2025
…#224861)

# Backport

This will backport the following commits from `main` to `8.19`:
- [Register recommended ES|QL queries for Observability
(#224054)](#224054)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Giorgos
Bamparopoulos","email":"georgios.bamparopoulos@elastic.co"},"sourceCommit":{"committedDate":"2025-06-23T12:08:18Z","message":"Register
recommended ES|QL queries for Observability (#224054)\n\n##
Summary\nThis PRs builds on top of the [extension point added to the
E|QL\neditor](#221474) by
registering\nrecommended ES|QL queries from the Observability
plugin.\n\nIt registers two recommended queries for logs and two for
metrics:\n- Kubernetes pods sorted by memory usage\n- Kubernetes pods
sorted by CPU usage\n- Logs with \"error\" or \"warn\" messages\n- Error
occurrences by host name\n\n### Demo - Solution navigation
enabled\n\nhttps://github.com/user-attachments/assets/ca06b89b-5574-4775-8beb-157fa1eaa97a\n\n###
Demo - Solution navigation disabled\nOnly the default recommended ES|QL
queries are enabled in this
mode.\n\n\nhttps://github.com/user-attachments/assets/52043076-6467-44fa-a847-937d3e723afc\n\n##
Testing instructions\nThere is an environment deployed with
`/oblt-deploy` that has logs and\nmetrics (check last month in
Discover), you can find the credentials in\nthe issue posted by the bot
in the issue timeline below\n\nFor local testing:\n- Ingest logs and
metrics. This can be done by running a Synthtrace\nscenario, e.g. `node
scripts/synthtrace.js logs_traces_hosts`\n- Enable the Observability
solution navigation\n- Start writing an ES|QL query targeting indices
that match the `logs-*`\nor `metrics-*` patterns\n\n## Open questions\n-
Should the `KQL` command be used in the queries?\n\nCloses
https://github.com/elastic/kibana/issues/224027\n\n---------\n\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"3a235538a01654b22ca9b177c5e6159ae7df588e","branchLabelMapping":{"^v9.1.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","Team:obs-ux-management","backport:version","v9.1.0","v8.19.0"],"title":"Register
recommended ES|QL queries for
Observability","number":224054,"url":"https://github.com/elastic/kibana/pull/224054","mergeCommit":{"message":"Register
recommended ES|QL queries for Observability (#224054)\n\n##
Summary\nThis PRs builds on top of the [extension point added to the
E|QL\neditor](#221474) by
registering\nrecommended ES|QL queries from the Observability
plugin.\n\nIt registers two recommended queries for logs and two for
metrics:\n- Kubernetes pods sorted by memory usage\n- Kubernetes pods
sorted by CPU usage\n- Logs with \"error\" or \"warn\" messages\n- Error
occurrences by host name\n\n### Demo - Solution navigation
enabled\n\nhttps://github.com/user-attachments/assets/ca06b89b-5574-4775-8beb-157fa1eaa97a\n\n###
Demo - Solution navigation disabled\nOnly the default recommended ES|QL
queries are enabled in this
mode.\n\n\nhttps://github.com/user-attachments/assets/52043076-6467-44fa-a847-937d3e723afc\n\n##
Testing instructions\nThere is an environment deployed with
`/oblt-deploy` that has logs and\nmetrics (check last month in
Discover), you can find the credentials in\nthe issue posted by the bot
in the issue timeline below\n\nFor local testing:\n- Ingest logs and
metrics. This can be done by running a Synthtrace\nscenario, e.g. `node
scripts/synthtrace.js logs_traces_hosts`\n- Enable the Observability
solution navigation\n- Start writing an ES|QL query targeting indices
that match the `logs-*`\nor `metrics-*` patterns\n\n## Open questions\n-
Should the `KQL` command be used in the queries?\n\nCloses
https://github.com/elastic/kibana/issues/224027\n\n---------\n\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"3a235538a01654b22ca9b177c5e6159ae7df588e"}},"sourceBranch":"main","suggestedTargetBranches":["8.19"],"targetPullRequestStates":[{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/224054","number":224054,"mergeCommit":{"message":"Register
recommended ES|QL queries for Observability (#224054)\n\n##
Summary\nThis PRs builds on top of the [extension point added to the
E|QL\neditor](#221474) by
registering\nrecommended ES|QL queries from the Observability
plugin.\n\nIt registers two recommended queries for logs and two for
metrics:\n- Kubernetes pods sorted by memory usage\n- Kubernetes pods
sorted by CPU usage\n- Logs with \"error\" or \"warn\" messages\n- Error
occurrences by host name\n\n### Demo - Solution navigation
enabled\n\nhttps://github.com/user-attachments/assets/ca06b89b-5574-4775-8beb-157fa1eaa97a\n\n###
Demo - Solution navigation disabled\nOnly the default recommended ES|QL
queries are enabled in this
mode.\n\n\nhttps://github.com/user-attachments/assets/52043076-6467-44fa-a847-937d3e723afc\n\n##
Testing instructions\nThere is an environment deployed with
`/oblt-deploy` that has logs and\nmetrics (check last month in
Discover), you can find the credentials in\nthe issue posted by the bot
in the issue timeline below\n\nFor local testing:\n- Ingest logs and
metrics. This can be done by running a Synthtrace\nscenario, e.g. `node
scripts/synthtrace.js logs_traces_hosts`\n- Enable the Observability
solution navigation\n- Start writing an ES|QL query targeting indices
that match the `logs-*`\nor `metrics-*` patterns\n\n## Open questions\n-
Should the `KQL` command be used in the queries?\n\nCloses
https://github.com/elastic/kibana/issues/224027\n\n---------\n\nCo-authored-by:
Elastic Machine
<elasticmachine@users.noreply.github.com>\nCo-authored-by: kibanamachine
<42973632+kibanamachine@users.noreply.github.com>","sha":"3a235538a01654b22ca9b177c5e6159ae7df588e"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Giorgos Bamparopoulos <georgios.bamparopoulos@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request Jun 25, 2025
## Summary

Adds the ability to register recommended fields per datasource. It
follows the logic [of the recommended queries]
(elastic#221474). 

These fields will get prioritized in the editor. So the fields
prioritization goes like:

- recommended fields from the registry first
- ecs schema second
- everything else

### How to register them

- Use the esql server side registry as described in the aforementioned
PR
- Register the fields like that

```
    this.extensionsRegistry.setRecommendedFields(
      [
        {
          name: 'log_level',
          pattern: 'logs*', // This field is relevant for any index starting with 'logs...'
        },
        {
          name: 'host.ip.keyword',
          pattern: 'logs-apache_error', // This field is specific to 'logs-apache_error'
        },
        {
          name: 'http.request.method',
          pattern: 'logs*',
        },
      ],
      'oblt'
    );
```

You will see that the fields are getting prioritized in the editor

<img width="730" alt="image"
src="https://github.com/user-attachments/assets/c4941579-c04f-4c5f-875d-369355420a3d"
/>


As a bonus: The good thing with the recommended fields is that we are
prioritizing fields that exist in the datasource. So this means that a
solution team can register fields per datasource and not afraid if these
fields do not exist at the instance


### Checklist

- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
akowalska622 pushed a commit to akowalska622/kibana that referenced this pull request Jun 25, 2025
## Summary
This PRs builds on top of the [extension point added to the E|QL
editor](elastic#221474) by registering
recommended ES|QL queries from the Observability plugin.

It registers two recommended queries for logs and two for metrics:
- Kubernetes pods sorted by memory usage
- Kubernetes pods sorted by CPU usage
- Logs with "error" or "warn" messages
- Error occurrences by host name

### Demo - Solution navigation enabled

https://github.com/user-attachments/assets/ca06b89b-5574-4775-8beb-157fa1eaa97a

### Demo - Solution navigation disabled
Only the default recommended ES|QL queries are enabled in this mode.


https://github.com/user-attachments/assets/52043076-6467-44fa-a847-937d3e723afc

## Testing instructions
There is an environment deployed with `/oblt-deploy` that has logs and
metrics (check last month in Discover), you can find the credentials in
the issue posted by the bot in the issue timeline below

For local testing:
- Ingest logs and metrics. This can be done by running a Synthtrace
scenario, e.g. `node scripts/synthtrace.js logs_traces_hosts`
- Enable the Observability solution navigation
- Start writing an ES|QL query targeting indices that match the `logs-*`
or `metrics-*` patterns

## Open questions
- Should the `KQL` command be used in the queries?

Closes elastic#224027

---------

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels Feature:ES|QL ES|QL related features in Kibana release_note:skip Skip the PR/issue when compiling release notes Team:ESQL ES|QL related features in Kibana t// v8.19.0 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ES|QL] Contextual recommended queries [MVP]

4 participants