[ES|QL] Update grammars by kibanamachine · Pull Request #213006 · elastic/kibana

kibanamachine · 2025-03-04T01:03:05Z

This PR updates the ES|QL grammars (lexer and parser) to match the latest version in Elasticsearch.

…ost-scripts

elasticmachine · 2025-03-04T01:03:08Z

Pinging @elastic/kibana-esql (Team:ESQL)

stratoula

Looks great, thanx for working on this. I know it was hard 👏

stratoula · 2025-03-18T09:31:40Z

The failure in the jest test is very alarming 🤔

…ana into esql_grammar_sync_1741050179

drewdaemon · 2025-03-18T16:14:20Z

@stratoula that test used an invalid query (extra parenthesis).

The parser used be more forgiving of that kind of mistake, so I think the problem with the tested query got missed because it still generated an AST. With the changes here from Elasticsearch, the parser instead throws a "Stack empty" error and gives up, leading to an empty result.

So, it's just a matter of correcting the query in that particular test case.

stratoula · 2025-03-18T16:15:47Z

Ah great catch! Opening and closing parenthesis are so difficult to follow sometimes

drewdaemon · 2025-03-18T17:38:08Z

@elasticmachine merge upstream

elasticmachine · 2025-03-18T19:26:17Z

💚 Build Succeeded

Buildkite Build
Commit: 2069e48

Metrics [docs]

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`kbnUiSharedDeps-srcJs`	3.6MB	3.6MB	+7.9KB

History

cc @drewdaemon

kibanamachine · 2025-03-18T19:26:21Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/13931820929

This PR updates the ES|QL grammars (lexer and parser) to match the latest version in Elasticsearch. --------- Co-authored-by: drewdaemon <drew.tate@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> (cherry picked from commit 681295f)

kibanamachine · 2025-03-18T19:32:33Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

# Backport This will backport the following commits from `main` to `8.x`: - [[ES|QL] Update grammars (#213006)](#213006)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport)

This PR updates the ES|QL grammars (lexer and parser) to match the latest version in Elasticsearch. --------- Co-authored-by: drewdaemon <drew.tate@elastic.co> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co>

@ioanatia

## Summary Part of #215092 ## Considerations ### `RRF` may change in the future @ioanatia has stated that the command may change in the future as it's not fully implemented yet. ### Licence check Will be addressed at #216791 https://github.com/user-attachments/assets/759470a2-4afa-4d20-adbf-cb3001895e95 ## New command support checklist ### AST support - [x] Make sure that the new command is in the local Kibana grammar definition. The ANTLR lexer and parser files are updated every Monday from the source definition of the language at Elasticsearch (via a manually merged, automatically generated [PR](#213006)). - [x] Create a factory for generating the new node. The new node should satisfy the `ESQLCommand<Name>` interface. If the syntax of your command cannot be decomposed only in parameters, you can hold extra information by extending the `ESQLCommand` interface. I.E., check the Rerank command. - [x] While ANTLR is parsing the text query, we create our own AST by using `onExit` listeners at `kbn-esql-ast/src/parser/esql_ast_builder_listener.ts`. Implement the `onExit<COMMAND_NAME>` method based on the interface autogenerated by ANTLR and push the new node into the AST. - [x] Create unit tests checking that the correct AST nodes are generated when parsing your command. - [x] Add a dedicated [visitor callback](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-ast/src/visitor/README.md) for the new command. - [x] Verify that the `Walker` API can visit the new node. - [x] Verify that the `Synth` API can construct the new node. ### Validating that the command works well when prettifying the query - [x] Validate that the prettifier works correctly. - [ ] Adjust the basic pretty printer and the wrapping pretty printer if needed. - [x] Add unit tests validating that the queries are correctly formatted (even if no adjustment has been done). ### Creating the command definition - [x] Add the definition of the new command at `kbn-esql-validation-autocomplete/src/definitions/commands.ts`. ### Adding the corresponding client-side validations - [x] Add a custom validation if needed. - [x] Add tests checking the behavior of the validation following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#the-new-way). ### Adding the autocomplete suggestions - [x] Add the suggestions to be shown when **positioned at** the new command. - [x] Create a new folder at `kbn-esql-validation-autocomplete/src/autocomplete/commands` for your command. - [x] Export a `suggest` function that should return an array of suggestions and set it up into the command definition. - [ ] Optionally, we must filter or incorporate fields suggestions after a command is executed, this is supported by adding the `fieldsSuggestionsAfter` method to the command definition. Read this documentation to understand how it works. - [x] If the new command must be suggested only in particular situations, modify the corresponding suggestions to make it possible. - [x] Add tests following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#automated-testing). ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> Co-authored-by: Drew Tate <drewctate@gmail.com>

@ioanatia

## Summary Part of elastic#215092 ## Considerations ### `RRF` may change in the future @ioanatia has stated that the command may change in the future as it's not fully implemented yet. ### Licence check Will be addressed at elastic#216791 https://github.com/user-attachments/assets/759470a2-4afa-4d20-adbf-cb3001895e95 ## New command support checklist ### AST support - [x] Make sure that the new command is in the local Kibana grammar definition. The ANTLR lexer and parser files are updated every Monday from the source definition of the language at Elasticsearch (via a manually merged, automatically generated [PR](elastic#213006)). - [x] Create a factory for generating the new node. The new node should satisfy the `ESQLCommand<Name>` interface. If the syntax of your command cannot be decomposed only in parameters, you can hold extra information by extending the `ESQLCommand` interface. I.E., check the Rerank command. - [x] While ANTLR is parsing the text query, we create our own AST by using `onExit` listeners at `kbn-esql-ast/src/parser/esql_ast_builder_listener.ts`. Implement the `onExit<COMMAND_NAME>` method based on the interface autogenerated by ANTLR and push the new node into the AST. - [x] Create unit tests checking that the correct AST nodes are generated when parsing your command. - [x] Add a dedicated [visitor callback](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-ast/src/visitor/README.md) for the new command. - [x] Verify that the `Walker` API can visit the new node. - [x] Verify that the `Synth` API can construct the new node. ### Validating that the command works well when prettifying the query - [x] Validate that the prettifier works correctly. - [ ] Adjust the basic pretty printer and the wrapping pretty printer if needed. - [x] Add unit tests validating that the queries are correctly formatted (even if no adjustment has been done). ### Creating the command definition - [x] Add the definition of the new command at `kbn-esql-validation-autocomplete/src/definitions/commands.ts`. ### Adding the corresponding client-side validations - [x] Add a custom validation if needed. - [x] Add tests checking the behavior of the validation following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#the-new-way). ### Adding the autocomplete suggestions - [x] Add the suggestions to be shown when **positioned at** the new command. - [x] Create a new folder at `kbn-esql-validation-autocomplete/src/autocomplete/commands` for your command. - [x] Export a `suggest` function that should return an array of suggestions and set it up into the command definition. - [ ] Optionally, we must filter or incorporate fields suggestions after a command is executed, this is supported by adding the `fieldsSuggestionsAfter` method to the command definition. Read this documentation to understand how it works. - [x] If the new command must be suggested only in particular situations, modify the corresponding suggestions to make it possible. - [x] Add tests following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#automated-testing). ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> Co-authored-by: Drew Tate <drewctate@gmail.com> (cherry picked from commit 0794030) # Conflicts: # src/platform/packages/shared/kbn-esql-validation-autocomplete/src/definitions/commands.ts

@ioanatia

## Summary Part of elastic#215092 ## Considerations ### `RRF` may change in the future @ioanatia has stated that the command may change in the future as it's not fully implemented yet. ### Licence check Will be addressed at elastic#216791 https://github.com/user-attachments/assets/759470a2-4afa-4d20-adbf-cb3001895e95 ## New command support checklist ### AST support - [x] Make sure that the new command is in the local Kibana grammar definition. The ANTLR lexer and parser files are updated every Monday from the source definition of the language at Elasticsearch (via a manually merged, automatically generated [PR](elastic#213006)). - [x] Create a factory for generating the new node. The new node should satisfy the `ESQLCommand<Name>` interface. If the syntax of your command cannot be decomposed only in parameters, you can hold extra information by extending the `ESQLCommand` interface. I.E., check the Rerank command. - [x] While ANTLR is parsing the text query, we create our own AST by using `onExit` listeners at `kbn-esql-ast/src/parser/esql_ast_builder_listener.ts`. Implement the `onExit<COMMAND_NAME>` method based on the interface autogenerated by ANTLR and push the new node into the AST. - [x] Create unit tests checking that the correct AST nodes are generated when parsing your command. - [x] Add a dedicated [visitor callback](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-ast/src/visitor/README.md) for the new command. - [x] Verify that the `Walker` API can visit the new node. - [x] Verify that the `Synth` API can construct the new node. ### Validating that the command works well when prettifying the query - [x] Validate that the prettifier works correctly. - [ ] Adjust the basic pretty printer and the wrapping pretty printer if needed. - [x] Add unit tests validating that the queries are correctly formatted (even if no adjustment has been done). ### Creating the command definition - [x] Add the definition of the new command at `kbn-esql-validation-autocomplete/src/definitions/commands.ts`. ### Adding the corresponding client-side validations - [x] Add a custom validation if needed. - [x] Add tests checking the behavior of the validation following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#the-new-way). ### Adding the autocomplete suggestions - [x] Add the suggestions to be shown when **positioned at** the new command. - [x] Create a new folder at `kbn-esql-validation-autocomplete/src/autocomplete/commands` for your command. - [x] Export a `suggest` function that should return an array of suggestions and set it up into the command definition. - [ ] Optionally, we must filter or incorporate fields suggestions after a command is executed, this is supported by adding the `fieldsSuggestionsAfter` method to the command definition. Read this documentation to understand how it works. - [x] If the new command must be suggested only in particular situations, modify the corresponding suggestions to make it possible. - [x] Add tests following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#automated-testing). ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> Co-authored-by: Drew Tate <drewctate@gmail.com> (cherry picked from commit 0794030) # Conflicts: # src/platform/packages/shared/kbn-esql-validation-autocomplete/src/definitions/commands.ts

@ioanatia

## Summary Part of elastic#215092 ## Considerations ### `RRF` may change in the future @ioanatia has stated that the command may change in the future as it's not fully implemented yet. ### Licence check Will be addressed at elastic#216791 https://github.com/user-attachments/assets/759470a2-4afa-4d20-adbf-cb3001895e95 ## New command support checklist ### AST support - [x] Make sure that the new command is in the local Kibana grammar definition. The ANTLR lexer and parser files are updated every Monday from the source definition of the language at Elasticsearch (via a manually merged, automatically generated [PR](elastic#213006)). - [x] Create a factory for generating the new node. The new node should satisfy the `ESQLCommand<Name>` interface. If the syntax of your command cannot be decomposed only in parameters, you can hold extra information by extending the `ESQLCommand` interface. I.E., check the Rerank command. - [x] While ANTLR is parsing the text query, we create our own AST by using `onExit` listeners at `kbn-esql-ast/src/parser/esql_ast_builder_listener.ts`. Implement the `onExit<COMMAND_NAME>` method based on the interface autogenerated by ANTLR and push the new node into the AST. - [x] Create unit tests checking that the correct AST nodes are generated when parsing your command. - [x] Add a dedicated [visitor callback](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-ast/src/visitor/README.md) for the new command. - [x] Verify that the `Walker` API can visit the new node. - [x] Verify that the `Synth` API can construct the new node. ### Validating that the command works well when prettifying the query - [x] Validate that the prettifier works correctly. - [ ] Adjust the basic pretty printer and the wrapping pretty printer if needed. - [x] Add unit tests validating that the queries are correctly formatted (even if no adjustment has been done). ### Creating the command definition - [x] Add the definition of the new command at `kbn-esql-validation-autocomplete/src/definitions/commands.ts`. ### Adding the corresponding client-side validations - [x] Add a custom validation if needed. - [x] Add tests checking the behavior of the validation following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#the-new-way). ### Adding the autocomplete suggestions - [x] Add the suggestions to be shown when **positioned at** the new command. - [x] Create a new folder at `kbn-esql-validation-autocomplete/src/autocomplete/commands` for your command. - [x] Export a `suggest` function that should return an array of suggestions and set it up into the command definition. - [ ] Optionally, we must filter or incorporate fields suggestions after a command is executed, this is supported by adding the `fieldsSuggestionsAfter` method to the command definition. Read this documentation to understand how it works. - [x] If the new command must be suggested only in particular situations, modify the corresponding suggestions to make it possible. - [x] Add tests following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#automated-testing). ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> Co-authored-by: Drew Tate <drewctate@gmail.com> (cherry picked from commit 0794030)

@ioanatia

## Summary Part of elastic#215092 ## Considerations ### `RRF` may change in the future @ioanatia has stated that the command may change in the future as it's not fully implemented yet. ### Licence check Will be addressed at elastic#216791 https://github.com/user-attachments/assets/759470a2-4afa-4d20-adbf-cb3001895e95 ## New command support checklist ### AST support - [x] Make sure that the new command is in the local Kibana grammar definition. The ANTLR lexer and parser files are updated every Monday from the source definition of the language at Elasticsearch (via a manually merged, automatically generated [PR](elastic#213006)). - [x] Create a factory for generating the new node. The new node should satisfy the `ESQLCommand<Name>` interface. If the syntax of your command cannot be decomposed only in parameters, you can hold extra information by extending the `ESQLCommand` interface. I.E., check the Rerank command. - [x] While ANTLR is parsing the text query, we create our own AST by using `onExit` listeners at `kbn-esql-ast/src/parser/esql_ast_builder_listener.ts`. Implement the `onExit<COMMAND_NAME>` method based on the interface autogenerated by ANTLR and push the new node into the AST. - [x] Create unit tests checking that the correct AST nodes are generated when parsing your command. - [x] Add a dedicated [visitor callback](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-ast/src/visitor/README.md) for the new command. - [x] Verify that the `Walker` API can visit the new node. - [x] Verify that the `Synth` API can construct the new node. ### Validating that the command works well when prettifying the query - [x] Validate that the prettifier works correctly. - [ ] Adjust the basic pretty printer and the wrapping pretty printer if needed. - [x] Add unit tests validating that the queries are correctly formatted (even if no adjustment has been done). ### Creating the command definition - [x] Add the definition of the new command at `kbn-esql-validation-autocomplete/src/definitions/commands.ts`. ### Adding the corresponding client-side validations - [x] Add a custom validation if needed. - [x] Add tests checking the behavior of the validation following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#the-new-way). ### Adding the autocomplete suggestions - [x] Add the suggestions to be shown when **positioned at** the new command. - [x] Create a new folder at `kbn-esql-validation-autocomplete/src/autocomplete/commands` for your command. - [x] Export a `suggest` function that should return an array of suggestions and set it up into the command definition. - [ ] Optionally, we must filter or incorporate fields suggestions after a command is executed, this is supported by adding the `fieldsSuggestionsAfter` method to the command definition. Read this documentation to understand how it works. - [x] If the new command must be suggested only in particular situations, modify the corresponding suggestions to make it possible. - [x] Add tests following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#automated-testing). ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> Co-authored-by: Drew Tate <drewctate@gmail.com>

@ioanatia

## Summary Part of elastic#215092 ## Considerations ### `RRF` may change in the future @ioanatia has stated that the command may change in the future as it's not fully implemented yet. ### Licence check Will be addressed at elastic#216791 https://github.com/user-attachments/assets/759470a2-4afa-4d20-adbf-cb3001895e95 ## New command support checklist ### AST support - [x] Make sure that the new command is in the local Kibana grammar definition. The ANTLR lexer and parser files are updated every Monday from the source definition of the language at Elasticsearch (via a manually merged, automatically generated [PR](elastic#213006)). - [x] Create a factory for generating the new node. The new node should satisfy the `ESQLCommand<Name>` interface. If the syntax of your command cannot be decomposed only in parameters, you can hold extra information by extending the `ESQLCommand` interface. I.E., check the Rerank command. - [x] While ANTLR is parsing the text query, we create our own AST by using `onExit` listeners at `kbn-esql-ast/src/parser/esql_ast_builder_listener.ts`. Implement the `onExit<COMMAND_NAME>` method based on the interface autogenerated by ANTLR and push the new node into the AST. - [x] Create unit tests checking that the correct AST nodes are generated when parsing your command. - [x] Add a dedicated [visitor callback](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-ast/src/visitor/README.md) for the new command. - [x] Verify that the `Walker` API can visit the new node. - [x] Verify that the `Synth` API can construct the new node. ### Validating that the command works well when prettifying the query - [x] Validate that the prettifier works correctly. - [ ] Adjust the basic pretty printer and the wrapping pretty printer if needed. - [x] Add unit tests validating that the queries are correctly formatted (even if no adjustment has been done). ### Creating the command definition - [x] Add the definition of the new command at `kbn-esql-validation-autocomplete/src/definitions/commands.ts`. ### Adding the corresponding client-side validations - [x] Add a custom validation if needed. - [x] Add tests checking the behavior of the validation following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#the-new-way). ### Adding the autocomplete suggestions - [x] Add the suggestions to be shown when **positioned at** the new command. - [x] Create a new folder at `kbn-esql-validation-autocomplete/src/autocomplete/commands` for your command. - [x] Export a `suggest` function that should return an array of suggestions and set it up into the command definition. - [ ] Optionally, we must filter or incorporate fields suggestions after a command is executed, this is supported by adding the `fieldsSuggestionsAfter` method to the command definition. Read this documentation to understand how it works. - [x] If the new command must be suggested only in particular situations, modify the corresponding suggestions to make it possible. - [x] Add tests following this [guide](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-esql-validation-autocomplete/README.md#automated-testing). ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/src/platform/packages/shared/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the [docker list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker) - [ ] This was checked for breaking HTTP API changes, and any breaking changes have been approved by the breaking-change committee. The `release_note:breaking` label should be applied in these situations. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) ### Identify risks Does this PR introduce any risks? For example, consider risks like hard to test bugs, performance regression, potential of data loss. Describe the risk, its severity, and mitigation for each identified risk. Invite stakeholders and evaluate how to proceed before merging. - [ ] [See some risk examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) - [ ] ... --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Stratoula Kalafateli <efstratia.kalafateli@elastic.co> Co-authored-by: Drew Tate <drewctate@gmail.com>

drewdaemon and others added 22 commits February 6, 2025 16:42

Merge branch 'main' of github.com:elastic/kibana

0e937a9

Merge branch 'main' of github.com:elastic/kibana

7ffbf4c

Merge branch 'main' of github.com:elastic/kibana

b426318

Merge branch 'main' of github.com:elastic/kibana

511ccaa

Merge branch 'main' of github.com:elastic/kibana

2d71300

Merge branch 'main' of github.com:elastic/kibana

0076de9

Merge branch 'main' of github.com:elastic/kibana

2898ce1

add post processing

df6f876

add post processing

969c075

add post processing

2535b1c

something

767f5cf

Merge branch 'main' of github.com:elastic/kibana into fix-esql-sync-p…

ce1e507

…ost-scripts

change

d0a3958

lolz

5d15ab8

lolz

208813f

lolz

bc62fc6

lolz

663c6fa

lolz

4cf954b

lolz

3cbfa00

lolz

1c4397e

lolz

b89348b

Update ES|QL grammars

b7369b8

kibanamachine requested a review from a team as a code owner March 4, 2025 01:03

kibanamachine added release_note:skip Skip the PR/issue when compiling release notes Team:ESQL ES|QL related features in Kibana t// labels Mar 4, 2025

Update postbuild.sh

b530e41

drewdaemon added backport:version Backport to applied version labels v9.1.0 v8.19.0 labels Mar 4, 2025

Merge branch 'main' into esql_grammar_sync_1741050179

576a0f0

stratoula approved these changes Mar 18, 2025

View reviewed changes

eokoneyo approved these changes Mar 18, 2025

View reviewed changes

drewdaemon added 2 commits March 18, 2025 10:07

Merge branch 'esql_grammar_sync_1741050179' of github.com:elastic/kib…

0ef2d5e

…ana into esql_grammar_sync_1741050179

fix a last test

a9d0007

Merge branch 'main' into esql_grammar_sync_1741050179

2069e48

drewdaemon enabled auto-merge (squash) March 18, 2025 18:21

drewdaemon merged commit 681295f into main Mar 18, 2025
9 checks passed

drewdaemon deleted the esql_grammar_sync_1741050179 branch March 18, 2025 19:25

kibanamachine mentioned this pull request Mar 18, 2025

[8.x] [ES|QL] Update grammars (#213006) #215058

Merged

sddonne mentioned this pull request May 23, 2025

[ES|QL] Add support for RRF #221349

Merged

28 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ES|QL] Update grammars#213006

[ES|QL] Update grammars#213006
drewdaemon merged 41 commits intomainfrom
esql_grammar_sync_1741050179

kibanamachine commented Mar 4, 2025 •

edited

Loading

Uh oh!

elasticmachine commented Mar 4, 2025

Uh oh!

stratoula left a comment

Uh oh!

stratoula commented Mar 18, 2025

Uh oh!

drewdaemon commented Mar 18, 2025 •

edited

Loading

Uh oh!

stratoula commented Mar 18, 2025

Uh oh!

drewdaemon commented Mar 18, 2025

Uh oh!

Uh oh!

elasticmachine commented Mar 18, 2025

Uh oh!

kibanamachine commented Mar 18, 2025

Uh oh!

kibanamachine commented Mar 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

kibanamachine commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Mar 4, 2025

Uh oh!

stratoula left a comment

Choose a reason for hiding this comment

Uh oh!

stratoula commented Mar 18, 2025

Uh oh!

drewdaemon commented Mar 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stratoula commented Mar 18, 2025

Uh oh!

drewdaemon commented Mar 18, 2025

Uh oh!

Uh oh!

elasticmachine commented Mar 18, 2025

💚 Build Succeeded

Metrics [docs]

Page load bundle

History

Uh oh!

kibanamachine commented Mar 18, 2025

Uh oh!

kibanamachine commented Mar 18, 2025

💚 All backports created successfully

Questions ?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kibanamachine commented Mar 4, 2025 •

edited

Loading

drewdaemon commented Mar 18, 2025 •

edited

Loading