Skip to content

Add ES|QL full text functions challenges#759

Merged
carlosdelest merged 13 commits intoelastic:masterfrom
carlosdelest:feature/esql-full-text-functions
Mar 21, 2025
Merged

Add ES|QL full text functions challenges#759
carlosdelest merged 13 commits intoelastic:masterfrom
carlosdelest:feature/esql-full-text-functions

Conversation

@carlosdelest
Copy link
Member

Adds ES|QL full text functions challenges to existing tracks:

Search use case

Check that full text functions behave correctly for a full text search scenario with enough text data. Sorting by score will be used in all queries.

  • Use wikipedia rally track as a base (23M docs), using 1 node / 1 shard / force merge to a single segment.
  • Perform queries using ESQL full text functions (MATCH, QSTR, KQL).
  • Perform equivalent queries using the query DSL

Observability use case

Perform filtering using full text functions for partial word matches, with no scoring.

  • Use elastic/logs rally track - it can provide many shards and it has been already used for ESQL scale testing
  • Define full text function queries. Use as a base the text queries done in the different track workflows (searching for exceptions, SQL logs, etc) and add filtering based on time ranges

Closes elastic/elasticsearch#124649

Copy link

@ioanatia ioanatia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking at the changes in the track.py the ES|QL and query DSL queries should be equivalent (same queries are being used, same queried fields, same source fields being retrieved, same size etc).



def register(registry):
registry.register_param_source("query-string-search", QueryParamSource)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any impact from this renaming?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The param-source attribute for the query-string-search operation has been changed - no other changes are needed as just the param source is changed and not the actual query name.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo ES@QL

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤦 Done in fe9c146

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's a linting CI error that we should fix before merging - there are some extra lines here too that we prob want to remove

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right 🤦 Executed linting in fe9c146

@carlosdelest carlosdelest requested review from a team and luigidellaquila March 21, 2025 09:27
Copy link
Contributor

@luigidellaquila luigidellaquila left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@carlosdelest carlosdelest merged commit ef11d3e into elastic:master Mar 21, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ES|QL Full Text Functions benchmarking

4 participants

Comments