-
Notifications
You must be signed in to change notification settings - Fork 2.3k
Approximation framework to support numeric search_after queries
#18896
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Approximation framework to support numeric search_after queries
#18896
Conversation
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
Signed-off-by: Prudhvi Godithi <[email protected]>
|
❌ Gradle check result for 3c6596a: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
server/src/main/java/org/opensearch/index/mapper/DateFieldMapper.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Prudhvi Godithi <[email protected]>
|
❌ Gradle check result for 9137fe5: FAILURE Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change? |
Signed-off-by: Prudhvi Godithi <[email protected]>
|
For some reason @kkewwei 's approval didn't count towards merge. Approving the PR while we figure out the codeowners issue. |
6468464
into
opensearch-project:main
…ensearch-project#18896) * Initial commit for search_after queries Signed-off-by: Prudhvi Godithi <[email protected]> * Test with increment and decrement with search_after Signed-off-by: Prudhvi Godithi <[email protected]> * search_after queries Signed-off-by: Prudhvi Godithi <[email protected]> * Support framework for search_after queries Signed-off-by: Prudhvi Godithi <[email protected]> * Fix gradle precommit issues Signed-off-by: Prudhvi Godithi <[email protected]> * Add comments Signed-off-by: Prudhvi Godithi <[email protected]> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <[email protected]> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <[email protected]> * Update CHANGELOG.md Signed-off-by: Prudhvi Godithi <[email protected]> * Update CHANGELOG.md and fetch upstream Signed-off-by: Prudhvi Godithi <[email protected]> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <[email protected]> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <[email protected]> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <[email protected]> * Update code with comments Signed-off-by: Prudhvi Godithi <[email protected]> * Add encode tests Signed-off-by: Prudhvi Godithi <[email protected]> * Fix conflicts Signed-off-by: Prudhvi Godithi <[email protected]> * Fix spotless Signed-off-by: Prudhvi Godithi <[email protected]> * Code clean up Signed-off-by: Prudhvi Godithi <[email protected]> * Upstream fetch Signed-off-by: Prudhvi Godithi <[email protected]> * Upstream Fetch Signed-off-by: Prudhvi Godithi <[email protected]> * Add clamps Signed-off-by: Prudhvi Godithi <[email protected]> * Upstream Fetch to resolve conflicts Signed-off-by: Prudhvi Godithi <[email protected]> --------- Signed-off-by: Prudhvi Godithi <[email protected]>
…ensearch-project#18896) * Initial commit for search_after queries Signed-off-by: Prudhvi Godithi <[email protected]> * Test with increment and decrement with search_after Signed-off-by: Prudhvi Godithi <[email protected]> * search_after queries Signed-off-by: Prudhvi Godithi <[email protected]> * Support framework for search_after queries Signed-off-by: Prudhvi Godithi <[email protected]> * Fix gradle precommit issues Signed-off-by: Prudhvi Godithi <[email protected]> * Add comments Signed-off-by: Prudhvi Godithi <[email protected]> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <[email protected]> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <[email protected]> * Update CHANGELOG.md Signed-off-by: Prudhvi Godithi <[email protected]> * Update CHANGELOG.md and fetch upstream Signed-off-by: Prudhvi Godithi <[email protected]> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <[email protected]> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <[email protected]> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <[email protected]> * Update code with comments Signed-off-by: Prudhvi Godithi <[email protected]> * Add encode tests Signed-off-by: Prudhvi Godithi <[email protected]> * Fix conflicts Signed-off-by: Prudhvi Godithi <[email protected]> * Fix spotless Signed-off-by: Prudhvi Godithi <[email protected]> * Code clean up Signed-off-by: Prudhvi Godithi <[email protected]> * Upstream fetch Signed-off-by: Prudhvi Godithi <[email protected]> * Upstream Fetch Signed-off-by: Prudhvi Godithi <[email protected]> * Add clamps Signed-off-by: Prudhvi Godithi <[email protected]> * Upstream Fetch to resolve conflicts Signed-off-by: Prudhvi Godithi <[email protected]> --------- Signed-off-by: Prudhvi Godithi <[email protected]>
…ensearch-project#18896) * Initial commit for search_after queries Signed-off-by: Prudhvi Godithi <[email protected]> * Test with increment and decrement with search_after Signed-off-by: Prudhvi Godithi <[email protected]> * search_after queries Signed-off-by: Prudhvi Godithi <[email protected]> * Support framework for search_after queries Signed-off-by: Prudhvi Godithi <[email protected]> * Fix gradle precommit issues Signed-off-by: Prudhvi Godithi <[email protected]> * Add comments Signed-off-by: Prudhvi Godithi <[email protected]> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <[email protected]> * Attempt to fix the test Signed-off-by: Prudhvi Godithi <[email protected]> * Update CHANGELOG.md Signed-off-by: Prudhvi Godithi <[email protected]> * Update CHANGELOG.md and fetch upstream Signed-off-by: Prudhvi Godithi <[email protected]> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <[email protected]> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <[email protected]> * Update tests validating with lucene searchAfter Signed-off-by: Prudhvi Godithi <[email protected]> * Update code with comments Signed-off-by: Prudhvi Godithi <[email protected]> * Add encode tests Signed-off-by: Prudhvi Godithi <[email protected]> * Fix conflicts Signed-off-by: Prudhvi Godithi <[email protected]> * Fix spotless Signed-off-by: Prudhvi Godithi <[email protected]> * Code clean up Signed-off-by: Prudhvi Godithi <[email protected]> * Upstream fetch Signed-off-by: Prudhvi Godithi <[email protected]> * Upstream Fetch Signed-off-by: Prudhvi Godithi <[email protected]> * Add clamps Signed-off-by: Prudhvi Godithi <[email protected]> * Upstream Fetch to resolve conflicts Signed-off-by: Prudhvi Godithi <[email protected]> --------- Signed-off-by: Prudhvi Godithi <[email protected]>
This PR targets to support
search_afterqueries from the solution described in this issue #18546.ASCsort and upper bound forDESCsort. Example:Adds new
byte[] encodePoint(Object value, boolean roundUp);inNumericPointEncoderas todaysearch_aftervalue is returned as object and requires converting it into the appropriate field specific byte[].search_afterwithASCsort we want values greater than thesearch_aftervalue and forDESCsort we want values less than thesearch_after.Handles the search_after conversation for dates with date math rounding operations, here is the sample query. Note the existing
parseToLongis for date math rounding operations and not for creatingsearch_afterbounds.Coming from this PR https://github.com/opensearch-project/OpenSearch/pull/18763/files where approximation code path is disabled for multi sort, hence
search_aftershould work as expected when dealing with multiple tie breakersearch_aftervaluesRegarding benchmarks following are the queries we should see improvement from the existing workloads
desc_sort_with_after_timestamp,asc_sort_with_after_timestampdesc_sort_with_after_timestamp,asc_sort_with_after_timestampasc_sort_with_after_population,desc_sort_with_after_geonameid,asc_sort_with_after_geonameidDescription
[Describe what this change achieves]
Related Issues
Resolves #18546
Check List
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.