Enable Fuzzy codec for doc id fields using a bloom filter#11027

Merged

shwetathareja merged 9 commits intoopensearch-project:mainfrom

mgodwan:fuzzy-filter

Feb 5, 2024

Member

mgodwan commented Oct 31, 2023 •

edited

Loading

Description

Enable Fuzzy codec for doc id fields using a bloom filter. This provides added performance advantage for update and get operations.

Related Issues

Resolves #4489

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)
Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions bot added discuss enhancement Indexing & Search labels

Member Author

mgodwan commented Oct 31, 2023

Looking into the failing gradle check due to FuzzyFilter tests
Adding upgrade tests

mgodwan requested review from nknize and shwetathareja

October 31, 2023 13:31

mgodwan commented

View reviewed changes

server/src/main/java/org/opensearch/index/codec/fuzzy/IndexInputLongArray.java Outdated

mgodwan marked this pull request as ready for review

October 31, 2023 13:35

mgodwan requested review from Bukhtawar, CEHENKLE, Rishikesh1159, abbashus, anasalkouz, andrross, dblock, dbwiddis, dreamer-89, gbbafna, kartg, kotwanikunal, mch2, msfroh, owaiskazi19, peternied, reta, ryanbogan, sachinpkale, saratvemulapalli, setiah and sohami as code owners

October 31, 2023 13:35

Contributor

github-actions bot commented Feb 1, 2024

❌ Gradle check result for bd51005: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Feb 1, 2024

❌ Gradle check result for 7f82842: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Feb 1, 2024

❌ Gradle check result for 36925d2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          Address PR comments

886757e

Signed-off-by: mgodwan <mgodwan@amazon.com>

Contributor

github-actions bot commented Feb 1, 2024

❌ Gradle check result for 886757e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          Address PR comments

278dfea

Signed-off-by: mgodwan <mgodwan@amazon.com>

Contributor

github-actions bot commented Feb 1, 2024

❌ Gradle check result for 278dfea: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Feb 1, 2024

❌ Gradle check result for 0866ee4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          Fix tests after feature flag

b898785

Signed-off-by: mgodwan <mgodwan@amazon.com>

mgodwan force-pushed the fuzzy-filter branch from 0866ee4 to b898785 Compare

February 1, 2024 16:25

Contributor

github-actions bot commented Feb 1, 2024

❌ Gradle check result for b898785: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?


          Fix upgrade tests after feature flag

fed8863

Signed-off-by: mgodwan <mgodwan@amazon.com>

mgodwan force-pushed the fuzzy-filter branch from af65b96 to fed8863 Compare

February 1, 2024 17:55

Contributor

github-actions bot commented Feb 1, 2024

❌ Gradle check result for af65b96: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Contributor

github-actions bot commented Feb 1, 2024

✅ Gradle check result for fed8863: SUCCESS

mgodwan requested a review from shwetathareja

February 2, 2024 06:21

shwetathareja approved these changes

View reviewed changes

qa/rolling-upgrade/src/test/java/org/opensearch/upgrades/IndexingIT.java


          Merge branch 'main' into fuzzy-filter

b7fb54d

Signed-off-by: shwetathareja <shwetathareja@live.com>

shwetathareja approved these changes

View reviewed changes

Contributor

github-actions bot commented Feb 5, 2024

❕ Gradle check result for b7fb54d: UNSTABLE

TEST FAILURES:

      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

shwetathareja merged commit 0a88963 into opensearch-project:main

shwetathareja added the backport 2.x label

opensearch-trigger-bot bot pushed a commit that referenced this pull request


          Enable Fuzzy codec for doc id fields using a bloom filter (#11027)

09e2bdd

* Enable Fuzzy codec for doc id fields using a bloom filter

Signed-off-by: mgodwan <mgodwan@amazon.com>
(cherry picked from commit 0a88963)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

opensearch-trigger-bot bot mentioned this pull request

[Backport 2.x] Enable Fuzzy codec for doc id fields using a bloom filter #12171

Merged

shwetathareja pushed a commit that referenced this pull request


          Enable Fuzzy codec for doc id fields using a bloom filter (#11027) (#…

4bf568f

…12171)

* Enable Fuzzy codec for doc id fields using a bloom filter


(cherry picked from commit 0a88963)

Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

peteralfonsi pushed a commit to peteralfonsi/OpenSearch that referenced this pull request


          Enable Fuzzy codec for doc id fields using a bloom filter (opensearch…

e8a5a1c

…-project#11027)

* Enable Fuzzy codec for doc id fields using a bloom filter

Signed-off-by: mgodwan <mgodwan@amazon.com>

rayshrey pushed a commit to rayshrey/OpenSearch that referenced this pull request


          Enable Fuzzy codec for doc id fields using a bloom filter (opensearch…

ffbbbf1

…-project#11027)

* Enable Fuzzy codec for doc id fields using a bloom filter

Signed-off-by: mgodwan <mgodwan@amazon.com>

shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request


          Enable Fuzzy codec for doc id fields using a bloom filter (opensearch…

9247d99

…-project#11027)

* Enable Fuzzy codec for doc id fields using a bloom filter

Signed-off-by: mgodwan <mgodwan@amazon.com>
Signed-off-by: Shivansh Arora <hishiv@amazon.com>

kkewwei mentioned this pull request

Adding bloom_filter_enable field mapping parameter to allow field level bloom filter for Parquet #20404

Open

shwetathareja mentioned this pull request

Add @mgodwan as maintainer #20774

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

shwetathareja shwetathareja approved these changes

nknize Awaiting requested review from nknize

abbashus Awaiting requested review from abbashus

anasalkouz Awaiting requested review from anasalkouz

andrross Awaiting requested review from andrross andrross is a code owner

Bukhtawar Awaiting requested review from Bukhtawar Bukhtawar is a code owner

CEHENKLE Awaiting requested review from CEHENKLE

dblock Awaiting requested review from dblock

dbwiddis Awaiting requested review from dbwiddis dbwiddis is a code owner

dreamer-89 Awaiting requested review from dreamer-89

gbbafna Awaiting requested review from gbbafna gbbafna is a code owner

kartg Awaiting requested review from kartg

kotwanikunal Awaiting requested review from kotwanikunal kotwanikunal is a code owner

mch2 Awaiting requested review from mch2 mch2 is a code owner

msfroh Awaiting requested review from msfroh msfroh is a code owner

owaiskazi19 Awaiting requested review from owaiskazi19 owaiskazi19 is a code owner

peternied Awaiting requested review from peternied

reta Awaiting requested review from reta reta is a code owner

Rishikesh1159 Awaiting requested review from Rishikesh1159

ryanbogan Awaiting requested review from ryanbogan

sachinpkale Awaiting requested review from sachinpkale sachinpkale is a code owner

saratvemulapalli Awaiting requested review from saratvemulapalli saratvemulapalli is a code owner

setiah Awaiting requested review from setiah

sohami Awaiting requested review from sohami sohami is a code owner

VachaShah Awaiting requested review from VachaShah

adnapibar Awaiting requested review from adnapibar

tlfeng Awaiting requested review from tlfeng

+3 more reviewers

ketanv3 ketanv3 left review comments

vikasvb90 vikasvb90 left review comments

abiesps abiesps left review comments

Labels

backport 2.x discuss enhancement Indexing & Search v2.12.0