Allow searching 1K+ IP/masks in DV-only fields via terms_query

[mkhludnev]

Description

Avoid clauses limit what searching terms_query for IP/masks over docValues fields.

Related Issues

Resolves #17394

Check List

• Functionality includes testing.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for bf21595: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 9493dd7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for d88fea8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 4ad17d8: SUCCESS

[codecov]

Codecov Report

Attention: Patch coverage is 81.03448% with 11 lines in your changes missing coverage. Please review.

Project coverage is 72.66%. Comparing base (d52cefa) to head (6fb5c76).
Report is 33 commits behind head on main.

Files with missing lines	Patch %	Lines
...ava/org/opensearch/index/mapper/IpFieldMapper.java	81.03%	5 Missing and 6 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #18357      +/-   ##
============================================
- Coverage     72.74%   72.66%   -0.09%     
+ Complexity    67767    67726      -41     
============================================
  Files          5497     5504       +7     
  Lines        311815   311928     +113     
  Branches      45261    45284      +23     
============================================
- Hits         226822   226648     -174     
- Misses        66504    66781     +277     
- Partials      18489    18499      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

❌ Gradle check result for 6fb5c76: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[mkhludnev]

org.opensearch.upgrades.FullClusterRestartIT.testRecovery
Seems like an irrelevant flaky test failure

[mkhludnev]

Hello,
Kindly asking for considering it?

[rishabhmaurya]

@mkhludnev thanks for the lucene contribution and using it here. I wondering what other places we can exploit the use of DocValuesMultiRangeQuery?

[mkhludnev]

Thanks @rishabhmaurya I also though about it, at the first glimpse there is no many field types letting to pass a range into terms query. Let me double check ranges and especially IP range.

[github-actions]

✅ Gradle check result for 6fb5c76: SUCCESS

[mkhludnev]

Thanks for merging this. Aren't there a backport duty now?

[rishabhmaurya]

looks like 2.x still pointing to old lucene version -

OpenSearch/gradle/libs.versions.toml

Line 3 in 0aea92b

lucene = "9.12.1"

[mkhludnev]

Right. Got it now. Thanks

[mkhludnev]

I wondering what other places we can exploit the use of DocValuesMultiRangeQuery

Cant' find a low hanging fruit here. Potentially it could be:

• rewriting bool {should [{range:{}}, ...{range:{}}] } into DV multirange. Not sure if users need it.
• it could be possible if we introduce range mini-syntax for terms:{"fld":["1 TO 2", "2 TO 3"]}, nothing like this even for range fields exists now.
• even for IP range fields there's no CIDR syntax support for terms:{"ip_range_fld":["1.2.3.4/24", "1.2.3.5/22"]}

So, looks like we can put it on-hold until we get a request from users.

[rishabhmaurya]

@mkhludnev I like these ideas especially rewriting the boolean queries. Do you think this approach can result in better performance for some of these boolean queries?

[mkhludnev]

I'm afraid it's a quite rare case. Let's wait for a particular user.