Disable request cache for queries on fields with non-default use_similarity or split_queries_on_whitespace

[peteralfonsi]

Description

Some dynamically updateable parameters can change query results, for example "use_similarity" and "split_queries_on_whitespace" for keyword indices. This means that after changing them, users can get stale/incorrect values from the request cache. This PR prevents this by disabling caching queries which are on fields with non-default values for these parameters. This should be an acceptable solution as users opting into the non-default parameter should be fairly rare.

Related Issues

#19279

Check List

• Functionality includes testing.
• [N/A] API changes companion pull request created, if applicable.
• [N/A] Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 5c5f4d4: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 6157c8a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[sgup432]

@peteralfonsi Instead of trying to find ways to invalidate the cache by listening on mapper events, should we instead not cache the queries for such indices which uses use_similarity in its mapping? I think it will simplify it by not having to write the extra logic to handle specific scenarios. I don't think its worth caching for such indices.

In IndicesService.canCache, it does have access to indexShard/indexSettings, and I think we can use that to check whether index has this mapping and accordingly not cache for such indices.

[peteralfonsi]

@sgup432 I think it makes more sense to keep these queries cacheable for a couple reasons:

• There are at least 2 parameters where changing them affects query results, the other one is "split_queries_on_whitespace" on keyword queries, and there may be more (for example in field mappers from other plugins), so we don't want the solution to be specific to "use_similarity"
• For "use_similarity", every keyword field does technically have this defined, it's just the default is false. The problem is really in switching between the 2 states, not it being true or false. We can imagine some complex query where one component is on a field with "use_similarity", and it'd be unfortunate if the whole thing skipped the cache unnecessarily

[sgup432]

Okay, I was hoping that we avoiding such indices to be cached might be a more simpler solution. As I don't think we will gain a lot overall caching such queries.

But seems like even with my approach, we might have to tackle this case by case which I was trying to avoid in the first place.

[github-actions]

❌ Gradle check result for 624d2b8: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for dc2c277: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[peteralfonsi]

Flaky test (org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT.test {p0=search/310_match_bool_prefix/multi_match single field complete term} is mentioned specifically in the issue): #14294

[github-actions]

❌ Gradle check result for 27f5b42: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for 274c90f: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.00%. Comparing base (e0ee3b4) to head (274c90f).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #19385      +/-   ##
============================================
+ Coverage     72.81%   73.00%   +0.18%     
- Complexity    69854    69930      +76     
============================================
  Files          5676     5676              
  Lines        321048   321067      +19     
  Branches      46420    46421       +1     
============================================
+ Hits         233774   234383     +609     
+ Misses        68376    67677     -699     
- Partials      18898    19007     +109     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.