Introduced new setting search.query.max_query_string_length

[nibix]

Description

This introduces a new cluster wide setting search.query.max_query_string_length which can be used to limit the max length of simple, Lucene-style query strings. This provides a simple but robust way to control inputs from potentially untrusted sources.

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

❌ Gradle check result for 9a0481e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

✅ Gradle check result for 9d5cef2: SUCCESS

[codecov]

Codecov Report

❌ Patch coverage is 63.63636% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.06%. Comparing base (32b6873) to head (9d5cef2).
⚠️ Report is 114 commits behind head on main.

Files with missing lines	Patch %	Lines
...pensearch/index/search/QueryStringQueryParser.java	42.85%	3 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #19491      +/-   ##
============================================
+ Coverage     72.99%   73.06%   +0.06%     
- Complexity    70416    70434      +18     
============================================
  Files          5708     5708              
  Lines        322492   322502      +10     
  Branches      46704    46705       +1     
============================================
+ Hits         235412   235645     +233     
+ Misses        68161    67832     -329     
- Partials      18919    19025     +106     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[kkhatua]

@nibix ( cc: @cwperks , @jainankitk )

Just a thought... this can be a breaking change if users were to move to the latest version.

Can we add support where this check can run in a monitor_only mode and log the query (and at least the size) ? This way, an upgrade would not break anyone who is able to run queries above 32_000 bytes? That warning can then be acted upon and the sysAdmin can resize it to a more appropriate threshold.

[nibix]

@kkhatua

Can we add support where this check can run in a monitor_only mode and log the query (and at least the size) ? This way, an upgrade would not break anyone who is able to run queries above 32_000 bytes? That warning can then be acted upon and the sysAdmin can resize it to a more appropriate threshold.

Do you mean, monitor_only shall be the default for the beginning? At what point in time would we then globally switch from that default to an "error" default? If we follow clean deprecation rules, that would be only possible in OpenSearch 4, right?

Still, as the new setting can be changed using the cluster settings REST API, cluster admins have actually a quite good control over the limit. If they encounter the error message, this actually even mentions the name of the setting. So, even if someone hits the limit, they can very quickly react and change it.

[kkhatua]

We go either either options.... though I am in favor of having this default enforced since this is a resiliency feature. Having queries fails is preferred over risking a cluster to potential OOME-triggered node drops.

Interestingly, a similar resiliency feature, Search backpressure, has search_backpressure.mode was with monitor_only by default, which makes sense.
https://docs.opensearch.org/latest/tuning-your-cluster/availability-and-recovery/search-backpressure/#search-backpressure-settings:~:text=Description-,search_backpressure.mode,-monitor_only

In production, the monitor_only mode serves as an important tool for settings discovery. If the setting is arbitrarily low for the cluster's workload, the setting can be discovered via the logs and raised to a more appropriate level without breaking the experience for good actors in the cluster.

[opensearch-trigger-bot]

The backport to 2.19 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/OpenSearch/backport-2.19 2.19
# Navigate to the new working tree
pushd ../.worktrees/OpenSearch/backport-2.19
# Create a new branch
git switch --create backport/backport-19491-to-2.19
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 3677d8f9314bd17b814c29ff539c737e051e4e46
# Push it to GitHub
git push --set-upstream origin backport/backport-19491-to-2.19
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/OpenSearch/backport-2.19

Then, create a pull request where the base branch is 2.19 and the compare/head branch is backport/backport-19491-to-2.19.

[cwperks]

Creating a manual backport