Add combined_fields query to utilize Lucene's CombinedField (BM25F Text)

[harrisonengel]

Description

When querying across multiple text fields, for example a title and a body, you can often achieve better relevance by treating them as a single 'combined field' and scoring them by BM25F. Lucene provides the CombinedFieldQuery for this purpose. It effectively treats all fields as combined into one for matching and ranking purposes, where each field can be weighted higher or lower.

Related Issues

Resolves #3996

Check List

• Functionality includes testing.
• API changes companion pull request created, if applicable.
• Public documentation issue/PR created, if applicable. [DOC] Document combined_fields query documentation-website#10209

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the
best of my knowledge, is covered under an appropriate open
source license and I have the right under that license to
submit that work with modifications, whether created in whole
or in part by me, under the same open source license (unless
I am permitted to submit under a different license), as
Indicated in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including
all personal information I submit with it, including my
sign-off) is maintained indefinitely and may be redistributed
consistent with this project or the open source license(s)
involved.

Signed-off-by: Harrison Engel <[email protected], [email protected]>

[github-actions]

❌ Gradle check result for 3ea57e2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 7bbc948: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for ea81f54: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for 74cd236: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❕ Gradle check result for 4189d76: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

[mch2]

Thanks @harrisonengel, This is looking good to me.
@msfroh @prudhvigodithi would you guys like to take a pass here as you had prior context on combined fields.

[github-actions]

❌ Gradle check result for b080ba3: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[github-actions]

❌ Gradle check result for b080ba3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

[prudhvigodithi]

LGTM, while reviewing this example, I was initially curious why CombinedFieldsQueryBuilder (part of this PR) uses a BooleanQuery internally, rather than adding multiple terms directly to a single CombinedFieldQuery.Builder using addTerm. Since Lucene's CombinedFieldQuery supports multiple terms. However, it now makes sense creating a separate CombinedFieldQuery for each term and wrapping them in a BooleanQuery as it provides explicit operator control making it easy to switch between AND and OR choice.

@harrisonengel also another reason to wrap in a BooleanQuery is to have also have per-term boosting using the boost field and also support of minimum_should_match?

[github-actions]

✅ Gradle check result for b080ba3: SUCCESS

[harrisonengel]

@harrisonengel also another reason to wrap in a BooleanQuery is to have also have per-term boosting using the boost field and also support of minimum_should_match?

Also my hands were kind of tied, Lucene stopped supporting multi-term CombinedFieldQuery.Builder recently-ish. Probably to promote this pattern since multi-term CombinedFieldsQuery is just a less flexible version of BooleanQuery with CombinedFieldQuery.