[AUTO] Add release notes for 3.7.0

[opensearch-ci-bot]

Add release notes for 3.7.0

Borderline Calls

• Add Data Format aware engine #21181 (DataFormatAwareEngine): Categorized as Feature since it introduces a net-new engine type. Many follow-up PRs (DataFormat-aware NRT replication engine and remote-store wiring #21311, Dynamic Mapping support for Pluggable Data Formats #21444, Add segments and segments stats support for DataFormatAwareEngine #21696, etc.) are categorized as Enhancements since they build on this foundation.

• Add failure handling for composite multi-format indexing engine #21417 (failure handling for composite engine): Included despite being reverted by Revert "Add per-doc failure handling for DataFormatAwareEngine (#21417)" #21842 — the revert landed in the same release window, but the feature PR description indicates it was re-merged. If the revert is the final state, both should be excluded.

• Use caching to avoid excessive creation of NonClosingReaderWrapper instances #20921 (NonClosingReaderWrapper cache): Excluded as it was reverted by Revert NonClosingReaderWrapper cache (#20921) #21830. The revert is included as a Bug Fix.

• BitsetFilterCache refactor: Change it to a node level cache with a size limit #21179 (BitsetFilterCache refactor): Categorized as Enhancement rather than Breaking Change — the cache behavior changes but the API surface is preserved.

• Added queryTimeout to the IndexSearcher to ensure that queries that uses this timeout during search exit when queries times out #21316 (queryTimeout on IndexSearcher): Categorized as Enhancement — it fixes a gap where KNN queries ignored timeouts, but the mechanism is an enhancement to existing timeout infrastructure.

• native memory based admission control #21191 (native memory admission control): Categorized as Feature — introduces a new admission controller type.

• Tighten memory guard to prevent OOM #21814 (tighten memory guard): Categorized as Enhancement — improves existing memory management rather than introducing new functionality.

• Speed up Script Engine initialization #21463 (Script Engine init speedup): Categorized as Enhancement — performance improvement to existing functionality.

• fixed accumaltes size when multi files share the same extension  #21000 (file sizes accumulation fix): Categorized as Enhancement — fixes incorrect behavior in stats reporting.

• Many analytics-engine PPL function PRs (Enable PPL JSON scalar functions on the analytics-engine route #21513, Adding sweep of MATH scalar functions to analytics-engine. #21520, Wire PPL string scalar functions with analytics-backend-datafusion #21543, Enable PPL datetime scalar functions on the analytics-engine route #21556, etc.) are grouped by function category rather than listed individually to keep the notes readable. Each is still linked.

• Remove Unsafe class injection from Java agent #21542 (Remove Unsafe from agent): Categorized as Maintenance — internal implementation change with no user-facing behavior difference.

• Replace mimalloc with jemalloc as global allocator for native sandbox plugins #21497 (mimalloc→jemalloc): Categorized as Maintenance — allocator swap is an internal implementation detail.

• Excluded PRs: Add back getSearchSettings to fix japicmp #21500 (japicmp fix), Add dedicated CI check for sandbox running on Github #21175/Include sandbox only when explicitly enabled via -Dsandbox.enabled=true #21187/Disable sandbox by default for publication, require passing -Dsandbox.enabled=true #21125 (CI/sandbox gating), Add plugin option for gradle run to AGENTS.md #21198 (AGENTS.md), Add dependency on native rust lib build for composite plugin test task #21238/Fix worker gone test failure in sandbox #21270/Muting broken test DatafusionSearchExecEngineTests #21196/Add fix for the tests #21657/fix sandbox compilation #21663/Fix flaky dynamic mapping IT due to merges #21691/mute flaky sandbox test #21713/fix sandbox failures #21761/Mute Broken Sandbox tests #21774/Mute sandbox SearchCancellationIT #21777 (test fixes/mutes), Wire Rust native library build into assemble lifecycle #21475/Allow external override of the sandbox native library path #21592/Bump maven snapshots publishing to jdk25 to unlock sandbox jar publishing #21568/Update maven snapshots publishing to include sandbox jars #21567/update sandbox check to publish latest UnifiedQueryAPI jars to maven local #21569/Onboarding calcite opensearch revision publishing setups #21549/Update calcite workflow name to reflect calcite #21551/Revert "[Sandbox SQL snapshot] Pin org.opensearch.query:* (unified-qu… #21580/[Sandbox SQL snapshot] Pin org.opensearch.query:* (unified-query-*) snapshots to mavenLocal #21578/[Sandbox] Make sqlUnifiedQueryVersion overridable; force mavenLocal precedence #21589/[Sandbox] Add calcite-patched module: Gradle-driven patched Calcite build #21501 (build/CI/publish machinery), Bump jenkins lib to 12.0.0 and update secrets ref #21651/Add maven cache before mavenCentral to reduce calls made to maven central from jenkins runners #21682 (Jenkins/Maven cache), Add issues write permission to untriaged label workflow #21762/Pin actions/github-script to exact commit SHA #21790/Remove scratch notes files accidentally committed #21784 (workflow/docs cleanup), Fix DiskUsageTests NodeStats constructor call #21770/Fix NodeResourceUsageStats constructor to include nativeMemoryUtilizationPercent param #21714 (compile fixes for other PRs), Fix breaking changes by delegating and deprecating #21194/Fix breaking changes check #21171 (breaking changes check fixes), Link to the security response process in SECURITY.md #21641 (SECURITY.md link), Document security setup for Arrow Flight RPC transport #21314 (Arrow Flight security docs), Remove invalid test with field in both _source includes and excludes #21231 (test removal), dissect: fix 'occured' -> 'occurred' typo in DissectMatch#getResults javadoc #21241 (typo fix), Extend OpenSearchTestCase in BlockIndexInputTests #21355/Fix BlockTransferManagerTests to extend OpenSearchTestCase instead of LuceneTestCase #21363/Split gradle check into parallel check and bwc jobs #21106 (test base class fixes), improve unit test make sure thread context propagate to executor thread #21200 (test improvement), Onboard issue dedupe workflow (OpenSearch) #21405 (workflow onboard), Fix Detect Breaking Changes version selection logic for previous released version #21529 (CI logic fix), Update Transport.assertDefaultThreadContext to allow all Task.REQUEST_HEADERS #21462 (transport assertion fix), Extract analytics-api module for JDK 21 downstream consumers #21574/Extract arrow-base plugin #21465 (internal module extraction), Skip root transfer for byte-serialized via handler marker #21454/Transfer Flight batch to a response-owned root before returning #21437 (internal Arrow transport plumbing — covered by Native Arrow path support in stream transport with zero-copy transfer #21253/Stream Arrow batches on the data-node fragment execution path #21418), numerous analytics-engine internal refactors and QA-only PRs.

• Relax field-level meta validation constraints #20578 (relax field meta): Labeled "enhancement" + "Indexing" — kept as Enhancement.

• feat(hunspell): Add reload_cached_resources in TokenFilterFactory to support dictionary hot-reload #21559 (hunspell hot-reload): Categorized as Feature — introduces new capability (hot-reload without restart).

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 8292941)

Here are some key observations to aid the review process:

🧪 No relevant tests

🔒 No security concerns identified

✅ No TODO sections

🔀 No multiple PR themes

⚡ No major issues detected

[github-actions]

Persistent review updated to latest commit 69f30ea

[github-actions]

Persistent review updated to latest commit 73c3afa

[andrross]

#21417 (failure handling for composite engine): Included despite being reverted by #21842 — the revert landed in the same release window, but the feature PR description indicates it was re-merged. If the revert is the final state, both should be excluded.

#20921 (NonClosingReaderWrapper cache): Excluded as it was reverted by #21830. The revert is included as a Bug Fix.

I don't know why the AI got confused here. These commit/revert pairs result in net no change, so nothing should be included in the release notes about them. I'll fix this.

[github-actions]

Persistent review updated to latest commit 8292941