Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 44 additions & 42 deletions release-notes/opensearch.release-notes-3.6.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,78 +7,80 @@ Compatible with OpenSearch and OpenSearch Dashboards version 3.6.0
* Add bitmap64 query support ([#20606](https://github.com/opensearch-project/OpenSearch/pull/20606))
* Add warmup phase for pull-based ingestion to prevent serving stale data before catching up with the streaming source ([#20526](https://github.com/opensearch-project/OpenSearch/pull/20526))
* Implement field_mapping ingestion message mapper for pull-based ingestion ([#20729](https://github.com/opensearch-project/OpenSearch/pull/20729))
* Add mapper_settings support and field_mapping mapper type for pull-based ingestion ([#20722](https://github.com/opensearch-project/OpenSearch/pull/20722))
* Remove experimental tag for pull-based ingestion, marking it as public API ([#20704](https://github.com/opensearch-project/OpenSearch/pull/20704))

### Enhancements

* Add index warmer support for replica shards using segment replication ([#20650](https://github.com/opensearch-project/OpenSearch/pull/20650))
* Add indices to search request slow log for easier request identification ([#20588](https://github.com/opensearch-project/OpenSearch/pull/20588))
* Add mapper_settings support and field_mapping mapper type for pull-based ingestion ([#20722](https://github.com/opensearch-project/OpenSearch/pull/20722))
* Add new `Sensitive` setting property for tiering dynamic settings authorization ([#20901](https://github.com/opensearch-project/OpenSearch/pull/20901))
* Add node-level JVM and CPU runtime metrics following OpenTelemetry semantic conventions ([#20844](https://github.com/opensearch-project/OpenSearch/pull/20844))
* Add new `Sensitive` setting property for tiering dynamic settings authorization ([#20901](https://github.com/opensearch-project/OpenSearch/pull/20901))
* Add adaptive shard selection for bulk writes on append-only indices ([#20065](https://github.com/opensearch-project/OpenSearch/pull/20065))
* Add support for expected remote cluster name validation in CCS sniff mode ([#20532](https://github.com/opensearch-project/OpenSearch/pull/20532))
* Add Scroll API support for workload management rule-based autotagging ([#20151](https://github.com/opensearch-project/OpenSearch/pull/20151))
* Add custom search settings support for workload management groups, starting with timeout ([#20536](https://github.com/opensearch-project/OpenSearch/pull/20536))
* Expose wrapped scorer in `ProfileScorer` via `getWrappedScorer` method for plugin access ([#20549](https://github.com/opensearch-project/OpenSearch/pull/20549))
* Fix `ProfileScorer.getChildren()` to expose wrapped scorer in the scorer tree hierarchy ([#20607](https://github.com/opensearch-project/OpenSearch/pull/20607))
* Fallback to Netty HTTP client when AWS CRT client is unavailable on the target platform ([#20698](https://github.com/opensearch-project/OpenSearch/pull/20698))
* Add `search_settings` support to WLM workload groups with initial `timeout` setting ([#20536](https://github.com/opensearch-project/OpenSearch/pull/20536))
* Add scroll API support for workload management rule-based autotagging ([#20151](https://github.com/opensearch-project/OpenSearch/pull/20151))
* Add stream request flag to SearchRequestContext for plugin consumption ([#20530](https://github.com/opensearch-project/OpenSearch/pull/20530))
* Support Docker distribution builds for ppc64le, arm64, and s390x architectures ([#20678](https://github.com/opensearch-project/OpenSearch/pull/20678))
* Add intra-segment concurrency support for single-value metric aggregations ([#20503](https://github.com/opensearch-project/OpenSearch/pull/20503))
* Add `ref_path` support for package-based Hunspell dictionary loading ([#20840](https://github.com/opensearch-project/OpenSearch/pull/20840))
* Add streaming request flag to `SearchRequestContext` for plugin consumption ([#20530](https://github.com/opensearch-project/OpenSearch/pull/20530))
* Make telemetry `Tags` immutable with sorted parallel arrays and precomputed hash for improved performance ([#20788](https://github.com/opensearch-project/OpenSearch/pull/20788))
* Add indexer interface to decouple `IndexShard` from underlying engine implementations ([#20675](https://github.com/opensearch-project/OpenSearch/pull/20675))
* Fallback to Netty HTTP client when AWS CRT client is unavailable on the target platform ([#20698](https://github.com/opensearch-project/OpenSearch/pull/20698))
* Add intra-segment support for single-value metric aggregations (sum, min, max, avg, stats, cardinality, value_count) ([#20503](https://github.com/opensearch-project/OpenSearch/pull/20503))
* Expose wrapped scorer in ProfileScorer for plugin access to custom scorer methods ([#20549](https://github.com/opensearch-project/OpenSearch/pull/20549))
* Fix ProfileScorer.getChildren() to expose wrapped scorer in the scorer tree hierarchy ([#20607](https://github.com/opensearch-project/OpenSearch/pull/20607))
* Remove X-Request-Id format restrictions and make maximum length configurable ([#21048](https://github.com/opensearch-project/OpenSearch/pull/21048))
* Make telemetry Tags immutable with allocation-efficient factories and content-based equality ([#20788](https://github.com/opensearch-project/OpenSearch/pull/20788))
* Add `ref_path` support for package-based Hunspell dictionary loading with multi-tenant isolation ([#20840](https://github.com/opensearch-project/OpenSearch/pull/20840))
* Prevent criteria field updates for context-aware indices to simplify version management ([#20250](https://github.com/opensearch-project/OpenSearch/pull/20250))
* Add indexer interface to decouple IndexShard from Engine for pluggable engine architectures ([#20675](https://github.com/opensearch-project/OpenSearch/pull/20675))
* Use `ReadAdviseByContext` for MMapDirectory instead of Lucene default read advise ([#21062](https://github.com/opensearch-project/OpenSearch/pull/21062))

### Bug Fixes

* Add range validations in query builder and field mapper ([#20518](https://github.com/opensearch-project/OpenSearch/pull/20518))
* Expose `STRICT_EXPAND_OPEN_HIDDEN_FORBID_CLOSED` index option via static method ([#20980](https://github.com/opensearch-project/OpenSearch/pull/20980))
* Fix `opensearch.cgroups.hierarchy.override` causing `SecurityException` due to missing file permissions ([#20565](https://github.com/opensearch-project/OpenSearch/pull/20565))
* Relax `updatedAt` validation for workload group creation to handle minor clock skew across nodes ([#20486](https://github.com/opensearch-project/OpenSearch/pull/20486))
* Fix `copy_to` functionality for `geo_point` fields with object or array values ([#20542](https://github.com/opensearch-project/OpenSearch/pull/20542))
* Fix `copy_to` functionality for geo_point fields with object and array values ([#20542](https://github.com/opensearch-project/OpenSearch/pull/20542))
* Fix `field_caps` returning empty results for `disable_objects` mappings and field name corruption ([#20814](https://github.com/opensearch-project/OpenSearch/pull/20814))
* Fix `listBlobsByPrefixInSortedOrder` in `EncryptedBlobContainer` to respect limit and avoid loading all files into memory ([#20514](https://github.com/opensearch-project/OpenSearch/pull/20514))
* Fix terms lookup subquery to use cluster `max_clause_count` setting instead of hardcoded fallback ([#20823](https://github.com/opensearch-project/OpenSearch/pull/20823))
* Fix terms aggregation performance regression on high-cardinality fields by introducing a max cardinality setting ([#20623](https://github.com/opensearch-project/OpenSearch/pull/20623))
* Fix index template pattern collision detection producing false positives for multi-wildcard patterns ([#20702](https://github.com/opensearch-project/OpenSearch/pull/20702))
* Fix terms aggregation performance regression on high-cardinality fields by adding a max cardinality setting ([#20623](https://github.com/opensearch-project/OpenSearch/pull/20623))
* Fix terms aggregation performance regression using segment-to-global ordinals mapping ([#20683](https://github.com/opensearch-project/OpenSearch/pull/20683))
* Fix `array_index_out_of_bounds_exception` with wildcard and aggregations under concurrent access ([#20842](https://github.com/opensearch-project/OpenSearch/pull/20842))
* Add range validations in query builder and field mapper ([#20518](https://github.com/opensearch-project/OpenSearch/pull/20518))
* Fix synonym_graph filter failure with word_delimiter_graph by handling analyzer dependency ordering ([#19248](https://github.com/opensearch-project/OpenSearch/pull/19248))
* Fix index template pattern collision false positives for multi-wildcard patterns ([#20702](https://github.com/opensearch-project/OpenSearch/pull/20702))
* Fix `SecurityException` when using `opensearch.cgroups.hierarchy.override` setting ([#20565](https://github.com/opensearch-project/OpenSearch/pull/20565))
* Fix SLF4J component error caused by slf4j-api version mismatch with log4j binding ([#20587](https://github.com/opensearch-project/OpenSearch/pull/20587))
* Fix JSON escaping in task details log metadata ([#20802](https://github.com/opensearch-project/OpenSearch/pull/20802))
* Fix SLF4J component error by updating to `log4j-slf4j2-impl` for SLF4J 2.x compatibility ([#20587](https://github.com/opensearch-project/OpenSearch/pull/20587))
* Fix `array_index_out_of_bounds_exception` with wildcard fields and aggregations under concurrent access ([#20842](https://github.com/opensearch-project/OpenSearch/pull/20842))
* Delegate `getMin`/`getMax` methods for `ExitableTerms` to avoid slow path in field sort ([#20775](https://github.com/opensearch-project/OpenSearch/pull/20775))
* Fix `CriteriaBasedCodec` to work with delegate codec ([#20442](https://github.com/opensearch-project/OpenSearch/pull/20442))
* Fix synonym_graph filter failure with word_delimiter_graph when using custom synonym analyzers ([#19248](https://github.com/opensearch-project/OpenSearch/pull/19248))
* Improve exception messaging when encountering legacy Elasticsearch version IDs during upgrade ([#20512](https://github.com/opensearch-project/OpenSearch/pull/20512))
* Fix `listBlobsByPrefixInSortedOrder` in EncryptedBlobContainer to respect limit and prevent JVM exhaustion ([#20514](https://github.com/opensearch-project/OpenSearch/pull/20514))
* Fix batched deletion of stale cluster metadata manifests to prevent remote storage pile-up ([#20566](https://github.com/opensearch-project/OpenSearch/pull/20566))
* Fix segment replication infinite retry caused by stale metadata checkpoint ([#20551](https://github.com/opensearch-project/OpenSearch/pull/20551))
* Show heap percent threshold instead of moving-average value in search cancellation messages ([#20779](https://github.com/opensearch-project/OpenSearch/pull/20779))
* Defer stored field reader initialization in `SourceLookup` to avoid unnecessary `madvise` calls ([#20827](https://github.com/opensearch-project/OpenSearch/pull/20827))
* Fix terms aggregation performance regression by leveraging segment-to-global ordinal mapping ([#20683](https://github.com/opensearch-project/OpenSearch/pull/20683))
* Fix batched deletion of stale cluster metadata manifests in remote cluster state cleanup ([#20566](https://github.com/opensearch-project/OpenSearch/pull/20566))
* Fix stale segments cleanup to also trigger based on map size threshold ([#20976](https://github.com/opensearch-project/OpenSearch/pull/20976))
* Fix `ExitableTerms` to delegate `getMin`/`getMax` methods avoiding slow path in field sort ([#20775](https://github.com/opensearch-project/OpenSearch/pull/20775))
* Lazily initialize stored field reader in SourceLookup to avoid unnecessary `madvise` calls ([#20827](https://github.com/opensearch-project/OpenSearch/pull/20827))
* Fix CriteriaBasedCodec to work with delegate codec ([#20442](https://github.com/opensearch-project/OpenSearch/pull/20442))
* Show heap percent threshold instead of moving average in search cancellation messages ([#20779](https://github.com/opensearch-project/OpenSearch/pull/20779))
* Relax `updatedAt` validation for WLM workload group creation to handle minor clock skew ([#20486](https://github.com/opensearch-project/OpenSearch/pull/20486))
* Improve exception messaging when encountering legacy Elasticsearch version IDs during upgrade ([#20512](https://github.com/opensearch-project/OpenSearch/pull/20512))
* Fix Windows service startup with OpenJDK by updating procrun to 1.5.1 ([#20615](https://github.com/opensearch-project/OpenSearch/pull/20615))
* Move `Randomness` from server to `lib/common` to reduce FIPS demo installer size ([#20570](https://github.com/opensearch-project/OpenSearch/pull/20570))
* Expose `STRICT_EXPAND_OPEN_HIDDEN_FORBID_CLOSED` index option via static method ([#20980](https://github.com/opensearch-project/OpenSearch/pull/20980))
* Fix stale segments cleanup to also trigger based on segments map size threshold ([#20976](https://github.com/opensearch-project/OpenSearch/pull/20976))
* Move `Randomness` from server to lib/common to reduce FIPS demo installer size ([#20570](https://github.com/opensearch-project/OpenSearch/pull/20570))
* Add `accessUnixDomainSocket` permission for transport-grpc on Windows ([#20649](https://github.com/opensearch-project/OpenSearch/pull/20649))
* Harden HTTP/3 support detection by verifying Quic native library availability for the target platform ([#20680](https://github.com/opensearch-project/OpenSearch/pull/20680))
* Harden HTTP/3 support detection by checking Quic native library availability for the target platform ([#20680](https://github.com/opensearch-project/OpenSearch/pull/20680))
* Use `OPENSEARCH_FIPS_MODE` environment variable to enable FIPS enforced mode instead of checking for bcFIPS jars ([#20625](https://github.com/opensearch-project/OpenSearch/pull/20625))
* Update MMapDirectory to use `ReadAdviseByContext` rather than default read advise ([#21062](https://github.com/opensearch-project/OpenSearch/pull/21062))
* Ensure transient `ThreadContext` headers with propagators survive context restore ([#20854](https://github.com/opensearch-project/OpenSearch/pull/20854))
* Fix Arrow Flight transport TLS certificate hot-reload ([#20700](https://github.com/opensearch-project/OpenSearch/pull/20700))
* Fix stream transport TLS certificate hot-reload by using live SSLContext from SecureTransportSettingsProvider ([#20734](https://github.com/opensearch-project/OpenSearch/pull/20734))
* Ensure transient ThreadContext headers with propagators survive context restore ([#20854](https://github.com/opensearch-project/OpenSearch/pull/20854))

### Maintenance

* Bump Apache Lucene from 10.3.2 to 10.4.0 ([#20735](https://github.com/opensearch-project/OpenSearch/pull/20735))
* Bump OpenTelemetry to 1.60.1 and OpenTelemetry Semconv to 1.40.0 ([#20737](https://github.com/opensearch-project/OpenSearch/pull/20737), [#20797](https://github.com/opensearch-project/OpenSearch/pull/20797))
* Bump Project Reactor to 3.8.4 and Reactor Netty to 1.3.4 ([#20589](https://github.com/opensearch-project/OpenSearch/pull/20589), [#20834](https://github.com/opensearch-project/OpenSearch/pull/20834))
* Bump Netty to 4.2.12.Final ([#20586](https://github.com/opensearch-project/OpenSearch/pull/20586), [#20997](https://github.com/opensearch-project/OpenSearch/pull/20997), [#20998](https://github.com/opensearch-project/OpenSearch/pull/20998))
* Update Netty to 4.2.12.Final ([#20586](https://github.com/opensearch-project/OpenSearch/pull/20586)), ([#20997](https://github.com/opensearch-project/OpenSearch/pull/20997)), ([#20998](https://github.com/opensearch-project/OpenSearch/pull/20998))
* Update Jackson to 2.21.2 ([#20989](https://github.com/opensearch-project/OpenSearch/pull/20989))
* Bump `ch.qos.logback:logback-classic` to 1.5.32 and `logback-core` to 1.5.27 ([#20525](https://github.com/opensearch-project/OpenSearch/pull/20525), [#20761](https://github.com/opensearch-project/OpenSearch/pull/20761))
* Bump OpenTelemetry to 1.60.1 and OpenTelemetry Semconv to 1.40.0 ([#20737](https://github.com/opensearch-project/OpenSearch/pull/20737)), ([#20797](https://github.com/opensearch-project/OpenSearch/pull/20797))
* Bump Project Reactor to 3.8.4 and Reactor Netty to 1.3.4 ([#20589](https://github.com/opensearch-project/OpenSearch/pull/20589)), ([#20834](https://github.com/opensearch-project/OpenSearch/pull/20834))
* Bump `ch.qos.logback:logback-classic` to 1.5.32 and `logback-core` to 1.5.27 ([#20525](https://github.com/opensearch-project/OpenSearch/pull/20525)), ([#20761](https://github.com/opensearch-project/OpenSearch/pull/20761))
* Bump `com.nimbusds:nimbus-jose-jwt` from 10.7 to 10.8 ([#20715](https://github.com/opensearch-project/OpenSearch/pull/20715))
* Bump `com.sun.xml.bind:jaxb-impl` from 4.0.6 to 4.0.7 ([#20886](https://github.com/opensearch-project/OpenSearch/pull/20886))
* Bump `com.netflix.nebula.ospackage-base` from 12.2.0 to 12.3.0 ([#20799](https://github.com/opensearch-project/OpenSearch/pull/20799))
* Bump `org.apache.commons:commons-text` from 1.14.0 to 1.15.0 ([#20576](https://github.com/opensearch-project/OpenSearch/pull/20576))
* Bump `org.jline:jline` from 3.30.6 to 4.0.0 ([#20800](https://github.com/opensearch-project/OpenSearch/pull/20800))
* Bump `org.jruby.jcodings:jcodings` from 1.0.63 to 1.0.64 ([#20713](https://github.com/opensearch-project/OpenSearch/pull/20713))
* Bump `org.jruby.joni:joni` from 2.2.3 to 2.2.7 ([#20714](https://github.com/opensearch-project/OpenSearch/pull/20714), [#20759](https://github.com/opensearch-project/OpenSearch/pull/20759))
* Bump `org.jruby.joni:joni` from 2.2.3 to 2.2.7 ([#20714](https://github.com/opensearch-project/OpenSearch/pull/20714)), ([#20759](https://github.com/opensearch-project/OpenSearch/pull/20759))
* Bump `org.tukaani:xz` from 1.11 to 1.12 ([#20760](https://github.com/opensearch-project/OpenSearch/pull/20760))
* Bump `shadow-gradle-plugin` from 8.3.9 to 9.3.1 ([#20569](https://github.com/opensearch-project/OpenSearch/pull/20569))
* Bump `com.netflix.nebula:gradle-info-plugin` to 16.2.1 ([#20825](https://github.com/opensearch-project/OpenSearch/pull/20825))
* Bump shadow-gradle-plugin from 8.3.9 to 9.3.1 ([#20569](https://github.com/opensearch-project/OpenSearch/pull/20569))
Loading