Merge branch 'main' into indices-admin-client

Signed-off-by: Craig Perkins <[email protected]>

Author: cwperks

.ci/bwcVersions

@@ -46,6 +46,7 @@ BWC_VERSION:
   - "2.19.1"
   - "2.19.2"
   - "2.19.3"
+  - "2.19.4"
   - "3.0.0"
   - "3.1.0"
   - "3.2.0"

.gitattributes

@@ -10,5 +10,6 @@
 *.bcfks binary
 *.crt binary
 *.p12 binary
+*.ttf binary
 *.txt text=auto
 CHANGELOG.md merge=union

CHANGELOG.md

@@ -7,16 +7,19 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 ### Added
 - Expand fetch phase profiling to support inner hits and top hits aggregation phases  ([##18936](https://github.com/opensearch-project/OpenSearch/pull/18936))
 - Add temporal routing processors for time-based document routing ([#18920](https://github.com/opensearch-project/OpenSearch/issues/18920))
+- Implement Query Rewriting Infrastructure ([#19060](https://github.com/opensearch-project/OpenSearch/pull/19060))
 - The dynamic mapping parameter supports false_allow_templates ([#19065](https://github.com/opensearch-project/OpenSearch/pull/19065))
 - Add a toBuilder method in EngineConfig to support easy modification of configs([#19054](https://github.com/opensearch-project/OpenSearch/pull/19054))
 - Add StoreFactory plugin interface for custom Store implementations([#19091](https://github.com/opensearch-project/OpenSearch/pull/19091))
 - Use S3CrtClient for higher throughput while uploading files to S3 ([#18800](https://github.com/opensearch-project/OpenSearch/pull/18800))
 - Add a dynamic setting to change skip_cache_factor and min_frequency for querycache ([#18351](https://github.com/opensearch-project/OpenSearch/issues/18351))
+- Add overload constructor for Translog to accept Channel Factory as a parameter ([#18918](https://github.com/opensearch-project/OpenSearch/pull/18918))
 
 ### Changed
 - Add CompletionStage variants to methods in the Client Interface and default to ActionListener impl ([#18998](https://github.com/opensearch-project/OpenSearch/pull/18998))
 - IllegalArgumentException when scroll ID references a node not found in Cluster ([#19031](https://github.com/opensearch-project/OpenSearch/pull/19031))
 - Adding ScriptedAvg class to painless spi to allowlist usage from plugins ([#19006](https://github.com/opensearch-project/OpenSearch/pull/19006))
+- Replace centos:8 with almalinux:8 since centos docker images are deprecated ([#19154](https://github.com/opensearch-project/OpenSearch/pull/19154))
 - Add CompletionStage variants to IndicesAdminClient as an alternative to ActionListener ([#19161](https://github.com/opensearch-project/OpenSearch/pull/19161))
 
 ### Fixed
@@ -35,12 +38,17 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 - Bump `commons-cli:commons-cli` from 1.9.0 to 1.10.0 ([#19021](https://github.com/opensearch-project/OpenSearch/pull/19021))
 - Bump `org.jline:jline` from 3.30.4 to 3.30.5 ([#19013](https://github.com/opensearch-project/OpenSearch/pull/19013))
 - Bump `com.github.spotbugs:spotbugs-annotations` from 4.9.3 to 4.9.4 ([#19015](https://github.com/opensearch-project/OpenSearch/pull/19015))
-- Bump `com.azure:azure-storage-common` from 12.29.1 to 12.30.1 ([#19016](https://github.com/opensearch-project/OpenSearch/pull/19016))
+- Bump `com.azure:azure-storage-common` from 12.29.1 to 12.30.2 ([#19016](https://github.com/opensearch-project/OpenSearch/pull/19016), [#19145](https://github.com/opensearch-project/OpenSearch/pull/19145))
 - Update OpenTelemetry to 1.53.0 and OpenTelemetry SemConv to 1.34.0 ([#19068](https://github.com/opensearch-project/OpenSearch/pull/19068))
 - Bump `1password/load-secrets-action` from 2 to 3 ([#19100](https://github.com/opensearch-project/OpenSearch/pull/19100))
 - Bump `com.nimbusds:nimbus-jose-jwt` from 10.3 to 10.4.2 ([#19099](https://github.com/opensearch-project/OpenSearch/pull/19099), [#19101](https://github.com/opensearch-project/OpenSearch/pull/19101))
 - Bump netty from 4.1.121.Final to 4.1.124.Final ([#19103](https://github.com/opensearch-project/OpenSearch/pull/19103))
-- Bump google cloud storage from 1.113.1 to 2.55.0 ([#4547](https://github.com/opensearch-project/OpenSearch/pull/4547))
+- Bump Google Cloud Storage SDK from 1.113.1 to 2.55.0 ([#18922](https://github.com/opensearch-project/OpenSearch/pull/18922))
+- Bump `com.google.auth:google-auth-library-oauth2-http` from 1.37.1 to 1.38.0 ([#19144](https://github.com/opensearch-project/OpenSearch/pull/19144))
+- Bump `com.squareup.okio:okio` from 3.15.0 to 3.16.0 ([#19146](https://github.com/opensearch-project/OpenSearch/pull/19146))
+- Bump Slf4j from 1.7.36 to 2.0.17 ([#19136](https://github.com/opensearch-project/OpenSearch/pull/19136))
+- Bump `org.apache.tika` from 2.9.2 to 3.2.2 ([#19125](https://github.com/opensearch-project/OpenSearch/pull/19125))
+- Bump `org.apache.commons:commons-compress` from 1.26.1 to 1.28.0 ([#19125](https://github.com/opensearch-project/OpenSearch/pull/19125))
 
 ### Deprecated
 
@@ -49,4 +57,4 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 
 ### Security
 
-[Unreleased 3.x]: https://github.com/opensearch-project/OpenSearch/compare/3.1...main
+[Unreleased 3.x]: https://github.com/opensearch-project/OpenSearch/compare/3.2...main

buildSrc/src/main/java/org/opensearch/gradle/DockerBase.java

@@ -36,7 +36,7 @@
  * This class models the different Docker base images that are used to build Docker distributions of OpenSearch.
  */
 public enum DockerBase {
-    CENTOS("centos:8");
+    ALMALINUX("almalinux:8");
 
     private final String image;
 

client/rest/build.gradle

@@ -105,9 +105,6 @@ testingConventions {
 thirdPartyAudit {
   ignoreMissingClasses(
       'org.conscrypt.Conscrypt',
-      'org.slf4j.impl.StaticLoggerBinder',
-      'org.slf4j.impl.StaticMDCBinder',
-      'org.slf4j.impl.StaticMarkerBinder',
       //commons-logging optional dependencies
       'org.apache.avalon.framework.logger.Logger',
       'org.apache.log.Hierarchy',

client/rest/licenses/slf4j-api-1.7.36.jar.sha1

@@ -0,0 +1 @@
+d9e58ac9c7779ba3bf8142aff6c830617a7fe60f

client/rest/licenses/slf4j-api-2.0.17.jar.sha1

@@ -132,8 +132,8 @@ project.ext {
 }
 
 void addCopyDockerContextTask(Architecture architecture, DockerBase base) {
-  if (base != DockerBase.CENTOS) {
-    throw new GradleException("The only allowed docker base image for builds is CENTOS")
+  if (base != DockerBase.ALMALINUX) {
+    throw new GradleException("The only allowed docker base image for builds is ALMALINUX")
   }
 
   tasks.register(taskName("copy", architecture, base, "DockerContext"), Sync) {
@@ -181,8 +181,8 @@ opensearch_distributions {
 tasks.named("preProcessFixture").configure {
   dependsOn opensearch_distributions.docker
   // always run the task, otherwise the folders won't be created
-  outputs.upToDateWhen { 
-    false 
+  outputs.upToDateWhen {
+    false
   }
   doLast {
     // tests expect to have an empty repo
@@ -208,8 +208,8 @@ tasks.named("check").configure {
 }
 
 void addBuildDockerImage(Architecture architecture, DockerBase base) {
-  if (base != DockerBase.CENTOS) {
-    throw new GradleException("The only allowed docker base image for builds is CENTOS")
+  if (base != DockerBase.ALMALINUX) {
+    throw new GradleException("The only allowed docker base image for builds is ALMALINUX")
   }
 
   final TaskProvider<DockerBuildTask> buildDockerImageTask =
@@ -232,9 +232,9 @@ void addBuildDockerImage(Architecture architecture, DockerBase base) {
 }
 
 for (final Architecture architecture : Architecture.values()) {
-  // We only create Docker images for the distribution on CentOS.
+  // We only create Docker images for the distribution on AlmaLinux.
   for (final DockerBase base : DockerBase.values()) {
-    if (base == DockerBase.CENTOS) {
+    if (base == DockerBase.ALMALINUX) {
       addCopyDockerContextTask(architecture, base)
       addBuildDockerImage(architecture, base)
     }
@@ -257,7 +257,7 @@ subprojects { Project subProject ->
     apply plugin: 'distribution'
 
     final Architecture architecture = subProject.name.contains('arm64-') ? Architecture.ARM64 : Architecture.X64
-    final DockerBase base = DockerBase.CENTOS
+    final DockerBase base = DockerBase.ALMALINUX
 
     final String arch = architecture == Architecture.ARM64 ? '-arm64' : ''
     final String extension = 'docker.tar'

distribution/docker/build.gradle

@@ -19,7 +19,7 @@ tasks.register("buildDockerBuildContext", Tar) {
   archiveClassifier = "docker-build-context"
   archiveBaseName = "opensearch"
   // Non-local builds don't need to specify an architecture.
-  with dockerBuildContext(null, DockerBase.CENTOS, false)
+  with dockerBuildContext(null, DockerBase.ALMALINUX, false)
 }
 
 tasks.named("assemble").configure { dependsOn "buildDockerBuildContext" }

distribution/docker/docker-build-context/build.gradle

@@ -63,16 +63,12 @@ FROM ${base_image}
 
 ENV OPENSEARCH_CONTAINER true
 
-RUN  sed -i 's/mirrorlist/#mirrorlist/g' /etc/yum.repos.d/CentOS-Linux-* && \\
-     sed -i 's|#baseurl=http://mirror.centos.org|baseurl=http://vault.epel.cloud|g' /etc/yum.repos.d/CentOS-Linux-* && \\
-     for iter in {1..10}; do \\
-      ${package_manager} update --setopt=tsflags=nodocs -y && \\
-      ${package_manager} install --setopt=tsflags=nodocs -y \\
-        nc shadow-utils zip unzip && \\
-      ${package_manager} clean all && exit_code=0 && break || exit_code=\$? && echo "${package_manager} error: retry \$iter in 10s" && \\
-      sleep 10; \\
-    done; \\
-    (exit \$exit_code)
+RUN set -e \\
+     && dnf -y update \\
+     && dnf -y install --setopt=tsflags=nodocs \\
+          nmap-ncat shadow-utils zip unzip \\
+     && dnf clean all \\
+     && rm -rf /var/cache/dnf
 
 RUN groupadd -g 1000 opensearch && \\
     adduser -u 1000 -g 1000 -G 0 -d /usr/share/opensearch opensearch && \\