Skip to content

HDDS-6163. Change env vars for docker container image #2967

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
adoroszlai merged 4 commits into from
Jan 20, 2022
Merged

HDDS-6163. Change env vars for docker container image #2967

adoroszlai merged 4 commits into from
Jan 20, 2022

Conversation

@kmizumar
Copy link
Contributor

@kmizumar kmizumar commented Jan 7, 2022

What changes were proposed in this pull request?

Change environment variables assigned to self-built Docker image generated by

$ mvn -Pdocker -Ddocker.image=<name>:<version> -DskipTests -Pdist clean package

Without these changes, StatefulSets which reside in kubernetes/examples/ozone never start since the init container fails in finding the file ozone in its $PATH:

$ kubectl logs scm-0 -c init
/opt/hadoop/libexec/entrypoint.sh: line 162: exec: ozone: not found

According to the apache/ozone:1.2.0 container image, I believe that

  • HADDOP_LOG_DIR and HADOOP_CONF_DIR should NOT be defined
  • OZONE_LOG_DIR and OZONE_CONF_DIR should be defined instead

but it seems there's no way to unset environment variables already defined in the underlying container image. So I changed the values of HADOOP_LOG_DIR and HADOOP_CONF_DIR to empty value.

$ docker inspect apache/ozone:1.2.0
(snip)
            "Env": [
                "PATH=/opt/hadoop/libexec:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/hadoop/bin",
                "JAVA_HOME=/usr/lib/jvm/jre/",
                "LD_LIBRARY_PATH=/usr/local/lib",
                "OZONE_LOG_DIR=/var/log/hadoop",
                "OZONE_CONF_DIR=/etc/hadoop"
            ],
(snip)

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-6163

How was this patch tested?

Manually tested with kind.

$ mvn -Pdocker -Ddocker.image=pfnmaru/ozone:1.3.0-SNAPSHOT -DskipTests -Pdist clean package
$ kind load docker-image pfnmaru/ozone:1.3.0-SNAPSHOT
$ cd hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/kubernetes/examples/ozone
$ kubectl apply -f config-configmap.yaml
configmap/config created

$ kubectl apply -f scm-statefulset.yaml
statefulset.apps/scm created

$ kubectl get pods
NAME    READY   STATUS    RESTARTS   AGE
scm-0   1/1     Running   0          20s

$ kubectl logs scm-0 -c init
No '-XX:...' jvm parameters are set. Adding safer GC settings '-XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled' to the OZONE_OPTS
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
2022-01-07 12:12:17 INFO  StorageContainerManagerStarter:112 - STARTUP_MSG: 
/************************************************************
STARTUP_MSG: Starting StorageContainerManager
STARTUP_MSG:   host = scm-0/10.244.7.6
STARTUP_MSG:   args = [--init]
STARTUP_MSG:   version = 1.3.0-SNAPSHOT
STARTUP_MSG:   classpath = /opt/hadoop/etc/hadoop:/opt/hadoop/share/ozone/lib/jackson-annotations-2.12.1.jar:/opt/hadoop/share/ozone/lib/kerb-simplekdc-1.0.1.jar:/opt/hadoop/share/ozone/lib/slf4j-log4j12-1.7.30.jar:/opt/hadoop/share/ozone/lib/protobuf-java-2.5.0.jar:/opt/hadoop/share/ozone/lib/jaxb-core-2.3.0.1.jar:/opt/hadoop/share/ozone/lib/istack-commons-runtime-3.0.5.jar:/opt/hadoop/share/ozone/lib/commons-validator-1.6.jar:/opt/hadoop/share/ozone/lib/bcpkix-jdk15on-1.67.jar:/opt/hadoop/share/ozone/lib/hdds-interface-admin-1.3.0-SNAPSHOT.jar:/opt/hadoop/share/ozone/lib/jetty-security-9.4.43.v20210629.jar:/opt/hadoop/share/ozone/lib/kerb-core-1.0.1.jar:/opt/hadoop/share/ozone/lib/hadoop-shaded-protobuf_3_7-1.1.1.jar:/opt/hadoop/share/ozone/lib/hdds-container-service-1.3.0-SNAPSHOT.jar:/opt/hadoop/share/ozone/lib/okhttp-4.9.0.jar:/opt/hadoop/share/ozone/lib/jsr305-3.0.0.jar:/opt/hadoop/share/ozone/lib/j2objc-annotations-1.3.jar:/opt/hadoop/share/ozone/lib/json-smart-2.3.1.jar:/opt/hadoop/share/ozone/lib/picocli-4.6.1.jar:/opt/hadoop/share/ozone/lib/javassist-3.21.0-GA.jar:/opt/hadoop/share/ozone/lib/jaxb-api-2.3.0.jar:/opt/hadoop/share/ozone/lib/htrace-core4-4.1.0-incubating.jar:/opt/hadoop/share/ozone/lib/kerb-util-1.0.1.jar:/opt/hadoop/share/ozone/lib/kotlin-stdlib-common-1.4.31.jar:/opt/hadoop/share/ozone/lib/checker-qual-3.8.0.jar:/opt/hadoop/share/ozone/lib/hadoop-shaded-guava-1.1.1.jar:/opt/hadoop/share/ozone/lib/log4j-core-2.17.1.jar:/opt/hadoop/share/ozone/lib/commons-logging-1.2.jar:/opt/hadoop/share/ozone/lib/kerby-xdr-1.0.1.jar:/opt/hadoop/share/ozone/lib/netty-buffer-4.1.63.Final.jar:/opt/hadoop/share/ozone/lib/txw2-2.3.0.1.jar:/opt/hadoop/share/ozone/lib/jackson-datatype-jsr310-2.12.1.jar:/opt/hadoop/share/ozone/lib/httpcore-4.4.13.jar:/opt/hadoop/share/ozone/lib/opentracing-tracerresolver-0.1.8.jar:/opt/hadoop/share/ozone/lib/commons-net-3.6.jar:/opt/hadoop/share/ozone/lib/libthrift-0.14.1.jar:/opt/hadoop/share/ozone/lib/snakeyaml-1.26.jar:/opt/hadoop/share/ozone/lib/stax-ex-1.7.8.jar:/opt/hadoop/share/ozone/lib/error_prone_annotations-2.2.0.jar:/opt/hadoop/share/ozone/lib/httpclient-4.5.13.jar:/opt/hadoop/share/ozone/lib/jetty-servlet-9.4.43.v20210629.jar:/opt/hadoop/share/ozone/lib/jaeger-client-1.6.0.jar:/opt/hadoop/share/ozone/lib/kerby-config-1.0.1.jar:/opt/hadoop/share/ozone/lib/commons-io-2.11.0.jar:/opt/hadoop/share/ozone/lib/metrics-core-3.2.4.jar:/opt/hadoop/share/ozone/lib/kerby-pkix-1.0.1.jar:/opt/hadoop/share/ozone/lib/hdds-interface-client-1.3.0-SNAPSHOT.jar:/opt/hadoop/share/ozone/lib/netty-codec-4.1.63.Final.jar:/opt/hadoop/share/ozone/lib/jsr311-api-1.1.1.jar:/opt/hadoop/share/ozone/lib/commons-beanutils-1.9.4.jar:/opt/hadoop/share/ozone/lib/jetty-webapp-9.4.43.v20210629.jar:/opt/hadoop/share/ozone/lib/rocksdbjni-6.25.3.jar:/opt/hadoop/share/ozone/lib/hdds-server-framework-1.3.0-SNAPSHOT.jar:/opt/hadoop/share/ozone/lib/javax.servlet-api-3.1.0.jar:/opt/hadoop/share/ozone/lib/kerb-server-1.0.1.jar:/opt/hadoop/share/ozone/lib/guava-30.1.1-jre.jar:/opt/hadoop/share/ozone/lib/hdds-common-1.3.0-SNAPSHOT.jar:/opt/hadoop/share/ozone/lib/kerb-client-1.0.1.jar:/opt/hadoop/share/ozone/lib/jackson-core-2.12.1.jar:/opt/hadoop/share/ozone/lib/netty-transport-4.1.63.Final.jar:/opt/hadoop/share/ozone/lib/ratis-grpc-2.2.0.jar:/opt/hadoop/share/ozone/lib/ratis-common-2.2.0.jar:/opt/hadoop/share/ozone/lib/netty-common-4.1.63.Final.jar:/opt/hadoop/share/ozone/lib/disruptor-3.4.2.jar:/opt/hadoop/share/ozone/lib/reflections-0.9.11.jar:/opt/hadoop/share/ozone/lib/jetty-io-9.4.43.v20210629.jar:/opt/hadoop/share/ozone/lib/hdds-interface-server-1.3.0-SNAPSHOT.jar:/opt/hadoop/share/ozone/lib/hadoop-hdfs-3.3.1.jar:/opt/hadoop/share/ozone/lib/annotations-13.0.jar:/opt/hadoop/share/ozone/lib/jetty-server-9.4.43.v20210629.jar:/opt/hadoop/share/ozone/lib/simpleclient_common-0.7.0.jar:/opt/hadoop/share/ozone/lib/jackson-databind-2.12.1.jar:/opt/hadoop/share/ozone/lib/jaeger-thrift-1.6.0.jar:/opt/hadoop/share/ozone/lib/hadoop-common-3.3.1.jar:/opt/hadoop/share/ozone/lib/dnsjava-2.1.7.jar:/opt/hadoop/share/ozone/lib/bcprov-jdk15on-1.67.jar:/opt/hadoop/share/ozone/lib/commons-configuration2-2.1.1.jar:/opt/hadoop/share/ozone/lib/ratis-proto-2.2.0.jar:/opt/hadoop/share/ozone/lib/ratis-client-2.2.0.jar:/opt/hadoop/share/ozone/lib/token-provider-1.0.1.jar:/opt/hadoop/share/ozone/lib/log4j-1.2.17.jar:/opt/hadoop/share/ozone/lib/commons-daemon-1.0.13.jar:/opt/hadoop/share/ozone/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/opt/hadoop/share/ozone/lib/jaeger-tracerresolver-1.6.0.jar:/opt/hadoop/share/ozone/lib/jetty-xml-9.4.43.v20210629.jar:/opt/hadoop/share/ozone/lib/ratis-netty-2.2.0.jar:/opt/hadoop/share/ozone/lib/commons-lang3-3.7.jar:/opt/hadoop/share/ozone/lib/re2j-1.1.jar:/opt/hadoop/share/ozone/lib/commons-math3-3.1.1.jar:/opt/hadoop/share/ozone/lib/ratis-thirdparty-misc-0.7.0.jar:/opt/hadoop/share/ozone/lib/simpleclient_dropwizard-0.7.0.jar:/opt/hadoop/share/ozone/lib/simpleclient-0.7.0.jar:/opt/hadoop/share/ozone/lib/asm-5.0.4.jar:/opt/hadoop/share/ozone/lib/kerby-util-1.0.1.jar:/opt/hadoop/share/ozone/lib/failureaccess-1.0.1.jar:/opt/hadoop/share/ozone/lib/okio-2.8.0.jar:/opt/hadoop/share/ozone/lib/hdds-config-1.3.0-SNAPSHOT.jar:/opt/hadoop/share/ozone/lib/jsp-api-2.1.jar:/opt/hadoop/share/ozone/lib/jsch-0.1.54.jar:/opt/hadoop/share/ozone/lib/jetty-http-9.4.43.v20210629.jar:/opt/hadoop/share/ozone/lib/jakarta.activation-api-1.2.1.jar:/opt/hadoop/share/ozone/lib/slf4j-api-1.7.30.jar:/opt/hadoop/share/ozone/lib/hadoop-hdfs-client-3.3.1.jar:/opt/hadoop/share/ozone/lib/nimbus-jose-jwt-7.9.jar:/opt/hadoop/share/ozone/lib/commons-codec-1.11.jar:/opt/hadoop/share/ozone/lib/commons-pool2-2.6.0.jar:/opt/hadoop/share/ozone/lib/gson-2.2.4.jar:/opt/hadoop/share/ozone/lib/jetty-util-9.4.43.v20210629.jar:/opt/hadoop/share/ozone/lib/netty-handler-4.1.63.Final.jar:/opt/hadoop/share/ozone/lib/opentracing-api-0.33.0.jar:/opt/hadoop/share/ozone/lib/jetty-util-ajax-9.4.43.v20210629.jar:/opt/hadoop/share/ozone/lib/jcip-annotations-1.0-1.jar:/opt/hadoop/share/ozone/lib/kerb-admin-1.0.1.jar:/opt/hadoop/share/ozone/lib/netty-resolver-4.1.63.Final.jar:/opt/hadoop/share/ozone/lib/kotlin-stdlib-1.4.31.jar:/opt/hadoop/share/ozone/lib/opentracing-noop-0.33.0.jar:/opt/hadoop/share/ozone/lib/jersey-core-1.19.jar:/opt/hadoop/share/ozone/lib/commons-compress-1.21.jar:/opt/hadoop/share/ozone/lib/jaxb-runtime-2.3.0.1.jar:/opt/hadoop/share/ozone/lib/ratis-metrics-2.2.0.jar:/opt/hadoop/share/ozone/lib/accessors-smart-2.3.1.jar:/opt/hadoop/share/ozone/lib/ratis-server-api-2.2.0.jar:/opt/hadoop/share/ozone/lib/opentracing-util-0.33.0.jar:/opt/hadoop/share/ozone/lib/hdds-client-1.3.0-SNAPSHOT.jar:/opt/hadoop/share/ozone/lib/hdds-hadoop-dependency-client-1.3.0-SNAPSHOT.jar:/opt/hadoop/share/ozone/lib/snappy-java-1.1.8.2.jar:/opt/hadoop/share/ozone/lib/commons-cli-1.2.jar:/opt/hadoop/share/ozone/lib/log4j-api-2.17.1.jar:/opt/hadoop/share/ozone/lib/woodstox-core-5.0.3.jar:/opt/hadoop/share/ozone/lib/javax.annotation-api-1.2.jar:/opt/hadoop/share/ozone/lib/kerby-asn1-1.0.1.jar:/opt/hadoop/share/ozone/lib/hadoop-annotations-3.3.1.jar:/opt/hadoop/share/ozone/lib/stax2-api-3.1.4.jar:/opt/hadoop/share/ozone/lib/commons-digester-1.8.1.jar:/opt/hadoop/share/ozone/lib/commons-collections-3.2.2.jar:/opt/hadoop/share/ozone/lib/jaeger-core-1.6.0.jar:/opt/hadoop/share/ozone/lib/commons-text-1.4.jar:/opt/hadoop/share/ozone/lib/jersey-servlet-1.19.jar:/opt/hadoop/share/ozone/lib/ratis-server-2.2.0.jar:/opt/hadoop/share/ozone/lib/hadoop-auth-3.3.1.jar:/opt/hadoop/share/ozone/lib/kerb-identity-1.0.1.jar:/opt/hadoop/share/ozone/lib/FastInfoset-1.2.13.jar:/opt/hadoop/share/ozone/lib/kerb-common-1.0.1.jar:/opt/hadoop/share/ozone/lib/jersey-server-1.19.jar:/opt/hadoop/share/ozone/lib/hdds-hadoop-dependency-server-1.3.0-SNAPSHOT.jar:/opt/hadoop/share/ozone/lib/kerb-crypto-1.0.1.jar:/opt/hadoop/share/ozone/web:/opt/hadoop/share/ozone/lib/hdds-server-scm-1.3.0-SNAPSHOT.jar
STARTUP_MSG:   build = git@github.com:kmizumar/ozone.git/62c818b390ff0a4aa777203cf23e15a6c4f5e0ac ; compiled by 'maru' on 2022-01-07T11:51Z
STARTUP_MSG:   java = 11.0.10
************************************************************/
2022-01-07 12:12:17 INFO  StorageContainerManagerStarter:90 - registered UNIX signal handlers for [TERM, HUP, INT]
2022-01-07 12:12:17 WARN  ServerUtils:148 - ozone.scm.db.dirs is not configured. We recommend adding this setting. Falling back to ozone.metadata.dirs instead.
2022-01-07 12:12:17 INFO  SCMHANodeDetails:157 - ServiceID for StorageContainerManager is null
2022-01-07 12:12:17 INFO  SCMHANodeDetails:162 - ozone.scm.default.service.id is not defined, falling back to ozone.scm.service.ids to find serviceID for StorageContainerManager if it is HA enabled cluster
2022-01-07 12:12:17 INFO  AbstractLayoutVersionManager:79 - Initializing Layout version manager with metadata layout = SCM_HA (version = 2), software layout = SCM_HA (version = 2)
2022-01-07 12:12:17 INFO  Reflections:232 - Reflections took 57 ms to scan 3 urls, producing 103 keys and 217 values 
2022-01-07 12:12:17 INFO  StorageContainerManager:1115 - SCM already initialized. Reusing existing cluster id for sd=/data/metadata/scm;cid=CID-71c88d22-135e-4ddb-89b5-519110b2266b; layoutVersion=2; HAEnabled=false
2022-01-07 12:12:17 INFO  StorageContainerManagerStarter:124 - SHUTDOWN_MSG: 
/************************************************************
SHUTDOWN_MSG: Shutting down StorageContainerManager at scm-0/10.244.7.6
************************************************************/

adoroszlai
adoroszlai reviewed Jan 10, 2022
View reviewed changes
Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @kmizumar for the patch.

kmizumar and others added 3 commits January 11, 2022 01:39
@kmizumar @adoroszlai
Update hadoop-ozone/dist/src/main/docker/Dockerfile
3278ca3 
Co-authored-by: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>
@kmizumar @adoroszlai
Update hadoop-ozone/dist/src/main/docker/Dockerfile
74e46cf 
Co-authored-by: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>
@kmizumar
[EMPTY] trigger CI
0f34bb7
@adoroszlai adoroszlai merged commit a99ec03 into apache:master Jan 20, 2022
@adoroszlai
Copy link
Contributor

adoroszlai commented Jan 20, 2022

Thanks @kmizumar for the fix.

errose28 added a commit to errose28/ozone that referenced this pull request Feb 4, 2022
@errose28
Merge branch 'master' into HDDS-6234-fix-container-metadata
10ea8bd 
* master: (27 commits)
  HDDS-6206. Application errors must not flood system log (apache#3001)
  HDDS-6181. Change SCMHAInvocationHandler#invokeRatis() logging to TRACE (apache#2992)
  HDDS-5529. For any IOexception from @REPLicated method we should throw it (apache#2788)
  HDDS-6201. Fix NPE for DataScanner with scanned container deleted by others. (apache#3005)
  HDDS-6239. ozonesecure-mr failing with No URLs in mirrorlist (apache#3029)
  HDDS-6227. Test helpers should observe naming conditions (apache#3020)
  HDDS-6205. Add CLI command to display the latest Replication Manager report (apache#3013)
  HDDS-6192. feature/Observability.md translated to Chinese (apache#2994)
  HDDS-6219. Switch to RATIS ReplicationType from STAND_ALONE in our tests. (apache#3014)
  HDDS-6211. [Docs] Image styling on deployed site does not replicate local builds. (apache#3007)
  HDDS-6177. Extend container info command to include replica details  (apache#2995)
  HDDS-6128. CLI tool that downloads all the block replicas and creates a manifest file (apache#2987)
  HDDS-6191. Intermittent failure in TestDeleteWithSlowFollower (apache#3015)
  HDDS-6216. Move OMOpenKeysDeleteRequest to package om.request.key (apache#3011)
  HDDS-6203. CleanUp incomplete gz files during Container move (apache#3000)
  HDDS-3408. Rename ChunkLayOutVersion to ContainerLayoutVersion. (apache#2983)
  HDDS-6109. Preserve the underlying exception raised in client lib. (apache#2989)
  HDDS-6135. SCM Container DB bootstrap on Recon startup for SCM HA. (apache#2972)
  HDDS-6202. Avoid using jmh-generator-annprocess since it is GPL2.0. (apache#2998)
  HDDS-6163. Fix PATH in docker image (apache#2967)
  ...

Conflicts:
    hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/keyvalue/TestKeyValueContainerCheck.java
errose28 added a commit to errose28/ozone that referenced this pull request Feb 7, 2022
@errose28
Merge branch 'master' into HDDS-6235-empty-container-rep
3f27e27 
* master: (27 commits)
  HDDS-6206. Application errors must not flood system log (apache#3001)
  HDDS-6181. Change SCMHAInvocationHandler#invokeRatis() logging to TRACE (apache#2992)
  HDDS-5529. For any IOexception from @REPLicated method we should throw it (apache#2788)
  HDDS-6201. Fix NPE for DataScanner with scanned container deleted by others. (apache#3005)
  HDDS-6239. ozonesecure-mr failing with No URLs in mirrorlist (apache#3029)
  HDDS-6227. Test helpers should observe naming conditions (apache#3020)
  HDDS-6205. Add CLI command to display the latest Replication Manager report (apache#3013)
  HDDS-6192. feature/Observability.md translated to Chinese (apache#2994)
  HDDS-6219. Switch to RATIS ReplicationType from STAND_ALONE in our tests. (apache#3014)
  HDDS-6211. [Docs] Image styling on deployed site does not replicate local builds. (apache#3007)
  HDDS-6177. Extend container info command to include replica details  (apache#2995)
  HDDS-6128. CLI tool that downloads all the block replicas and creates a manifest file (apache#2987)
  HDDS-6191. Intermittent failure in TestDeleteWithSlowFollower (apache#3015)
  HDDS-6216. Move OMOpenKeysDeleteRequest to package om.request.key (apache#3011)
  HDDS-6203. CleanUp incomplete gz files during Container move (apache#3000)
  HDDS-3408. Rename ChunkLayOutVersion to ContainerLayoutVersion. (apache#2983)
  HDDS-6109. Preserve the underlying exception raised in client lib. (apache#2989)
  HDDS-6135. SCM Container DB bootstrap on Recon startup for SCM HA. (apache#2972)
  HDDS-6202. Avoid using jmh-generator-annprocess since it is GPL2.0. (apache#2998)
  HDDS-6163. Fix PATH in docker image (apache#2967)
  ...
arp7 pushed a commit that referenced this pull request Feb 17, 2022
@neils-dev @adoroszlai
@lokeshj1703 @aswinshakil @jojochuang @symious @bharatviswa504 @sodonnel @GeorgeJahad @siddhantsangwan @JyotinderSingh @bshashikant @szetszwo @kmizumar @kerneltime @Xushaohong @kaijchen @dombizita @fapifta @steinsgateted @sky76093016 @kuenishi @smengcl
HDDS-6303. Merge from master to resolve CI workflow issues & hrpcOmTr…
b88a45e 
…ansport support (#3074)

* HDDS-6149. Remove unused keytabs (#2960)

* HDDS-6094. Some unit tests are skipped due to using JUnit4 runner (#2909)

* HDDS-6075. OzoneConfiguration constructor overrides input configuration keys. (#2921)

* HDDS-4177. SCM Container DB bootstrap on Recon startup (#2942)

* HDDS-6086. Compute MD5MD5CRC file checksum using chunk checksums from DataNodes (#2919)

* HDDS-6148. Validate ContainerBalancerConfiguration before start ContainerBalancer (#2957)

* HDDS-6161. SCM StateMachine failing to reinitialize doesn't terminate the process. (#2971)

* HDDS-6134. Move replication-specific config to ReplicationServer (#2943)

* HDDS-4010. S3G startup fails when multiple service ids are configured. (#2976)

* HDDS-6170. Add metrics to replication manager to track container health states (#2975)

* HDDS-3231. Cleanup KeyManagerImpl (#2961)

* HDDS-5927. Improve defaults in ContainerBalancerConfiguration (#2892)

* HDDS-6157. More consistent synchronization in InputStreams (#2965)

* HDDS-4743. [FSO] Add FSO variant of ITestOzoneContractDistcp. (#2980)

* HDDS-6114. Intermittent error due to Failed to init RocksDB (#2947)

* HDDS-6175. Use s3Auth during proxy during decrypt in RpcClient. (#2981)

* HDDS-6175. Use s3Auth during proxy during decrypt in RpcClient.

* HDDS-5740. Enable ratis by default for SCM. (#2637)

* HDDS-6183. Intermittent failure in TestKeyDeletingService.checkIfDeleteServiceWithFailingSCM. (#2991)

* HDDS-4190. Intermittent failure in TestOMVolumeSetOwnerRequest and TestOMVolumeSetQuotaRequest. (#2982)

* HDDS-6120. Compute block checksum using chunk checksums (#2930)

* HDDS-6147. Add ability in OM to get limited delta updates (#2956)

* HDDS-6195. Remove unused jmh-core dependency. (#2997)

* HDDS-6167. Update ozone-runner version to 20211202-1 (#2969)

* HDDS-6171. Create an API to change Bucket Owner (#2988)

* HDDS-6163. Fix PATH in docker image (#2967)

* HDDS-6202. Avoid using jmh-generator-annprocess since it is GPL2.0. (#2998)

* HDDS-6135. SCM Container DB bootstrap on Recon startup for SCM HA. (#2972)

* HDDS-6109. Preserve the underlying exception raised in client lib. (#2989)

* HDDS-3408. Rename ChunkLayOutVersion to ContainerLayoutVersion. (#2983)

* HDDS-6203. CleanUp incomplete gz files during Container move (#3000)

* HDDS-6216. Move OMOpenKeysDeleteRequest to package om.request.key (#3011)

* HDDS-6191. Intermittent failure in TestDeleteWithSlowFollower (#3015)

* HDDS-6128. CLI tool that downloads all the block replicas and creates a manifest file (#2987)

* HDDS-6177. Extend container info command to include replica details  (#2995)

* HDDS-6211. [Docs] Image styling on deployed site does not replicate local builds. (#3007)

* HDDS-6219. Switch to RATIS ReplicationType from STAND_ALONE in our tests. (#3014)

* HDDS-6192. feature/Observability.md translated to Chinese (#2994)

* HDDS-6205. Add CLI command to display the latest Replication Manager report (#3013)

* HDDS-6227. Test helpers should observe naming conditions (#3020)

* HDDS-6239. ozonesecure-mr failing with No URLs in mirrorlist (#3029)

* HDDS-6201. Fix NPE for DataScanner with scanned container deleted by others. (#3005)

* HDDS-5529. For any IOexception from @REPLicated method we should throw it (#2788)

* HDDS-6181. Change SCMHAInvocationHandler#invokeRatis() logging to TRACE (#2992)

* HDDS-6206. Application errors must not flood system log (#3001)

* HDDS-6245. Add BucketLayout logging to Audit Logs (#3040)

* HDDS-6238 Reduce memory requirements for list keys. (#3032)

* HDDS-2919. Intermittent failure in TestRDBStore (#3028)

* HDDS-6253. Unnecessary duplicate smoketest after defaulting to FSO (#3036)

* HDDS-6204. Cleanup handling malformed authorization header (#2999)

* HDDS-6169. Selective checks: skip junit tests on ozone-runner image update (#2974)

* HDDS-6270. Use a dedicated file instead of /etc/passwd for xcompat acceptance test (#3050)

* HDDS-6273. Amend doc SecuringTDE.md (#3047)

* HDDS-6140. Selective checks: skip unit check for integration-test changes (#2948)

* HDDS-6215. Recon get limited delta updates from OM (#3009)

* HDDS-6125. Recon get limited delta updates from OM

* HDDS-6215. Fix unit test

* trigger new CI check

* HDDS-6215. Fix typo

* trigger new CI check

Co-authored-by: Symious <yiyang0203@gmail.com>

* HDDS-6226. Run tests for selective CI checks in CI (#3019)

* HDDS-6247. Avoid logging stack trace for user input problems (#3039)

* HDDS-6208. New checkstyle: WhitespaceAround (#3003)

* HDDS-6289. Upgrade acceptance test log flooded with parse error (#3063)

Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>

* Trigger Build

* Fix integration test for added configuation field for selecting OmTransport for s3 gateway - TestOzoneConfigurationFields (added config key not in xml).

Co-authored-by: Doroszlai, Attila <6454655+adoroszlai@users.noreply.github.com>
Co-authored-by: Lokesh Jain <ljain@apache.org>
Co-authored-by: Aswin Shakil Balasubramanian <aswinshakilbalu@gmail.com>
Co-authored-by: Wei-Chiu Chuang <weichiu@apache.org>
Co-authored-by: Symious <yiyang0203@foxmail.com>
Co-authored-by: Bharat Viswanadham <bharat@apache.org>
Co-authored-by: Stephen O'Donnell <stephen.odonnell@gmail.com>
Co-authored-by: GeorgeJahad <github@blackbirdsystems.net>
Co-authored-by: Siddhant Sangwan <siddhantsangwan027@gmail.com>
Co-authored-by: Jyotinder Singh <jyotindrsingh@gmail.com>
Co-authored-by: Shashikant Banerjee <shashikant@apache.org>
Co-authored-by: Tsz-Wo Nicholas Sze <szetszwo@apache.org>
Co-authored-by: Kiyoshi Mizumaru <kiyoshi.mizumaru@gmail.com>
Co-authored-by: Ritesh H Shukla <kerneltime@gmail.com>
Co-authored-by: Nibiru <axcsd3692@qq.com>
Co-authored-by: Kaijie Chen <chen@kaijie.org>
Co-authored-by: Zita Dombi <50611074+dombizita@users.noreply.github.com>
Co-authored-by: Istvan Fajth <pifta@cloudera.com>
Co-authored-by: steinsgateted <71066027+steinsgateted@users.noreply.github.com>
Co-authored-by: Gui Hecheng <markgui@tencent.com>
Co-authored-by: Jackson Yao <jacksonyao@tencent.com>
Co-authored-by: Keyi Song <72794035+sky76093016@users.noreply.github.com>
Co-authored-by: UENISHI Kota <kuenishi@users.noreply.github.com>
Co-authored-by: Siyao Meng <50227127+smengcl@users.noreply.github.com>
Co-authored-by: Symious <yiyang0203@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@adoroszlai adoroszlai adoroszlai approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kmizumar @adoroszlai