[DO NOT MERGE] Backport some fixes, performance optimizations from master to ozone-1.4 #6929#6964

Merged

xichen01 merged 53 commits intoapache:ozone-1.4from

xichen01:ozone-1.4-backport

Jul 24, 2024

Commits on Jul 17, 2024

HDDS-8882. Manage status of DeleteBlocksCommand in SCM to avoid sending duplicates to Datanode (#4988)
xichen01
committed
HDDS-6152. Migrate TestOzoneFileSystem to JUnit5 (#5795)

adoroszlai
authored and
xichen01
committed
HDDS-9966. Bump maven-shade-plugin to 3.5.1 (#5823)

dependabot[bot]
authored and
xichen01
committed
HDDS-9807. Consider volume committed space when checking if datanode can host new container (#5721)

vtutrinov
authored and
xichen01
committed
HDDS-9922. Migrate TestOzoneFileInterfaces to JUnit5 (#5838)

adoroszlai
authored and
xichen01
committed
HDDS-10027. NPE in VolumeInfoMetrics.getCommitted() (#5885)

adoroszlai
authored and
xichen01
committed
HDDS-10007. Rename ManagedSstFileReader in rocksdb-checkpoint-differ (#5859)

adoroszlai
authored and
xichen01
committed
HDDS-9959. Propagate group remove to other datanodes during pipeline close (#5827)

ivandika3
authored and
xichen01
committed
HDDS-9883. Recon - Improve the performance of processing IncrementalContainerReport from DN (#5793)

devmadhuu
authored and
xichen01
committed
HDDS-10046. Replace PrecomputedVolumeSpace with SpaceUsageSource.Fixed (#5904)

adoroszlai
authored and
xichen01
committed
HDDS-8982. Log flooded by WritableRatisContainerProvider if pipeline's nodes are not found (#5911)

adoroszlai
authored and
xichen01
committed
HDDS-10070. Intermittent failure in TestWritableRatisContainerProvider (#5930)

adoroszlai
authored and
xichen01
committed
HDDS-8888. Consider Datanode queue capacity when sending DeleteBlocks command (#4939)
xichen01
committed
HDDS-10178. Shaded Jar build failure in case-insensitive filesystem (#6045)

adoroszlai
authored and
xichen01
committed
HDDS-10219. Bump frontend-maven-plugin to 1.15.0 (#6104)

dependabot[bot]
authored and
xichen01
committed

Commits on Jul 18, 2024

HDDS-10225. Speed up TestSCMHAManagerImpl. (#6109)

adoroszlai
authored and
xichen01
committed
HDDS-10029. Improved logs for SCMDeletedBlockTransactionStatusManager (#5888)
xichen01
committed
HDDS-10246. Remove KeyValueHandler.checkContainerIsHealthy to improve read performance (#6127)

whbing
authored and
xichen01
committed
HDDS-10262. Encapsulate SnapshotCache inside OmSnapshotManager (#6135)

Cyrill
authored and
xichen01
committed
HDDS-10250. Use SnapshotId as key in SnapshotCache (#6139)

hemantk-12
authored and
xichen01
committed
HDDS-7810. Support namespace summaries (du, dist & counts) for OBJECT_STORE buckets. (#4245)

ArafatKhan2198
authored and
xichen01
committed
HDDS-10504. Remove unused VolumeInfo#configuredCapacity (#6363)

adoroszlai
authored and
xichen01
committed
HDDS-10505. Move space reservation logic to VolumeUsage (#6370)

adoroszlai
authored and
xichen01
committed
HDDS-5865. Make read retry interval and attempts in BlockInputStream configurable (#6408)

SaketaChalamchala
authored and
xichen01
committed
HDDS-9534. Support namespace summaries (du, dist & counts) for LEGACY buckets with file system disabled (#5517)

ArafatKhan2198
authored and
xichen01
committed
HDDS-10206. Expose jmx metrics for snapshot cache size on the ozone manager. (#6138)

ceekay47
authored and
xichen01
committed
HDDS-10452. Improve Recon Disk Usage to fetch and display Top N records based on size. (#6318)

ArafatKhan2198
authored and
xichen01
committed
HDDS-10156. Optimize Snapshot Cache get and eviction (#6024)

swamirishi
authored and
xichen01
committed
HDDS-10652. EC Reconstruction fails with "IOException: None of the block data have checksum" after upgrade (#6520)

siddhantsangwan
authored and
xichen01
committed
HDDS-10614. Avoid decreasing cached space usage below zero (#6508)

ArafatKhan2198
authored and
xichen01
committed
HDDS-10783. Close SstFileReaderIterator in RocksDBCheckpointDiffer (#6616)

hemantk-12
authored and
xichen01
committed
HDDS-10784. Multipart upload to encrypted bucket fails with ClassCastException (#6618)

adoroszlai
authored and
xichen01
committed
HDDS-10792. Bump Netty to 4.1.109.Final (#6622)

rohit-kb
authored and
xichen01
committed
HDDS-10720. Datanode volume DU reserved percent should have a non-zero default value. (#6561)

errose28
authored and
xichen01
committed
HDDS-10787. Updated rocksdb-checkpoint-differ to use managed RocksDB objects (#6625)

hemantk-12
authored and
xichen01
committed
HDDS-10806. Bump Bouncy Castle to 1.78.1 (#6632)

dependabot[bot]
authored and
xichen01
committed
HDDS-10803. HttpServer fails to start with wildcard principal (#6631)

adoroszlai
authored and
xichen01
committed
HDDS-10815. Bump Spring Framework to 5.3.34 (#6643)

rohit-kb
authored and
xichen01
committed
HDDS-10834. Revert snapshot diff output change added in HDDS-9360 (#6660)

hemantk-12
authored and
xichen01
committed
HDDS-10608. Recon can't get full key when using Recon API. (#6492)

ArafatKhan2198
authored and
xichen01
committed
HDDS-10696. Fix test failure caused by empty snapshot installation (#6659)

hemantk-12
authored and
xichen01
committed
HDDS-10781. Do not use OFSPath in O3FS BasicOzoneClientAdapterImpl (#6614)

chungen0126
authored and
xichen01
committed
HDDS-10371. NPE in OzoneAclUtils.isOwner (#6676)

adoroszlai
authored and
xichen01
committed
HDDS-10875. XceiverRatisServer#getRaftPeersInPipeline should be called before XceiverRatisServer#removeGroup (#6696)

ivandika3
authored and
xichen01
committed
HDDS-10832. Client should switch to streaming based on OpenKeySession replication (#6683)

adoroszlai
authored and
xichen01
committed
HDDS-10924. TestSCMHAManagerImpl#testAddSCM fails on ratis master (#6753)

adoroszlai
authored and
xichen01
committed
HDDS-10999. Remove dependency on ratis-server from Ozone Client (#6800)

adoroszlai
authored and
xichen01
committed
HDDS-11013. Ensure version is always set in ContainerCommandRequestProto (#6812)

swamirishi
authored and
xichen01
committed
HDDS-10983. EC Key read corruption when the replica index of container in DN mismatches (#6779)

swamirishi
authored and
xichen01
committed
HDDS-10910. Bump Ratis to 3.1.0 (#6872)

adoroszlai
authored and
xichen01
committed
HDDS-11104. Bump maven-dependency-plugin to 3.7.1 (#6903)

dependabot[bot]
authored and
xichen01
committed
HDDS-11172. Bump vite to 4.5.3 (#6918)

dependabot[bot]
authored and
xichen01
committed
HDDS-11186. First container log missing from bundle (#6952)

adoroszlai
authored and
xichen01
committed