HDDS-12960. SST Statistics-Based RocksDB Compaction Scheduling#2

Closed

peterxcli wants to merge 32 commits intomasterfrom

hdds12960-implement-finer-compaction-and-provide-benchmark-data

Commits on Jun 12, 2025

Support cacheIterator method with startKey for full table cache
peterxcli
committed
Support CF key range compaction and SST properties retrieval for rocksDB
peterxcli
committed
Add range compaction service skelton
peterxcli
committed
Refactor compaction logic to utilize StringUtils for key conversion and enhance KeyRangeStats to use long for entry counts. Introduce FSOTableCompactor and OBSTableCompactor classes for improved co…
peterxcli
committed
Enhance FullTableCache iterator to handle null startKey and add getKeyPrefixUpperBound method in AbstractCompactor for improved key range handling. Update FSOTableCompactor and OBSTableCompactor to…
peterxcli
committed
comment out FSO table compactor
peterxcli
committed
Add CompoundKeyRangeStats class for managing key range statistics and enhance KeyRange constructor to support byte array inputs. Update OBSTableCompactor to utilize new CompoundKeyRangeStats for im…
peterxcli
committed
Refactor range compaction configuration by renaming maxEntriesSum to maxCompactionEntries and adding rangesPerRun parameter. Update related classes to accommodate new configuration keys and improve…
peterxcli
committed
Enhance StringUtils with new methods for key prefix upper bound, max, and min string comparisons. Refactor KeyRange and RDBStore to use updated method names. Update OBSTableCompactor to utilize Str…
peterxcli
committed
fix pmd
peterxcli
committed
Rename "deletedDirTable" to "deletedDirectoryTable" in RangeCompactionService for improved clarity.
peterxcli
committed
Refactor ManagedRange to implement AutoCloseable, ensuring proper resource management by closing ManagedSlices. Update KeyRange and RocksDatabase to handle exceptions and close resources appropriat…
peterxcli
committed
Update Javadoc in DBStore to clarify parameter naming from 'columnFamily' to 'tableName' for improved documentation accuracy.
peterxcli
committed
Add range compaction service configuration to ozone-default.xml
peterxcli
committed
Minor adjustment and fixes
peterxcli
committed
Refactor key range handling by removing ManagedRange and replacing it with RocksDB's Range. Update RDBStore and RocksDatabase to utilize the new Range implementation, ensuring proper resource manag…
peterxcli
committed
Update rocksJava JNI patch to fix GetPropertiesOfTableInRange
peterxcli
committed
try to fix ci
peterxcli
committed
Add disk space cleanup script for ci
peterxcli
committed
fix checkstyle and pmd
peterxcli
committed
Update pom.xml for rocksdb version and add GitHub repository configuration; enhance CI workflows with GitHub actor and token; implement Maven dependency plugin for rocksdbjni unpacking; improve log…
peterxcli
committed
Remove disk space cleanup script and related workflow steps from CI configuration
peterxcli
committed
Enhance getCompoundKeyRangeStatsFromRange method in OBSTableCompactor by ensuring proper closure of key range stats list, improving the accuracy of compound key range statistics retrieval.
peterxcli
committed
Fix some logic in OBS table compactor
peterxcli
committed
add log
peterxcli
committed
Update Grafana dashboard JSON files to replace Prometheus datasource UID from "${DS_PROMETHEUS}" to "-- Grafana --" for consistency.
peterxcli
committed
Remove uid: "--Grafana--"
peterxcli
committed
Disable the range compaction service by default and add a new benchmark generator for key operations in Ozone. Update related configuration keys and ensure proper handling of bucket bounds in the c…
peterxcli
committed
Add max live keys limit to OmKeyBenchGenerator and optimize key retrieval. Enhance logging in AbstractCompactor and synchronize start method in RangeCompactionService.
peterxcli
committed

Commits on Jul 4, 2025

Implement FSO compaction logic in FSOTableCompactor, introducing directory-aware range processing and adaptive splitting based on directory hierarchy. Enhance logging for better traceability during…
peterxcli
committed

Commits on Aug 28, 2025