Skip to content

HDDS-13009. Background snapshot defrag service #9227

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
smengcl wants to merge 26 commits into from
Closed

HDDS-13009. Background snapshot defrag service #9227

smengcl wants to merge 26 commits into from

Conversation

@smengcl
Copy link
Contributor

@smengcl smengcl commented Oct 31, 2025

What changes were proposed in this pull request?

Implement Background Snapshot Defragmentation Service outlined in the design

Some commits are cherry-picked from the POC and rebased/changed.

Based on draft v1 #9117, #9133 and various improvements. Rebased and addressed most comments in the draft v1.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-13009

How was this patch tested?

  • New integration test.

@smengcl smengcl requested a review from Copilot October 31, 2025 06:18
@smengcl smengcl added the snapshot https://issues.apache.org/jira/browse/HDDS-6517 label Oct 31, 2025
@smengcl smengcl mentioned this pull request Oct 31, 2025
Closed
2 tasks
Copilot AI reviewed Oct 31, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements snapshot defragmentation functionality to optimize storage for Ozone snapshots. The defragmentation service processes snapshots in the active chain, creating compacted versions that contain only necessary data for tracked column families.

Key changes:

  • Implemented full and incremental defragmentation logic in SnapshotDefragService
  • Added support for defragmented snapshot directories alongside regular checkpoint state
  • Enhanced database and snapshot utilities to support defragmentation operations
  • Added comprehensive integration tests for snapshot defragmentation

Reviewed Changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 14 comments.

Show a summary per file
File Description
SnapshotDefragService.java Implements core defragmentation logic including full and incremental defrag operations, DB integrity verification, and metadata updates
SnapshotDiffManager.java Made getDeltaFiles public and added null check for defrag job tracking
OmSnapshotLocalDataManager.java Sets needsDefrag flag when snapshot versions are modified
OmSnapshotManager.java Updated to pass defrag flag when creating snapshot metadata managers
OmMetadataManagerImpl.java Added support for loading from defragged checkpoint directories
RDBStore.java Creates defraggedSnapshotsParentDir directory structure
RocksDatabase.java Made checkpoint methods public and added proper resource cleanup
OzoneConsts.java Fixed defraggedSnapshotsParentDir constant to include proper path prefix
TestSnapshotDefragService.java Added integration tests for defragmentation functionality

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

...one/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/SnapshotDiffManager.java
SnapshotDiffResponse.SubStatus subStatus) {
SnapshotDiffJob snapshotDiffJob = snapDiffJobTable.get(jobKey);
if (snapshotDiffJob == null) {
// TODO: Record activity for defrag jobs as well somehow
Copy link

Copilot AI Oct 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The null check for snapshotDiffJob indicates that defrag jobs are not being tracked in the same way as snapshot diff jobs. The TODO suggests this should be addressed. Consider implementing a separate tracking mechanism for defrag jobs or extending the existing job table to handle both job types.

Copilot uses AI. Check for mistakes.
smengcl added 18 commits October 31, 2025 00:21
@smengcl
Move const; findbugs; address copilot comments
01f38f2
@smengcl
pmd
b5de4e3
@smengcl
Add version suffix to defragged DB dirs; cleanup
62fd5f4
@smengcl
delete old DB; fix append version suffix to defrag DB dir names
0c2f696
@smengcl
Merge remote-tracking branch 'asf' into HDDS-13009-defrag-service-reb…
aaaf0c0 
…ase-v2

Conflicts:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/snapshot/OmSnapshotLocalDataManager.java
@smengcl
improve error handling
4fba9fa
@smengcl
Use FileUtils.deleteDirectory for cleanup
6dd3876
@smengcl
logging
c07898f
@smengcl
throw when failing to ingest SST file
b12d41c
@smengcl
fix previous defragmented DB existence check
108f35d
@smengcl
add debug logging in OmMetadataManagerImpl
aa90ce8
@smengcl
add debug logging in SnapshotCache#invalidate
2dc5309
@smengcl
throw exception when previous snapshot is not defragged in incrementa…
553acdb 
…l defrag, instead of just return
@smengcl
fix checkpoint debug logging
24fbb1c
@smengcl
Invalidate snapshot cache before deleting old DB
1689158 
Added cache invalidation for the snapshot entry before deleting the old database directory to prevent issues with lingering DB handles.
@smengcl
Update log message order in OmMetadataManagerImpl
edc1eb6 
Swapped the order of variables in a debug log statement to display snapshotLocalDataVersion before dbName for improved clarity.
@smengcl
Update snapshot defrag test configuration and assertions
dae1162 
Introduces a static variable for SNAPSHOT_DEFRAG_LIMIT_PER_TASK_VALUE and uses it in configuration and assertions. Also increases the await timeout and refines the defragmentation completion condition for improved test reliability.
@smengcl
cleanup
1766882 
Added explicit invalidation of SnapshotCache entry before deleting old snapshot DBs to prevent cache from holding stale DB handles. Improved error handling for invalid oldVersion and clarified comments regarding cache cleanup.
@jojochuang jojochuang marked this pull request as ready for review November 2, 2025 04:29
jojochuang
jojochuang reviewed Nov 2, 2025
View reviewed changes
Copy link
Contributor

@jojochuang jojochuang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test failure looks related:

Error: Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 45.06 s <<< FAILURE! -- in org.apache.hadoop.ozone.om.TestOMDbCheckpointServlet
Error: org.apache.hadoop.ozone.om.TestOMDbCheckpointServlet.testWithoutACL -- Time elapsed: 30.68 s <<< FAILURE!
org.opentest4j.AssertionFailedError: expected snapshot files not found ==> expected: <[db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000058.sst, db.snapshots/diffState/compaction-sst-backup, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000053.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/MANIFEST-000005, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f.yaml, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000063.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/MANIFEST-000005, db.snapshots/diffState/compaction-log/expected.log, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/CURRENT, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000054.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/OPTIONS-000051, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000056.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082, db.snapshots/checkpointStateDefragged, db.snapshots/diffState/compaction-log/_README.txt, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000057.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000053.sst, db.snapshots/diffState/compaction-log, db.snapshots/diffState, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000057.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000061.sst, db.snapshots/diffState/snapDiff/_README.txt, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000061.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000066.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000056.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000065.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000060.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082.yaml, db.snapshots/diffState/compaction-sst-backup/CURRENT, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000058.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000060.sst, db.snapshots/diffState/snapDiff, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/OPTIONS-000051, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000067.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000059.sst, db.snapshots/diffState/compaction-sst-backup/expected.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000054.sst, db.snapshots/checkpointState, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000064.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/CURRENT, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000059.sst]> but was: <[db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000058.sst, db.snapshots/diffState/compaction-sst-backup, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000053.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/MANIFEST-000005, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f.yaml, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000063.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/MANIFEST-000005, db.snapshots/diffState/compaction-log/expected.log, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/CURRENT, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000054.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/OPTIONS-000051, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000056.sst, db.snapshots/diffState/compaction-log/_README.txt, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000057.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000053.sst, db.snapshots/diffState/compaction-log, db.snapshots/diffState, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000057.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000061.sst, db.snapshots/diffState/snapDiff/_README.txt, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000066.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000061.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000056.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000065.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000060.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082.yaml, db.snapshots/diffState/compaction-sst-backup/CURRENT, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000060.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000058.sst, db.snapshots/diffState/snapDiff, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/OPTIONS-000051, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000067.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000059.sst, db.snapshots/diffState/compaction-sst-backup/expected.sst, db.snapshots/checkpointState, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000054.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000064.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/CURRENT, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000059.sst]>
at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182)
at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1156)
at org.apache.hadoop.ozone.om.TestOMDbCheckpointServlet.testWriteDbDataToStream(TestOMDbCheckpointServlet.java:502)
at org.apache.hadoop.ozone.om.TestOMDbCheckpointServlet.testWithoutACL(TestOMDbCheckpointServlet.java:252)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

@smengcl
Copy link
Contributor Author

smengcl commented Nov 3, 2025

Note #9208 may have introduced conflict with this PR in how defrag DB path (dir location) is handled.

@smengcl
Copy link
Contributor Author

smengcl commented Nov 3, 2025

Test failure looks related:

Error: Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 45.06 s <<< FAILURE! -- in org.apache.hadoop.ozone.om.TestOMDbCheckpointServlet Error: org.apache.hadoop.ozone.om.TestOMDbCheckpointServlet.testWithoutACL -- Time elapsed: 30.68 s <<< FAILURE! org.opentest4j.AssertionFailedError: expected snapshot files not found ==> expected: <[db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000058.sst, db.snapshots/diffState/compaction-sst-backup, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000053.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/MANIFEST-000005, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f.yaml, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000063.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/MANIFEST-000005, db.snapshots/diffState/compaction-log/expected.log, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/CURRENT, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000054.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/OPTIONS-000051, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000056.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082, db.snapshots/checkpointStateDefragged, db.snapshots/diffState/compaction-log/_README.txt, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000057.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000053.sst, db.snapshots/diffState/compaction-log, db.snapshots/diffState, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000057.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000061.sst, db.snapshots/diffState/snapDiff/_README.txt, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000061.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000066.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000056.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000065.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000060.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082.yaml, db.snapshots/diffState/compaction-sst-backup/CURRENT, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000058.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000060.sst, db.snapshots/diffState/snapDiff, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/OPTIONS-000051, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000067.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000059.sst, db.snapshots/diffState/compaction-sst-backup/expected.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000054.sst, db.snapshots/checkpointState, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000064.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/CURRENT, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000059.sst]> but was: <[db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000058.sst, db.snapshots/diffState/compaction-sst-backup, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000053.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/MANIFEST-000005, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f.yaml, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000063.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/MANIFEST-000005, db.snapshots/diffState/compaction-log/expected.log, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/CURRENT, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000054.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/OPTIONS-000051, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000056.sst, db.snapshots/diffState/compaction-log/_README.txt, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000057.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000053.sst, db.snapshots/diffState/compaction-log, db.snapshots/diffState, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000057.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000061.sst, db.snapshots/diffState/snapDiff/_README.txt, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000066.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000061.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000056.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000065.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000060.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082.yaml, db.snapshots/diffState/compaction-sst-backup/CURRENT, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000060.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000058.sst, db.snapshots/diffState/snapDiff, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/OPTIONS-000051, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000067.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000059.sst, db.snapshots/diffState/compaction-sst-backup/expected.sst, db.snapshots/checkpointState, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/000054.sst, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000064.sst, db.snapshots/checkpointState/om.db-34a4a5d7-a672-4024-8f63-53248c82e94f/CURRENT, db.snapshots/checkpointState/om.db-5f94a3d6-08e4-4e0e-a7d5-a61c4e19b082/000059.sst]> at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151) at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132) at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197) at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:182) at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:1156) at org.apache.hadoop.ozone.om.TestOMDbCheckpointServlet.testWriteDbDataToStream(TestOMDbCheckpointServlet.java:502) at org.apache.hadoop.ozone.om.TestOMDbCheckpointServlet.testWithoutACL(TestOMDbCheckpointServlet.java:252) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

Ah. It could be related to the change in getSnapshotPath. But since Swami has the patch that modifies it in a different way. I might need to undo that part of change anyway.

@adoroszlai adoroszlai marked this pull request as draft November 13, 2025 11:02
@smengcl
Copy link
Contributor Author

smengcl commented Nov 24, 2025

Closing this in favor of #9324

TestSnapshotDefragService could still be useful.

@smengcl smengcl closed this Nov 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jojochuang jojochuang jojochuang left review comments

Copilot code review Copilot Copilot left review comments

Assignees

No one assigned

Labels

snapshot https://issues.apache.org/jira/browse/HDDS-6517

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@smengcl @jojochuang