Skip to content

Conversation

@tanvipenumudy
Copy link
Contributor

What changes were proposed in this pull request?

  • This change adds a persisted replicatedSizeOfFiles field to NSSummary to track the total expected replicated size of a directory's sub-files.
  • The value is dynamically updated upon PUT and DELETE key events.
  • The endpoint: /keys/deletePending/dirs is also updated to include the replicated size information.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-13180

How was this patch tested?

Manual Testing:

bash-5.1$ ozone sh volume create vol1
bash-5.1$ ozone sh bucket create vol1/buck1 --layout=FILE_SYSTEM_OPTIMIZED
bash-5.1$ ozone fs -mkdir -p ofs://om/vol1/buck1/dir1/dir2/dir3

# Create 1 MB file:
bash-5.1$ dd if=/dev/zero of=test.txt bs=1M count=1
bash-5.1$ ozone fs -put test.txt ofs://om/vol1/buck1/dir1/file1
bash-5.1$ ozone fs -put test.txt ofs://om/vol1/buck1/dir1/dir2/file2
bash-5.1$ ozone fs -put test.txt ofs://om/vol1/buck1/dir1/dir2/dir3/file3

Directory structure:

dir1/
    ├── file1 (1 MB)
    └── dir2/
        ├── file2 (1 MB)
        └── dir3/
            └── file3 (1 MB)

# Recursively delete top-level directory:
bash-5.1$ ozone fs -rm -r ofs://om/vol1/buck1/dir1

GET /api/v1/keys/deletePending/dirs/:

{
  "lastKey": "/-9223372036854775552/-9223372036854775040/-9223372036854767870/250608184600/-9223372036854751487",
  "replicatedDataSize": 9437184,
  "unreplicatedDataSize": 3145728,
  "deletedDirInfo": [
    {
      "key": "250608184600",
      "path": "vol1/buck1/.Trash/hadoop/250608184600",
      "inStateSince": 1749408310990,
      "size": 3145728,
      "replicatedSize": 9437184,
      "replicationInfo": {
        "replicationFactor": "ONE",
        "requiredNodes": 1,
        "minimumNodes": 1,
        "replicationType": "RATIS"
      },
      "creationTime": 1749408437900,
      "modificationTime": 1749408437900,
      "isKey": false
    }
  ],
  "status": "OK"
}

Unreplicated size: 3145728 (3 MB)
Replicated size (RATIS THREE): 9437184 (9 MB)

@tanvipenumudy
Copy link
Contributor Author

Closed the current PR, changed the base branch from master to HDDS-13177 on the existing PR: #8568

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant