Skip to content

HDDS-11414. Key listing for FSO buckets fails with forward client #7161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sumitagrawl merged 5 commits into from
Sep 9, 2024
Merged

HDDS-11414. Key listing for FSO buckets fails with forward client #7161

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e803b1e
HDDS-11414. Key listing for FSO buckets fails with forward client
tanvipenumudy Sep 5, 2024
895c25a
Add omVersion checks to listStatusLight instead of listStatus
tanvipenumudy Sep 6, 2024
d8e2629
Add acceptance-compat tests for list
tanvipenumudy Sep 6, 2024
10bc76a
Fix acceptance-compat test
tanvipenumudy Sep 6, 2024
71be2d5
Fix acceptance-compat test
tanvipenumudy Sep 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions hadoop-hdds/common/src/main/java/org/apache/hadoop/ozone/OzoneManagerVersion.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@ public enum OzoneManagerVersion implements ComponentVersion {

ATOMIC_REWRITE_KEY(6, "OzoneManager version that supports rewriting key as atomic operation"),
HBASE_SUPPORT(7, "OzoneManager version that supports HBase integration"),
LIGHTWEIGHT_LIST_STATUS(8, "OzoneManager version that supports lightweight"
+ " listStatus API."),

FUTURE_VERSION(-1, "Used internally in the client when the server side is "
+ " newer and an unknown server version has arrived to the client.");
Expand Down
13 changes: 10 additions & 3 deletions hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2315,9 +2315,16 @@ public List<OzoneFileStatusLight> listStatusLight(String volumeName,
String bucketName, String keyName, boolean recursive, String startKey,
long numEntries, boolean allowPartialPrefixes) throws IOException {
OmKeyArgs keyArgs = prepareOmKeyArgs(volumeName, bucketName, keyName);
return ozoneManagerClient
.listStatusLight(keyArgs, recursive, startKey, numEntries,
allowPartialPrefixes);
if (omVersion.compareTo(OzoneManagerVersion.LIGHTWEIGHT_LIST_STATUS) >= 0) {
return ozoneManagerClient.listStatusLight(keyArgs, recursive, startKey,
numEntries, allowPartialPrefixes);
} else {
return ozoneManagerClient.listStatus(keyArgs, recursive, startKey,
numEntries, allowPartialPrefixes)
.stream()
.map(OzoneFileStatusLight::fromOzoneFileStatus)
.collect(Collectors.toList());
}
}

/**
Expand Down
4 changes: 4 additions & 0 deletions hadoop-ozone/dist/src/main/smoketest/compatibility/read.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ Key Can Be Read
Dir Can Be Listed
Execute ozone fs -ls o3fs://bucket1.vol1/dir-${SUFFIX}

Dir Can Be Listed Using Shell
${result} = Execute ozone sh key list /vol1/bucket1
Should Contain ${result} key-${SUFFIX}

File Can Be Get
Execute ozone fs -get o3fs://bucket1.vol1/dir-${SUFFIX}/file-${SUFFIX} /tmp/
Execute diff -q ${TESTFILE} /tmp/file-${SUFFIX}
Expand Down