Skip to content

HDDS-8424. Restore legacy bucket behavior for directory objects. #4576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kerneltime merged 1 commit into from
Apr 19, 2023
+14 −0
Merged

HDDS-8424. Restore legacy bucket behavior for directory objects. #4576

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions hadoop-ozone/dist/src/main/smoketest/s3/commonawslib.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,11 @@ Create bucket with name
${result} = Execute AWSS3APICli create-bucket --bucket ${bucket}
Should contain ${result} Location
Should contain ${result} ${bucket}
Create legacy bucket
${postfix} = Generate Ozone String
${legacy_bucket} = Set Variable legacy-bucket-${postfix}
${result} = Execute and checkrc ozone sh bucket create -l LEGACY s3v/${legacy_bucket} 0
[Return] ${legacy_bucket}

Setup s3 tests
Return From Keyword if ${OZONE_S3_TESTS_SET_UP}
Expand Down
6 changes: 6 additions & 0 deletions hadoop-ozone/dist/src/main/smoketest/s3/objecthead.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,12 @@ Head object in non existing bucket
${result} = Execute AWSS3APICli and checkrc head-object --bucket ${BUCKET}-non-existent --key ${PREFIX}/headobject/key=value/f1 255
Should contain ${result} 404
Should contain ${result} Not Found
Head object where path is a directory
${legacy-bucket} = Create legacy bucket
${result} = Execute AWSS3APICli and checkrc put-object --bucket ${legacy-bucket} --key ${PREFIX}/headobject/keyvalue/f1 --body /tmp/testfile 0
${result} = Execute AWSS3APICli and checkrc head-object --bucket ${legacy-bucket} --key ${PREFIX}/headobject/keyvalue/ 255
Should contain ${result} 404
Should contain ${result} Not Found

Head non existing key
${result} = Execute AWSS3APICli and checkrc head-object --bucket ${BUCKET} --key ${PREFIX}/non-existent 255
Expand Down
3 changes: 3 additions & 0 deletions hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -405,6 +405,9 @@ private OmKeyInfo readKeyInfo(OmKeyArgs args) throws IOException {
value = getOmKeyInfoFSO(volumeName, bucketName, keyName);
} else {
value = getOmKeyInfoDirectoryAware(volumeName, bucketName, keyName);
if (bucketLayout.isLegacy() && value != null && !value.isFile()) {
Copy link
Contributor

@duongkame duongkame Apr 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it right to put the check here? I think it's an S3 logic. OM always returns key requests, and it's up to the clients (S3 or OFS) to determine how the result is intercepted.

Copy link
Contributor Author

@kerneltime kerneltime Apr 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is true, but the intention here is not to change the behavior of LEGACY buckets. I considered doing it only in S3G, but there are 2 issues there, it (keeps the regression) changes the behavior of LEGACY without documenting and communicating it widely. Secondly, detecting directory objects is much more involved at the S3G level.
There need to be more work as part of the OBS and FSO bucket evolution to address these issues. For now, let's keep LEGACY the way it was.

value = null; // Legacy buckets do not report key info for directories
}
}
} catch (IOException ex) {
if (ex instanceof OMException) {
Expand Down