Skip to content

Conversation

@bogthe
Copy link
Contributor

@bogthe bogthe commented Feb 2, 2022

Picking the added changes straight into branch-3.3.2 for ease.

Idea is we can release this feature in 3.3.2 and then can clean up branch-3.3 by cherry-picking these changes on top of it. See rebase PR.

@sunchao @steveloughran

Add support for S3 Access Points. This provides extra security as it
ensures applications are not working with buckets belong to third parties.

To bind a bucket to an access point, set the access point (ap) ARN,
which must be done for each specific bucket, using the pattern

fs.s3a.bucket.$BUCKET.accesspoint.arn = ARN

* The global/bucket option `fs.s3a.accesspoint.required` to
mandate that buckets must declare their access point.
* This is not compatible with S3Guard.

Consult the documentation for further details.

Contributed by Bogdan Stolojan
…he#3516)


Follow-on to HADOOP-17198. Support S3 Access Points

Contributed by Bogdan Stolojan
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 9m 55s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 1s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 markdownlint 0m 0s markdownlint was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 8 new or modified test files.
_ branch-3.3.2 Compile Tests _
+0 🆗 mvndep 3m 34s Maven dependency ordering for branch
-1 ❌ mvninstall 29m 6s /branch-mvninstall-root.txt root in branch-3.3.2 failed.
-1 ❌ compile 11m 34s /branch-compile-root.txt root in branch-3.3.2 failed.
-0 ⚠️ checkstyle 2m 12s /buildtool-branch-checkstyle-root.txt The patch fails to run checkstyle in root
+1 💚 mvnsite 2m 19s branch-3.3.2 passed
+1 💚 javadoc 2m 11s branch-3.3.2 passed
+1 💚 spotbugs 3m 39s branch-3.3.2 passed
+1 💚 shadedclient 23m 26s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+0 🆗 mvndep 0m 29s Maven dependency ordering for patch
+1 💚 mvninstall 1m 38s the patch passed
-1 ❌ compile 11m 5s /patch-compile-root.txt root in the patch failed.
-1 ❌ javac 11m 5s /patch-compile-root.txt root in the patch failed.
+1 💚 blanks 0m 0s The patch has no blanks issues.
-0 ⚠️ checkstyle 2m 8s /buildtool-patch-checkstyle-root.txt The patch fails to run checkstyle in root
+1 💚 mvnsite 2m 9s the patch passed
+1 💚 xml 0m 1s The patch has no ill-formed XML file.
+1 💚 javadoc 1m 28s hadoop-common in the patch passed.
+1 💚 javadoc 0m 34s hadoop-tools_hadoop-aws generated 0 new + 62 unchanged - 1 fixed = 62 total (was 63)
+1 💚 spotbugs 3m 54s the patch passed
+1 💚 shadedclient 23m 33s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 17m 25s hadoop-common in the patch passed.
+1 💚 unit 2m 20s hadoop-aws in the patch passed.
+1 💚 asflicense 0m 39s The patch does not generate ASF License warnings.
156m 31s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3955/1/artifact/out/Dockerfile
GITHUB PR #3955
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml spotbugs checkstyle markdownlint
uname Linux 65b6d7522301 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision branch-3.3.2 / f1b4dec
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~18.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3955/1/testReport/
Max. process+thread count 3149 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common hadoop-tools/hadoop-aws U: .
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3955/1/console
versions git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@steveloughran
Copy link
Contributor

i have just checked out and tested this, it works for me

@steveloughran steveloughran changed the title Rebase Access Point feature onto branch-3.3.2 HADOOP-17198. Support S3 Access Points (#3260)branch-3.3.2) Feb 4, 2022
@steveloughran steveloughran changed the title HADOOP-17198. Support S3 Access Points (#3260)branch-3.3.2) HADOOP-17198. Support S3 Access Points (#3260) (branch-3.3.2) Feb 4, 2022
@sunchao
Copy link
Member

sunchao commented Feb 4, 2022

Cool, thanks for verifying @steveloughran . I think we can merge this then.

@sunchao sunchao merged commit 8cd8e43 into apache:branch-3.3.2 Feb 4, 2022
@sunchao
Copy link
Member

sunchao commented Feb 4, 2022

Thanks @bogthe ! merged to branch-3.3.2

@bogthe
Copy link
Contributor Author

bogthe commented Feb 7, 2022

Awesome, thanks for all the help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants