Skip to content

HDFS-16628 RBF: Correct target directory when move to trash for kerberos login user.#4424

Merged
Hexiaoqiao merged 2 commits intoapache:trunkfrom
zhangxiping1:HDFS-16628
Jun 15, 2022
Merged

HDFS-16628 RBF: Correct target directory when move to trash for kerberos login user.#4424
Hexiaoqiao merged 2 commits intoapache:trunkfrom
zhangxiping1:HDFS-16628

Conversation

@zhangxiping1
Copy link
Copy Markdown
Contributor

HDFS-16628 RBF: kerbose user remove Non-default namespace data failed

@hadoop-yetus
Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 41s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 1s codespell was not available.
+0 🆗 detsecrets 0m 1s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 37m 30s trunk passed
+1 💚 compile 1m 1s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 compile 0m 57s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 0m 53s trunk passed
+1 💚 mvnsite 1m 2s trunk passed
+1 💚 javadoc 1m 7s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javadoc 1m 19s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 1m 48s trunk passed
+1 💚 shadedclient 21m 5s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 41s the patch passed
+1 💚 compile 0m 44s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javac 0m 44s the patch passed
+1 💚 compile 0m 40s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 0m 40s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 27s the patch passed
+1 💚 mvnsite 0m 41s the patch passed
+1 💚 javadoc 0m 42s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javadoc 0m 57s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 1m 28s the patch passed
+1 💚 shadedclient 20m 32s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 21m 41s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 0m 54s The patch does not generate ASF License warnings.
118m 57s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4424/1/artifact/out/Dockerfile
GITHUB PR #4424
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux a003555ec08e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 61dbbdc
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4424/1/testReport/
Max. process+thread count 2752 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4424/1/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

Copy link
Copy Markdown
Contributor

@Hexiaoqiao Hexiaoqiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhangxiping1 Thanks for your report and contribution. Good catch here.
Leave some comments inline. FYI.

}

@Test
public void testMoveToTrashNoMountPointWithKerBoersUser() throws IOException,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

testMoveToTrashNoMountPointWithKerBoersUser -> testMoveToTrasWithKerberosUser?
a. IMO, this case is not related with NoMountPoint or not, right?
b. KerBoers -> Kerberos.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, thank you for your correction.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for a , yes

public void testMoveToTrashNoMountPointWithKerBoersUser() throws IOException,
URISyntaxException, InterruptedException {
//Constructs the structure of the KerBoers user name
String kerBoersUser = "randomUser/dev@HADOOP.COM";
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest to correct KerBoers to Kerberos. Both of the following spell issues.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha, ok

DFSClient currentUserClientNs1 = nn1Context.getClient();

currentUserClientNs0.setOwner("/", ugi.getShortUserName(), ugi.getShortUserName());
currentUserClientNs1.setOwner("/", ugi.getShortUserName(), ugi.getShortUserName());
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If only one DFSClient and NamenodeContext is enough for this unit test?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is a good idea to set the test user permissions for both subsets. Give test users permission to create both subCluster Trash directories, otherwise there may be some false positives.

Configuration routerConf = routerContext.getConf();
FileSystem fs = DFSTestUtil.getFileSystemAs(ugi, routerConf);
Trash trash = new Trash(fs, routerConf);
assertTrue(trash.moveToTrash(filePath));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest to check if the final trash directory entries are expected: /user/randomUser/.Trash/?

@zhangxiping1 zhangxiping1 changed the title HDFS-16628 RBF: kerbose user remove Non-default namespace data failed. HDFS-16628 RBF: kerberos user remove Non-default namespace data failed. Jun 13, 2022
@hadoop-yetus
Copy link
Copy Markdown

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 1m 22s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 40m 7s trunk passed
+1 💚 compile 0m 59s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 compile 0m 56s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 checkstyle 0m 49s trunk passed
+1 💚 mvnsite 0m 59s trunk passed
+1 💚 javadoc 1m 4s trunk passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javadoc 1m 7s trunk passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 1m 53s trunk passed
+1 💚 shadedclient 22m 39s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 0m 43s the patch passed
+1 💚 compile 0m 44s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javac 0m 44s the patch passed
+1 💚 compile 0m 41s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 javac 0m 41s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 checkstyle 0m 26s the patch passed
+1 💚 mvnsite 0m 44s the patch passed
+1 💚 javadoc 0m 43s the patch passed with JDK Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1
+1 💚 javadoc 1m 0s the patch passed with JDK Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
+1 💚 spotbugs 1m 34s the patch passed
+1 💚 shadedclient 22m 58s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 38m 48s hadoop-hdfs-rbf in the patch passed.
+1 💚 asflicense 0m 51s The patch does not generate ASF License warnings.
143m 2s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4424/2/artifact/out/Dockerfile
GITHUB PR #4424
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname Linux f24d6d7524f0 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 70ad0e0
Default Java Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Private Build-11.0.15+10-Ubuntu-0ubuntu0.20.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_312-8u312-b07-0ubuntu1~20.04-b07
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4424/2/testReport/
Max. process+thread count 2729 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs-rbf U: hadoop-hdfs-project/hadoop-hdfs-rbf
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4424/2/console
versions git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@zhangxiping1
Copy link
Copy Markdown
Contributor Author

cc @Hexiaoqiao It has been modified. Please help to merge it. Thank you very much

Copy link
Copy Markdown
Contributor

@Hexiaoqiao Hexiaoqiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. +1. Will commit if no more other comments for a while.

@zhangxiping1
Copy link
Copy Markdown
Contributor Author

Ok, thank you @Hexiaoqiao

@Hexiaoqiao Hexiaoqiao changed the title HDFS-16628 RBF: kerberos user remove Non-default namespace data failed. HDFS-16628 RBF: Correct target directory when move to trash for kerberos login user. Jun 15, 2022
@Hexiaoqiao Hexiaoqiao merged commit f8c7e67 into apache:trunk Jun 15, 2022
@Hexiaoqiao
Copy link
Copy Markdown
Contributor

Committed to trunk. Thanks @zhangxiping1 for your report and contribution.

xinglin pushed a commit to xinglin/hadoop that referenced this pull request Oct 5, 2022
…ros login user. (apache#4424). Contributed by Xiping Zhang.

(cherry picked from commit f8c7e67)
xinglin pushed a commit to xinglin/hadoop that referenced this pull request Oct 9, 2022
…ros login user. (apache#4424). Contributed by Xiping Zhang.

(cherry picked from commit f8c7e67)
HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants