HADOOP-17548. ABFS: Toggle Store Mkdirs request overwrite parameter (#2729) #2781

sumangala-patki · 2021-03-17T11:49:49Z

Contributed by Sumangala Patki.

(cherry picked from commit fe633d4)

…pache#2729) Contributed by Sumangala Patki. (cherry picked from commit fe633d4)

hadoop-yetus · 2021-03-17T13:26:19Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	21m 45s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	32m 19s		branch-3.3 passed
+1 💚	compile	0m 32s		branch-3.3 passed
+1 💚	checkstyle	0m 28s		branch-3.3 passed
+1 💚	mvnsite	0m 40s		branch-3.3 passed
+1 💚	javadoc	0m 29s		branch-3.3 passed
+1 💚	spotbugs	0m 59s		branch-3.3 passed
+1 💚	shadedclient	15m 48s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 29s		the patch passed
+1 💚	compile	0m 26s		the patch passed
+1 💚	javac	0m 26s		the patch passed
+1 💚	blanks	0m 1s		The patch has no blanks issues.
+1 💚	checkstyle	0m 17s		the patch passed
+1 💚	mvnsite	0m 29s		the patch passed
+1 💚	javadoc	0m 20s		the patch passed
+1 💚	spotbugs	1m 0s		the patch passed
+1 💚	shadedclient	15m 52s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 56s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 34s		The patch does not generate ASF License warnings.
		95m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/1/artifact/out/Dockerfile
GITHUB PR	#2781
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 2457a5eba562 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `4432e09`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~18.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/1/testReport/
Max. process+thread count	668 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/1/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-03-25T11:36:27Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	25m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	32m 41s		branch-3.3 passed
+1 💚	compile	0m 34s		branch-3.3 passed
+1 💚	checkstyle	0m 28s		branch-3.3 passed
+1 💚	mvnsite	0m 39s		branch-3.3 passed
+1 💚	javadoc	0m 30s		branch-3.3 passed
+1 💚	spotbugs	0m 59s		branch-3.3 passed
+1 💚	shadedclient	16m 5s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 29s		the patch passed
+1 💚	compile	0m 26s		the patch passed
+1 💚	javac	0m 26s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 18s		the patch passed
+1 💚	mvnsite	0m 29s		the patch passed
+1 💚	javadoc	0m 20s		the patch passed
+1 💚	spotbugs	1m 0s		the patch passed
+1 💚	shadedclient	15m 57s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 58s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 32s		The patch does not generate ASF License warnings.
		99m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/2/artifact/out/Dockerfile
GITHUB PR	#2781
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 7b26bfeb0012 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `492a040`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~18.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/2/testReport/
Max. process+thread count	534 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/2/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-03-29T13:05:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	24m 47s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	32m 36s		branch-3.3 passed
+1 💚	compile	0m 35s		branch-3.3 passed
+1 💚	checkstyle	0m 28s		branch-3.3 passed
+1 💚	mvnsite	0m 40s		branch-3.3 passed
+1 💚	javadoc	0m 31s		branch-3.3 passed
+1 💚	spotbugs	0m 59s		branch-3.3 passed
+1 💚	shadedclient	15m 58s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s		the patch passed
+1 💚	compile	0m 26s		the patch passed
+1 💚	javac	0m 26s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 17s		the patch passed
+1 💚	mvnsite	0m 29s		the patch passed
+1 💚	javadoc	0m 20s		the patch passed
+1 💚	spotbugs	1m 1s		the patch passed
+1 💚	shadedclient	15m 52s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 57s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 33s		The patch does not generate ASF License warnings.
		98m 54s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/3/artifact/out/Dockerfile
GITHUB PR	#2781
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux b651d22c9165 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `0258df9`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~18.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/3/testReport/
Max. process+thread count	612 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/3/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-03-31T10:08:54Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 4s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	37m 30s		branch-3.3 passed
+1 💚	compile	0m 30s		branch-3.3 passed
+1 💚	checkstyle	0m 22s		branch-3.3 passed
+1 💚	mvnsite	0m 35s		branch-3.3 passed
+1 💚	javadoc	0m 30s		branch-3.3 passed
+1 💚	spotbugs	1m 5s		branch-3.3 passed
+1 💚	shadedclient	17m 49s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 28s		the patch passed
+1 💚	compile	0m 23s		the patch passed
+1 💚	javac	0m 23s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 15s		the patch passed
+1 💚	mvnsite	0m 28s		the patch passed
+1 💚	javadoc	0m 18s		the patch passed
+1 💚	spotbugs	1m 0s		the patch passed
+1 💚	shadedclient	17m 21s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 2s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 30s		The patch does not generate ASF License warnings.
		82m 54s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/4/artifact/out/Dockerfile
GITHUB PR	#2781
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 63237ccda86f 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `288e5f8`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~18.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/4/testReport/
Max. process+thread count	511 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/4/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

sumangala-patki · 2021-04-12T05:02:20Z

TEST RESULTS

HNS Account Location: East US 2
NonHNS Account Location: East US 2, Central US

HNS-OAuth

[INFO] Tests run: 93, Failures: 0, Errors: 0, Skipped: 0
[ERROR]   ITestAzureBlobFileSystemFileStatus.testLastModifiedTime:140->Assert.assertTrue:42->Assert.fail:89 lastModifiedTime should be before createEndTime
[ERROR] Tests run: 513, Failures: 1, Errors: 0, Skipped: 70
[ERROR] Errors: 
[ERROR]   ITestAbfsReadWriteAndSeek.testReadAndWriteWithDifferentBufferSizesAndSeek:62->testReadWriteAndSeek:78 » TestTimedOut
[ERROR]   ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 » TestTimedOut
[ERROR] Tests run: 261, Failures: 0, Errors: 2, Skipped: 50

HNS-SharedKey

[INFO] Tests run: 93, Failures: 0, Errors: 0, Skipped: 0
[ERROR] Failures: 
[ERROR]   ITestAzureBlobFileSystemFileStatus.testLastModifiedTime:140->Assert.assertTrue:42->Assert.fail:89 lastModifiedTime should be before createEndTime
[ERROR] Tests run: 513, Failures: 1, Errors: 0, Skipped: 26
[ERROR] Errors: 
[ERROR]   ITestAbfsReadWriteAndSeek.testReadAndWriteWithDifferentBufferSizesAndSeek:62->testReadWriteAndSeek:78 » TestTimedOut
[ERROR]   ITestAbfsFileSystemContractDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 » TestTimedOut
[ERROR]   ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 » TestTimedOut 
[ERROR] Tests run: 261, Failures: 0, Errors: 3, Skipped: 40

NonHNS-SharedKey

[INFO] Tests run: 93, Failures: 0, Errors: 0, Skipped: 0
[ERROR] Failures: 
[ERROR]   ITestAzureBlobFileSystemFileStatus.testLastModifiedTime:140->Assert.assertTrue:42->Assert.fail:89 lastModifiedTime should be before createEndTime
[ERROR] Tests run: 513, Failures: 1, Errors: 0, Skipped: 248
[ERROR] Errors: 
[ERROR]   ITestAbfsReadWriteAndSeek.testReadAndWriteWithDifferentBufferSizesAndSeek:62->testReadWriteAndSeek:78 » TestTimedOut
[ERROR]   ITestAbfsFileSystemContractDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 » TestTimedOut
[ERROR]   ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 » TestTimedOut
[ERROR] Tests run: 261, Failures: 0, Errors: 3, Skipped: 40

Note: The timeouts in the above tests are commonly observed due to network issues and are not related to this PR. LMT failure is caused by diff in clock time between client and server (transient error).
Existing JIRAs for tracking the timeout failures in these tests: ITestAzureBlobFileSystemFileStatus LMT, AbstractContractDistCpTest, ITestAbfsReadWriteAndSeek

hadoop-yetus · 2021-04-16T13:03:15Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	21m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ branch-3.3 Compile Tests _
+1 💚	mvninstall	33m 40s		branch-3.3 passed
+1 💚	compile	0m 34s		branch-3.3 passed
+1 💚	checkstyle	0m 28s		branch-3.3 passed
+1 💚	mvnsite	0m 39s		branch-3.3 passed
+1 💚	javadoc	0m 30s		branch-3.3 passed
+1 💚	spotbugs	0m 58s		branch-3.3 passed
+1 💚	shadedclient	16m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s		the patch passed
+1 💚	compile	0m 26s		the patch passed
+1 💚	javac	0m 26s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 17s		the patch passed
+1 💚	mvnsite	0m 30s		the patch passed
+1 💚	javadoc	0m 19s		the patch passed
+1 💚	spotbugs	0m 59s		the patch passed
+1 💚	shadedclient	15m 43s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	2m 6s		hadoop-azure in the patch passed.
+1 💚	asflicense	0m 32s		The patch does not generate ASF License warnings.
		96m 46s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/6/artifact/out/Dockerfile
GITHUB PR	#2781
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 3dfb8bd08083 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	branch-3.3 / `374dc4f`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~18.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/6/testReport/
Max. process+thread count	694 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-azure U: hadoop-tools/hadoop-azure
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2781/6/console
versions	git=2.17.1 maven=3.6.0 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

sumangala-patki · 2021-04-19T05:54:26Z

TEST RESULTS ON CANARY ACCOUNTS

HNS-OAuth

[INFO] Tests run: 93, Failures: 0, Errors: 0, Skipped: 0
[WARNING] Tests run: 514, Failures: 0, Errors: 0, Skipped: 70
[ERROR] Errors: 
[ERROR]   ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 » TestTimedOut
[ERROR] Tests run: 261, Failures: 0, Errors: 1, Skipped: 50

HNS-SharedKey

[ERROR] Tests run: 93, Failures: 0, Errors: 0, Skipped: 0
[ERROR] Tests run: 514, Failures: 0, Errors: 0, Skipped: 26
[ERROR] Errors: 
[ERROR]   ITestAbfsFileSystemContractDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 » TestTimedOut
[ERROR]   ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 » TestTimedOut 
[ERROR] Tests run: 261, Failures: 0, Errors: 2, Skipped: 40

NonHNS-SharedKey

[INFO] Tests run: 93, Failures: 0, Errors: 0, Skipped: 0
[WARNING] Tests run: 514, Failures: 0, Errors: 0, Skipped: 248
[ERROR] Errors: 
[ERROR]   ITestAbfsFileSystemContractDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 » TestTimedOut
[ERROR]   ITestAbfsFileSystemContractSecureDistCp>AbstractContractDistCpTest.testDistCpWithIterator:631 » TestTimedOut
[ERROR] Tests run: 261, Failures: 0, Errors: 2, Skipped: 40

JIRAs for tracking timeout failures: AbstractContractDistCpTest

surendralilhore · 2021-05-10T06:19:27Z

+1

… parameter (apache#2729) (apache#2781) Contributed by Sumangala Patki. (cherry picked from commit fe633d4) Change-Id: Ibbd82f3cf2d28fa5438c9b1ee6b0e28217dc18df

HADOOP-17548. ABFS: Toggle Store Mkdirs request overwrite parameter (a…

4432e09

…pache#2729) Contributed by Sumangala Patki. (cherry picked from commit fe633d4)

Merge branch 'branch-3.3' into HADOOP-17548-backport

492a040

Merge branch 'branch-3.3' into HADOOP-17548-backport

0258df9

Merge branch 'branch-3.3' into HADOOP-17548-backport

288e5f8

sumangala17 added 2 commits April 7, 2021 11:47

Merge branch 'branch-3.3' into HADOOP-17548-backport

86731e9

Merge branch 'branch-3.3' into HADOOP-17548-backport

e2c885b

Merge branch 'branch-3.3' into HADOOP-17548-backport

374dc4f

sumangala-patki marked this pull request as ready for review April 19, 2021 05:54

surendralilhore merged commit b20bc66 into apache:branch-3.3 May 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-17548. ABFS: Toggle Store Mkdirs request overwrite parameter (#2729) #2781

HADOOP-17548. ABFS: Toggle Store Mkdirs request overwrite parameter (#2729) #2781

Uh oh!

sumangala-patki commented Mar 17, 2021

Uh oh!

hadoop-yetus commented Mar 17, 2021

Uh oh!

hadoop-yetus commented Mar 25, 2021

Uh oh!

hadoop-yetus commented Mar 29, 2021

Uh oh!

hadoop-yetus commented Mar 31, 2021

Uh oh!

sumangala-patki commented Apr 12, 2021 •

edited

Loading

Uh oh!

hadoop-yetus commented Apr 16, 2021

Uh oh!

sumangala-patki commented Apr 19, 2021

Uh oh!

surendralilhore commented May 10, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

HADOOP-17548. ABFS: Toggle Store Mkdirs request overwrite parameter (#2729) #2781

HADOOP-17548. ABFS: Toggle Store Mkdirs request overwrite parameter (#2729) #2781

Uh oh!

Conversation

sumangala-patki commented Mar 17, 2021

Uh oh!

hadoop-yetus commented Mar 17, 2021

Uh oh!

hadoop-yetus commented Mar 25, 2021

Uh oh!

hadoop-yetus commented Mar 29, 2021

Uh oh!

hadoop-yetus commented Mar 31, 2021

Uh oh!

sumangala-patki commented Apr 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hadoop-yetus commented Apr 16, 2021

Uh oh!

sumangala-patki commented Apr 19, 2021

Uh oh!

surendralilhore commented May 10, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sumangala-patki commented Apr 12, 2021 •

edited

Loading