HDFS-16128: Added support for saving/loading an FS Image for fine-grain locking #3201

xinglin · 2021-07-13T22:28:22Z

Test:
We create 1 million dirs and then stop and re-start the namenode. Namenode restarts successfully.

xinglin@xinglin-mn1 ~/p/h/h/t/hadoop-3.4.0-SNAPSHOT (fgl-saveloadfs)> ./bin/hadoop org.apache.hadoop.hdfs.server.namenode.NNThroughputBenchmark -fs hdfs://localhost:9000 -op mkdirs -threads 50 -dirs 1000000 -dirsPerDir 100 -keepResults
2021-07-13 14:59:39,703 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2021-07-13 14:59:40,264 INFO namenode.NNThroughputBenchmark: Starting benchmark: mkdirs
2021-07-13 14:59:40,412 INFO namenode.NNThroughputBenchmark: Generate 1000000 inputs for mkdirs
2021-07-13 14:59:40,669 INFO namenode.NNThroughputBenchmark: Log level = ERROR
2021-07-13 14:59:40,894 INFO namenode.NNThroughputBenchmark: Starting 1000000 mkdirs(s).
2021-07-13 15:00:15,524 INFO namenode.NNThroughputBenchmark:
2021-07-13 15:00:15,524 INFO namenode.NNThroughputBenchmark: --- mkdirs inputs ---
2021-07-13 15:00:15,524 INFO namenode.NNThroughputBenchmark: nrDirs = 1000000
2021-07-13 15:00:15,524 INFO namenode.NNThroughputBenchmark: nrThreads = 50
2021-07-13 15:00:15,524 INFO namenode.NNThroughputBenchmark: nrDirsPerDir = 100
2021-07-13 15:00:15,524 INFO namenode.NNThroughputBenchmark: --- mkdirs stats  ---
2021-07-13 15:00:15,524 INFO namenode.NNThroughputBenchmark: # operations: 1000000
2021-07-13 15:00:15,524 INFO namenode.NNThroughputBenchmark: Elapsed Time: 33637
2021-07-13 15:00:15,524 INFO namenode.NNThroughputBenchmark:  Ops per sec: 29729.16728602432
2021-07-13 15:00:15,524 INFO namenode.NNThroughputBenchmark: Average Time: 1
xinglin@xinglin-mn1 ~/p/h/h/t/hadoop-3.4.0-SNAPSHOT (fgl-saveloadfs)> ./bin/hdfs dfs -ls -R / | wc
2021-07-13 15:01:23,813 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 1010002 8080016 156814967
xinglin@xinglin-mn1 ~/p/h/h/t/hadoop-3.4.0-SNAPSHOT (fgl-saveloadfs)> ./sbin/stop-dfs.sh
Stopping namenodes on [localhost]
Stopping datanodes
Stopping secondary namenodes [localhost]
2021-07-13 15:09:47,518 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
xinglin@xinglin-mn1 ~/p/h/h/t/hadoop-3.4.0-SNAPSHOT (fgl-saveloadfs)> ./sbin/start-dfs.sh
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [localhost]
2021-07-13 15:10:05,055 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
xinglin@xinglin-mn1 ~/p/h/h/t/hadoop-3.4.0-SNAPSHOT (fgl-saveloadfs)> jps
61745 DataNode
61635 NameNode
61878 SecondaryNameNode
95382 CmdLineAppRunner
65082
62030 Jps
xinglin@xinglin-mn1 ~/p/h/h/t/hadoop-3.4.0-SNAPSHOT (fgl-saveloadfs)> ./bin/hdfs dfs -ls -R / | wc
2021-07-13 15:10:20,998 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
 1010002 8080016 156814967

Log output from the namenode.

xinglin@xinglin-mn1 ~/p/h/h/t/h/logs (fgl-saveloadfs)> grep LOADING hadoop-xinglin-namenode-xinglin-mn1.linkedin.biz.log
INFO org.apache.hadoop.hdfs.server.namenode.FSImage: LOADING_FSIMAGE: loaded 1 inodes into inodeMap
INFO org.apache.hadoop.hdfs.server.namenode.FSImage: LOADING_EDITS: loaded 1 inodes into inodeMap
INFO org.apache.hadoop.hdfs.server.namenode.FSImage: LOADING_FSIMAGE: loaded 1010003 inodes into inodeMap
INFO org.apache.hadoop.hdfs.server.namenode.FSImage: LOADING_EDITS: loaded 1010003 inodes into inodeMap

hadoop-yetus · 2021-07-13T22:58:50Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	13m 12s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 1 new or modified test files.
			_ fgl Compile Tests _
+0 🆗	mvndep	3m 33s		Maven dependency ordering for branch
-1 ❌	mvninstall	0m 35s	/branch-mvninstall-root.txt	root in fgl failed.
-1 ❌	compile	0m 17s	/branch-compile-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	root in fgl failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	compile	0m 18s	/branch-compile-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root in fgl failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-0 ⚠️	checkstyle	0m 18s	/buildtool-branch-checkstyle-root.txt	The patch fails to run checkstyle in root
-1 ❌	mvnsite	1m 18s	/branch-mvnsite-hadoop-common-project_hadoop-common.txt	hadoop-common in fgl failed.
-1 ❌	mvnsite	0m 43s	/branch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in fgl failed.
-1 ❌	javadoc	0m 16s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-common in fgl failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	javadoc	0m 14s	/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-hdfs in fgl failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	javadoc	0m 13s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-common in fgl failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌	javadoc	0m 13s	/branch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-hdfs in fgl failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌	spotbugs	0m 13s	/branch-spotbugs-hadoop-common-project_hadoop-common.txt	hadoop-common in fgl failed.
-1 ❌	spotbugs	0m 13s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in fgl failed.
-1 ❌	shadedclient	4m 5s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s		Maven dependency ordering for patch
-1 ❌	mvninstall	0m 13s	/patch-mvninstall-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	mvninstall	0m 12s	/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	compile	0m 18s	/patch-compile-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	root in the patch failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	javac	0m 18s	/patch-compile-root-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	root in the patch failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	compile	0m 18s	/patch-compile-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌	javac	0m 18s	/patch-compile-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 15s	/buildtool-patch-checkstyle-root.txt	The patch fails to run checkstyle in root
-1 ❌	mvnsite	0m 13s	/patch-mvnsite-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	mvnsite	0m 13s	/patch-mvnsite-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	javadoc	0m 14s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	javadoc	0m 14s	/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkUbuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.txt	hadoop-hdfs in the patch failed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04.
-1 ❌	javadoc	0m 13s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-common in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌	javadoc	0m 13s	/patch-javadoc-hadoop-hdfs-project_hadoop-hdfs-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	hadoop-hdfs in the patch failed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.
-1 ❌	spotbugs	0m 13s	/patch-spotbugs-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	spotbugs	0m 13s	/patch-spotbugs-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
-1 ❌	shadedclient	3m 35s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 13s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
-1 ❌	unit	0m 13s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch failed.
+1 💚	asflicense	0m 27s		The patch does not generate ASF License warnings.
		29m 15s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/1/artifact/out/Dockerfile
GITHUB PR	#3201
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 15cfafd79475 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	fgl / b5d03cea45b0d6fd505413da59190e31d36e24dc
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/1/testReport/
Max. process+thread count	90 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/1/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

…y Xing Lin. (apache#3197)

…nedGSet

hadoop-yetus · 2021-07-17T13:48:50Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ fgl Compile Tests _
+0 🆗	mvndep	13m 0s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 1s		fgl passed
+1 💚	compile	20m 58s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	18m 10s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 49s		fgl passed
+1 💚	mvnsite	3m 16s		fgl passed
+1 💚	javadoc	2m 19s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 28s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	2m 29s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in fgl has 3 extant spotbugs warnings.
-1 ❌	spotbugs	3m 23s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html	hadoop-hdfs-project/hadoop-hdfs in fgl has 2 extant spotbugs warnings.
-1 ❌	shadedclient	16m 34s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 7s		the patch passed
+1 💚	compile	20m 17s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	20m 17s		the patch passed
+1 💚	compile	18m 2s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	18m 2s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 1 new + 1792 unchanged - 1 fixed = 1793 total (was 1793)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 42s	/results-checkstyle-root.txt	root: The patch generated 6 new + 116 unchanged - 0 fixed = 122 total (was 116)
+1 💚	mvnsite	3m 14s		the patch passed
+1 💚	javadoc	2m 17s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 23s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	3m 37s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 2 new + 2 unchanged - 0 fixed = 4 total (was 2)
-1 ❌	shadedclient	16m 40s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 2s		hadoop-common in the patch passed.
-1 ❌	unit	280m 10s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
-1 ❌	asflicense	1m 26s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		482m 47s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	Dead store to inode in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long) At INodeMap.java:org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long) At INodeMap.java:[line 250]
	Found reliance on default encoding in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long):in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long): String.getBytes() At INodeMap.java:[line 300]
Failed junit tests	hadoop.hdfs.TestReadStripedFileWithDecodingDeletedData
	hadoop.hdfs.server.namenode.TestAddBlock
	hadoop.hdfs.TestDFSShell
	hadoop.fs.viewfs.TestViewFSOverloadSchemeWithMountTableConfigInHDFS
	hadoop.hdfs.server.datanode.TestReadOnlySharedStorage
	hadoop.hdfs.server.namenode.TestRedudantBlocks
	hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistPolicy
	hadoop.hdfs.TestErasureCodingExerciseAPIs
	hadoop.hdfs.server.namenode.TestFsckWithMultipleNameNodes
	hadoop.hdfs.TestClientProtocolForPipelineRecovery
	hadoop.hdfs.TestBlockMissingException
	hadoop.fs.TestUrlStreamHandler
	hadoop.hdfs.TestEncryptionZonesWithHA
	hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS
	hadoop.hdfs.TestSnapshotCommands
	hadoop.hdfs.server.datanode.TestDataNodeVolumeMetrics
	hadoop.fs.TestFcHdfsCreateMkdir
	hadoop.hdfs.TestPipelines
	hadoop.hdfs.shortcircuit.TestShortCircuitCache
	hadoop.hdfs.server.datanode.fsdataset.impl.TestFsVolumeList
	hadoop.hdfs.server.namenode.TestReencryption
	hadoop.hdfs.TestGetFileChecksum
	hadoop.hdfs.TestRestartDFS
	hadoop.hdfs.TestLargeBlock
	hadoop.hdfs.TestDFSMkdirs
	hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocks
	hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy
	hadoop.hdfs.TestWriteReadStripedFile
	hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery
	hadoop.hdfs.TestDFSStripedOutputStreamWithFailure
	hadoop.hdfs.server.datanode.fsdataset.impl.TestReplicaCachingGetSpaceUsed
	hadoop.hdfs.TestFileConcurrentReader
	hadoop.hdfs.TestDataStream
	hadoop.hdfs.TestErasureCodingPoliciesWithRandomECPolicy
	hadoop.hdfs.tools.TestDFSAdmin
	hadoop.hdfs.server.namenode.TestNameNodeXAttr
	hadoop.hdfs.client.impl.TestClientBlockVerification
	hadoop.fs.contract.hdfs.TestHDFSContractConcat
	hadoop.hdfs.TestSafeModeWithStripedFile
	hadoop.hdfs.security.TestDelegationToken
	hadoop.hdfs.TestBlockStoragePolicy
	hadoop.hdfs.TestHFlush
	hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash
	hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
	hadoop.hdfs.TestDistributedFileSystem
	hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter
	hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer
	hadoop.hdfs.TestClientReportBadBlock
	hadoop.hdfs.TestMultiThreadedHflush
	hadoop.hdfs.server.datanode.fsdataset.impl.TestScrLazyPersistFiles
	hadoop.hdfs.TestErasureCodeBenchmarkThroughput
	hadoop.hdfs.server.datanode.TestDnRespectsBlockReportSplitThreshold
	hadoop.hdfs.server.namenode.web.resources.TestWebHdfsCreatePermissions
	hadoop.hdfs.server.datanode.fsdataset.impl.TestProvidedImpl
	hadoop.hdfs.security.TestDelegationTokenForProxyUser
	hadoop.hdfs.TestDFSStripedOutputStream
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshot
	hadoop.hdfs.server.namenode.TestParallelImageWrite
	hadoop.tools.TestJMXGet
	hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
	hadoop.hdfs.server.namenode.snapshot.TestUpdatePipelineWithSnapshots
	hadoop.hdfs.server.namenode.ha.TestFailoverWithBlockTokensEnabled
	hadoop.hdfs.server.namenode.TestNameNodeRecovery
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForStoragePolicy
	hadoop.hdfs.server.datanode.TestDataNodeMetrics
	hadoop.hdfs.server.namenode.TestFSImageWithAcl
	hadoop.hdfs.TestDFSStripedInputStreamWithRandomECPolicy
	hadoop.hdfs.TestDFSStorageStateRecovery
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotFileLength
	hadoop.fs.viewfs.TestViewFsFileStatusHdfs
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
	hadoop.fs.contract.hdfs.TestHDFSContractSeek
	hadoop.hdfs.TestRenameWhileOpen
	hadoop.hdfs.server.namenode.TestFSNamesystem
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
	hadoop.hdfs.TestStoragePolicyPermissionSettings
	hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup
	hadoop.fs.TestFcHdfsSetUMask
	hadoop.hdfs.TestEncryptedTransfer
	hadoop.hdfs.TestStateAlignmentContextWithHA
	hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory
	hadoop.hdfs.server.datanode.TestCachingStrategy
	hadoop.hdfs.TestPread
	hadoop.hdfs.client.impl.TestBlockReaderLocal
	hadoop.cli.TestHDFSCLI
	hadoop.hdfs.server.namenode.ha.TestDNFencing
	hadoop.hdfs.TestWriteRead
	hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
	hadoop.fs.contract.hdfs.TestHDFSContractOpen
	hadoop.hdfs.TestFileCorruption
	hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages
	hadoop.hdfs.TestReadStripedFileWithDecodingCorruptData
	hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaPlacement
	hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics
	hadoop.hdfs.server.namenode.ha.TestPendingCorruptDnMessages
	hadoop.hdfs.web.TestFSMainOperationsWebHdfs
	hadoop.hdfs.server.namenode.sps.TestStoragePolicySatisfierWithStripedFile
	hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyWriter
	hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
	hadoop.hdfs.server.namenode.metrics.TestNNMetricFilesInGetListingOps
	hadoop.hdfs.TestDFSOutputStream
	hadoop.hdfs.TestRead
	hadoop.hdfs.TestDFSInotifyEventInputStream
	hadoop.hdfs.TestFileAppend3
	hadoop.hdfs.TestCrcCorruption
	hadoop.hdfs.TestApplyingStoragePolicy
	hadoop.hdfs.server.diskbalancer.TestDiskBalancerRPC
	hadoop.hdfs.server.namenode.TestListCorruptFileBlocks
	hadoop.hdfs.server.namenode.TestFSImage
	hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing
	hadoop.hdfs.TestFileCreationDelete
	hadoop.hdfs.server.namenode.snapshot.TestOrderedSnapshotDeletion
	hadoop.hdfs.TestDFSUpgrade
	hadoop.hdfs.server.namenode.TestStorageRestore
	hadoop.hdfs.server.datanode.TestBlockRecovery2
	hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
	hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
	hadoop.fs.TestFcHdfsPermission
	hadoop.hdfs.web.TestWebHDFS
	hadoop.hdfs.server.namenode.TestPersistentStoragePolicySatisfier
	hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeWithHdfsScheme
	hadoop.hdfs.TestDecommissionWithBackoffMonitor
	hadoop.hdfs.server.blockmanagement.TestComputeInvalidateWork
	hadoop.hdfs.TestParallelShortCircuitLegacyRead
	hadoop.hdfs.server.namenode.ha.TestHAAppend
	hadoop.hdfs.server.namenode.ha.TestHASafeMode
	hadoop.security.TestPermission
	hadoop.hdfs.TestLease
	hadoop.fs.TestSymlinkHdfsFileContext
	hadoop.hdfs.TestMaintenanceState
	hadoop.hdfs.TestFileAppend
	hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
	hadoop.hdfs.TestRollingUpgradeRollback
	hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor
	hadoop.fs.viewfs.TestViewFileSystemLinkRegex
	hadoop.hdfs.TestTrashWithEncryptionZones
	hadoop.fs.viewfs.TestViewFileSystemHdfs
	hadoop.hdfs.TestClose
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy
	hadoop.hdfs.server.datanode.TestDataNodeMXBean
	hadoop.hdfs.server.namenode.TestHostsFiles
	hadoop.hdfs.server.balancer.TestBalancerService
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl
	hadoop.fs.TestSWebHdfsFileContextMainOperations
	hadoop.hdfs.tools.TestStoragePolicyCommands
	hadoop.hdfs.server.datanode.fsdataset.impl.TestDatanodeRestart
	hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
	hadoop.hdfs.TestSetTimes
	hadoop.fs.viewfs.TestViewFileSystemOverloadSchemeHdfsFileSystemContract
	hadoop.hdfs.server.namenode.TestFileContextXAttr
	hadoop.hdfs.server.balancer.TestBalancerRPCDelay
	hadoop.hdfs.server.namenode.snapshot.TestFSImageWithOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.ha.TestHAMetrics
	hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
	hadoop.hdfs.server.namenode.ha.TestPipelinesFailover
	hadoop.hdfs.TestFileCreation
	hadoop.hdfs.client.impl.TestBlockReaderFactory
	hadoop.hdfs.server.datanode.TestDeleteBlockPool
	hadoop.hdfs.TestUnsetAndChangeDirectoryEcPolicy
	hadoop.hdfs.server.namenode.TestHDFSConcat
	hadoop.hdfs.TestDFSRemove
	hadoop.hdfs.TestSafeMode
	hadoop.hdfs.server.namenode.TestFSDirectory
	hadoop.hdfs.server.blockmanagement.TestPendingReconstruction
	hadoop.hdfs.TestDataTransferKeepalive
	hadoop.hdfs.TestFileLengthOnClusterRestart
	hadoop.hdfs.server.namenode.TestGetBlockLocations
	hadoop.hdfs.TestDistributedFileSystemWithECFileWithRandomECPolicy
	hadoop.hdfs.TestDFSRollback
	hadoop.fs.loadGenerator.TestLoadGenerator
	hadoop.hdfs.server.datanode.TestBPOfferService
	hadoop.hdfs.TestReconstructStripedFile
	hadoop.hdfs.server.namenode.TestBlockPlacementPolicyRackFaultTolerant
	hadoop.hdfs.TestLeaseRecovery2
	hadoop.hdfs.server.datanode.TestBlockHasMultipleReplicasOnSameDN
	hadoop.hdfs.TestDeadNodeDetection
	hadoop.hdfs.TestHDFSTrash
	hadoop.hdfs.server.namenode.TestRefreshBlockPlacementPolicy
	hadoop.hdfs.TestFileStatus
	hadoop.hdfs.server.namenode.TestLargeDirectoryDelete
	hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes
	hadoop.hdfs.TestGetBlocks
	hadoop.hdfs.server.namenode.ha.TestUpdateBlockTailing
	hadoop.hdfs.server.namenode.TestMetaSave
	hadoop.hdfs.TestDFSUpgradeFromImage
	hadoop.hdfs.crypto.TestHdfsCryptoStreams
	hadoop.hdfs.tools.TestDebugAdmin
	hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
	hadoop.hdfs.TestReservedRawPaths
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotReplication
	hadoop.hdfs.server.blockmanagement.TestBlockManager
	hadoop.hdfs.TestSeekBug
	hadoop.cli.TestAclCLI
	hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks
	hadoop.hdfs.server.namenode.TestNamenodeStorageDirectives
	hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer
	hadoop.hdfs.TestParallelShortCircuitRead
	hadoop.hdfs.TestDFSInputStreamBlockLocations
	hadoop.hdfs.TestFSOutputSummer
	hadoop.hdfs.server.diskbalancer.TestDiskBalancer
	hadoop.hdfs.TestDFSRename
	hadoop.hdfs.server.namenode.TestNameNodeMXBean
	hadoop.fs.contract.hdfs.TestHDFSContractCreate
	hadoop.hdfs.TestReconstructStripedFileWithValidator
	hadoop.hdfs.web.TestHttpsFileSystem
	hadoop.hdfs.TestMultipleNNPortQOP
	hadoop.hdfs.TestConnCache
	hadoop.fs.viewfs.TestViewFsHdfs
	hadoop.hdfs.server.datanode.TestIncrementalBrVariations
	hadoop.hdfs.server.blockmanagement.TestSequentialBlockGroupId
	hadoop.hdfs.TestInjectionForSimulatedStorage
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary
	hadoop.hdfs.server.namenode.TestDeleteRace
	hadoop.hdfs.TestDFSInputStream
	hadoop.hdfs.TestFileAppend2
	hadoop.hdfs.TestLeaseRecovery
	hadoop.hdfs.TestErasureCodingMultipleRacks
	hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistLockedMemory
	hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
	hadoop.hdfs.TestParallelUnixDomainRead
	hadoop.hdfs.TestDataTransferProtocol
	hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped
	hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
	hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles
	hadoop.fs.permission.TestStickyBit
	hadoop.hdfs.TestDecommission
	hadoop.hdfs.server.namenode.TestAddStripedBlocks
	hadoop.hdfs.server.namenode.TestAddBlockRetry
	hadoop.hdfs.server.namenode.snapshot.TestRenameWithOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.ha.TestBootstrapStandby
	hadoop.hdfs.server.namenode.TestSnapshotPathINodes
	hadoop.hdfs.TestSetrepDecreasing
	hadoop.hdfs.server.datanode.fsdataset.impl.TestSpaceReservation
	hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot
	hadoop.fs.contract.hdfs.TestHDFSContractMkdir
	hadoop.hdfs.server.blockmanagement.TestErasureCodingCorruption
	hadoop.hdfs.server.namenode.ha.TestStandbyIsHot
	hadoop.hdfs.TestAppendDifferentChecksum
	hadoop.hdfs.server.datanode.TestCorruptMetadataFile
	hadoop.fs.shell.TestHdfsTextCommand
	hadoop.hdfs.TestModTime
	hadoop.hdfs.TestDFSStripedInputStream
	hadoop.hdfs.server.blockmanagement.TestReconstructStripedBlocksWithRackAwareness
	hadoop.hdfs.TestDFSClientExcludedNodes
	hadoop.hdfs.TestBlockTokenWrappingQOP
	hadoop.hdfs.TestReplication
	hadoop.fs.TestSymlinkHdfsFileSystem
	hadoop.hdfs.TestViewDistributedFileSystemWithMountLinks
	hadoop.hdfs.server.namenode.TestNamenodeRetryCache
	hadoop.hdfs.TestReconstructStripedFileWithRandomECPolicy
	hadoop.hdfs.TestHdfsAdmin
	hadoop.hdfs.server.namenode.snapshot.TestGetContentSummaryWithSnapshot
	hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
	hadoop.fs.viewfs.TestViewFileSystemLinkFallback
	hadoop.hdfs.TestDFSClientFailover
	hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotMetrics
	hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
	hadoop.hdfs.TestTrashWithSecureEncryptionZones
	hadoop.hdfs.TestParallelRead
	hadoop.hdfs.web.TestWebHDFSXAttr
	hadoop.hdfs.server.blockmanagement.TestPendingDataNodeMessages
	hadoop.fs.viewfs.TestViewFileSystemWithTruncate
	hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
	hadoop.hdfs.TestEncryptionZonesWithKMS
	hadoop.hdfs.server.datanode.TestDataNodeFaultInjector
	hadoop.hdfs.server.datanode.TestBatchIbr
	hadoop.hdfs.server.namenode.TestNameNodeAcl
	hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication
	hadoop.hdfs.TestReadStripedFileWithDecoding
	hadoop.hdfs.server.namenode.TestAuditLoggerWithCommands
	hadoop.hdfs.TestListFilesInDFS
	hadoop.fs.contract.hdfs.TestHDFSContractAppend
	hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA
	hadoop.hdfs.client.impl.TestBlockReaderRemote
	hadoop.hdfs.TestReplaceDatanodeFailureReplication
	hadoop.cli.TestCryptoAdminCLI
	hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot
	hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
	hadoop.fs.TestSymlinkHdfsDisable
	hadoop.fs.TestEnhancedByteBufferAccess
	hadoop.hdfs.TestFsShellPermission
	hadoop.fs.contract.hdfs.TestHDFSContractMultipartUploader
	hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy
	hadoop.hdfs.server.namenode.TestEditLogRace
	hadoop.hdfs.server.namenode.TestBackupNode
	hadoop.hdfs.server.namenode.TestDecommissioningStatus
	hadoop.hdfs.server.blockmanagement.TestSequentialBlockId
	hadoop.hdfs.server.namenode.TestReconstructStripedBlocks
	hadoop.hdfs.server.namenode.TestReencryptionWithKMS
	hadoop.hdfs.server.namenode.TestFSEditLogLoader
	hadoop.hdfs.TestHDFSFileSystemContract
	hadoop.fs.TestUnbuffer
	hadoop.hdfs.TestDFSPermission
	hadoop.hdfs.TestErasureCodingPolicies
	hadoop.hdfs.server.datanode.TestDiskError
	hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus
	hadoop.hdfs.TestDFSFinalize
	hadoop.hdfs.server.datanode.TestDataNodeRollingUpgrade
	hadoop.hdfs.TestEncryptionZones
	hadoop.hdfs.TestFileAppendRestart
	hadoop.hdfs.TestFileStatusWithRandomECPolicy
	hadoop.hdfs.TestParallelShortCircuitReadNoChecksum
	hadoop.hdfs.TestDFSStripedOutputStreamWithRandomECPolicy
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.hdfs.TestDFSStripedOutputStreamUpdatePipeline
	hadoop.hdfs.server.blockmanagement.TestNodeCount
	hadoop.hdfs.server.datanode.fsdataset.impl.TestInterDatanodeProtocol
	hadoop.hdfs.server.namenode.TestCommitBlockWithInvalidGenStamp
	hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA
	hadoop.hdfs.server.namenode.TestProcessCorruptBlocks
	hadoop.hdfs.TestFSInputChecker
	hadoop.hdfs.web.TestWebHDFSForHA
	hadoop.hdfs.server.namenode.TestFileContextAcl
	hadoop.hdfs.server.datanode.TestNNHandlesCombinedBlockReport
	hadoop.hdfs.TestExtendedAcls
	hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs
	hadoop.hdfs.server.namenode.TestSaveNamespace
	hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots
	hadoop.hdfs.server.namenode.TestStripedINodeFile
	hadoop.hdfs.tools.TestViewFSStoragePolicyCommands
	hadoop.hdfs.server.namenode.TestDecommissioningStatusWithBackoffMonitor
	hadoop.hdfs.TestWriteConfigurationToDFS
	hadoop.hdfs.TestRollingUpgradeDowngrade
	hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff
	hadoop.hdfs.server.datanode.TestBlockReplacement
	hadoop.fs.viewfs.TestViewFsDefaultValue
	hadoop.hdfs.TestAppendSnapshotTruncate
	hadoop.hdfs.TestParallelShortCircuitReadUnCached
	hadoop.hdfs.server.namenode.TestStartup
	hadoop.hdfs.TestReplaceDatanodeOnFailure
	hadoop.hdfs.server.datanode.TestBlockScanner
	hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR
	hadoop.hdfs.server.namenode.TestListOpenFiles
	hadoop.hdfs.TestWriteBlockGetsBlockLengthHint
	hadoop.fs.contract.hdfs.TestHDFSContractPathHandle
	hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier
	hadoop.hdfs.server.mover.TestStorageMover
	hadoop.fs.contract.hdfs.TestHDFSContractDelete
	hadoop.hdfs.TestFileAppend4
	hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
	hadoop.hdfs.server.namenode.ha.TestQuotasWithHA
	hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureToleration
	hadoop.hdfs.TestAclsEndToEnd
	hadoop.hdfs.TestStripedFileAppend
	hadoop.hdfs.web.TestWebHdfsFileSystemContract
	hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes
	hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
	hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer
	hadoop.hdfs.server.datanode.TestHSync
	hadoop.hdfs.TestLocalDFS
	hadoop.hdfs.server.datanode.TestTriggerBlockReport
	hadoop.hdfs.TestMiniDFSCluster
	hadoop.fs.viewfs.TestViewFsAtHdfsRoot
	hadoop.hdfs.server.datanode.fsdataset.impl.TestPmemCacheRecovery
	hadoop.hdfs.server.namenode.TestNestedEncryptionZones
	hadoop.hdfs.TestQuota
	hadoop.hdfs.web.TestWebHdfsTokens
	hadoop.hdfs.tools.TestStoragePolicySatisfyAdminCommands
	hadoop.hdfs.server.datanode.TestTransferRbw
	hadoop.hdfs.TestReadStripedFileWithMissingBlocks
	hadoop.hdfs.tools.TestWebHDFSStoragePolicyCommands
	hadoop.hdfs.server.namenode.TestQuotaByStorageType
	hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
	hadoop.hdfs.TestExternalBlockReader
	hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
	hadoop.hdfs.TestDFSStartupVersions
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerWithStripedBlocks
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
	hadoop.hdfs.server.namenode.snapshot.TestSnapshot
	hadoop.hdfs.TestListFilesInFileContext
	hadoop.hdfs.server.namenode.TestAuditLogger
	hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs
	hadoop.hdfs.server.blockmanagement.TestCorruptionWithFailover
	hadoop.hdfs.TestDatanodeReport
	hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy
	hadoop.hdfs.TestDistributedFileSystemWithECFile
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters
	hadoop.fs.TestGlobPaths
	hadoop.hdfs.TestFileStatusWithDefaultECPolicy
	hadoop.fs.contract.hdfs.TestHDFSContractRename
	hadoop.cli.TestAclCLIWithPosixAclInheritance
	hadoop.hdfs.TestLeaseRecoveryStriped
	hadoop.security.TestPermissionSymlinks
	hadoop.hdfs.server.namenode.ha.TestHAStateTransitions
	hadoop.fs.contract.hdfs.TestHDFSContractUnbuffer
	hadoop.hdfs.server.namenode.TestFSNamesystemLock
	hadoop.hdfs.server.datanode.TestNNHandlesBlockReportPerStorage
	hadoop.hdfs.server.namenode.TestNameEditsConfigs
	hadoop.hdfs.server.blockmanagement.TestBlockInfoStriped
	hadoop.hdfs.TestSecureEncryptionZoneWithKMS
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.server.diskbalancer.command.TestDiskBalancerCommand
	hadoop.hdfs.server.namenode.TestFsck
	hadoop.hdfs.server.namenode.TestINodeFile
	hadoop.hdfs.server.namenode.snapshot.TestSetQuotaWithSnapshot
	hadoop.hdfs.server.namenode.TestFSImageWithXAttr
	hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
	hadoop.hdfs.web.TestWebHDFSAcl
	hadoop.hdfs.server.namenode.TestAuditLogs
	hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots
	hadoop.hdfs.TestAbandonBlock
	hadoop.hdfs.server.namenode.TestQuotaWithStripedBlocksWithRandomECPolicy
	hadoop.hdfs.TestDecommissionWithStriped
	hadoop.hdfs.server.balancer.TestBalancer
	hadoop.hdfs.server.namenode.TestGetContentSummaryWithPermission
	hadoop.hdfs.server.namenode.ha.TestStandbyBlockManagement
	hadoop.hdfs.TestDatanodeLayoutUpgrade
	hadoop.hdfs.server.namenode.TestLeaseManager
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotRename
	hadoop.hdfs.server.namenode.TestUpgradeDomainBlockPlacementPolicy
	hadoop.hdfs.server.datanode.fsdataset.impl.TestCacheByPmemMappableBlockLoader
	hadoop.hdfs.web.TestWebHdfsUrl
	hadoop.hdfs.server.blockmanagement.TestOverReplicatedBlocks
	hadoop.hdfs.server.datanode.TestFsDatasetCacheRevocation
	hadoop.hdfs.TestReadWhileWriting
	hadoop.hdfs.server.namenode.TestFileLimit
	hadoop.hdfs.server.namenode.TestCommitBlockSynchronization
	hadoop.hdfs.TestByteBufferPread
	hadoop.hdfs.TestDFSStripedOutputStreamWithFailureWithRandomECPolicy
	hadoop.fs.TestWebHdfsFileContextMainOperations
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.server.namenode.snapshot.TestINodeFileUnderConstructionWithSnapshot
	hadoop.hdfs.TestSmallBlock
	hadoop.hdfs.server.namenode.TestFileTruncate
	hadoop.hdfs.TestViewDistributedFileSystemContract
	hadoop.hdfs.TestDisableConnCache
	hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetCache
	hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean
	hadoop.hdfs.server.datanode.TestDirectoryScanner
	hadoop.hdfs.server.balancer.TestBalancerLongRunningTasks
	hadoop.fs.TestHDFSFileContextMainOperations
	hadoop.hdfs.server.namenode.TestFSNamesystemLockReport
	hadoop.hdfs.TestSetrepIncreasing
	hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality
	hadoop.hdfs.TestMissingBlocksAlert
	hadoop.hdfs.server.datanode.TestDataNodeTcpNoDelay
	hadoop.hdfs.TestBlocksScheduledCounter
	hadoop.fs.viewfs.TestViewFsLinkFallback
	hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead
	hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
	hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd
	hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/2/artifact/out/Dockerfile
GITHUB PR	#3201
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux addab1fa187b 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	fgl / `43f45ef`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/2/testReport/
Max. process+thread count	3378 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

...-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeMap.java

hadoop-yetus · 2021-07-19T08:07:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ fgl Compile Tests _
+0 🆗	mvndep	12m 48s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 3s		fgl passed
+1 💚	compile	20m 48s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	18m 8s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 49s		fgl passed
+1 💚	mvnsite	3m 14s		fgl passed
+1 💚	javadoc	2m 19s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 22s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	2m 25s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in fgl has 3 extant spotbugs warnings.
-1 ❌	spotbugs	3m 20s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html	hadoop-hdfs-project/hadoop-hdfs in fgl has 2 extant spotbugs warnings.
-1 ❌	shadedclient	16m 28s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 28s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 7s		the patch passed
+1 💚	compile	20m 6s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	20m 6s		the patch passed
+1 💚	compile	18m 11s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	18m 11s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 1 new + 1792 unchanged - 1 fixed = 1793 total (was 1793)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 45s	/results-checkstyle-root.txt	root: The patch generated 6 new + 116 unchanged - 0 fixed = 122 total (was 116)
+1 💚	mvnsite	3m 11s		the patch passed
+1 💚	javadoc	2m 17s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 22s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	3m 34s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
+1 💚	shadedclient	17m 5s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 1s		hadoop-common in the patch passed.
-1 ❌	unit	273m 58s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 10s		The patch does not generate ASF License warnings.
		475m 44s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	Found reliance on default encoding in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long):in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long): String.getBytes() At INodeMap.java:[line 300]
Failed junit tests	hadoop.hdfs.TestErasureCodingExerciseAPIs
	hadoop.hdfs.TestDistributedFileSystem
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeRecovery
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForStoragePolicy
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
	hadoop.hdfs.server.namenode.TestFSNamesystem
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
	hadoop.fs.TestFcHdfsSetUMask
	hadoop.hdfs.server.namenode.TestFSImage
	hadoop.hdfs.server.namenode.snapshot.TestOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.TestStorageRestore
	hadoop.security.TestPermission
	hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
	hadoop.hdfs.TestRollingUpgradeRollback
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy
	hadoop.hdfs.server.namenode.snapshot.TestFSImageWithOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
	hadoop.hdfs.TestDFSUpgradeFromImage
	hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
	hadoop.cli.TestAclCLI
	hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
	hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeAcl
	hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
	hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
	hadoop.hdfs.TestDFSPermission
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.hdfs.server.namenode.TestFileContextAcl
	hadoop.hdfs.TestExtendedAcls
	hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots
	hadoop.hdfs.TestRollingUpgradeDowngrade
	hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff
	hadoop.hdfs.server.namenode.TestStartup
	hadoop.hdfs.server.namenode.TestCacheDirectives
	hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
	hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
	hadoop.hdfs.server.namenode.snapshot.TestSnapshot
	hadoop.cli.TestAclCLIWithPosixAclInheritance
	hadoop.hdfs.server.namenode.TestFSNamesystemLock
	hadoop.hdfs.server.namenode.TestNameEditsConfigs
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
	hadoop.hdfs.web.TestWebHDFSAcl
	hadoop.hdfs.server.namenode.TestCommitBlockSynchronization
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.server.namenode.TestFileTruncate
	hadoop.hdfs.server.namenode.TestCacheDirectivesWithViewDFS
	hadoop.hdfs.server.namenode.TestFSNamesystemLockReport

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/3/artifact/out/Dockerfile
GITHUB PR	#3201
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux ca73b7f3d5f1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	fgl / `42697ae`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/3/testReport/
Max. process+thread count	3324 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

prasad-acit

Sorry for commening late, i have added 2 more comments but forgot to submit them. Plz take a look.

prasad-acit · 2021-07-19T05:30:29Z

...fs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java

+    LOG.debug("getInodeFromTempINodeMap: id={}, TempINodeMap.size={}",
+        id, inodeMapTemp.size());
+    INode inode = new INodeWithAdditionalFields(id, null,
+        new PermissionStatus("", "", new FsPermission((short) 0)), 0, 0) {


PermissionStatus - a static object can be used here instead of creating new one for every call.

If we define a PermissionStatus object as a static object, then it looks like we have to define that static object inside the class, instead of this function. Would that create some unnecessary confusion, given we need that object only for this function?

Happy to make it a static object if we can define it inside this function.

...fs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java

shvachko

Hey @xinglin could you take care of some checkstyle and spotbug warnings from the Jenkins build.
Would be good to have a unit test, which saves, loads, and reloads the image. Unless the one already exists.

...fs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java

hadoop-yetus · 2021-07-22T01:38:58Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ fgl Compile Tests _
+0 🆗	mvndep	13m 14s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 45s		fgl passed
+1 💚	compile	24m 35s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	21m 44s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	4m 10s		fgl passed
+1 💚	mvnsite	3m 13s		fgl passed
+1 💚	javadoc	2m 18s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 24s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	2m 49s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in fgl has 3 extant spotbugs warnings.
-1 ❌	spotbugs	3m 38s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html	hadoop-hdfs-project/hadoop-hdfs in fgl has 2 extant spotbugs warnings.
-1 ❌	shadedclient	17m 27s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 20s		the patch passed
+1 💚	compile	23m 10s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	23m 10s		the patch passed
+1 💚	compile	22m 3s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	22m 3s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 1 new + 1792 unchanged - 1 fixed = 1793 total (was 1793)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 55s	/results-checkstyle-root.txt	root: The patch generated 6 new + 116 unchanged - 0 fixed = 122 total (was 116)
+1 💚	mvnsite	3m 29s		the patch passed
+1 💚	javadoc	2m 27s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 38s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	3m 55s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
+1 💚	shadedclient	17m 38s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 26s		hadoop-common in the patch passed.
-1 ❌	unit	288m 56s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 13s		The patch does not generate ASF License warnings.
		513m 27s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	Found reliance on default encoding in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long):in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long): String.getBytes() At INodeMap.java:[line 300]
Failed junit tests	hadoop.hdfs.TestErasureCodingExerciseAPIs
	hadoop.hdfs.TestDistributedFileSystem
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeRecovery
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForStoragePolicy
	hadoop.hdfs.server.namenode.TestFSImageWithAcl
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
	hadoop.hdfs.server.namenode.TestFSNamesystem
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
	hadoop.fs.TestFcHdfsSetUMask
	hadoop.hdfs.server.namenode.TestFSImage
	hadoop.hdfs.server.namenode.snapshot.TestOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.TestStorageRestore
	hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
	hadoop.security.TestPermission
	hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
	hadoop.hdfs.TestRollingUpgradeRollback
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy
	hadoop.hdfs.server.namenode.snapshot.TestFSImageWithOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
	hadoop.hdfs.TestDFSUpgradeFromImage
	hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
	hadoop.cli.TestAclCLI
	hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
	hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeAcl
	hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
	hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
	hadoop.hdfs.server.namenode.TestEditLogRace
	hadoop.hdfs.TestDFSPermission
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.hdfs.server.namenode.TestFileContextAcl
	hadoop.hdfs.TestExtendedAcls
	hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots
	hadoop.hdfs.TestRollingUpgradeDowngrade
	hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff
	hadoop.hdfs.server.namenode.TestStartup
	hadoop.hdfs.server.namenode.TestCacheDirectives
	hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
	hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
	hadoop.hdfs.server.namenode.snapshot.TestSnapshot
	hadoop.cli.TestAclCLIWithPosixAclInheritance
	hadoop.hdfs.server.namenode.TestFSNamesystemLock
	hadoop.hdfs.server.namenode.TestNameEditsConfigs
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
	hadoop.hdfs.web.TestWebHDFSAcl
	hadoop.hdfs.server.namenode.TestCommitBlockSynchronization
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.server.namenode.TestFileTruncate
	hadoop.hdfs.server.namenode.TestCacheDirectivesWithViewDFS
	hadoop.hdfs.server.namenode.TestFSNamesystemLockReport

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/4/artifact/out/Dockerfile
GITHUB PR	#3201
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux ef95c2d0f389 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	fgl / `17bd453`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/4/testReport/
Max. process+thread count	2977 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-07-22T13:30:41Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 1s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ fgl Compile Tests _
+0 🆗	mvndep	11m 41s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 53s		fgl passed
+1 💚	compile	23m 32s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	21m 17s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	4m 1s		fgl passed
+1 💚	mvnsite	3m 33s		fgl passed
+1 💚	javadoc	2m 32s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 39s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	2m 40s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in fgl has 3 extant spotbugs warnings.
-1 ❌	spotbugs	3m 38s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html	hadoop-hdfs-project/hadoop-hdfs in fgl has 2 extant spotbugs warnings.
-1 ❌	shadedclient	17m 21s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 29s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 33s		the patch passed
+1 💚	compile	23m 38s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	23m 38s		the patch passed
+1 💚	compile	20m 38s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	20m 38s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 1 new + 1792 unchanged - 1 fixed = 1793 total (was 1793)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 0s		the patch passed
+1 💚	mvnsite	3m 27s		the patch passed
+1 💚	javadoc	2m 30s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 34s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	3m 57s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
+1 💚	shadedclient	17m 50s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 4s		hadoop-common in the patch passed.
-1 ❌	unit	280m 6s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 15s		The patch does not generate ASF License warnings.
		500m 29s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	Found reliance on default encoding in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long):in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long): String.getBytes() At INodeMap.java:[line 300]
Failed junit tests	hadoop.hdfs.TestErasureCodingExerciseAPIs
	hadoop.hdfs.TestDistributedFileSystem
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeRecovery
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForStoragePolicy
	hadoop.hdfs.server.namenode.TestFSImageWithAcl
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
	hadoop.hdfs.server.namenode.TestFSNamesystem
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
	hadoop.fs.TestFcHdfsSetUMask
	hadoop.hdfs.server.namenode.TestFSImage
	hadoop.hdfs.server.namenode.snapshot.TestOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.TestStorageRestore
	hadoop.hdfs.server.namenode.TestPersistentStoragePolicySatisfier
	hadoop.security.TestPermission
	hadoop.hdfs.TestRollingUpgradeRollback
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy
	hadoop.hdfs.server.namenode.snapshot.TestFSImageWithOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
	hadoop.hdfs.TestDFSUpgradeFromImage
	hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
	hadoop.cli.TestAclCLI
	hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
	hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeAcl
	hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
	hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
	hadoop.hdfs.server.namenode.TestEditLogRace
	hadoop.hdfs.TestDFSPermission
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.hdfs.server.namenode.TestFileContextAcl
	hadoop.hdfs.TestExtendedAcls
	hadoop.hdfs.TestRollingUpgradeDowngrade
	hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff
	hadoop.hdfs.server.namenode.TestStartup
	hadoop.hdfs.server.blockmanagement.TestReplicationPolicy
	hadoop.hdfs.server.namenode.TestCacheDirectives
	hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
	hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
	hadoop.hdfs.server.namenode.snapshot.TestSnapshot
	hadoop.cli.TestAclCLIWithPosixAclInheritance
	hadoop.hdfs.server.namenode.TestFSNamesystemLock
	hadoop.hdfs.server.namenode.TestNameEditsConfigs
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
	hadoop.hdfs.web.TestWebHDFSAcl
	hadoop.hdfs.server.namenode.TestCommitBlockSynchronization
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.server.namenode.TestFileTruncate
	hadoop.hdfs.server.namenode.TestCacheDirectivesWithViewDFS
	hadoop.hdfs.server.namenode.TestFSNamesystemLockReport

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/5/artifact/out/Dockerfile
GITHUB PR	#3201
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 56d08f5a3065 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	fgl / `9c741a1`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/5/testReport/
Max. process+thread count	3223 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

…ion.

hadoop-yetus · 2021-07-23T16:26:28Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 35s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ fgl Compile Tests _
+0 🆗	mvndep	13m 12s		Maven dependency ordering for branch
+1 💚	mvninstall	22m 40s		fgl passed
+1 💚	compile	25m 7s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	20m 59s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	4m 6s		fgl passed
+1 💚	mvnsite	3m 26s		fgl passed
+1 💚	javadoc	2m 22s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 44s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	2m 46s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in fgl has 3 extant spotbugs warnings.
-1 ❌	spotbugs	3m 42s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html	hadoop-hdfs-project/hadoop-hdfs in fgl has 2 extant spotbugs warnings.
-1 ❌	shadedclient	18m 39s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 31s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 28s		the patch passed
+1 💚	compile	24m 21s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	24m 21s		the patch passed
+1 💚	compile	21m 16s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	21m 16s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 1 new + 1792 unchanged - 1 fixed = 1793 total (was 1793)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	4m 5s		the patch passed
+1 💚	mvnsite	3m 30s		the patch passed
+1 💚	javadoc	2m 29s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 41s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	4m 6s	/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs.html	hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 2 fixed = 1 total (was 2)
+1 💚	shadedclient	19m 3s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 16s		hadoop-common in the patch passed.
-1 ❌	unit	519m 53s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 34s		The patch does not generate ASF License warnings.
		747m 49s

Reason	Tests
SpotBugs	module:hadoop-hdfs-project/hadoop-hdfs
	Found reliance on default encoding in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long):in org.apache.hadoop.hdfs.server.namenode.INodeMap.get(long): String.getBytes() At INodeMap.java:[line 301]
Failed junit tests	hadoop.hdfs.TestErasureCodingExerciseAPIs
	hadoop.hdfs.TestDistributedFileSystem
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeRecovery
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForStoragePolicy
	hadoop.hdfs.server.namenode.TestFSImageWithAcl
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
	hadoop.hdfs.server.namenode.TestFSNamesystem
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
	hadoop.fs.TestFcHdfsSetUMask
	hadoop.hdfs.server.namenode.TestFSImage
	hadoop.hdfs.server.namenode.snapshot.TestOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.TestStorageRestore
	hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
	hadoop.hdfs.server.namenode.TestPersistentStoragePolicySatisfier
	hadoop.security.TestPermission
	hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes
	hadoop.hdfs.TestRollingUpgradeRollback
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy
	hadoop.hdfs.server.namenode.snapshot.TestFSImageWithOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
	hadoop.hdfs.TestDFSUpgradeFromImage
	hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
	hadoop.cli.TestAclCLI
	hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
	hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeAcl
	hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
	hadoop.hdfs.server.namenode.TestEditLogRace
	hadoop.hdfs.TestDFSPermission
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.hdfs.server.namenode.TestFileContextAcl
	hadoop.hdfs.TestExtendedAcls
	hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots
	hadoop.hdfs.TestRollingUpgradeDowngrade
	hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff
	hadoop.hdfs.server.namenode.TestStartup
	hadoop.hdfs.server.namenode.TestCacheDirectives
	hadoop.hdfs.server.namenode.ha.TestObserverNode
	hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
	hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
	hadoop.hdfs.server.namenode.snapshot.TestSnapshot
	hadoop.cli.TestAclCLIWithPosixAclInheritance
	hadoop.hdfs.server.namenode.TestFSNamesystemLock
	hadoop.hdfs.server.namenode.TestNameEditsConfigs
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
	hadoop.hdfs.web.TestWebHDFSAcl
	hadoop.hdfs.server.namenode.TestCommitBlockSynchronization
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.server.namenode.TestFileTruncate
	hadoop.hdfs.server.namenode.TestCacheDirectivesWithViewDFS
	hadoop.hdfs.server.namenode.TestFSNamesystemLockReport

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/6/artifact/out/Dockerfile
GITHUB PR	#3201
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux f6ccd84c4771 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	fgl / `832b96c`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/6/testReport/
Max. process+thread count	2999 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/6/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

shvachko · 2021-07-23T19:51:22Z

+1
Checked that TestSaveNamespace is passing now.

…deMap.get(long inode).

…odeWithAdditionalFields> for tempINodeMap.

xinglin

Hi Konstantin,

I should have addressed your four specific comments. The only comment which has not been addressed is to add unit tests. Will work on that later on.

…rtitionedGSet. Contributed by Xing Lin. (#3201)

hadoop-yetus · 2021-07-24T05:23:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 36s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ fgl Compile Tests _
+0 🆗	mvndep	13m 19s		Maven dependency ordering for branch
+1 💚	mvninstall	19m 57s		fgl passed
+1 💚	compile	20m 42s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	18m 26s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 50s		fgl passed
+1 💚	mvnsite	3m 20s		fgl passed
+1 💚	javadoc	2m 15s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 22s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	2m 28s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in fgl has 3 extant spotbugs warnings.
-1 ❌	spotbugs	3m 32s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html	hadoop-hdfs-project/hadoop-hdfs in fgl has 2 extant spotbugs warnings.
-1 ❌	shadedclient	16m 49s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 26s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 11s		the patch passed
+1 💚	compile	22m 42s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	22m 42s		the patch passed
+1 💚	compile	19m 34s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	19m 34s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 1 new + 1792 unchanged - 1 fixed = 1793 total (was 1793)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 1s	/results-checkstyle-root.txt	root: The patch generated 2 new + 121 unchanged - 0 fixed = 123 total (was 121)
+1 💚	mvnsite	3m 22s		the patch passed
+1 💚	javadoc	2m 8s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 27s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	2m 38s		hadoop-common-project/hadoop-common generated 0 new + 2 unchanged - 1 fixed = 2 total (was 3)
+1 💚	spotbugs	3m 31s		hadoop-hdfs-project/hadoop-hdfs generated 0 new + 0 unchanged - 2 fixed = 0 total (was 2)
+1 💚	shadedclient	17m 34s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 22s		hadoop-common in the patch passed.
-1 ❌	unit	284m 44s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 14s		The patch does not generate ASF License warnings.
		492m 37s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestErasureCodingExerciseAPIs
	hadoop.hdfs.TestDistributedFileSystem
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeRecovery
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForStoragePolicy
	hadoop.hdfs.server.namenode.TestFSImageWithAcl
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
	hadoop.hdfs.server.namenode.TestFSNamesystem
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
	hadoop.fs.TestFcHdfsSetUMask
	hadoop.hdfs.server.namenode.TestFSImage
	hadoop.hdfs.server.namenode.snapshot.TestOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.TestStorageRestore
	hadoop.hdfs.server.namenode.TestPersistentStoragePolicySatisfier
	hadoop.security.TestPermission
	hadoop.hdfs.TestRollingUpgradeRollback
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy
	hadoop.hdfs.server.namenode.snapshot.TestFSImageWithOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
	hadoop.hdfs.TestDFSUpgradeFromImage
	hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
	hadoop.cli.TestAclCLI
	hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
	hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeAcl
	hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
	hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
	hadoop.hdfs.TestDFSPermission
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.hdfs.server.namenode.TestFileContextAcl
	hadoop.hdfs.TestExtendedAcls
	hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots
	hadoop.hdfs.TestRollingUpgradeDowngrade
	hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff
	hadoop.hdfs.server.namenode.TestStartup
	hadoop.hdfs.server.namenode.TestCacheDirectives
	hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
	hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
	hadoop.hdfs.server.namenode.snapshot.TestSnapshot
	hadoop.cli.TestAclCLIWithPosixAclInheritance
	hadoop.hdfs.server.namenode.TestFSNamesystemLock
	hadoop.hdfs.server.namenode.TestNameEditsConfigs
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
	hadoop.hdfs.web.TestWebHDFSAcl
	hadoop.hdfs.server.namenode.TestCommitBlockSynchronization
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.server.namenode.TestFileTruncate
	hadoop.hdfs.server.namenode.TestCacheDirectivesWithViewDFS
	hadoop.hdfs.server.namenode.TestFSNamesystemLockReport

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/7/artifact/out/Dockerfile
GITHUB PR	#3201
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 19c2698267da 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	fgl / `5cd228b`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/7/testReport/
Max. process+thread count	3066 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/7/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-07-24T06:47:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 52s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ fgl Compile Tests _
+0 🆗	mvndep	12m 56s		Maven dependency ordering for branch
+1 💚	mvninstall	21m 19s		fgl passed
+1 💚	compile	22m 53s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	20m 2s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	3m 44s		fgl passed
+1 💚	mvnsite	3m 21s		fgl passed
+1 💚	javadoc	2m 9s		fgl passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 20s		fgl passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	spotbugs	2m 36s	/branch-spotbugs-hadoop-common-project_hadoop-common-warnings.html	hadoop-common-project/hadoop-common in fgl has 3 extant spotbugs warnings.
-1 ❌	spotbugs	3m 32s	/branch-spotbugs-hadoop-hdfs-project_hadoop-hdfs-warnings.html	hadoop-hdfs-project/hadoop-hdfs in fgl has 2 extant spotbugs warnings.
-1 ❌	shadedclient	17m 4s		branch has errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 55s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 13s		the patch passed
+1 💚	compile	21m 58s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	21m 58s		the patch passed
+1 💚	compile	21m 0s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
-1 ❌	javac	21m 0s	/results-compile-javac-root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10.txt	root-jdkPrivateBuild-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~~20.04-b10 generated 1 new + 1792 unchanged - 1 fixed = 1793 total (was 1793)
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	4m 10s	/results-checkstyle-root.txt	root: The patch generated 7 new + 121 unchanged - 0 fixed = 128 total (was 121)
+1 💚	mvnsite	3m 23s		the patch passed
+1 💚	javadoc	2m 19s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	3m 24s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	2m 49s		hadoop-common-project/hadoop-common generated 0 new + 2 unchanged - 1 fixed = 2 total (was 3)
+1 💚	spotbugs	4m 2s		hadoop-hdfs-project/hadoop-hdfs generated 0 new + 0 unchanged - 2 fixed = 0 total (was 2)
+1 💚	shadedclient	19m 4s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	17m 48s		hadoop-common in the patch passed.
-1 ❌	unit	294m 7s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	1m 4s		The patch does not generate ASF License warnings.
		510m 28s

Reason	Tests
Failed junit tests	hadoop.hdfs.TestErasureCodingExerciseAPIs
	hadoop.hdfs.TestDistributedFileSystem
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeRecovery
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForStoragePolicy
	hadoop.hdfs.server.namenode.TestFSImageWithAcl
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap
	hadoop.hdfs.server.namenode.TestFSNamesystem
	hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewer
	hadoop.fs.TestFcHdfsSetUMask
	hadoop.hdfs.server.namenode.TestFSImage
	hadoop.hdfs.server.namenode.snapshot.TestOrderedSnapshotDeletion
	hadoop.hdfs.server.namenode.TestStorageRestore
	hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
	hadoop.security.TestPermission
	hadoop.hdfs.TestRollingUpgradeRollback
	hadoop.hdfs.TestErasureCodingPolicyWithSnapshotWithRandomECPolicy
	hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
	hadoop.hdfs.TestDFSUpgradeFromImage
	hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot
	hadoop.cli.TestAclCLI
	hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot
	hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot
	hadoop.hdfs.server.namenode.TestNameNodeAcl
	hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics
	hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots
	hadoop.hdfs.server.namenode.TestEditLogRace
	hadoop.hdfs.TestDFSPermission
	hadoop.hdfs.TestViewDistributedFileSystem
	hadoop.hdfs.server.namenode.TestFileContextAcl
	hadoop.hdfs.TestExtendedAcls
	hadoop.hdfs.server.namenode.snapshot.TestRandomOpsWithSnapshots
	hadoop.hdfs.TestRollingUpgradeDowngrade
	hadoop.hdfs.server.namenode.snapshot.TestSnapRootDescendantDiff
	hadoop.hdfs.server.namenode.TestStartup
	hadoop.hdfs.server.namenode.TestCacheDirectives
	hadoop.hdfs.server.namenode.TestSecondaryNameNodeUpgrade
	hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion
	hadoop.hdfs.server.namenode.snapshot.TestSnapshot
	hadoop.cli.TestAclCLIWithPosixAclInheritance
	hadoop.hdfs.server.namenode.TestFSNamesystemLock
	hadoop.hdfs.server.namenode.TestNameEditsConfigs
	hadoop.hdfs.server.namenode.snapshot.TestSnapshotDiffReport
	hadoop.hdfs.TestFileChecksum
	hadoop.hdfs.server.namenode.TestFSImageWithSnapshot
	hadoop.hdfs.web.TestWebHDFSAcl
	hadoop.hdfs.server.namenode.TestCommitBlockSynchronization
	hadoop.hdfs.TestRollingUpgrade
	hadoop.hdfs.server.namenode.TestFileTruncate
	hadoop.hdfs.server.namenode.TestCacheDirectivesWithViewDFS
	hadoop.hdfs.server.namenode.TestFSNamesystemLockReport

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/8/artifact/out/Dockerfile
GITHUB PR	#3201
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux d3c76ff78a28 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	fgl / `d43f1f0`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/8/testReport/
Max. process+thread count	3743 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3201/8/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

…rtitionedGSet. Contributed by Xing Lin. (#3201)

shvachko · 2021-07-31T20:13:15Z

I re-committed PR to reflect latest changes.
We can close this.

xinglin · 2021-07-31T21:02:04Z

close it. Appreciated all the comments I received for this commit and Thanks @shvachko for committing it!

…rtitionedGSet. Contributed by Xing Lin. (#3201)

shvachko and others added 3 commits July 16, 2021 13:08

INodeMap with PartitionedGSet and per-partition locking.

0e7d216

Add namespace key for INode. (shv)

1829b58

HDFS-16125. [FGL] Fix the iterator for PartitionedGSet. Contributed b…

b1e2c07

…y Xing Lin. (apache#3197)

asfgit force-pushed the fgl branch from 9a60e53 to b1e2c07 Compare July 16, 2021 21:30

HDFS-16128: Added support for saving/loading an FS Image for Partitio…

43f45ef

…nedGSet

xinglin force-pushed the fgl-saveloadfs branch from b5d03ce to 43f45ef Compare July 17, 2021 05:44

xinglin marked this pull request as ready for review July 17, 2021 05:45

prasad-acit reviewed Jul 18, 2021

View reviewed changes

...-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeMap.java Outdated Show resolved Hide resolved

...-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeMap.java Outdated Show resolved Hide resolved

fixed a bug in using a wrong variable: use inode, instead of key here.

42697ae

prasad-acit reviewed Jul 21, 2021

View reviewed changes

Replace the use of an anonymous INode subclass with INodeDirectory.

17bd453

shvachko reviewed Jul 21, 2021

View reviewed changes

fixes for hadoop checkstyle test.

9c741a1

fixed two spotbugs related with String encoding for internationalizat…

832b96c

…ion.

Xing Lin added 3 commits July 23, 2021 14:08

fixed two spotbugs and replaced anonymous class with INodeFile in INo…

5cd228b

…deMap.get(long inode).

renamed inodeMapTemp to tempInodeMap.

5715ce5

Replaced GSet<INode, INodeWithAdditionalFields> with HashMap<Long, IN…

d43f1f0

…odeWithAdditionalFields> for tempINodeMap.

xinglin commented Jul 23, 2021

View reviewed changes

asfgit pushed a commit that referenced this pull request Jul 23, 2021

HDFS-16128. [FGL] Added support for saving/loading an FS Image for Pa…

0a73bde

…rtitionedGSet. Contributed by Xing Lin. (#3201)

asfgit force-pushed the fgl branch from b1e2c07 to 7598a68 Compare July 23, 2021 22:30

xinglin force-pushed the fgl-saveloadfs branch from 8a9cb44 to d43f1f0 Compare July 24, 2021 00:02

xinglin changed the title ~~Fgl saveloadfs~~ HDFS-16128: Added support for saving/loading an FS Image for fine-grain locking Jul 24, 2021

asfgit pushed a commit that referenced this pull request Jul 31, 2021

HDFS-16128. [FGL] Added support for saving/loading an FS Image for Pa…

b135d26

…rtitionedGSet. Contributed by Xing Lin. (#3201)

asfgit force-pushed the fgl branch from 7598a68 to 0f1a9a8 Compare July 31, 2021 20:06

xinglin closed this Jul 31, 2021

asfgit pushed a commit that referenced this pull request Aug 13, 2021

HDFS-16128. [FGL] Added support for saving/loading an FS Image for Pa…

4610e1d

…rtitionedGSet. Contributed by Xing Lin. (#3201)

HDFS-16128: Added support for saving/loading an FS Image for fine-grain locking #3201

HDFS-16128: Added support for saving/loading an FS Image for fine-grain locking #3201

Uh oh!

Conversation

xinglin commented Jul 13, 2021

Uh oh!

hadoop-yetus commented Jul 13, 2021

Uh oh!

hadoop-yetus commented Jul 17, 2021

Uh oh!

Uh oh!

Uh oh!

hadoop-yetus commented Jul 19, 2021

Uh oh!

prasad-acit left a comment

Choose a reason for hiding this comment

Uh oh!

prasad-acit Jul 19, 2021

Choose a reason for hiding this comment

Uh oh!

xinglin Jul 21, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

shvachko left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hadoop-yetus commented Jul 22, 2021

Uh oh!

hadoop-yetus commented Jul 22, 2021

Uh oh!

hadoop-yetus commented Jul 23, 2021

Uh oh!

shvachko commented Jul 23, 2021

Uh oh!

xinglin left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Jul 24, 2021

Uh oh!

hadoop-yetus commented Jul 24, 2021

Uh oh!

shvachko commented Jul 31, 2021

Uh oh!

xinglin commented Jul 31, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants