YARN-10924. Clean up CapacityScheduler#initScheduler by szilard-nemeth · Pull Request #3581 · apache/hadoop

szilard-nemeth · 2021-10-23T15:01:50Z

Description of PR

How was this patch tested?

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2021-10-23T17:22:22Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 3s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 21s		trunk passed
+1 💚	compile	1m 4s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	0m 57s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	0m 49s		trunk passed
+1 💚	mvnsite	1m 1s		trunk passed
+1 💚	javadoc	0m 50s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	0m 44s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	1m 51s		trunk passed
+1 💚	shadedclient	20m 33s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 52s		the patch passed
+1 💚	compile	0m 54s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	0m 54s		the patch passed
+1 💚	compile	0m 49s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	0m 49s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 38s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 11 new + 82 unchanged - 0 fixed = 93 total (was 82)
+1 💚	mvnsite	0m 50s		the patch passed
+1 💚	javadoc	0m 38s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	0m 36s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	1m 53s		the patch passed
-1 ❌	shadedclient	20m 0s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	50m 40s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch passed.
-1 ❌	asflicense	0m 34s	/results-asflicense.txt	The patch generated 1 ASF License warnings.
		139m 12s

Reason	Tests
Failed junit tests	hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCResponseId
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueMappings
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNewQueueAutoCreation
	hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels
	hadoop.yarn.server.resourcemanager.TestRMHATimelineCollectors
	hadoop.yarn.server.resourcemanager.TestLeaderElectorService
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCSAllocateCustomResource
	hadoop.yarn.server.resourcemanager.rmapp.TestNodesListManager
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodes
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueMappingFactory
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueState
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoCreatedQueuePreemption
	hadoop.yarn.server.resourcemanager.TestWorkPreservingUnmanagedAM
	hadoop.yarn.server.resourcemanager.TestRMRestart
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAsyncScheduling
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodeLabels
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesNodes
	hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRMRPCNodeUpdates
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueParsing
	hadoop.yarn.server.resourcemanager.federation.TestFederationRMStateStoreService
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesReservation
	hadoop.yarn.server.resourcemanager.rmapp.attempt.TestAMLivelinessMonitor
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestSchedulingRequestContainerAllocation
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServices
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebApp
	hadoop.yarn.server.resourcemanager.TestReservationSystemWithRMHA
	hadoop.yarn.server.resourcemanager.TestRMStoreCommands
	hadoop.yarn.server.resourcemanager.TestResourceManagerMXBean
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAutoCreatedQueueDeletionPolicy
	hadoop.yarn.server.resourcemanager.TestRMHAForAsyncScheduler
	hadoop.yarn.server.resourcemanager.TestApplicationCleanup
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerQueueACLs
	hadoop.yarn.server.resourcemanager.resource.TestResourceProfiles
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities
	hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl
	hadoop.yarn.server.resourcemanager.webapp.TestAppPage
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMaxParallelApps
	hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementProcessor
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriorityACLs
	hadoop.yarn.server.resourcemanager.scheduler.policy.TestFairOrderingPolicy
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivitiesWithMultiNodesEnabled
	hadoop.yarn.server.resourcemanager.reservation.TestReservationSystem
	hadoop.yarn.server.resourcemanager.scheduler.constraint.algorithm.TestLocalAllocationTagsManager
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestIncreaseAllocationExpirer
	hadoop.yarn.server.resourcemanager.TestResourceManager
	hadoop.yarn.server.resourcemanager.reservation.TestCapacitySchedulerPlanFollower
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSchedulingRequestUpdate
	hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId
	hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestWorkPreservingRMRestartForNodeLabel
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate
	hadoop.yarn.server.resourcemanager.TestNodeBlacklistingOnAMFailures
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWithMultiResourceTypes
	hadoop.yarn.server.resourcemanager.TestResourceTrackerService
	hadoop.yarn.server.resourcemanager.TestClientRMService
	hadoop.yarn.server.resourcemanager.TestOpportunisticContainerAllocatorAMService
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerSurgicalPreemption
	hadoop.yarn.server.resourcemanager.security.TestAMRMTokens
	hadoop.yarn.server.resourcemanager.TestRMProxyUsersConf
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerLazyPreemption
	hadoop.yarn.server.resourcemanager.nodelabels.TestRMDelegatedNodeLabelsUpdater
	hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceCapacity
	hadoop.yarn.server.resourcemanager.TestSignalContainer
	hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWorkflowPriorityMapping
	hadoop.yarn.server.resourcemanager.volume.csi.TestVolumeProcessor
	hadoop.yarn.server.resourcemanager.TestRM
	hadoop.yarn.server.resourcemanager.scheduler.TestAbstractYarnScheduler
	hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueConfigurationAutoRefreshPolicy
	hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
	hadoop.yarn.server.resourcemanager.TestContainerResourceUsage
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimitsByPartition
	hadoop.yarn.server.resourcemanager.ahs.TestRMApplicationHistoryWriter
	hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerHealth
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationLimits
	hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler
	hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceFair
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAutoQueueCreation
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestSchedulingRequestContainerAllocationAsync
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestReservations
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAbsoluteResourceWithAutoQueue
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched
	hadoop.yarn.server.resourcemanager.TestRMHA
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueStateManager
	hadoop.yarn.server.resourcemanager.monitor.capacity.TestProportionalCapacityPreemptionPolicy
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerMultiNodesWithPreemption
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestApplicationPriority
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerAmbiguousLeafs
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestAbsoluteResourceConfiguration
	hadoop.yarn.server.resourcemanager.webapp.TestNodesPage
	hadoop.yarn.server.resourcemanager.TestAppManager
	hadoop.yarn.server.resourcemanager.monitor.TestSchedulingMonitor
	hadoop.yarn.server.resourcemanager.scheduler.constraint.TestAllocationTagsManager
	hadoop.yarn.server.resourcemanager.scheduler.TestSchedulerUtils
	hadoop.yarn.server.resourcemanager.scheduler.constraint.TestPlacementConstraintsUtil
	hadoop.yarn.server.resourcemanager.nodelabels.TestRMNodeLabelsManager
	hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher
	hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWeightMode
	hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestZKConfigurationStore
	hadoop.yarn.server.resourcemanager.TestRMEmbeddedElector
	hadoop.yarn.server.resourcemanager.webapp.TestRedirectionErrorPage
	hadoop.yarn.server.resourcemanager.placement.TestPlacementManager
	hadoop.yarn.server.resourcemanager.TestRMTimelineService
	hadoop.yarn.server.resourcemanager.TestDecommissioningNodesWatcher
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerConfigValidator
	hadoop.yarn.server.resourcemanager.scheduler.fair.converter.TestFSConfigToCSConfigConverterMain
	hadoop.yarn.server.resourcemanager.TestCapacitySchedulerMetrics
	hadoop.yarn.server.resourcemanager.rmapp.TestApplicationLifetimeMonitor
	hadoop.yarn.server.resourcemanager.applicationsmanager.TestAMRestart
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerOvercommit
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueManagementDynamicEditPolicy
	hadoop.yarn.server.resourcemanager.TestAMAuthorization
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesForCSWithPartitions
	hadoop.yarn.server.resourcemanager.TestRMHAMetrics
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification
	hadoop.yarn.server.resourcemanager.TestApplicationACLs
	hadoop.yarn.server.resourcemanager.resourcetracker.TestNMReconnect
	hadoop.yarn.server.resourcemanager.security.TestClientToAMTokens
	hadoop.yarn.server.resourcemanager.scheduler.fair.converter.TestConvertedConfigValidator
	hadoop.yarn.server.resourcemanager.webapp.TestRMWebServiceAppsNodelabel
	hadoop.yarn.server.resourcemanager.TestRMAdminService

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/1/artifact/out/Dockerfile
GITHUB PR	#3581
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 7ea2eff0cc74 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `06f2eae`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/1/testReport/
Max. process+thread count	5410 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-10-24T00:05:29Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 59s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	31m 45s		trunk passed
+1 💚	compile	1m 3s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	0m 55s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	0m 48s		trunk passed
+1 💚	mvnsite	1m 2s		trunk passed
+1 💚	javadoc	0m 49s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	0m 45s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	1m 55s		trunk passed
+1 💚	shadedclient	20m 24s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 51s		the patch passed
+1 💚	compile	0m 54s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	0m 54s		the patch passed
+1 💚	compile	0m 48s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	0m 48s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 39s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 10 new + 82 unchanged - 0 fixed = 92 total (was 82)
+1 💚	mvnsite	0m 49s		the patch passed
+1 💚	javadoc	0m 38s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	0m 37s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	1m 54s		the patch passed
+1 💚	shadedclient	20m 21s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	99m 23s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 31s		The patch does not generate ASF License warnings.
		187m 13s

Reason	Tests
Failed junit tests	hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy
	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWithMultiResourceTypes

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/2/artifact/out/Dockerfile
GITHUB PR	#3581
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 57bb299111f1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `b4c75d3`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/2/testReport/
Max. process+thread count	959 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/2/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-10-24T12:31:12Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 42s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	31m 54s		trunk passed
+1 💚	compile	1m 2s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	0m 57s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	0m 48s		trunk passed
+1 💚	mvnsite	0m 59s		trunk passed
+1 💚	javadoc	0m 49s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	0m 45s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	1m 52s		trunk passed
+1 💚	shadedclient	20m 19s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 49s		the patch passed
+1 💚	compile	0m 54s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	0m 54s		the patch passed
+1 💚	compile	0m 47s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	0m 47s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 39s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 10 new + 82 unchanged - 0 fixed = 92 total (was 82)
+1 💚	mvnsite	0m 51s		the patch passed
+1 💚	javadoc	0m 36s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	0m 35s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	1m 52s		the patch passed
+1 💚	shadedclient	20m 3s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	96m 10s	/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 34s		The patch does not generate ASF License warnings.
		183m 25s

Reason	Tests
Failed junit tests	hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerWithMultiResourceTypes

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/3/artifact/out/Dockerfile
GITHUB PR	#3581
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux 06f134703dd0 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `55d684a`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/3/testReport/
Max. process+thread count	958 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/3/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

...java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java

hadoop-yetus · 2021-10-25T14:59:21Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 39s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 1s		trunk passed
+1 💚	compile	1m 3s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	0m 56s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	0m 48s		trunk passed
+1 💚	mvnsite	1m 1s		trunk passed
+1 💚	javadoc	0m 51s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	0m 43s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	1m 54s		trunk passed
+1 💚	shadedclient	20m 30s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 52s		the patch passed
+1 💚	compile	0m 55s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	0m 55s		the patch passed
+1 💚	compile	0m 47s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	0m 47s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 37s	/results-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 10 new + 82 unchanged - 0 fixed = 92 total (was 82)
+1 💚	mvnsite	0m 52s		the patch passed
+1 💚	javadoc	0m 37s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	0m 35s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	1m 53s		the patch passed
+1 💚	shadedclient	20m 7s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	96m 0s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		183m 42s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/4/artifact/out/Dockerfile
GITHUB PR	#3581
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux c3a0d5cf6145 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `53fbd87`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/4/testReport/
Max. process+thread count	955 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/4/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tomicooler

Thanks for the patch @szilard-nemeth. Other than the checkStyle issues, non-binding +1.

hadoop-yetus · 2021-10-26T10:51:36Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 53s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 3s		trunk passed
+1 💚	compile	1m 4s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	0m 58s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	checkstyle	0m 48s		trunk passed
+1 💚	mvnsite	1m 2s		trunk passed
+1 💚	javadoc	0m 49s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	0m 45s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	1m 53s		trunk passed
+1 💚	shadedclient	20m 14s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 53s		the patch passed
+1 💚	compile	0m 56s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	0m 56s		the patch passed
+1 💚	compile	0m 48s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	0m 48s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 40s		the patch passed
+1 💚	mvnsite	0m 52s		the patch passed
+1 💚	javadoc	0m 39s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	0m 34s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	spotbugs	1m 52s		the patch passed
+1 💚	shadedclient	20m 17s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	96m 1s		hadoop-yarn-server-resourcemanager in the patch passed.
+1 💚	asflicense	0m 35s		The patch does not generate ASF License warnings.
		184m 23s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/5/artifact/out/Dockerfile
GITHUB PR	#3581
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell
uname	Linux a702f13ae201 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `53dc5c6`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/5/testReport/
Max. process+thread count	955 (vs. ulimit of 5500)
modules	C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3581/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

shuzirra · 2021-10-27T15:14:43Z

@szilard-nemeth thank you for the patch LGTM+1.

szilard-nemeth · 2021-10-27T15:17:01Z

Thanks @shuzirra

…ntributed by Szilard Nemeth

…che#3581) Contributed by Szilard Nemeth (cherry picked from commit 66ac476) Change-Id: Ic437e521052dbd81ff323c7d696381134d1b4426

YARN-10924

06f2eae

Fix NPE

b4c75d3

retrigger build

55d684a

tomicooler suggested changes Oct 25, 2021

View reviewed changes

...java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java Outdated Show resolved Hide resolved

Fix CapacityScheduler#initResourceCalculator

53fbd87

tomicooler approved these changes Oct 26, 2021

View reviewed changes

fix checkstyle

53dc5c6

shuzirra merged commit 66ac476 into apache:trunk Oct 27, 2021

shuzirra self-requested a review October 27, 2021 15:14

HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022

YARN-10924. Clean up CapacityScheduler#initScheduler (apache#3581) Co…

278614a

…ntributed by Szilard Nemeth

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

YARN-10924. Clean up CapacityScheduler#initScheduler#3581

YARN-10924. Clean up CapacityScheduler#initScheduler#3581
shuzirra merged 5 commits intoapache:trunkfrom
szilard-nemeth:YARN-10924

szilard-nemeth commented Oct 23, 2021

Uh oh!

hadoop-yetus commented Oct 23, 2021

Uh oh!

hadoop-yetus commented Oct 24, 2021

Uh oh!

hadoop-yetus commented Oct 24, 2021

Uh oh!

Uh oh!

hadoop-yetus commented Oct 25, 2021

Uh oh!

tomicooler left a comment

Uh oh!

hadoop-yetus commented Oct 26, 2021

Uh oh!

shuzirra commented Oct 27, 2021

Uh oh!

szilard-nemeth commented Oct 27, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

szilard-nemeth commented Oct 23, 2021

Description of PR

How was this patch tested?

For code changes:

Uh oh!

hadoop-yetus commented Oct 23, 2021

Uh oh!

hadoop-yetus commented Oct 24, 2021

Uh oh!

hadoop-yetus commented Oct 24, 2021

Uh oh!

Uh oh!

hadoop-yetus commented Oct 25, 2021

Uh oh!

tomicooler left a comment

Choose a reason for hiding this comment

Uh oh!

hadoop-yetus commented Oct 26, 2021

Uh oh!

shuzirra commented Oct 27, 2021

Uh oh!

szilard-nemeth commented Oct 27, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants