HDFS-15716. WaitforReplication in TestUpgradeDomainBlockPlacementPolicy #2528
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
💔 -1 overall
This message was automatically generated. |
amahussein
force-pushed
the
hdfs-15716
branch
from
7efa58e to
147f2ca
Compare
|
💔 -1 overall
This message was automatically generated. |
Member
Does this is done outside TestUpgradeDomainBlockPlacementPolicy? Your patch looks applicable to branch-2.10 too. Since branch-2.10 still support Java 7, cherry-picking would be easy by avoiding lambda here. I think the lambda does not improve readability here. @amahussein |
Contributor
Author
Good point. I will remove the lambda. |
amahussein
force-pushed
the
hdfs-15716
branch
from
147f2ca to
def653a
Compare
jbrennan333
approved these changes
Dec 8, 2020
Contributor
jbrennan333
left a comment
jbrennan333
left a comment
There was a problem hiding this comment.
+1 this looks good to me.
|
💔 -1 overall
This message was automatically generated. |
asfgit
pushed a commit
that referenced
this pull request
Dec 8, 2020
…cy (#2528) Contributed by Ahmed Hussein
goiri
reviewed
Dec 11, 2020
| BlockPlacementStatus status = cluster.getNamesystem().getBlockManager(). | ||
| getBlockPlacementPolicy().verifyBlockPlacement( | ||
| block.getLocations(), REPLICATION_FACTOR); | ||
| assertTrue(status.isPlacementPolicySatisfied()); |
Member
There was a problem hiding this comment.
Why not leave this static import?
Contributor
Author
There was a problem hiding this comment.
Sorry @goiri I did not know that static imports are preferred. I remembered it used to be not favorable because of code readability.
I will take a note of that for future changes.
Member
|
The list of failed unit tests in the last few days is getting worse and worse. |
Contributor
Author
Thanks @goiri. Test Result (23 failures / -45)
org.apache.hadoop.hdfs.server.balancer.TestBalancer.testBalancerWithStripedFile
org.apache.hadoop.hdfs.server.balancer.TestBalancer.testBalancerWithIncludeListWithPorts
org.apache.hadoop.hdfs.server.balancer.TestBalancer.testBalancerWithSortTopNodes
org.apache.hadoop.hdfs.server.balancer.TestBalancer.testBalancerCliWithIncludeListWithPorts
org.apache.hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks.testSetRepIncWithUnderReplicatedBlocks
org.apache.hadoop.hdfs.server.namenode.TestFsck.testECFsck
org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater.testNodeDecommision
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testUpdateAppStateXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testAppQueueXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testAppTimeoutsXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testGetContainersXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testUpdateAppPriorityXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testUpdateAppQueueXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testAppTimeoutXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testAppAttemptXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testGetAppAttemptXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testAppXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testAppStateXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testAppPriorityXML
org.apache.hadoop.yarn.server.router.webapp.TestRouterWebServicesREST.testGetAppsMultiThread
org.apache.hadoop.tools.dynamometer.TestDynamometerInfra.org.apache.hadoop.tools.dynamometer.TestDynamometerInfra
org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithOpportunisticContainers
org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell.testDSShellWithEnforceExecutionType
By the way, I found that TestDistributedShell does not clean at all. The problem that the two failing unit tests leave several processes running for sometime. It could be one of the reasons the system crashes as the background containers are sucking memory and CPU resources. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In some slow runs
TestUpgradeDomainBlockPlacementPolicy#testPlacementandTestUpgradeDomainBlockPlacementPolicy#testPlacementAfterDecommissionfail.On branch-2.10, this was fixed by waiting for the replication to be complete.