Skip to content

Conversation

@iwasakims
Copy link
Member

@iwasakims iwasakims commented Nov 17, 2020

https://issues.apache.org/jira/browse/HDFS-15674

The test intermittently fails if count of blocks reported by IBR does not reached to expected value(4000).

[ERROR] Failures:
[ERROR]   TestBPOfferService.testMissBlocksWhenReregister:341 Timed out wait for IBR counts FBRCount = 2441, IBRCount = 3999; expected = 4000. Exception: Timed out waiting for condition. Thread diagnostics:
Timestamp: 2020-11-16 12:45:57,425
{noformat}

This waiting condition does not make sense because pending IBRs are possible to be deleted by clearIBRs() in BPServiceActor#reRegister. It is ok because succeeding full block report covers them based on the discussion of HDFS-15113.

The tests should check that all blocks are reported rather than counting number of reported blocks in FBR/IBR independently.

@iwasakims
Copy link
Member Author

The failure could be reproduced within 10 tries on my local environment. I checked that pending IBRs are cleared before sending heartbeat in failed case by adding debug logging to IncrementalBlockReportManager#clearIBRs. (The logging is not including the PR.)

$ cd hadoop-hdfs-project/hadoop-hdfs
$ for i in `seq 100` ; do echo $i && mvn test -Dtest=TestBPOfferService#testMissBlocksWhenReregister -DignoreTestFailure=false  || break ; done

@iwasakims iwasakims changed the title HDFS-15674. TestBPOfferService#testMissBlocksWhenReregister fails on … HDFS-15674. TestBPOfferService#testMissBlocksWhenReregister fails on trunk. Nov 17, 2020
@hadoop-yetus
Copy link

💔 -1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 26m 14s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 0m 0s test4tests The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 33m 48s trunk passed
+1 💚 compile 1m 19s trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚 compile 1m 13s trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚 checkstyle 0m 49s trunk passed
+1 💚 mvnsite 1m 18s trunk passed
+1 💚 shadedclient 21m 35s branch has no errors when building and testing our client artifacts.
+1 💚 javadoc 0m 59s trunk passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚 javadoc 1m 31s trunk passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+0 🆗 spotbugs 3m 6s Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚 findbugs 3m 4s trunk passed
_ Patch Compile Tests _
+1 💚 mvninstall 1m 14s the patch passed
+1 💚 compile 1m 9s the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚 javac 1m 9s the patch passed
+1 💚 compile 1m 8s the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚 javac 1m 8s the patch passed
+1 💚 checkstyle 0m 40s the patch passed
+1 💚 mvnsite 1m 11s the patch passed
+1 💚 whitespace 0m 0s The patch has no whitespace issues.
+1 💚 shadedclient 14m 49s patch has no errors when building and testing our client artifacts.
+1 💚 javadoc 0m 48s the patch passed with JDK Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04
+1 💚 javadoc 1m 21s the patch passed with JDK Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
+1 💚 findbugs 3m 1s the patch passed
_ Other Tests _
-1 ❌ unit 100m 16s /patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt hadoop-hdfs in the patch passed.
+1 💚 asflicense 0m 39s The patch does not generate ASF License warnings.
219m 57s
Reason Tests
Failed junit tests hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate
hadoop.hdfs.server.namenode.TestAddStripedBlockInFBR
hadoop.hdfs.TestDecommissionWithBackoffMonitor
hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
Subsystem Report/Notes
Docker ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2467/1/artifact/out/Dockerfile
GITHUB PR #2467
Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname Linux f861655c6c83 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / b57f04c
Default Java Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Multi-JDK versions /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.9.1+1-Ubuntu-0ubuntu1.18.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_275-8u275-b01-0ubuntu1~18.04-b01
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2467/1/testReport/
Max. process+thread count 4925 (vs. ulimit of 5500)
modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2467/1/console
versions git=2.17.1 maven=3.6.0 findbugs=4.0.6
Powered by Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

@Hexiaoqiao
Copy link
Contributor

Thanks @iwasakims for your works. +1 from my side.

@iwasakims iwasakims merged commit 425996e into apache:trunk Nov 18, 2020
@iwasakims
Copy link
Member Author

I merged this. Thanks, @Hexiaoqiao and @goiri .

xinglin pushed a commit to xinglin/hadoop that referenced this pull request Oct 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants