TEZ-4603: Bucket Map Join can hang if the source vertex parallelism i…

[mayankkunwar]

…s changed by reducer autoparallelism
Description
The changes in this PR are w.r.t. when auto reduce parallelism kicks in, we try to updates downstream vertex's expected number of input, so that the execution hang could be avoided.
In this case, the type of edge between reducer and downstream vertices (Mapper) is a CUSTOM edge.
[Reducer --(C)--> Mapper]

TODO: Unit Tests

[tez-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	25m 23s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 0s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s	The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
		_ branch-0.10.4 Compile Tests _
+0 🆗	mvndep	7m 6s	Maven dependency ordering for branch
+1 💚	mvninstall	15m 48s	branch-0.10.4 passed
+1 💚	compile	1m 55s	branch-0.10.4 passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚	compile	1m 44s	branch-0.10.4 passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚	checkstyle	2m 7s	branch-0.10.4 passed
+1 💚	javadoc	1m 51s	branch-0.10.4 passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚	javadoc	1m 29s	branch-0.10.4 passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+0 🆗	spotbugs	1m 19s	Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	4m 7s	branch-0.10.4 passed
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	1m 8s	the patch passed
+1 💚	compile	1m 21s	the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚	javac	1m 21s	the patch passed
+1 💚	compile	1m 7s	the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
+1 💚	javac	1m 7s	the patch passed
-0 ⚠️	checkstyle	0m 15s	tez-runtime-library: The patch generated 1 new + 24 unchanged - 0 fixed = 25 total (was 24)
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	0m 53s	the patch passed with JDK Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04
+1 💚	javadoc	0m 51s	the patch passed with JDK Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
-1 ❌	findbugs	1m 15s	tez-dag generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
		_ Other Tests _
+1 💚	unit	2m 17s	tez-api in the patch passed.
+1 💚	unit	8m 39s	tez-runtime-library in the patch passed.
+1 💚	unit	4m 51s	tez-dag in the patch passed.
+1 💚	asflicense	0m 35s	The patch does not generate ASF License warnings.
		89m 6s

Reason	Tests
FindBugs	module:tez-dag
	Load of known null value in org.apache.tez.dag.app.dag.impl.VertexImpl.setParallelismWrapper(int, VertexLocationHint, Map, Map, boolean) At VertexImpl.java:in org.apache.tez.dag.app.dag.impl.VertexImpl.setParallelismWrapper(int, VertexLocationHint, Map, Map, boolean) At VertexImpl.java:[line 2004]

Subsystem	Report/Notes
Docker	ClientAPI=1.47 ServerAPI=1.47 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-391/1/artifact/out/Dockerfile
GITHUB PR	#391
JIRA Issue	TEZ-4603
Optional Tests	dupname asflicense javac javadoc unit spotbugs findbugs checkstyle compile
uname	Linux ff1045aac604 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	branch-0.10.4 / 5b5fff6
Default Java	Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.26+4-post-Ubuntu-1ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_442-8u442-b06us1-0ubuntu122.04-b06
checkstyle	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-391/1/artifact/out/diff-checkstyle-tez-runtime-library.txt
findbugs	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-391/1/artifact/out/new-findbugs-tez-dag.html
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-391/1/testReport/
Max. process+thread count	1090 (vs. ulimit of 5500)
modules	C: tez-api tez-runtime-library tez-dag U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-391/1/console
versions	git=2.34.1 maven=3.6.3 findbugs=3.0.1
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

[mayankkunwar]

Closing this PR, will create a similar PR for master branch