TEZ-4470: typo fixes #264

mlorek · 2023-02-04T10:13:09Z

minor fixes in var,method names and comments

abstractdog · 2023-02-06T07:16:33Z

not sure what's happening in the precommit, need to check this week:
https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/1/console

mlorek · 2023-02-07T09:09:58Z

not sure what's happening in the precommit, need to check this week: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/1/console

anything i can do to help?

abstractdog · 2023-02-15T07:54:42Z

tez-dag/src/main/java/org/apache/tez/dag/app/dag/impl/ScatterGatherEdgeManager.java

    int physicalOutputs = getContext().getDestinationVertexNumTasks();
    Preconditions.checkArgument(physicalOutputs >= 0,
-        "ScatteGather edge manager must have destination vertex task parallelism specified");
+        "ScatterGather edge manager must have destination vertex task parallelism specified");


please adjust the asserted message in TestEdge.testScatterGatherManager as well

abstractdog · 2023-02-15T07:55:30Z

thanks @mlorek for taking care of this
could you please check how the mvnsite failures are related to the patch (if they're related)?

mlorek · 2023-02-22T08:53:39Z

mvnsite errors are related to:

Loading source files for package org.apache.tez.examples...
Constructing Javadoc information...
IncludePublicAnnotationsStandardDoclet
...
[ERROR] Exit code: 1 - javadoc: error - No public or protected classes found to document.

mlorek · 2023-02-24T09:31:43Z

@abstractdog all good?

# Conflicts: # tez-runtime-library/src/main/java/org/apache/tez/runtime/library/common/shuffle/orderedgrouped/ShuffleScheduler.java

mlorek · 2023-03-14T08:21:05Z

@abstractdog can you take another look?

abstractdog · 2023-03-25T19:42:59Z

@mlorek : have you had the chance to check the root cause of test failures in some modules?

mlorek · 2023-03-28T08:50:32Z

@abstractdog I see only 1 failed test and it's not related to my changes:

Error Message
port out of range:-1
Stacktrace
java.lang.IllegalArgumentException: port out of range:-1
at org.apache.tez.test.TestAMRecovery.runDAGAndVerify(TestAMRecovery.java:416)
at org.apache.tez.test.TestAMRecovery.testVertexCompletelyFinished_Broadcast(TestAMRecovery.java:242)

tez-yetus · 2023-05-16T15:51:02Z

💔 -1 overall

Vote	Subsystem	Runtime	Comment
+0 🆗	reexec	25m 24s	Docker mode activated.
		_ Prechecks _
+1 💚	dupname	0m 1s	No case conflicting files found.
+1 💚	@author	0m 0s	The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s	The patch appears to include 1 new or modified test files.
		_ master Compile Tests _
+0 🆗	mvndep	6m 9s	Maven dependency ordering for branch
+1 💚	mvninstall	10m 17s	master passed
+1 💚	compile	6m 30s	master passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu122.04
+1 💚	compile	6m 16s	master passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~22.04-b09
+1 💚	checkstyle	5m 35s	master passed
-1 ❌	mvnsite	0m 31s	tez-examples in master failed.
-1 ❌	mvnsite	0m 41s	tez-dag in master failed.
-1 ❌	mvnsite	0m 34s	tez-tests in master failed.
-1 ❌	mvnsite	0m 31s	tez-aux-services in master failed.
-1 ❌	mvnsite	0m 35s	job-analyzer in master failed.
+1 💚	javadoc	5m 36s	master passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu122.04
+1 💚	javadoc	5m 22s	master passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~22.04-b09
+0 🆗	spotbugs	0m 44s	Used deprecated FindBugs config; considering switching to SpotBugs.
+0 🆗	findbugs	0m 47s	branch/tez-ui no findbugs output file (findbugsXml.xml)
+0 🆗	findbugs	0m 23s	branch/docs no findbugs output file (findbugsXml.xml)
-0 ⚠️	patch	2m 0s	Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary.
		_ Patch Compile Tests _
+0 🆗	mvndep	0m 10s	Maven dependency ordering for patch
+1 💚	mvninstall	5m 49s	the patch passed
+1 💚	jshint	53m 18s	There were no new jshint issues.
+1 💚	compile	6m 24s	the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu122.04
+1 💚	javac	6m 24s	the patch passed
+1 💚	compile	6m 1s	the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~22.04-b09
+1 💚	javac	6m 1s	the patch passed
+1 💚	checkstyle	0m 29s	tez-api: The patch generated 0 new + 198 unchanged - 1 fixed = 198 total (was 199)
+1 💚	checkstyle	0m 23s	The patch passed checkstyle in tez-common
+1 💚	checkstyle	0m 24s	The patch passed checkstyle in tez-runtime-internals
-0 ⚠️	checkstyle	0m 36s	tez-runtime-library: The patch generated 6 new + 373 unchanged - 13 fixed = 379 total (was 386)
+1 💚	checkstyle	0m 21s	The patch passed checkstyle in tez-examples
-0 ⚠️	checkstyle	0m 45s	tez-dag: The patch generated 5 new + 509 unchanged - 5 fixed = 514 total (was 514)
+1 💚	checkstyle	0m 24s	The patch passed checkstyle in tez-tests
+1 💚	checkstyle	0m 19s	The patch passed checkstyle in tez-ui
+1 💚	checkstyle	0m 23s	The patch passed checkstyle in tez-history-parser
+1 💚	checkstyle	0m 23s	The patch passed checkstyle in tez-aux-services
-0 ⚠️	checkstyle	0m 22s	tez-tools/analyzers/job-analyzer: The patch generated 1 new + 13 unchanged - 1 fixed = 14 total (was 14)
+1 💚	checkstyle	0m 19s	The patch passed checkstyle in docs
+1 💚	markdownlint	0m 2s	There were no new markdownlint issues.
-1 ❌	mvnsite	0m 30s	tez-examples in the patch failed.
-1 ❌	mvnsite	0m 38s	tez-dag in the patch failed.
-1 ❌	mvnsite	0m 34s	tez-tests in the patch failed.
-1 ❌	mvnsite	0m 32s	tez-aux-services in the patch failed.
-1 ❌	mvnsite	0m 33s	job-analyzer in the patch failed.
+1 💚	whitespace	0m 0s	The patch has no whitespace issues.
+1 💚	javadoc	5m 29s	the patch passed with JDK Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu122.04
+1 💚	javadoc	5m 18s	the patch passed with JDK Private Build-1.8.0_362-8u362-ga-0ubuntu1~22.04-b09
+0 🆗	findbugs	0m 45s	tez-ui has no data from findbugs
+0 🆗	findbugs	0m 23s	docs has no data from findbugs
		_ Other Tests _
+1 💚	unit	2m 26s	tez-api in the patch passed.
+1 💚	unit	0m 38s	tez-common in the patch passed.
+1 💚	unit	0m 51s	tez-runtime-internals in the patch passed.
+1 💚	unit	5m 54s	tez-runtime-library in the patch passed.
+1 💚	unit	0m 27s	tez-examples in the patch passed.
+1 💚	unit	5m 21s	tez-dag in the patch passed.
+1 💚	unit	44m 15s	tez-tests in the patch passed.
+1 💚	unit	1m 40s	tez-ui in the patch passed.
+1 💚	unit	2m 41s	tez-history-parser in the patch passed.
+1 💚	unit	3m 5s	tez-aux-services in the patch passed.
+1 💚	unit	3m 0s	job-analyzer in the patch passed.
+1 💚	unit	0m 24s	docs in the patch passed.
+1 💚	asflicense	4m 44s	The patch does not generate ASF License warnings.
		278m 48s

Subsystem	Report/Notes
Docker	ClientAPI=1.42 ServerAPI=1.42 base: https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/Dockerfile
GITHUB PR	#264
JIRA Issue	TEZ-4470
Optional Tests	dupname asflicense markdownlint mvnsite javac javadoc unit spotbugs findbugs checkstyle compile jshint
uname	Linux c4bd388351c5 4.15.0-206-generic #217-Ubuntu SMP Fri Feb 3 19:10:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/tez.sh
git revision	master / `ccad71d`
Default Java	Private Build-1.8.0_362-8u362-ga-0ubuntu1~22.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.18+10-post-Ubuntu-0ubuntu122.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_362-8u362-ga-0ubuntu1~22.04-b09
mvnsite	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/branch-mvnsite-tez-examples.txt
mvnsite	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/branch-mvnsite-tez-dag.txt
mvnsite	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/branch-mvnsite-tez-tests.txt
mvnsite	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/branch-mvnsite-tez-plugins_tez-aux-services.txt
mvnsite	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/branch-mvnsite-tez-tools_analyzers_job-analyzer.txt
checkstyle	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/diff-checkstyle-tez-runtime-library.txt
checkstyle	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/diff-checkstyle-tez-dag.txt
checkstyle	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/diff-checkstyle-tez-tools_analyzers_job-analyzer.txt
mvnsite	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/patch-mvnsite-tez-examples.txt
mvnsite	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/patch-mvnsite-tez-dag.txt
mvnsite	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/patch-mvnsite-tez-tests.txt
mvnsite	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/patch-mvnsite-tez-plugins_tez-aux-services.txt
mvnsite	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/artifact/out/patch-mvnsite-tez-tools_analyzers_job-analyzer.txt
Test Results	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/testReport/
Max. process+thread count	2100 (vs. ulimit of 5500)
modules	C: tez-api tez-common tez-runtime-internals tez-runtime-library tez-examples tez-dag tez-tests tez-ui tez-plugins/tez-history-parser tez-plugins/tez-aux-services tez-tools/analyzers/job-analyzer docs U: .
Console output	https://ci-hadoop.apache.org/job/tez-multibranch/job/PR-264/10/console
versions	git=2.34.1 maven=3.6.3 findbugs=3.0.1 jshint=2.12.0 markdownlint=0.23.2
Powered by	Apache Yetus 0.12.0 https://yetus.apache.org

This message was automatically generated.

abstractdog · 2023-06-12T09:50:37Z

this patch waits for a long time, let me check the mvn site problems in the scope of TEZ-4495
I don't think they are related as I was able to reproduce it on master too, so I'm about to approve this PR if every failure is unrelated and tracked
UPDATE: confirmed, none of the checkstyle problems was caused by this patch, they were already present

abstractdog

+1

typos fix

ec1e989