MAPREDUCE-7461: Fixed assertionComparision failure by resolving xml path for 'name'

[kavvya97]

Setup:
Java version: openjdk 11.0.20.1
Maven version: Apache Maven 3.6.3

Issue: https://issues.apache.org/jira/browse/MAPREDUCE-7461

Description of PR

The following tests can fail due to flakiness while comparing the contents of the generated XML response.
Module: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs#testJobIdXML
org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs#testJobsXML

Module: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs
org.apache.hadoop.mapreduce.v2.hs.webapp.TestHsWebServicesJobs#testJobIdXML

Steps to reproduce

1. git clone https://github.com/apache/hadoop
2. mvn install -pl hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app -am -DskipTests
3. Run the tests
   mvn -pl hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app test -Dtests=org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs#testJobIdXML
4. Run the test with the Nondex tool and observe the test results
   mvn -pl hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs#testJobsIdXML
   • Test Fails when Running Nondex in ONE mode (Assumes deterministic implementation of code but shuffled once different from underlying implementation) -DnondexMode=ONE & FULL Mode -DnondexMode=FULL (shuffles differently for each call)

Root Cause

The test attempts to send a HTTP GET request to a specific URL and expects a response in XML format. However, elements in the XML response order is not necessarily guaranteed. The contents of the XML and the tags are compared with Job contents from appContext based on job Id in verifyAMJobXML / verifyHsJobXML. When comparing the the XML contents, The <name> tag occurs in multiple places inside <job> field. The current test utlization WebServicesTestUtils.java getXmlString always takes the first tag irrespective of whether it is nested tag or in the direct child tag which causes the test to fail and become flaky.

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 8.377 s <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs
[ERROR] testJobIdXML(org.apache.hadoop.mapreduce.v2.app.webapp.TestAMWebServicesJobs)  Time elapsed: 8.361 s  <<< FAILURE!
java.lang.AssertionError:
[name]
Expecting:
 "mapreduce.job.acl-view-job"
to match pattern:
 "RandomWriter"

Fix

Since WebServicesTestUtils.java getXmlString compares the first tag only which might not necessary be the direct child tag, The Fix uses Xpath to resolve the conflicts and identify the direct child tag. Thus the test passes since the direct child tag is always retrieved irrespective of xml order.

How was this patch tested?

The fix was tested by running the Nondex plugin again and ensuring that all the tests pass in FULL Mode and ONE Mode of the Nondex runs.

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	17m 22s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 56s		Maven dependency ordering for branch
+1 💚	mvninstall	33m 49s		trunk passed
+1 💚	compile	1m 46s		trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	compile	1m 36s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	checkstyle	1m 14s		trunk passed
+1 💚	mvnsite	1m 16s		trunk passed
+1 💚	javadoc	1m 16s		trunk passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	1m 8s		trunk passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	1m 56s		trunk passed
+1 💚	shadedclient	33m 9s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 52s		the patch passed
+1 💚	compile	1m 36s		the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javac	1m 36s		the patch passed
+1 💚	compile	1m 27s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	javac	1m 27s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	1m 1s	/results-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt	hadoop-mapreduce-project/hadoop-mapreduce-client: The patch generated 4 new + 3 unchanged - 2 fixed = 7 total (was 5)
+1 💚	mvnsite	0m 57s		the patch passed
+1 💚	javadoc	0m 52s		the patch passed with JDK Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 50s		the patch passed with JDK Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
+1 💚	spotbugs	1m 56s		the patch passed
+1 💚	shadedclient	33m 46s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	8m 40s		hadoop-mapreduce-client-app in the patch passed.
+1 💚	unit	4m 44s		hadoop-mapreduce-client-hs in the patch passed.
+1 💚	asflicense	0m 38s		The patch does not generate ASF License warnings.
		172m 58s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6252/1/artifact/out/Dockerfile
GITHUB PR	#6252
JIRA Issue	MAPREDUCE-7461
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 594f3c272237 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / c132a53
Default Java	Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.20.1+1-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_382-8u382-ga-1~20.04.1-b05
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6252/1/testReport/
Max. process+thread count	739 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs U: hadoop-mapreduce-project/hadoop-mapreduce-client
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6252/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

[kavvya97]

Requesting for review