[ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil #7416

PDavid · 2025-10-27T14:22:45Z

[ADDENDUM] TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build.

PDavid · 2025-10-27T14:24:03Z

hbase-server/src/test/java/org/apache/hadoop/hbase/master/http/TestMasterStatusUtil.java

    conf.setBoolean("hbase.master.ui.fragmentation.enabled", true);
    Map<String, Integer> fragmentationInfo = MasterStatusUtil.getFragmentationInfo(master, conf);
    assertNotNull(fragmentationInfo);
-    assertEquals(1, fragmentationInfo.size());


For me this locally was always 1 but in the nightly build it was 11:
https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1359/testReport/junit/org.apache.hadoop.hbase.master.http/TestMasterStatusUtil/health_checks___yetus_jdk17_hadoop3_backwards_compatibility_checks___testGetFragmentationInfoTurnedOn/

Is this expected or something is broken?

Thanks, very good question.

So TestMasterStatusUtil.getFragmentationInfo() was originally part of MasterStatusServlet and I migrated it to the static util class under #6875 and added the unit tests.

Locally I got different results than in the master build but I thought this can be OK because the code invokes FSUtils.getTableFragmentation() which does:

Runs through the HBase rootdir and checks how many stores for each table have more than one file in them. Checks -ROOT- and hbase:meta too. The total percentage across all tables is stored under the special key "-TOTAL-".

As I understand, this can be different in many cases so I just end up checking here that the map is not empty.

NihalJain · 2025-10-30T10:56:23Z

Hi @PDavid is this an addendum? you seem to be using an existing jira title here. Also is this the right fix, may be checkon what should be the expected assertion?

PDavid · 2025-10-30T18:46:35Z

Hi @PDavid is this an addendum? you seem to be using an existing jira title here.

Hi @NihalJain,
Thanks, yes this would need to be an addendum.

Also is this the right fix, may be checkon what should be the expected assertion?

To be completely honest I'm having trouble finding out the right value to expect in the assertion. Locally was always 1 but in the nightly build it was 11:
https://ci-hbase.apache.org/job/HBase%20Nightly/job/master/1359/testReport/junit/org.apache.hadoop.hbase.master.http/TestMasterStatusUtil/health_checks___yetus_jdk17_hadoop3_backwards_compatibility_checks___testGetFragmentationInfoTurnedOn/

I also thought about checking that the map is not empty. 🤔

TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build

…ationInfoTurnedOn

Apache-HBase · 2025-11-04T07:35:52Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 27s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+1 💚	mvninstall	3m 30s		master passed
+1 💚	compile	3m 21s		master passed
+1 💚	checkstyle	1m 1s		master passed
+1 💚	spotbugs	1m 39s		master passed
+1 💚	spotless	0m 50s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+1 💚	mvninstall	2m 58s		the patch passed
+1 💚	compile	3m 20s		the patch passed
+1 💚	javac	3m 20s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 57s		the patch passed
+1 💚	spotbugs	1m 39s		the patch passed
+1 💚	hadoopcheck	11m 27s		Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
+1 💚	spotless	1m 14s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 17s		The patch does not generate ASF License warnings.
		41m 13s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7416/4/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#7416
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 1dcd41e90786 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `bcd72e1`
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	85 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7416/4/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

Apache-HBase · 2025-11-04T11:21:01Z

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 31s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+1 💚	mvninstall	3m 41s		master passed
+1 💚	compile	1m 1s		master passed
+1 💚	javadoc	0m 30s		master passed
+1 💚	shadedjars	6m 22s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	3m 11s		the patch passed
+1 💚	compile	1m 0s		the patch passed
+1 💚	javac	1m 0s		the patch passed
+1 💚	javadoc	0m 28s		the patch passed
+1 💚	shadedjars	6m 21s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	238m 7s		hbase-server in the patch passed.
		266m 25s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7416/4/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7416
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux aac1c2242754 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / `bcd72e1`
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7416/4/testReport/
Max. process+thread count	3481 (vs. ulimit of 30000)
modules	C: hbase-server U: hbase-server
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7416/4/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build Signed-off-by: Nihal Jain <[email protected]> Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 8ef271f)

* HBASE-29223 Migrate Master Status Jamon page back to JSP (#6875) The JSP code is equivalent to the Jamon code, just changed the syntax back to JSP. Request attributes are used to transfer data between JSP pages. Tried to preserve the code as much as possible but did some changes: Sub-templates were usually extracted to separate JSP file (and included with `<jsp:include`), in some case it was extracted as Java method. Extracted some sections from master page to separate JSP pages: - Software Attributes - Warnings Extracted the long JavaScript from the master page which executes on page load to separate JS file. Extracted some frequently used static methods to a new util class: `MasterStatusUtil`. Also added unit tests for the static methods in `MasterStatusUtil`. Changed the Master Status page back to `/master.jsp` again. Now made sure that `/master-status` redirects to `/master.jsp`. Signed-off-by: Istvan Toth <[email protected]> (cherry picked from commit be40011) * HBASE-29223 spotless fix * [ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil (#7416) TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build Signed-off-by: Nihal Jain <[email protected]> Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 8ef271f)

* HBASE-29223 Migrate Master Status Jamon page back to JSP (#6875) The JSP code is equivalent to the Jamon code, just changed the syntax back to JSP. Request attributes are used to transfer data between JSP pages. Tried to preserve the code as much as possible but did some changes: Sub-templates were usually extracted to separate JSP file (and included with `<jsp:include`), in some case it was extracted as Java method. Extracted some sections from master page to separate JSP pages: - Software Attributes - Warnings Extracted the long JavaScript from the master page which executes on page load to separate JS file. Extracted some frequently used static methods to a new util class: `MasterStatusUtil`. Also added unit tests for the static methods in `MasterStatusUtil`. Changed the Master Status page back to `/master.jsp` again. Now made sure that `/master-status` redirects to `/master.jsp`. Signed-off-by: Istvan Toth <[email protected]> (cherry picked from commit be40011) * [ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil (#7416) TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build Signed-off-by: Nihal Jain <[email protected]> Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 8ef271f)

TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build Signed-off-by: Nihal Jain <[email protected]> Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 8ef271f)

apache#7478) * HBASE-29223 Migrate Master Status Jamon page back to JSP (apache#6875) The JSP code is equivalent to the Jamon code, just changed the syntax back to JSP. Request attributes are used to transfer data between JSP pages. Tried to preserve the code as much as possible but did some changes: Sub-templates were usually extracted to separate JSP file (and included with `<jsp:include`), in some case it was extracted as Java method. Extracted some sections from master page to separate JSP pages: - Software Attributes - Warnings Extracted the long JavaScript from the master page which executes on page load to separate JS file. Extracted some frequently used static methods to a new util class: `MasterStatusUtil`. Also added unit tests for the static methods in `MasterStatusUtil`. Changed the Master Status page back to `/master.jsp` again. Now made sure that `/master-status` redirects to `/master.jsp`. Signed-off-by: Istvan Toth <[email protected]> (cherry picked from commit be40011) * HBASE-29223 spotless fix * [ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil (apache#7416) TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build Signed-off-by: Nihal Jain <[email protected]> Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 8ef271f) (cherry picked from commit d481a46)

* HBASE-29531 Migrate RegionServer Status Jamon page back to JSP (#7371) This is the 2/3 step of the Jamon to JSP migration: the Region Server Status page. Did the migration the same way as for the Master Status page: #6875 Migrated the Jamon code to JSP as close as possible. Extracted the duplicated `formatZKString` method to new java class: ZKStringFormatter and added unit tests. Changed the Region Server Status page back to `/regionserver.jsp`. Made sure that `/rs-status` redirects to `/regionserver.jsp`. Extracted the BlockCache inline CSS styles to `hbase.css` file. Also extracted the large BlockCache Hit Ratio periods paging JavaScript code to separate .js file. Introduced a `src/main/resources/hbase-webapps/common` directory where we can place common JSP files which are used by both Master and RegionServer JSP pages. This required to adjust the JSP compiler Maven Antrun plugin a bit. Extracted the inline tablesorter initialization JavaScript code to separate file. Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 7892207) * [ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil (#7416) TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build Signed-off-by: Nihal Jain <[email protected]> Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 8ef271f)

apache#7478) * HBASE-29223 Migrate Master Status Jamon page back to JSP (apache#6875) The JSP code is equivalent to the Jamon code, just changed the syntax back to JSP. Request attributes are used to transfer data between JSP pages. Tried to preserve the code as much as possible but did some changes: Sub-templates were usually extracted to separate JSP file (and included with `<jsp:include`), in some case it was extracted as Java method. Extracted some sections from master page to separate JSP pages: - Software Attributes - Warnings Extracted the long JavaScript from the master page which executes on page load to separate JS file. Extracted some frequently used static methods to a new util class: `MasterStatusUtil`. Also added unit tests for the static methods in `MasterStatusUtil`. Changed the Master Status page back to `/master.jsp` again. Now made sure that `/master-status` redirects to `/master.jsp`. Signed-off-by: Istvan Toth <[email protected]> (cherry picked from commit be40011) * HBASE-29223 spotless fix * [ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil (apache#7416) TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build Signed-off-by: Nihal Jain <[email protected]> Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 8ef271f) (cherry picked from commit d481a46)

The JSP code is equivalent to the Jamon code, just changed the syntax back to JSP. Request attributes are used to transfer data between JSP pages. Tried to preserve the code as much as possible but did some changes: Sub-templates were usually extracted to separate JSP file (and included with `<jsp:include`), in some case it was extracted as Java method. Extracted some sections from master page to separate JSP pages: - Software Attributes - Warnings Extracted the long JavaScript from the master page which executes on page load to separate JS file. Extracted some frequently used static methods to a new util class: `MasterStatusUtil`. Also added unit tests for the static methods in `MasterStatusUtil`. Changed the Master Status page back to `/master.jsp` again. Now made sure that `/master-status` redirects to `/master.jsp`. Signed-off-by: Istvan Toth <[email protected]> (cherry picked from commit be40011) * [ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil (#7416) TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build Signed-off-by: Nihal Jain <[email protected]> Signed-off-by: Duo Zhang <[email protected]> (cherry picked from commit 8ef271f) (cherry picked from commit d481a46)

PDavid self-assigned this Oct 27, 2025

PDavid added the bug label Oct 27, 2025

PDavid commented Oct 27, 2025

View reviewed changes

PDavid marked this pull request as ready for review October 28, 2025 08:12

PDavid requested review from Apache9 and stoty October 28, 2025 12:01

PDavid changed the title ~~HBASE-29223 Fix TestMasterStatusUtil~~ [ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil Oct 31, 2025

PDavid added 2 commits October 31, 2025 13:29

HBASE-29223 Fix TestMasterStatusUtil

7ebbdf5

TestMasterStatusUtil.testGetFragmentationInfoTurnedOn failed in master nightly build

HBASE-29223 Improve assertion in TestMasterStatusUtil.testGetFragment…

bcd72e1

…ationInfoTurnedOn

PDavid force-pushed the HBASE-29223-TestMasterStatusUtil-fix branch from bba011e to bcd72e1 Compare October 31, 2025 12:29

NihalJain approved these changes Oct 31, 2025

View reviewed changes

This comment has been minimized.

Sign in to view

Apache9 approved these changes Nov 4, 2025

View reviewed changes

PDavid removed the bug label Nov 4, 2025

PDavid merged commit 8ef271f into apache:master Nov 4, 2025
1 check passed

PDavid deleted the HBASE-29223-TestMasterStatusUtil-fix branch November 4, 2025 13:21

PDavid mentioned this pull request Dec 1, 2025

HBASE-29531 Migrate RegionServer Status Jamon page back to JSP #7491

Merged

PDavid mentioned this pull request Dec 10, 2025

HBASE-29223 Migrate Master Status Jamon page back to JSP (#6875) #7534

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil #7416

[ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil #7416

Uh oh!

PDavid commented Oct 27, 2025 •

edited

Loading

Uh oh!

PDavid Oct 27, 2025

Uh oh!

Apache9 Nov 1, 2025

Uh oh!

PDavid Nov 4, 2025

Uh oh!

Apache9 Nov 4, 2025

Uh oh!

NihalJain commented Oct 30, 2025

Uh oh!

PDavid commented Oct 30, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Apache-HBase commented Nov 4, 2025

Uh oh!

Apache-HBase commented Nov 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil #7416

[ADDENDUM] HBASE-29223 Fix TestMasterStatusUtil #7416

Uh oh!

Conversation

PDavid commented Oct 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PDavid Oct 27, 2025

Choose a reason for hiding this comment

Uh oh!

Apache9 Nov 1, 2025

Choose a reason for hiding this comment

Uh oh!

PDavid Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

Apache9 Nov 4, 2025

Choose a reason for hiding this comment

Uh oh!

NihalJain commented Oct 30, 2025

Uh oh!

PDavid commented Oct 30, 2025

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Apache-HBase commented Nov 4, 2025

Uh oh!

Apache-HBase commented Nov 4, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

PDavid commented Oct 27, 2025 •

edited

Loading