[Enhancement] Implement metrics reporting for MemTrackerManager

[arin-mirza]

Why I'm doing:

There are currently no backend metrics reporting for memory pools.

I previously tried to add them by extending the workgroup metrics, but this turned out to be an incorrect approach:

• [Enhancement] Add mem_pool_use_ratio metric to be #67701

What I'm doing:

This PR implements metric reporting for MemTrackerManager and adds the following new metrics:

• mem_pool_mem_limit_bytes
• mem_pool_mem_usage_bytes
• mem_pool_mem_usage_ratio
• mem_pool_workgroup_count

The implementation follows the same locking structure that is present in WorkGroupManager.

• It was necessary to add a new mutex for MemTrackerManager because the update_metrics callback hook passed to MetricRegistry needs to be a closure which captures a write lock.
• The unlocked gap inside add_metrics method is unavoidable to AB-BA deadlock scenario with the metrics collector.
• Metrics entries are never deleted as it would complicate the thread synchronization even further. This is also the case for the existing implementation in WorkGroupManager.

Minor: Changed list_mem_trackers() method to not return the default memory pool name.

Tests and Docs

• I did not add any test cases as there were not any for workgroup metrics either. Let me know if this is necessary.
• I did not verify that the new metrics are being reported correctly by building and running the starrocks fe/be, as I am currently unable to build the engine locally.
• I updated the user documentation.
  • I am not a Chinese or Japanese speaker so I used AI for the translation. I would appreciate it if a native speaker could review my additions to ensure the tone is correct. :)

What type of PR is this:

• BugFix
• Feature
• Enhancement
• Refactor
• UT
• Doc
• Tool

Does this PR entail a change in behavior?

• Yes, this PR will result in a change in behavior.
• No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

• Interface/UI changes: syntax, type conversion, expression evaluation, display information
• Parameter changes: default values, similar parameters but with different default values
• Policy changes: use new policy to replace old one, functionality automatically enabled
• Feature removed
• Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

• I have added test cases for my bug fix or my new feature
• This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function
  • This pr needs auto generate documentation
• This is a backport pr

Bugfix cherry-pick branch check:

• I have checked the version labels which the pr will be auto-backported to the target branch
  • 4.1
  • 4.0
  • 3.5
  • 3.4

[StarRocks-Reviewer]

@cursor review

[StarRocks-Reviewer]

@cursor review

[arin-mirza]

@alvin-celerdata @kevincai I closed the previous PR where you were reviewers, can I get a review for this one, please? :)

[alvin-celerdata]

@codex review

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f7a13d9f6d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[StarRocks-Reviewer]

@cursor review

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

[github-actions]

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

[github-actions]

[FE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

[github-actions]

[BE Incremental Coverage Report]

✅ pass : 57 / 68 (83.82%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	be/src/exec/workgroup/mem_tracker_manager.cpp	55	66	83.33%	[120, 150, 154, 155, 156, 158, 159, 161, 162, 164, 165]
🔵	be/src/exec/workgroup/mem_tracker_manager.h	2	2	100.00%	[]

[arin-mirza]

@alvin-celerdata Can this PR be merged? Is there anything else that needs to be done?

[Copilot]

Pull request overview

This PR implements metrics reporting for MemTrackerManager to expose memory pool usage statistics. Previously, there were no backend metrics for memory pools. The implementation adds four new metrics: mem_pool_mem_limit_bytes, mem_pool_mem_usage_bytes, mem_pool_mem_usage_ratio, and mem_pool_workgroup_count. The implementation follows the established locking and metrics registration patterns from WorkGroupManager to avoid deadlocks with the metrics collector.

Changes:

• Added metrics infrastructure to MemTrackerManager with thread-safe registration and update mechanisms
• Updated list_mem_trackers() to exclude the default memory pool, improving consistency
• Added documentation in English, Chinese, and Japanese for the new metrics

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
be/src/exec/workgroup/mem_tracker_manager.h	Added MemTrackerMetrics struct, metrics-related private methods, and mutex for thread synchronization
be/src/exec/workgroup/mem_tracker_manager.cpp	Implemented metrics registration, update logic, and modified list_mem_trackers() to exclude default pool
be/test/exec/workgroup/work_group_manager_test.cpp	Updated test expectations to reflect that default memory pool is no longer included in the list, removed unnecessary sleep
docs/en/administration/management/monitoring/metrics.md	Added English documentation for the four new metrics
docs/zh/administration/management/monitoring/metrics.md	Added Chinese documentation for the four new metrics
docs/ja/administration/management/monitoring/metrics.md	Added Japanese documentation for the four new metrics