Skip to content

Conversation

@mergify
Copy link
Contributor

@mergify mergify bot commented Dec 31, 2025

Why I'm doing:

The mem_pool property for resource groups was recently introduced. See related pull requests below:

Currently, there is no way to display the usage metrics of the memory pools.

What I'm doing:

This pull request extends the result of show usage resource groups; command with the following:

Column Description
Id
Backend
BEInUseCpuCores
BEInUseMemBytes
BERunningQueries
+ BEMemLimitBytes Memory limit of the resource group.
+ BEMemPool Name of the memory pool the resource group belongs to.
+ BEMemPoolInUseMemBytes Current total memory usage of the mem_pool.
+ BEMemPoolMemLimitBytes Specified memory limit of the mem_pool.
  • The current implementation of memory pools requires configuring the memory limit of all resource groups (BEMemLimitBytes) under the same memory pool to the same value. This value also corresponds to the memory limit of the memory pool (BEMemPoolMemLimitBytes). The displayed values in these two columns will coincide due to this restriction, but it might change in the future.
  • The value of BEMemPoolInUseMemBytes is the sum of memory usages of each resource group that belongs to that memory pool. See the screenshots below for an example scenario.

Notes/Discussion

  • If two workgroups with the same id but different versions exist at the same time, the metrics to be reported for memory pools must belong to the latest version of that workgroup. This necessitates knowing which group_version a TResourceGroupUsage object belongs to. I tried two different approaches:
    • In resorce_group_usage_recorder.cpp, I added an auxillary VersionedResourceGroupUsage struct which had a group_version and a TResourceGroupUsage object as a pair. This solution worked, but it is just too messy for something as simple as this. See b9c4141
  • (Preferred) I added a group_version field to the ResourceUsage.thrift file and directly use it. This is much cleaner. See bba2165
    • The group_version is only used by resorce_group_usage_recorder.cpp in the backend, and is not read from the thrift file on the front end side.

Demo

demo-0
demo-1
demo-2
demo-3

Fixes #issue

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.0
    • 3.5
    • 3.4
    • 3.3

Note

Enhances resource group usage reporting with memory‑pool metrics end-to-end.

  • FE: Extends SHOW USAGE RESOURCE GROUPS with BEMemLimitBytes, BEMemPool, BEMemPoolInUseMemBytes, BEMemPoolMemLimitBytes; updates metadata, rendering, and tests
  • BE: ResourceGroupUsageRecorder now includes group_version, mem_limit_bytes, mem_pool, and mem‑pool limit/usage; merges per‑group stats preferring latest group_version
  • BE: WorkGroup exposes parent memory limit/usage helpers used for mem‑pool values
  • FE runtime: ComputeNode.ResourceGroupUsage extended to carry new fields and map from thrift
  • Thrift: adds fields to TResourceGroupUsage for mem limits/pool and group_version

Written by Cursor Bugbot for commit a07cc5e. This will update automatically on new commits. Configure here.


This is an automatic backport of pull request #66690 done by [Mergify](https://mergify.com).

…ics (#66690)

Signed-off-by: arin-mirza <[email protected]>
(cherry picked from commit f4a8df5)

# Conflicts:
#	fe/fe-core/src/main/java/com/starrocks/qe/ShowResultMetaFactory.java
#	fe/fe-core/src/main/java/com/starrocks/sql/ast/ShowResourceGroupUsageStmt.java
@mergify mergify bot added the conflicts label Dec 31, 2025
@mergify
Copy link
Contributor Author

mergify bot commented Dec 31, 2025

Cherry-pick of f4a8df5 has failed:

On branch mergify/bp/branch-4.0/pr-66690
Your branch is up to date with 'origin/branch-4.0'.

You are currently cherry-picking commit f4a8df573a.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   be/src/agent/resource_group_usage_recorder.cpp
	modified:   be/src/exec/workgroup/work_group.h
	modified:   be/test/agent/resource_group_usage_recorder_test.cpp
	modified:   fe/fe-core/src/main/java/com/starrocks/system/ComputeNode.java
	modified:   fe/fe-core/src/test/java/com/starrocks/qe/ShowStmtMetaTest.java
	modified:   fe/fe-core/src/test/java/com/starrocks/qe/scheduler/QueryQueueManagerTest.java
	modified:   gensrc/thrift/ResourceUsage.thrift

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   fe/fe-core/src/main/java/com/starrocks/qe/ShowResultMetaFactory.java
	both modified:   fe/fe-core/src/main/java/com/starrocks/sql/ast/ShowResourceGroupUsageStmt.java

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

@mergify
Copy link
Contributor Author

mergify bot commented Dec 31, 2025

@mergify[bot]: Backport conflict, please reslove the conflict and resubmit the pr

auto-merge was automatically disabled December 31, 2025 17:36

Pull request was closed

@mergify
Copy link
Contributor Author

mergify bot commented Dec 31, 2025

🧪 CI Insights

Here's what we observed from your CI run for a2dbdd9.

❌ Job Failures

Pipeline Job Health on branch-4.0 Retries 🔍 CI Insights 📄 Logs
PR CHECKER automerge-check Unknown 0 View View

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant