feat(vGPUmonitor): add missing GPU hardware health and power metrics

[devGPP23]

What this PR does:

This PR adds 5 new GPU hardware metrics to vGPUmonitor using NVML.
This will help users build much better and more detailed Grafana dashboards for monitoring cluster health.

The new metrics are:
hami_host_gpu_temperature_celsius (Temperature)
hami_host_gpu_power_usage_milliwatts (Current Power)
hami_host_gpu_power_limit_milliwatts (Power Limit)
hami_host_gpu_memory_total_bytes (Total Memory)
hami_host_gpu_memory_utilization_ratio (Memory Bandwidth Utilization)

How it works & safety:
I added explicit checks for nvml.ERROR_NOT_SUPPORTED.
If a GPU doesn't support reading temperature or power, it safely skips it without crashing the monitor.
It safely follows the exact same pattern already used for memory.Used.

Testing & Hardware Validation:: Passed local unit tests (TestDescribeCollectSync) which verifies the Prometheus descriptors.

Special Note for reviewer:
I do not currently have access to a physical NVIDIA GPU cluster. I have ensured compile-time safety and proper error handling, but I would really appreciate it if a maintainer could run this on a real GPU node to verify the final Prometheus output.

AI Assistance Disclosure:
I used an AI assistant to help research the NVML APIs and structure the code, but I have manually reviewed, checked the safety of the logic, and fully understand all the changes introduced in this PR.

Summary by CodeRabbit

• New Features
  • Added Prometheus metrics for GPU temperature, power usage, power limits, memory utilization, and total memory.
  • GPU health metrics are now collected and exposed for monitoring.
  • Unsupported temperature or power readings are handled gracefully without interrupting metric collection.

[hami-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: devGPP23
Once this PR has been reviewed and has the lgtm label, please assign wawa0210 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d8a2defc-5114-4d5c-b5bd-ba110ae425f4

📥 Commits

Reviewing files that changed from the base of the PR and between 3616313 and 37a72bb.

📒 Files selected for processing (1)

• cmd/vGPUmonitor/metrics.go

---

📝 Walkthrough

Walkthrough

vGPUmonitor adds Prometheus metrics for GPU memory, temperature, power usage, and power limits. GPU collection queries NVML health values, skips unsupported queries, logs other query failures, and continues metric collection.

Changes

GPU health metrics

Layer / File(s)	Summary
Metric descriptors and registration cmd/vGPUmonitor/metrics.go	Defines descriptors for GPU temperature, power usage, power limit, memory utilization, and total memory. Registers them with Prometheus.
GPU metric collection cmd/vGPUmonitor/metrics.go	Collects total memory and memory bandwidth utilization. Queries NVML for temperature and power metrics. Unsupported queries are skipped, while other failures are logged.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

• Project-HAMi/HAMi#2240: Both changes update GPU metric descriptors and Prometheus collection handling.
• Project-HAMi/HAMi#2452: Both changes add GPU memory metrics in cmd/vGPUmonitor/metrics.go.

Suggested reviewers: mesutoezdil, archlitchi

Poem

I hop through metrics, crisp and bright,
GPU health now fills the night.
Memory, power, heat all show,
NVML tells the values we know.
Unsupported calls step aside—
Prometheus gets the ride!

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the addition of GPU hardware health and power metrics.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches 💡 1

🛠️ Fix failing CI checks 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 8.62069% with 53 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
cmd/vGPUmonitor/metrics.go	8.62%	53 Missing ⚠️

Flag	Coverage Δ
unittests	63.97% <8.62%> (-0.24%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
cmd/vGPUmonitor/metrics.go	37.71% <8.62%> (-5.78%)	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[devGPP23]

cc/ @mesutoezdil @archlitchi @FouoF

The codecov/patch check is failing because the new NVML API calls are skipped in CI without physical GPUs. vGPUmonitor/metrics.go currently doesn't use an NVML mock interface.

Should I refactor the collector to support mocks to fix this coverage, or is it okay to bypass it for now?

[FouoF]

Thanks for the metrics work. This PR adds five public host metrics without behavior-level tests or user documentation. memory_total overlaps #2452, while the two _ratio metrics continue a 0–100 scale that conflicts with #2455's proposed 0–1 contract; patch coverage is also only 8.62% and failing. We should choose one public metrics schema before adding implementations, so we are closing this PR rather than publishing another competing contract.