fix(monitor): avoid uint64 underflow when deriving the legacy device-memory offset label - #2552
fix(monitor): avoid uint64 underflow when deriving the legacy device-memory offset label#2552Luffy-nani wants to merge 3 commits into
Conversation
…memory offset label Signed-off-by: Venkat <venkatedupuganti0311@gmail.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Luffy-nani The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Welcome @Luffy-nani! It looks like this is your first PR to Project-HAMi/HAMi 🎉 |
📝 WalkthroughWalkthroughThe legacy container memory metric now uses the recorded per-device memory offset. Test fixtures support indexed offsets, and a regression test verifies correct emission when component sizes exceed total memory. ChangesLegacy device-memory metrics
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cmd/vGPUmonitor/metrics_container_test.go`:
- Around line 158-159: Update the comment immediately above
TestCollectContainerMetricsLegacyOffsetNoUnderflow so it describes that test’s
legacy offset behavior and no-underflow validation, removing the stale
TestCheckBlocking_MultiDevice description.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 88e8e39a-c28b-4ed9-9318-bc62edc9995e
📒 Files selected for processing (3)
cmd/vGPUmonitor/feedback_test.gocmd/vGPUmonitor/metrics.gocmd/vGPUmonitor/metrics_container_test.go
aad979e to
954ce0a
Compare
954ce0a to
823d949
Compare
|
Thanks for investigating the underflow risk, but this implementation is not viable. The current head does not compile: the cleanup closure assigns legacyCtrDeviceMemorydesc = prev after the prev declaration was removed. More importantly, the pinned HAMi-core production writer never assigns a nonzero used[dev].offset; allocation updates total and the context/module/data fields instead. DeviceMemoryOffset() therefore reads zero from real caches, while the test invents an offset=7 producer state that does not exist. This change would silently replace a meaningful legacy residual with zero and does not solve the possible torn-read problem. A coherent snapshot/seqlock-style fix or another producer-backed contract would require a separate design and real reproduction, so we are closing this PR. |
|
got it! thanks |
What type of PR is this?
/kind bug
What this PR does / why we need it:
This fixes a possible uint64 underflow when collectContainerMetrics calculates the legacy Device_memory_desc_of_container metric's offset label
The values used for the calculation come from separate reads of shared memory, so they can become inconsistent if hami-core updates the memory information between reads. In that case, the subtraction can wrap around to a value close to math.MaxUint64
Instead of calculating the offset from those four values, this change uses the existing DeviceMemoryOffset() method, which returns the recorded offset directly and avoids the underflow
A regression test was also added to cover the torn-read scenario
Which issue(s) this PR fixes:
Fixes #2551
Special notes:
The full test suite could not be run locally because the environment could not access the Go module proxy required by the repository's Go toolchain version. gofmt was run on all touched files
The new test also avoids calling initLegacyDescriptors() because it modifies package-level Prometheus descriptors and could affect other tests
Does this PR introduce a user-facing change?:
No. This does not change the intended metric behavior. It prevents an invalid, wrapped offset value from being exposed when shared-memory values are read inconsistently
Summary by CodeRabbit