Skip to content

refactor: upgrade health check config from env vars to top-level options - #2151

Closed
shellyco-code wants to merge 1 commit into
Project-HAMi:masterfrom
shellyco-code:refactor-health-config
Closed

refactor: upgrade health check config from env vars to top-level options#2151
shellyco-code wants to merge 1 commit into
Project-HAMi:masterfrom
shellyco-code:refactor-health-config

Conversation

@shellyco-code

@shellyco-code shellyco-code commented Jul 27, 2026

Copy link
Copy Markdown

What type of PR is this?

/kind feature
/kind cleanup

What this PR does / why we need it:
This PR resolves the // TODO technical debt in pkg/device-plugin/nvidiadevice/nvinternal/rm/health.go by upgrading the health check configuration so it no longer dynamically reads environment variables (DP_DISABLE_HEALTHCHECKS and DP_ENABLE_HEALTHCHECKS) on the fly via os.Getenv().

Instead, the configuration is now parsed exactly once at startup during plugin initialization (pkg/device-plugin/nvidiadevice/nvinternal/plugin/factory.go) and explicitly passed down into the rm.ResourceManager instances as state variables. This eliminates hidden dependencies on os.Getenv() deep within the business logic and makes testing and mocking significantly cleaner. The test suite (health_test.go and health_checkhealth_test.go) has also been refactored to inject the configuration into the mock structs rather than relying on t.Setenv().

Which issue(s) this PR fixes:
Fixes #2150

Special notes for your reviewer:
The commit is signed-off (DCO) and all associated tests have been successfully updated to natively test the new configuration injection logic without environment variable mocking!

Does this PR introduce a user-facing change?:

NONE


<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

* **New Features**
  * Added environment-based configuration for enabling or disabling device health checks.
  * Health-check settings now apply consistently across supported NVIDIA and Tegra devices.
  * Explicitly enabled checks take precedence over disabled checks when both settings overlap.

* **Bug Fixes**
  * Improved health-check configuration handling to avoid relying on process-wide state during device monitoring.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: shellyco-code <shellychahar57@gmail.com>
@hami-robot

hami-robot Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: shellyco-code
Once this PR has been reviewed and has the lgtm label, please assign wawa0210 for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hami-robot
hami-robot Bot requested a review from archlitchi July 27, 2026 10:46
@hami-robot
hami-robot Bot requested a review from mesutoezdil July 27, 2026 10:46
@github-actions github-actions Bot added kind/enhancement New feature or request and removed kind/feature new function kind/cleanup labels Jul 27, 2026
@hami-robot hami-robot Bot added the size/M label Jul 27, 2026
@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The NVIDIA plugin now reads health-check environment variables during startup, passes them into NVML and Tegra resource managers, stores them in manager state, and uses them during health-check XID selection. Tests now provide configuration directly instead of modifying process-wide environment state.

Changes

NVIDIA health-check configuration

Layer / File(s) Summary
Resource-manager configuration state
pkg/device-plugin/nvidiadevice/nvinternal/rm/rm.go, pkg/device-plugin/nvidiadevice/nvinternal/rm/nvml_manager.go, pkg/device-plugin/nvidiadevice/nvinternal/rm/tegra_manager.go
Resource managers store disableHealthChecks and enableHealthChecks, and both constructors accept and propagate these values.
Plugin startup configuration wiring
pkg/device-plugin/nvidiadevice/nvinternal/plugin/factory.go
The factory reads both environment variables and passes them to the NVML or Tegra constructor.
Health-check consumption and validation
pkg/device-plugin/nvidiadevice/nvinternal/rm/health.go, pkg/device-plugin/nvidiadevice/nvinternal/rm/health_checkhealth_test.go, pkg/device-plugin/nvidiadevice/nvinternal/rm/health_test.go
Health-check XID selection uses explicit arguments and tests configure those values directly.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Suggested reviewers: archlitchi

Poem

I’m a rabbit with config tucked under my ear,
Health-check values now travel clear.
From startup fields to XID flight,
NVML and Tegra keep checks in sight.
No env vars hiding deep in the hay—
Tests hop along the same clean way.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the refactor from env vars to top-level health check config.
Linked Issues check ✅ Passed The changes match #2150 by removing dynamic env reads from health.go, parsing once in factory.go, and passing config into resource managers.
Out of Scope Changes check ✅ Passed The diff appears tightly scoped to health-check config wiring and related test updates, with no unrelated changes evident.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/device-plugin/nvidiadevice/nvinternal/rm/health.go`:
- Line 61: Move XID parsing out of checkHealth and into resource-manager
construction, creating and storing a parsed health-check configuration alongside
each manager. Update checkHealth and getHealthCheckXids usage to reuse that
stored configuration without lowercasing, splitting, or parsing raw environment
values at runtime, while preserving the existing enable/disable behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 82b5a7e1-e0db-465a-8d95-f334a4f039ff

📥 Commits

Reviewing files that changed from the base of the PR and between c343242 and c4847d0.

📒 Files selected for processing (7)
  • pkg/device-plugin/nvidiadevice/nvinternal/plugin/factory.go
  • pkg/device-plugin/nvidiadevice/nvinternal/rm/health.go
  • pkg/device-plugin/nvidiadevice/nvinternal/rm/health_checkhealth_test.go
  • pkg/device-plugin/nvidiadevice/nvinternal/rm/health_test.go
  • pkg/device-plugin/nvidiadevice/nvinternal/rm/nvml_manager.go
  • pkg/device-plugin/nvidiadevice/nvinternal/rm/rm.go
  • pkg/device-plugin/nvidiadevice/nvinternal/rm/tegra_manager.go

Comment thread pkg/device-plugin/nvidiadevice/nvinternal/rm/health.go
// TODO: We should not read the envvar here directly, but instead
// "upgrade" this to a top-level config option.
strings.Split(strings.ToLower(os.Getenv(envDisableHealthChecks)), ",")...,
strings.Split(strings.ToLower(disableHealthChecks), ",")...,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the os import at the top is unused now that both getenv calls are gone so the pkg doesnt compile, can u drop it?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Dropped the unused os import in health.go

}()

return rm.NewNVMLResourceManagers(o.infolib, o.nvmllib, o.devicelib, o.config.Config)
disableHealthChecks := os.Getenv("DP_DISABLE_HEALTHCHECKS")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the env names are hardcoded here (4 times) while envDisableHealthChecks/envEnableHealthChecks in rm/health.go are now dead, can u export those consts and read the env once before the switch?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I exported EnvDisableHealthChecks and EnvEnableHealthChecks in health.go and refactored getResourceManagers to read them just once at the top before the switch statement

@FouoF

FouoF commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Thanks for the configuration refactor. This branch does not address a demonstrated production failure, and the exact head still fails lint because of an unused os import, which prevents the remaining checks from running. Replies indicate that fixes were made, but those fixes are not present in the reviewable head. We are closing this stale branch. If the refactor is still needed, please first confirm the scope in an issue and then submit one minimal, clean, fully passing change from current master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Upgrade NVIDIA device health checks to use top-level configuration

3 participants