feat(scheduler): support per-pod device scoring weights

[blackdragoon26]

What type of PR is this?
/kind feature

What this PR does / why we need it:
This PR adds optional per-Pod resource weights for HAMi's device-level binpack and spread scoring.

A workload can configure the relative influence of virtual-device slots, accelerator cores, and device memory using:

metadata:
  annotations:
    hami.io/device-scoring-weights: "slot=1,core=1,memory=3"

When the annotation is absent, HAMi uses slot=1,core=1,memory=1, preserving the existing scoring behavior.
The annotation is parsed and validated once before device scoring. DeviceListsScore.ComputeScore receives only the validated typed weights and remains independent of Pod annotations and scheduler-policy composition.
The annotation must provide all three non-negative integer weights, with at least one positive value. Malformed, negative, incomplete, duplicate, or all-zero values are rejected.
This changes only the scalar device score. Existing device capacity checks, vendor Fit behavior, mutex handling, NUMA binding, topology-aware selection, and the established binpack/spread ordering remain unchanged.

Which issue(s) this PR fixes:
Fixes #2220
And this comment: #2220 (comment)

Special notes for your reviewer:
The implementation follows the revised scope discussed during the community meeting:

• No global or Helm configuration is introduced.
• The feature is opt-in per Pod.
• Missing annotations preserve the existing effective 1:1:1 calculation.
• Annotation parsing is kept outside ComputeScore for loose coupling.
• Memory-bandwidth-aware placement, node-level weights, migration, and eviction remain outside this PR.

Tests cover:

• annotation parsing and validation
• backward-compatible default scores
• memory-weighted and core-weighted scoring
• binpack and spread device preference
• NUMA precedence remaining independent of weighted scores
• scheduler rejection of invalid annotations

Verification completed:

make verify
go test ./pkg/util ./pkg/scheduler/policy ./pkg/scheduler

Does this PR introduce a user-facing change?:
Yes. Now, workloads can optionally influence HAMi's device-level placement by specifying slot, core, and memory scoring weights through a Pod annotation.
But, existing workloads without this annotation retain the current behaviour.

AI assistance notice: I used OpenAI Codex to help inspect the relevant scheduler paths, implement the parser, scoring changes, tests and documentation, and execute verification commands. I reviewed the resulting changes and take responsibility for the implementation.

Summary by CodeRabbit

Summary by CodeRabbit

• New Features

  • Added per-Pod GPU scoring weights for slot, core, and memory utilization.
  • Supports customized bin-packing, spreading, and topology tie-breaking decisions.
  • Equal weighting remains the default.

• Bug Fixes

  • Invalid or incomplete scoring annotations are rejected during scheduling.

• Documentation

  • Added configuration guidance, validation rules, and supported annotation formats.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d90f68eb-b04c-42db-bf64-4e413fa777d5

📥 Commits

Reviewing files that changed from the base of the PR and between 02d8882 and 0fb3083.

📒 Files selected for processing (2)

• pkg/scheduler/score.go
• pkg/scheduler/score_test.go

---

📝 Walkthrough

Walkthrough

The scheduler supports per-Pod GPU device scoring weights for slot, core, and memory utilization. It validates annotations, preserves equal-weight defaults, applies weights during device scoring, tests allocation and topology behavior, and documents the configuration.

Changes

Device scoring weights

Layer / File(s)	Summary
Weight contract and annotation parsing pkg/util/types.go, pkg/util/device_scoring_weights.go, pkg/util/device_scoring_weights_test.go	Defines the annotation key, weight structure, default weights, Pod lookup, parsing rules, validation errors, and unit tests.
Weighted device score flow pkg/scheduler/score.go, pkg/scheduler/policy/gpu_policy.go, pkg/scheduler/score_test.go, pkg/scheduler/policy/gpu_policy_test.go, docs/develop/scheduler-policy.md	Retrieves Pod-specific weights once, propagates them through init-container and application-container scoring, and applies them to device utilization scores. Tests cover default compatibility, allocation ordering, NUMA behavior, topology ties, and invalid annotations. The documentation describes the annotation and scoring rules.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant calcScoreWithOptions
  participant GetDeviceScoringWeightsByPod
  participant fitInDevices
  participant DeviceListsScore.ComputeScore
  calcScoreWithOptions->>GetDeviceScoringWeightsByPod: resolve Pod annotation
  GetDeviceScoringWeightsByPod-->>calcScoreWithOptions: return weights or validation error
  calcScoreWithOptions->>fitInDevices: pass DeviceScoringWeights
  fitInDevices->>DeviceListsScore.ComputeScore: compute weighted device score

Loading

Possibly related PRs

• Project-HAMi/HAMi#2011: Both changes modify device scoring and selection behavior in pkg/scheduler/policy/gpu_policy.go.
• Project-HAMi/HAMi#2012: Both changes modify GPU device scoring and score-based ordering.
• Project-HAMi/HAMi#2116: Both changes modify pkg/scheduler/score.go and init-container scoring paths.

Suggested reviewers: archlitchi, fouof, chaunceyjiang

Poem

A rabbit sets weights, neat and bright,
For slots, cores, and memory’s might.
Invalid values stop the run,
Equal defaults keep work as done,
While GPUs score each choice just right.

🚥 Pre-merge checks | ✅ 2 | ❌ 3

❌ Failed checks (3 warnings)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The PR adds weighted scoring, but [#2220] requires global scheduler configuration and Helm exposure, while this implementation uses only a Pod annotation.	Add global scheduler configuration and Helm support, or update [#2220] to explicitly approve the per-Pod annotation scope.
Out of Scope Changes check	⚠️ Warning	The per-Pod annotation override is explicitly outside the initial scope defined by [#2220].	Remove the Pod-level override from this PR, or obtain updated issue scope that explicitly includes per-Pod device scoring weights.
Docstring Coverage	⚠️ Warning	Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: per-Pod device scoring weights in the scheduler.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov]

Codecov Report

❌ Patch coverage is 96.36364% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
pkg/util/device_scoring_weights.go	95.00%	1 Missing and 1 partial ⚠️

Flag	Coverage Δ
unittests	62.46% <96.36%> (+0.10%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
pkg/scheduler/policy/gpu_policy.go	100.00% <100.00%> (ø)
pkg/scheduler/score.go	89.31% <100.00%> (+0.13%)	⬆️
pkg/util/types.go	100.00% <ø> (ø)
pkg/util/device_scoring_weights.go	95.00% <95.00%> (ø)

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[blackdragoon26]

@CodeRabbit review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[FouoF]

@blackdragoon26 This is a useful feature. Before merging, can you post a doc about how to use it at https://github.com/Project-HAMi/website?

[mesutoezdil]

@CodeRabbit review

i think i said it before, pls do NOT use manually coderrabbit.

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

@mesutoezdil Understood. I will not use manual @CodeRabbit review commands. I will use the configured automatic review flow only.

🐇

---

✏️ Learnings added

Learnt from: mesutoezdil
URL: https://github.com/Project-HAMi/HAMi/pull/2469

Timestamp: 2026-08-10T07:44:36.915Z
Learning: For the Project-HAMi/HAMi repository, do not trigger manual CodeRabbit reviews with `CodeRabbit review`. Use the configured automatic review flow only.

Note: Learnings are effective only in the context of similar code segments. To apply general coding guidelines, please configure review instructions. You can manage existing learnings in the app.

_{You are interacting with an AI system.}

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

@mesutoezdil Understood. I will not trigger manual @CodeRabbit review commands. I will use the configured automatic review flow only.

🐇

_{You are interacting with an AI system.}

[blackdragoon26]

@blackdragoon26 This is a useful feature. Before merging, can you post a doc about how to use it at https://github.com/Project-HAMi/website?

Yesss, Sure thing, I will be doing that.
Thank You

[blackdragoon26]

@CodeRabbit review

i think i said it before, pls do NOT use manually coderrabbit.

Oh yes, I am so sorry about that.
I mistakenly thought that Code Rabbit did not run for this PR, even for a single time.
I have already and will be keeping that in mind!
Thank You

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

@blackdragoon26 Thank you for the confirmation. No problem. The configured automatic review flow will handle future reviews.

🐇

_{You are interacting with an AI system.}

[coderabbitai]

Tip

For best results, initiate chat on the files or code changes.

@blackdragoon26 Thank you for the confirmation. The configured automatic review flow will handle future reviews.

🐇

_{You are interacting with an AI system.}

[blackdragoon26]

I have created the issue, at @FouoF Project-HAMi/website#758 for the same.

@blackdragoon26 This is a useful feature. Before merging, can you post a doc about how to use it at https://github.com/Project-HAMi/website?

Yesss, Sure thing, I will be doing that. Thank You

[FouoF]

/lgtm

[hami-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: blackdragoon26, FouoF

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [FouoF]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mesutoezdil]

/lgtm