feat: Support Pod Level Resources

[toVersus]

Fixes #2325

Description

This PR updates Karpenter to support Pod Level Resources. The Pod Level Resources feature is planned to graduate to beta in Kubernetes 1.34. Calculating pod resource requests and limits is complex, and to allow components like the scheduler and Node Autoscalers to share a common logic, the necessary functions have been published in k8s.io/component-helpers/resources. While Karpenter currently implements its own custom logic, this PR transitions it to use the shared library.

There is one difference between the shared library and the existing custom logic. The current custom logic includes behavior that merges limits into requests when only limits are specified. However, this merging is actually handled by the API server's defaulting logic. Some existing tests rely on this merging logic, and without the second commit, those tests fail. Personally, I think Karpenter doesn't need to implement this logic explicitly, but WDYT?

How was this change tested?

make presubmit

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[k8s-ci-robot]

Welcome @toVersus!

It looks like this is your first PR to kubernetes-sigs/karpenter 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/karpenter has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @toVersus. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[coveralls]

Pull Request Test Coverage Report for Build 17568207446

Details

• 3 of 3 (100.0%) changed or added relevant lines in 2 files are covered.
• 10 unchanged lines in 3 files lost coverage.
• Overall coverage decreased (-0.1%) to 81.572%

---

Files with Coverage Reduction	New Missed Lines	%
pkg/utils/resources/resources.go	1	91.09%
pkg/controllers/disruption/drift.go	2	87.76%
pkg/controllers/provisioning/scheduling/preferences.go	7	88.76%

Totals
Change from base Build 17559484392:	-0.1%
Covered Lines:	10677
Relevant Lines:	13089

---

💛 - Coveralls

[jmdeal]

Personally, I think Karpenter doesn't need to implement this logic explicitly, but WDYT?

So the issue is that the defaulting applies to the pods created for the daemonset, but not for the daemonset itself. Karpenter reads the daemonset spec to compute the "daemonset overhead" for a NodePool. We do this by creating fake daemonset pods based on the specs (ref) and then calling RequestsForPods against those pods (ref).

That being said, I think it's reasonable for this interface to expect that the pods provided adhere to the API server's defaulting logic. It should be the responsibility of any component generating "fake" pods to ensure that they meet those requirements. In this case, we would want to update our getDaemonsetPods function to handle that defaulting.

Other than that, this PR looks good to me and I'm happy to take on this shared dependency and drop our custom logic. Apologies that it took a while to get review, our bandwidth right now is really limited and we're trying to get more folks promoted to reviewer / approver. We are actually in a bit of a hurry to get this in for 1.34 - I was about to get a PR up but saw yours - so since the rest of the PR looks good I'm going to go ahead and approve then follow up with the change to getDaemonsetPods.

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jmdeal, toVersus

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [jmdeal]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment