perf: Typed Bucket Scaling #2420

ryan-mist · 2025-08-06T00:07:52Z

Fixes #N/A

Description

Scale typed buckets linearly with the number of reconciles.
- All of the todays typed buckets have a max tokens = 10 * qps. We keep this and have qps scale with the same ratio we have today between qps and reconciles.

The changes are as follows:

Node Termination Controller
* todays typed bucket
    * reconciles - 100
    * qps - 10
    * max tokens - 100
* with this PR: scale qps linearly at 10% of reconciles, max tokens = 10 * qps

Node Terminator Eviction Controller
* todays typed bucket
    * reconciles - 100
    * qps - 100
    * max tokens - 1000
* with this PR: scale qps linearly at 100% of reconciles, max tokens = 10 * qps

NodeClaim Lifecycle Controller
* todays typed bucket
    * reconciles - 1000
    * qps - 10
    * max tokens - 100  
* with this PR: scale qps linearly at 1% of reconciles, max tokens = 10 * qps

How was this change tested?

make presubmit

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

k8s-ci-robot · 2025-08-06T00:08:02Z

Hi @ryan-mist. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

pkg/controllers/node/termination/controller.go

pkg/controllers/node/termination/terminator/eviction.go

pkg/utils/controller/controller.go

coveralls · 2025-08-08T22:31:42Z

Pull Request Test Coverage Report for Build 17282406839

Details

6 of 16 (37.5%) changed or added relevant lines in 4 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.03%) to 81.702%

Changes Missing Coverage	Changed/Added Lines	%
pkg/controllers/node/termination/terminator/eviction.go	2	0.0%
pkg/controllers/node/termination/controller.go	4	0.0%
pkg/controllers/nodeclaim/lifecycle/controller.go	4	0.0%

Files with Coverage Reduction	New Missed Lines	%
pkg/controllers/provisioning/scheduling/volumetopology.go	2	84.0%

Totals
Change from base Build 17281949215:	-0.03%
Covered Lines:	10645
Relevant Lines:	13029

💛 - Coveralls

pkg/controllers/node/termination/controller.go

jmdeal · 2025-08-28T13:52:53Z

/lgtm
/approve

k8s-ci-robot · 2025-08-28T13:53:02Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jmdeal, ryan-mist

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [jmdeal]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* chore: bump go version to 1.24.4 (kubernetes-sigs#2298) * chore: Only log that the command succeeded when it actually did (kubernetes-sigs#2302) * fix: Fix bug with MarkForDeletion before creating replacements (kubernetes-sigs#2300) * perf: Refactor the eviction queue to be multithreaded (kubernetes-sigs#2252) * docs: Add Bizfly Cloud provider (kubernetes-sigs#2303) * chore: Bump lifecycle cache expiration to one hour (kubernetes-sigs#2307) * chore: Use cluster state to check replacement NodeClaim existence (kubernetes-sigs#2308) * chore(deps): bump github.com/samber/lo from 1.50.0 to 1.51.0 in the go-deps group (kubernetes-sigs#2315) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: bump operatorpkg (kubernetes-sigs#2314) * chore(deps): bump the k8s-go-deps group across 1 directory with 4 updates (kubernetes-sigs#2317) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: Refactor Orchestration Queue and Handle Mark/Unmark Deletion in Queue (kubernetes-sigs#2305) * chore(deps): bump the k8s-go-deps group with 7 updates (kubernetes-sigs#2326) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * perf: multithreaded orchestration queue (kubernetes-sigs#2293) * test: Add nodeclaim name when you have garbage collection (kubernetes-sigs#2333) * perf: Reduce multiple patch calls in instance termination (kubernetes-sigs#2324) * fix: add helm rbac for kwok-provider to update finalizers (kubernetes-sigs#2336) Signed-off-by: Max Cao <[email protected]> * feat: configure CRD status operator with larger histogram buckets (kubernetes-sigs#2328) * chore(deps): bump sigs.k8s.io/yaml from 1.4.0 to 1.5.0 in the k8s-go-deps group (kubernetes-sigs#2339) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump github.com/docker/docker from 28.2.2+incompatible to 28.3.0+incompatible in the go-deps group (kubernetes-sigs#2340) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: Fix re-retrieving object on retry (kubernetes-sigs#2337) * fix: Fix overriding error with patch call (kubernetes-sigs#2338) * fix: add missing rlock to disruption queue (kubernetes-sigs#2348) * test: allow e2e tests to output junit report (kubernetes-sigs#2334) Signed-off-by: Max Cao <[email protected]> * docs: Add Oracle Cloud Infrastructure (OCI) provider (kubernetes-sigs#2342) * fix: no longer allow the same hostname to take multiple capacity (kubernetes-sigs#2356) * feat: support auto relaxing min values (kubernetes-sigs#2299) * fix: update provider ID to ensure that Cloud Provider tests pass (kubernetes-sigs#2363) * fix: remove unsupported capacity_type label from karpenter_nodeclaims… (kubernetes-sigs#2364) * fix: update deletionTimestamp on terminating pods when after nodeDeletionTimestamp (kubernetes-sigs#2316) Co-authored-by: Amanuel Engeda <[email protected]> * chore: promote ReservedCapacity feature gate to beta (kubernetes-sigs#2365) * fix: flakiness in expiration tests (kubernetes-sigs#2366) * test: Bump the termination time for the deletion timestamp (kubernetes-sigs#2367) * chore(deps): bump github.com/docker/docker from 28.3.0+incompatible to 28.3.1+incompatible in the go-deps group (kubernetes-sigs#2355) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: pod errors when nodepool requirements filter all instance types (kubernetes-sigs#2341) * refactor: Create a NopValidator for the disruption testing (kubernetes-sigs#2369) * chore(deps): bump the go-deps group with 2 updates (kubernetes-sigs#2373) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * refactor: Update disruption testing from PR comments (kubernetes-sigs#2372) * feat: (BREAKING) addition of launch timeout for nodeclaim lifecycle (kubernetes-sigs#2349) * chore: Consider node.kubernetes.io/not-ready:NoExecute as ephemeral (kubernetes-sigs#2265) * perf: Optimistically delete from the cache after launch (kubernetes-sigs#2380) * docs: Node Overlay RFC (kubernetes-sigs#2166) * fix: handle multiple PDBs for the same pod more gracefully (kubernetes-sigs#2379) * docs: Add IBM Cloud provider (kubernetes-sigs#2396) Signed-off-by: Josephine Pfeiffer <[email protected]> * fix: rate limit eviction when PDBs are blocking (kubernetes-sigs#2399) * feat: Add the Node Overlay CRD (kubernetes-sigs#2296) * chore: ignore pods that use unsupported provisioner in the storageClass (kubernetes-sigs#2400) * feat: Add a feature flag for Node Overlay (kubernetes-sigs#2404) * feat: Add StaticCapacity feature flag (kubernetes-sigs#2405) * fix(BREAKING): update naming of karpenter_pods_drained_total (kubernetes-sigs#2421) * fix: pod metrics when pod is terminal (kubernetes-sigs#2417) * chore: ignore pods that have unbound pvc with volumeBindingMode immediate (kubernetes-sigs#2415) * docs: static capacity RFC (kubernetes-sigs#2309) * chore: bump go version to 1.24.6 (kubernetes-sigs#2432) * feat: Create optional operator arguments to leverage leader lease functionality (kubernetes-sigs#2433) * chore(deps): bump the go-deps group with 5 updates (kubernetes-sigs#2442) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/checkout from 4.2.2 to 5.0.0 in /.github/actions/install-pyroscope in the action-deps group (kubernetes-sigs#2428) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump the actions-deps group across 1 directory with 2 updates (kubernetes-sigs#2443) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/cache from 4.2.3 to 4.2.4 in /.github/actions/install-deps in the action-deps group (kubernetes-sigs#2425) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: do not block drifted nodes from being terminated if consolidation is disabled (kubernetes-sigs#2423) * chore: Pin GH action SHAs for run-bench-test (kubernetes-sigs#2448) * chore: update operatorpkg (kubernetes-sigs#2455) * chore: Track NodeClaims in NodePoolState (kubernetes-sigs#2449) * chore(deps): bump the k8s-go-deps group across 1 directory with 7 updates (kubernetes-sigs#2456) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * perf: Add flag to disable costly metrics controllers (kubernetes-sigs#2354) * perf: concurrent reconciles CPU-based scaling (kubernetes-sigs#2406) * perf: Disruption Queue Retry Duration Scaling (kubernetes-sigs#2411) * perf: Typed Bucket Scaling (kubernetes-sigs#2420) * ci: Include K8s version 1.33 and 1.34 in testing (kubernetes-sigs#2465) * chore: increase MaxInstanceTypes to give cloud-providers more control over instance type truncation (kubernetes-sigs#2430) * chore(deps): bump the go-deps group with 2 updates (kubernetes-sigs#2461) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump amannn/action-semantic-pull-request from 6.0.1 to 6.1.1 in the actions-deps group (kubernetes-sigs#2462) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * ci: revert k8s 1.34 addition (kubernetes-sigs#2475) * fix: Don't schedule a pod with DRA requirements (kubernetes-sigs#2384) * fix: support arbitrary reserved capacity labels for drift (kubernetes-sigs#2476) * chore(deps): bump actions/checkout from 4.2.2 to 5.0.0 in /.github/actions/install-prometheus in the action-deps group (kubernetes-sigs#2426) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * fix: Fix nil pointer exception for multiNodeConsolidation (kubernetes-sigs#2472) * fix: avoid hash collisions with duplicate match expressions (kubernetes-sigs#2479) * ci: enable k8s 1.34 tests (kubernetes-sigs#2481) * fix: Validate unsupported provisioners on bound PVs (kubernetes-sigs#2480) * refactor: use iterator for iterating state nodes (kubernetes-sigs#2483) * fix: make toolchain failing due to deletion of asciicheck (kubernetes-sigs#2485) * fix: Handle PVC edge cases handled by kube-scheduler (kubernetes-sigs#2488) * chore: Change appName from const to var (kubernetes-sigs#2489) * fix: Handle unbound volumes with volumeName defined (kubernetes-sigs#2487) * chore(deps): bump actions/setup-go from 5.5.0 to 6.0.0 in /.github/actions/install-deps in the action-deps group (kubernetes-sigs#2494) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump actions/setup-python from 5.6.0 to 6.0.0 in the actions-deps group (kubernetes-sigs#2493) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump the go-deps group with 6 updates (kubernetes-sigs#2491) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump the k8s-go-deps group with 4 updates (kubernetes-sigs#2492) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: remove duplicate reconcile logging (kubernetes-sigs#2496) * chore: bump operatorpkg version (kubernetes-sigs#2500) * perf: Update the Node Repair Controller for requeue time (kubernetes-sigs#2286) * feat: Add NodeOverlay Controller Support (kubernetes-sigs#2306) * chore(deps): bump the k8s-go-deps group with 3 updates (kubernetes-sigs#2504) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore: rolling back to 1.34 (kubernetes-sigs#2512) * fix: handle nil selector when hashing in topology (kubernetes-sigs#2511) * feat: Support Pod Level Resources (kubernetes-sigs#2383) Signed-off-by: Tsubasa Nagasawa <[email protected]> * fix: merge limits into requests when constructing ds pods (kubernetes-sigs#2514) * fix: default CPU_REQUESTS when non-positive value is provided (kubernetes-sigs#2516) * fix(node): prevent empty providerID causing false NodeClaim matches (kubernetes-sigs#2507) * feat: Support Static Capacity (kubernetes-sigs#2521) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Jason Deal <[email protected]> Co-authored-by: Jonathan Innis <[email protected]> Co-authored-by: Andrew Mitchell <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ryan Mistretta <[email protected]> * fix: over provisioning static nodeclaims during controller crashes (kubernetes-sigs#2534) * chore: drop consistency error to info log (kubernetes-sigs#2542) * fix: flaky static provisioning unit test (kubernetes-sigs#2546) * fix: nodepool crd definition should explicitly say replicas field as alpha (kubernetes-sigs#2554) * chore: Update NodeRegistrationHealthy SC to use a buffer mechanism (kubernetes-sigs#2520) --------- Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: Max Cao <[email protected]> Signed-off-by: Josephine Pfeiffer <[email protected]> Signed-off-by: Tsubasa Nagasawa <[email protected]> Co-authored-by: Derek Frank <[email protected]> Co-authored-by: Jonathan Innis <[email protected]> Co-authored-by: Lê Minh Quân <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Jigisha Patil <[email protected]> Co-authored-by: Amanuel Engeda <[email protected]> Co-authored-by: Max Cao <[email protected]> Co-authored-by: Aidan Rowe <[email protected]> Co-authored-by: Daniel Lopes <[email protected]> Co-authored-by: Saurav Agarwalla <[email protected]> Co-authored-by: cosimomeli <[email protected]> Co-authored-by: Jason Deal <[email protected]> Co-authored-by: Reed Schalo <[email protected]> Co-authored-by: Josephine Pfeiffer <[email protected]> Co-authored-by: Sumukha Radhakrishna <[email protected]> Co-authored-by: Andy Townsend <[email protected]> Co-authored-by: Sumukha Radhakrishna <[email protected]> Co-authored-by: ryan-mist <[email protected]> Co-authored-by: Brandon Wagner <[email protected]> Co-authored-by: Alima Azamat <[email protected]> Co-authored-by: Andrew Mitchell <[email protected]> Co-authored-by: Tsubasa Nagasawa <[email protected]> Co-authored-by: Neil <[email protected]>

perf: Typed Bucket Scaling

e5b6efe

k8s-ci-robot requested review from engedaam and tallaxes August 6, 2025 00:07

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 6, 2025

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 6, 2025

DerekFrank reviewed Aug 8, 2025

View reviewed changes

pkg/controllers/node/termination/controller.go Outdated Show resolved Hide resolved

pkg/controllers/node/termination/terminator/eviction.go Outdated Show resolved Hide resolved

pkg/utils/controller/controller.go Show resolved Hide resolved

abstract qps, bucket size function

b9baca7

ryan-mist added 4 commits August 11, 2025 10:14

cleanup testing

ac162d3

Merge branch 'main' into bucket-scaling

0593b03

update to reflect other PR changes

da69de7

Merge branch 'main' into bucket-scaling

13fe9d6

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Aug 27, 2025

fix rebase

bebdc66

jmdeal reviewed Aug 28, 2025

View reviewed changes

pkg/controllers/node/termination/controller.go Outdated Show resolved Hide resolved

ryan-mist added 2 commits August 27, 2025 17:43

consts for min/max reconciles & fix GetTypedBucketConfigs arg

9374b1a

Merge branch 'main' into bucket-scaling

544efcc

k8s-ci-robot assigned jmdeal Aug 28, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 28, 2025

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 28, 2025

k8s-ci-robot merged commit 06fedfc into kubernetes-sigs:main Aug 28, 2025
16 checks passed

ryan-mist deleted the bucket-scaling branch August 28, 2025 16:11

jigisha620 pushed a commit to jigisha620/karpenter that referenced this pull request Sep 19, 2025

perf: Typed Bucket Scaling (kubernetes-sigs#2420)

100d31d

DerekFrank mentioned this pull request Oct 9, 2025

chore: Add DerekFrank as reviewer #2568

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

perf: Typed Bucket Scaling #2420

perf: Typed Bucket Scaling #2420

Uh oh!

ryan-mist commented Aug 6, 2025 •

edited

Loading

Uh oh!

k8s-ci-robot commented Aug 6, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coveralls commented Aug 8, 2025 •

edited

Loading

Uh oh!

Uh oh!

jmdeal commented Aug 28, 2025

Uh oh!

k8s-ci-robot commented Aug 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

perf: Typed Bucket Scaling #2420

perf: Typed Bucket Scaling #2420

Uh oh!

Conversation

ryan-mist commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Aug 6, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

coveralls commented Aug 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Test Coverage Report for Build 17282406839

Details

💛 - Coveralls

Uh oh!

Uh oh!

jmdeal commented Aug 28, 2025

Uh oh!

k8s-ci-robot commented Aug 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ryan-mist commented Aug 6, 2025 •

edited

Loading

coveralls commented Aug 8, 2025 •

edited

Loading