Skip to content

Add maxInflightMountCalls and volumeAttachLimit args#1706

Merged
dankova22 merged 1 commit into
kubernetes-sigs:masterfrom
DavidXU12345:concurrency-control
Oct 3, 2025
Merged

Add maxInflightMountCalls and volumeAttachLimit args#1706
dankova22 merged 1 commit into
kubernetes-sigs:masterfrom
DavidXU12345:concurrency-control

Conversation

@DavidXU12345
Copy link
Copy Markdown
Contributor

Is this a bug fix or adding new feature?
Adding new feature.

What is this PR about? / Why do we need it?
Add maxInflightMountCalls and volumeAttachLimit args. We need them to prevent OOM killed issue for efs-plugin container since it takes 30MiB for mounting operation and 12MiB for each EFS volume due to efs-proxy process.

What testing is done?

  • Unit tests
  • e2e test
  • Manually deploy csi driver with this change locally to verify OOM issue can be solved with maxInflightMountCalls and volumeAttachLimit being set

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 16, 2025
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

Hi @DavidXU12345. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Sep 16, 2025
Comment thread cmd/main.go
klog.Fatalln(err)
}
drv := driver.NewDriver(*endpoint, etcAmazonEfs, *efsUtilsStaticFilesPath, *tags, *volMetricsOptIn, *volMetricsRefreshPeriod, *volMetricsFsRateLimit, *deleteAccessPointRootDir, *adaptiveRetryMode)
drv := driver.NewDriver(*endpoint, etcAmazonEfs, *efsUtilsStaticFilesPath, *tags, *volMetricsOptIn, *volMetricsRefreshPeriod, *volMetricsFsRateLimit, *deleteAccessPointRootDir, *adaptiveRetryMode, *maxInflightMountCallsOptIn, *maxInflightMountCalls, *volumeAttachLimitOptIn, *volumeAttachLimit)
Copy link
Copy Markdown
Contributor Author

@DavidXU12345 DavidXU12345 Sep 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably add all options into its own type (similar as EBS CSI driver) so that we can pass option around rather than passing each value into NewDriver for example. We can also encapsulate validation logic in option as well (e.g. check if maxInflightMountCallsOptIn is true, maxInflightMountCalls has to be a positive value).

I may consider to do the refactoring in next PR since the size of this PR is already very large.

@DavidXU12345 DavidXU12345 force-pushed the concurrency-control branch 3 times, most recently from 60029aa to 0fe78c3 Compare September 17, 2025 20:41
Comment thread cmd/main.go
tags = flag.String("tags", "", "Space separated key:value pairs which will be added as tags for EFS resources. For example, 'environment:prod region:us-east-1'")
adaptiveRetryMode = flag.Bool("adaptive-retry-mode", true, "Opt out to use standard sdk retry configuration. By default, adaptive retry mode will be used to more heavily client side rate limit EFS API requests.")
tags = flag.String("tags", "", "Space separated key:value pairs which will be added as tags for EFS resources. For example, 'environment:prod region:us-east-1'")
maxInflightMountCallsOptIn = flag.Bool("max-inflight-mount-calls-opt-in", false, "Opt in to use max inflight mount calls limit.")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we could simplify this by just creating one flag for each? If no is maxInflightCalls or volumeLimit is provided we just use current behavior (not limited at all)?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to keep the OptIn flag primarily for future-proofing. If we later implement default calculations for maxInflightCalls or volumeLimit, users could set OptIn=true without specifying values for parameters like maxInflightMountCalls.

While we could remove the OptIn feature gate later if we decide not to provide default calculations, adding it back would be more disruptive. From a user perspective, without OptIn=true, a configuration with maxInflightMountCalls=10 would be ignored, breaking backward compatibility.

Comment thread docs/README.md
Comment thread cmd/main.go
Comment thread pkg/driver/driver.go Outdated
Comment thread pkg/driver/inflight_checker.go Outdated
@dankova22
Copy link
Copy Markdown
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Oct 2, 2025
@dankova22
Copy link
Copy Markdown
Contributor

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 2, 2025
@dankova22
Copy link
Copy Markdown
Contributor

/retest

@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 2, 2025
@DavidXU12345 DavidXU12345 force-pushed the concurrency-control branch 3 times, most recently from c20bee8 to be64472 Compare October 2, 2025 18:47
1. maxInflightMountCalls with OptIn arg
In order to use maxInflightMountCalls, user has to set
maxInflightMountCallsOptIn to true and set a positive value for
maxInflightMountCalls.

2. volumeAttachLimit with OptIn arg
To use volumeAttachLimit, user has to set volumeAttachLimitOptIn to be
true and set volumeAttachLimit to a positive value.
@dankova22
Copy link
Copy Markdown
Contributor

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 3, 2025
@dankova22 dankova22 merged commit 68eb57d into kubernetes-sigs:master Oct 3, 2025
5 of 6 checks passed
@k8s-ci-robot
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dankova22, DavidXU12345

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants