Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improved context with timeout for DrainNode #920

Merged
merged 6 commits into from
Jul 10, 2024
Merged

Improved context with timeout for DrainNode #920

merged 6 commits into from
Jul 10, 2024

Conversation

sssash18
Copy link
Contributor

@sssash18 sssash18 commented Jun 24, 2024 •
edited
Loading

What this PR does / why we need it:
This PR introduces a drain Context in RunDrain with a timeout so that context cancels and machine can be force deleted later.
Which issue(s) this PR fixes:
Fixes #785

Special notes for your reviewer:

Release note:

Drain timeout is now correctly honored for Pod eviction during Machine Drain

@gardener-robot gardener-robot added needs/review Needs review size/xs Size of pull request is tiny (see gardener-robot robot/bots/size.py) labels Jun 24, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jun 24, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 added needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jun 24, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jun 25, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jun 25, 2024
@gardener-robot-ci-3 gardener-robot-ci-3 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jun 25, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 2, 2024
@sssash18 sssash18 marked this pull request as ready for review July 2, 2024 06:22
@sssash18 sssash18 requested a review from a team as a code owner July 2, 2024 06:22
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 2, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jul 2, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 2, 2024
@gardener-robot-ci-2 gardener-robot-ci-2 added reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) and removed reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) labels Jul 2, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 removed the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 2, 2024
@sssash18 sssash18 changed the title improved drain timeout calculation Improved context with timeout for DrainNode Jul 2, 2024
aaronfern
aaronfern reviewed Jul 2, 2024
View reviewed changes
Copy link
Contributor

@aaronfern aaronfern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@gardener-robot gardener-robot added the reviewed/lgtm Has approval for merging label Jul 2, 2024
@gardener-robot gardener-robot removed the needs/review Needs review label Jul 2, 2024
@gardener-robot-ci-1 gardener-robot-ci-1 added the reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) label Jul 2, 2024
@aaronfern
Copy link
Contributor

Please add a release note, something like below or feel free to come up with a different one
A bug in the drain process where the drain timeout was not respected has been fixed or
Timeout based context cancellation will now be followed for machine drains

elankath
elankath approved these changes Jul 9, 2024
View reviewed changes
Copy link
Contributor

@elankath elankath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can change the release note to : "Drain timeout is now correctly honored for Pod eviction during Machine Drain" ?

Can you do a test of this with some short timeouts and Pods with large number of PVs to see if there are no surprises ?

@sssash18
Copy link
Contributor Author

You can change the release note to : "Drain timeout is now correctly honored for Pod eviction during Machine Drain" ?

Can you do a test of this with some short timeouts and Pods with large number of PVs to see if there are no surprises ?

Performed the testing, the context cancellation leads to force delete even if pod with PVs are in process of eviction.

rishabh-11
rishabh-11 approved these changes Jul 10, 2024
View reviewed changes
Copy link
Contributor

@rishabh-11 rishabh-11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@rishabh-11 rishabh-11 merged commit e358db1 into gardener:master Jul 10, 2024
8 checks passed
@gardener-robot gardener-robot added the status/closed Issue is closed (either delivered or triaged) label Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@aaronfern aaronfern aaronfern left review comments

@elankath elankath elankath approved these changes

@rishabh-11 rishabh-11 rishabh-11 approved these changes

Assignees
No one assigned
Labels
needs/ok-to-test Needs approval for testing (check PR in detail before setting this label because PR is run on CI/CD) reviewed/lgtm Has approval for merging reviewed/ok-to-test Has approval for testing (check PR in detail before setting this label because PR is run on CI/CD) size/xs Size of pull request is tiny (see gardener-robot robot/bots/size.py) status/closed Issue is closed (either delivered or triaged)
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Drain Timeout is not respected during client side throttling
8 participants
@sssash18 @aaronfern @elankath @rishabh-11 @gardener-robot-ci-1 @gardener-robot-ci-2 @gardener-robot-ci-3 @gardener-robot