Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feature: support setting gracePeriodSeconds in DeschedulerPolicy #1537

Open
googs1025 opened this issue Oct 27, 2024 · 8 comments · May be fixed by #1538
Open

feature: support setting gracePeriodSeconds in DeschedulerPolicy #1537

googs1025 opened this issue Oct 27, 2024 · 8 comments · May be fixed by #1538
Assignees
@googs1025
Labels
kind/feature Categorizes issue or PR as related to a new feature.

Comments

@googs1025
Copy link
Member

Is your feature request related to a problem? Please describe.

  • support gracePeriodSeconds in DeschedulerPolicy

GracePeriodSeconds is important for setting up evictions because it ensures that Pods have enough time to perform necessary cleanup operations before being deleted, thereby ensuring data consistency and service availability. Depending on application requirements, setting GracePeriodSeconds appropriately can improve system stability and reliability.

type DeschedulerPolicy struct {
	metav1.TypeMeta

	// Profiles
	Profiles []DeschedulerProfile
	...
	// GracePeriodSeconds The duration in seconds before the object should be deleted. Value must be non-negative integer.
	// The value zero indicates delete immediately. If this value is nil, the default grace period for the
	// specified type will be used.
	// Defaults to a per object value if not specified. zero means delete immediately.
	GracePeriodSeconds int64
}

Describe the solution you'd like

  • In DeschedulerPolicy, the user can set it from the configuration file and pass it to the deleteOptions of evict.

descheduler/pkg/descheduler/evictions/evictions.go

Lines 199 to 222 in ef0c2c1

func evictPod(ctx context.Context, client clientset.Interface, pod *v1.Pod, policyGroupVersion string) error {
deleteOptions := &metav1.DeleteOptions{}
// GracePeriodSeconds ?
eviction := &policy.Eviction{
TypeMeta: metav1.TypeMeta{
APIVersion: policyGroupVersion,
Kind: eutils.EvictionKind,
},
ObjectMeta: metav1.ObjectMeta{
Name: pod.Name,
Namespace: pod.Namespace,
},
DeleteOptions: deleteOptions,
}
err := client.PolicyV1().Evictions(eviction.Namespace).Evict(ctx, eviction)
if apierrors.IsTooManyRequests(err) {
return fmt.Errorf("error when evicting pod (ignoring) %q: %v", pod.Name, err)
}
if apierrors.IsNotFound(err) {
return fmt.Errorf("pod not found when evicting %q: %v", pod.Name, err)
}
return err
}

Describe alternatives you've considered

What version of descheduler are you using?

descheduler version:

Additional context
@googs1025 googs1025 added the kind/feature Categorizes issue or PR as related to a new feature. label Oct 27, 2024
@googs1025
Copy link
Member Author

/assign

@googs1025
Copy link
Member Author

@damemi @ingvagabund @a7i
Do we need this configuration feature?

@googs1025 googs1025 changed the title feature: support gracePeriodSeconds in DeschedulerPolicy feature: support setting gracePeriodSeconds in DeschedulerPolicy Oct 27, 2024
@a7i
Copy link
Contributor

a7i commented Oct 27, 2024 •
edited
Loading

We've never had a use-case for it, typically eviction needs to happen immediately.

What is your use-case for this?

@googs1025
Copy link
Member Author

@a7i thanks for quick reply
Within our internal cluster, we have some long-running batch jobs or other business processes. However, when employing the descheduler for evictions (for instance, when the cluster reaches high utilization levels), the business holders wish to have the ability to set gracePeriodSeconds during evictions. This capability ensures that custom cleanup procedures have sufficient time to execute, thereby preventing issues arising from abrupt interruptions.

Furthermore, our SREs, during manual node maintenance using kubectl drain, typically include a reasonable --grace-period flag. We aim to have the same functionality available when utilizing the descheduler for rescheduling, maintaining consistency in the approach across both evictions and node maintenance operations.

@ingvagabund
Copy link
Contributor

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.31/#create-eviction-pod-v1-core eviction API does not currently provides means to specify gracePeriodSeconds.

@ingvagabund
Copy link
Contributor

the user can set it from the configuration file and pass it to the deleteOptions of evict.

@googs1025 I wonder how this works. Have you had a chance to implement it and see whether it works?

@googs1025
Copy link
Member Author

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.31/#create-eviction-pod-v1-core eviction API does not currently provides means to specify gracePeriodSeconds.

We currently use client.PolicyV1().Evictions(eviction.Namespace).Evict(ctx, eviction) to perform evictions, which contains the DeleteOptions option. Simply passing in the configuration should work.

https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.31/#eviction-v1-policy

@googs1025 googs1025 linked a pull request Oct 30, 2024 that will close this issue
Open
@googs1025
Copy link
Member Author

the user can set it from the configuration file and pass it to the deleteOptions of evict.

@googs1025 I wonder how this works. Have you had a chance to implement it and see whether it works?

This is a WIP pr:
#1538

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@googs1025 googs1025

Labels
kind/feature Categorizes issue or PR as related to a new feature.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feature(descheduler): add grace_period_seconds for DeschedulerPolicy googs1025/descheduler
3 participants
@ingvagabund @a7i @googs1025