Skip to content

RayJob Volcano Integration #3972

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rueian merged 12 commits into from
Oct 9, 2025
Merged

RayJob Volcano Integration #3972

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
6e4184a
modify batch scheduler interface to support CRD other than RayCluster
troychiu Aug 7, 2025
973fe82
[Feature] RayJob Volcano integration
win5923 Sep 4, 2025
2ee7e8a
Modify kai scheduler and sheduler plugins
win5923 Sep 11, 2025
92e7d95
Revert interface migration
win5923 Sep 24, 2025
dd38daa
Append submitter resources
win5923 Sep 25, 2025
a33a3b7
Remove empty ResourceList
win5923 Sep 26, 2025
42479c2
Add K8sJobMode check to prevent YuniKorn from adding submitter pod an…
win5923 Sep 27, 2025
597e57d
Apply Troy's comments
win5923 Oct 4, 2025
a64c011
Log more information
win5923 Oct 5, 2025
b8cb224
Add sidecarmode
win5923 Oct 8, 2025
84d39f9
fix
Future-Outlier Oct 9, 2025
cf6b48b
Update Rueian's advice
Future-Outlier Oct 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions ray-operator/controllers/ray/batchscheduler/volcano/volcano_scheduler.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,13 +78,23 @@ func (v *VolcanoBatchScheduler) handleRayJob(ctx context.Context, rayJob *rayv1.
// MinMember intentionally excludes the submitter pod to avoid a startup deadlock
// (submitter waits for cluster; gang would wait for submitter). We still add the
// submitter's resource requests into MinResources so capacity is reserved.
if rayJob.Spec.SubmissionMode == rayv1.K8sJobMode || rayJob.Spec.SubmissionMode == rayv1.SidecarMode {
submitterResource := getSubmitterResource(rayJob)
totalResourceList = append(totalResourceList, submitterResource)
return v.syncPodGroup(ctx, rayJob, minMember, utils.SumResourceList(totalResourceList))
}

func getSubmitterResource(rayJob *rayv1.RayJob) corev1.ResourceList {
if rayJob.Spec.SubmissionMode == rayv1.K8sJobMode {
Copy link
Collaborator

@rueian rueian Oct 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to do this for SidecarMode?

Copy link
Member

@Future-Outlier Future-Outlier Oct 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in SidecarMode, when calculating the head pod's resource, we will call the function CalculatePodResource, and this func will iterate all containers in the head pod spec like thisfor _, container := range podSpec.Containers.
so I think sidecar mode will work, but if we can get a screenshot about a test for sidecar mode I will appreciate it.

cc @win5923

Copy link
Member

@Future-Outlier Future-Outlier Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wait, @rueian is right.
when using sidecar mode, the Min Resources's CPU shoulde be 3.5, but this is not correct here.

Image

Copy link
Collaborator Author

@win5923 win5923 Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rueian!
I’ve already added the check for SidecarMode. ae2941f

image

submitterTemplate := common.GetSubmitterTemplate(&rayJob.Spec, rayJob.Spec.RayClusterSpec)
submitterResource := utils.CalculatePodResource(submitterTemplate.Spec)
totalResourceList = append(totalResourceList, submitterResource)
return utils.CalculatePodResource(submitterTemplate.Spec)
} else if rayJob.Spec.SubmissionMode == rayv1.SidecarMode {
submitterContainer := common.GetDefaultSubmitterContainer(rayJob.Spec.RayClusterSpec)
return corev1.ResourceList{
corev1.ResourceCPU: submitterContainer.Resources.Requests[corev1.ResourceCPU],
corev1.ResourceMemory: submitterContainer.Resources.Requests[corev1.ResourceMemory],
Copy link
Collaborator

@rueian rueian Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we take all the resource types into account here? We'd better not assume there are only CPU and memory.

Copy link
Member

@Future-Outlier Future-Outlier Oct 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just fix it here, thank you!
cf6b48b

}
}

return v.syncPodGroup(ctx, rayJob, minMember, utils.SumResourceList(totalResourceList))
return corev1.ResourceList{}
}

func getAppPodGroupName(object metav1.Object) string {
Expand Down
Loading