-
Notifications
You must be signed in to change notification settings - Fork 7.2k
[core] (3/n) [Removing GCS Centralized Scheduling] Removing GlobalGC from the GCS #60019
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[core] (3/n) [Removing GCS Centralized Scheduling] Removing GlobalGC from the GCS #60019
Conversation
ability to send global GC requests to all raylets in the cluster. Moving the GC throttler to the Raylet's package now that it's only used by Raylets. Signed-off-by: irabbani <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request effectively removes the GlobalGC functionality from the GCS, which is a good cleanup of an unused feature. The changes are clear and align with the goal of removing Centralized Actor Scheduling.
As part of this, the Throttler utility has been moved from src/ray/util to src/ray/raylet since it's now only used by the raylet. I've made a few suggestions to improve this move:
- Moving
Throttlerto theray::rayletnamespace for better code organization. - Removing a redundant header include.
Overall, this is a solid contribution to simplifying the codebase.
| #include "ray/core_worker_rpc_client/core_worker_client_pool.h" | ||
| #include "ray/flatbuffers/node_manager_generated.h" | ||
| #include "ray/raylet/local_object_manager_interface.h" | ||
| #include "ray/raylet/throttler.h" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Gemini, you need to talk to whoever wrote https://google.github.io/styleguide/cppguide.html#Include_What_You_Use at Google.
Signed-off-by: irabbani <[email protected]>
e17b2d2
into
irabbani/remove-centralized-actor-scheduling-2
…aseManager from GCS. (#60008) This PR stacks on #59979. This is 2/N and 3/N (#60019) in a series of PRs to remove Centralized Actor Scheduling by the GCS (introduced in #15943). The feature is off by default and no longer in use or supported. In this PR, 1. I remove use of the ClusterLeaseManager (from the Raylet's scheduler) in the GCS. 2. I remove GC requests published globally from the GCS to all Raylets through the RaySyncer. This feature was introduced as part of Centralized Scheduling through GCS in #28663. 3. I've moded the Throttler from util to the raylet package and made it's visibility private. The GC implementation used to check to see if there were pending leases for two consecutive intervals (of 10s) each from the ClusterLeaseManager in the GCS. If so, the GCS would trigger GlobalGC. Therefore, with GCS Centralized Scheduling disabled, the GCS would never send GlobalGC requests. It's safe to delete now. --------- Signed-off-by: irabbani <[email protected]>
…eManager from GCS (#60121) This PR stacks on #60019. This is 3/N in a series of PRs to remove Centralized Actor Scheduling by the GCS (introduced in #15943). The feature is off by default and no longer in use or supported. In this PR, I've removed the GCS's dependency on the LocalLeaseManager. I've also moved LocalLeaseManager to the raylet/scheduling package and made it's visibility private to the package. Also deleted the NoopLocalLeaseManager. The LocalLeaseManager is used by the ClusterLeaseManager to see if a task can scheduled locally by a Raylet. The GCS used only the Noop implementation. --------- Signed-off-by: irabbani <[email protected]> Signed-off-by: Ibrahim Rabbani <[email protected]>
…eManager from GCS (ray-project#60121) This PR stacks on ray-project#60019. This is 3/N in a series of PRs to remove Centralized Actor Scheduling by the GCS (introduced in ray-project#15943). The feature is off by default and no longer in use or supported. In this PR, I've removed the GCS's dependency on the LocalLeaseManager. I've also moved LocalLeaseManager to the raylet/scheduling package and made it's visibility private to the package. Also deleted the NoopLocalLeaseManager. The LocalLeaseManager is used by the ClusterLeaseManager to see if a task can scheduled locally by a Raylet. The GCS used only the Noop implementation. --------- Signed-off-by: irabbani <[email protected]> Signed-off-by: Ibrahim Rabbani <[email protected]> Signed-off-by: jeffery4011 <[email protected]>
This PR stacks on #60008.
This is 3/N in a series of PRs to remove Centralized Actor Scheduling by the GCS (introduced in #15943). The feature is off by default and no longer in use or supported.
In this PR, I remove GC requests published globally from the GCS to all Raylets through the RaySyncer. This feature was introduced as part of Centralized Scheduling through GCS in #28663.
The implementation used to check to see if there were pending leases for two consecutive intervals (of 10s) each from the ClusterLeaseManager in the GCS. If so, the GCS would trigger GlobalGC. Therefore, with GCS Centralized Scheduling disabled, the GCS would never send GlobalGC requests. It's safe to delete now.
I've also moved the Throttler from util to the raylet package and made it's visibility private.