Skip to content

[core] (cgroups) Use /proc/mounts if mount file is missing. #58577

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jjyao merged 4 commits into from
Nov 13, 2025
Merged

[core] (cgroups) Use /proc/mounts if mount file is missing. #58577

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5d7da0f
[core] (cgroups) Use /proc/mounts if glibc's MOUNTED points to a file
israbbani Nov 12, 2025
deb12d1
Making the unit test deterministic
israbbani Nov 12, 2025
8e1b6b2
Thnx robot
israbbani Nov 12, 2025
4928eed
error msg
israbbani Nov 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 19 additions & 3 deletions src/ray/common/cgroup2/sysfs_cgroup_driver.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,13 +46,29 @@

namespace ray {
Status SysFsCgroupDriver::CheckCgroupv2Enabled() {
FILE *fp = setmntent(mount_file_path_.c_str(), "r");
std::string mount_file_path = mount_file_path_;

int fd = open(mount_file_path.c_str(), O_RDONLY);

if (fd == -1) {
mount_file_path = fallback_mount_file_path_;
Copy link
Contributor Author

@israbbani israbbani Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could potentially add more error handling here (e.g. disambiguating ENOENT from other error codes), but I think it might be overkill.

Copy link
Collaborator

@jjyao jjyao Nov 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a log and print out the error

Copy link
Contributor Author

@israbbani israbbani Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Here's an example from the test output.

[2025-11-13 00:47:35,508 W 3722056 3722056] sysfs_cgroup_driver.cc:55: Failed to open mount fail at /does/not/exist because of error 'No such file or directory'. Using fallback mount file at /tmp/Bx6Fw0.

RAY_LOG(WARNING) << absl::StrFormat(
"Failed to open mount fail at %s because of error '%s'. Using fallback mount "
"file at %s.",
mount_file_path_,
strerror(errno),
fallback_mount_file_path_);
} else {
close(fd);
}

FILE *fp = setmntent(mount_file_path.c_str(), "r");

if (!fp) {
return Status::Invalid(
absl::StrFormat("Failed to open mount file at %s. Could not verify that "
"cgroupv2 was mounted correctly. \n%s",
mount_file_path_,
mount_file_path,
strerror(errno)));
}

Expand All @@ -71,7 +87,7 @@ Status SysFsCgroupDriver::CheckCgroupv2Enabled() {
return Status::Invalid(
absl::StrFormat("Failed to parse mount file at %s. Could not verify that "
"cgroupv2 was mounted correctly.",
mount_file_path_));
mount_file_path));
}

if (found_cgroupv1 && found_cgroupv2) {
Expand Down
9 changes: 7 additions & 2 deletions src/ray/common/cgroup2/sysfs_cgroup_driver.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,11 @@ class SysFsCgroupDriver : public CgroupDriverInterface {
/**
* @param mount_file_path only used for testing.
*/
explicit SysFsCgroupDriver(std::string mount_file_path = MOUNTED)
: mount_file_path_(std::move(mount_file_path)) {}
explicit SysFsCgroupDriver(
std::string mount_file_path = MOUNTED,
std::string fallback_mount_file_path = kFallbackMountsFilePath)
: mount_file_path_(std::move(mount_file_path)),
fallback_mount_file_path_(fallback_mount_file_path) {}

~SysFsCgroupDriver() override = default;
SysFsCgroupDriver(const SysFsCgroupDriver &other) = delete;
Expand Down Expand Up @@ -286,10 +289,12 @@ class SysFsCgroupDriver : public CgroupDriverInterface {

// Used for unit testing through the constructor.
std::string mount_file_path_;
std::string fallback_mount_file_path_;

static constexpr std::string_view kCgroupProcsFilename = "cgroup.procs";
static constexpr std::string_view kCgroupSubtreeControlFilename =
"cgroup.subtree_control";
static constexpr std::string_view kCgroupControllersFilename = "cgroup.controllers";
static inline std::string kFallbackMountsFilePath = "/proc/mounts";
};
} // namespace ray
9 changes: 9 additions & 0 deletions src/ray/common/cgroup2/tests/sysfs_cgroup_driver_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,15 @@ TEST(SysFsCgroupDriverTest,
ASSERT_TRUE(s.IsInvalid()) << s.ToString();
}

TEST(SysFsCgroupDriverTest,
CheckCgroupv2EnabledSucceedsIfMountFileNotFoundButFallbackFileIsCorrect) {
TempFile temp_fallback_mount_file;
temp_fallback_mount_file.AppendLine("cgroup2 /sys/fs/cgroup cgroup2 rw 0 0\n");
SysFsCgroupDriver driver("/does/not/exist", temp_fallback_mount_file.GetPath());
Status s = driver.CheckCgroupv2Enabled();
EXPECT_TRUE(s.ok()) << s.ToString();
}

TEST(SysFsCgroupDriverTest, CheckCgroupv2EnabledSucceedsIfOnlyCgroupv2Mounted) {
TempFile temp_mount_file;
temp_mount_file.AppendLine("cgroup2 /sys/fs/cgroup cgroup2 rw 0 0\n");
Expand Down