Skip to content

[None][Fix] Prevent resource leak warnings #3216

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yaox12 merged 8 commits into from
Feb 10, 2026
Merged

[None][Fix] Prevent resource leak warnings #3216

Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
5447886
[None][Fix] Prevent resource leak warnings
IanBoyanZhang Feb 3, 2026
5dfce30
Merge branch 'dev' into clean_up_close_warning
IanBoyanZhang Feb 5, 2026
bbe2c64
Update utils.py
yaox12 Feb 5, 2026
7be5064
Update utils.py
yaox12 Feb 9, 2026
49d1509
Merge branch 'dev' into clean_up_close_warning
yaox12 Feb 9, 2026
9c8366f
Update utils.py
yaox12 Feb 9, 2026
2ffbdd6
Update utils.py
yaox12 Feb 9, 2026
5833bce
Update utils.py
yaox12 Feb 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions megatron/core/datasets/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@


class Split(Enum):
"""Enum train, valid, and test split."""

train = 0
valid = 1
test = 2
Expand All @@ -26,7 +28,11 @@ def compile_helpers():
if subprocess.run(command).returncode != 0:
import sys

import torch.distributed as dist

log_single_rank(logger, logging.ERROR, "Failed to compile the C++ dataset helper functions")

dist.destroy_process_group()
sys.exit(1)


Expand Down
Loading