Move ray data out of experimental folder - #1619
Conversation
Signed-off-by: Onur Yilmaz <oyilmaz@nvidia.com>
Signed-off-by: Onur Yilmaz <oyilmaz@nvidia.com>
|
/ok to test acf301e |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
|
||
| def __init__(self, config: dict[str, Any] | None = None, ignore_head_node: bool = False): | ||
| super().__init__(config, ignore_head_node) |
There was a problem hiding this comment.
The nemo_curator.backends.experimental.ray_data module is completely removed without a deprecation redirect. Any downstream code that depended on the old import path — including anything not covered by this PR — will immediately receive an ImportError with no migration guidance.
While the experimental label signals that breaking changes are expected, providing a thin shim in the old location is still considered good practice:
# nemo_curator/backends/experimental/ray_data/__init__.py (re-created as a shim)
import warnings
warnings.warn(
"nemo_curator.backends.experimental.ray_data is deprecated. "
"Use nemo_curator.backends.ray_data instead.",
DeprecationWarning,
stacklevel=2,
)
from nemo_curator.backends.ray_data import RayDataExecutor # noqa: F401, E402
__all__ = ["RayDataExecutor"]This allows existing users to get a clear deprecation warning rather than a hard crash.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Signed-off-by: Onur Yilmaz <oyilmaz@nvidia.com>
|
/ok to test 2dcc3e8 |
|
@ayushdg Benchmarks also running without any issue. Can you please approve this? I guess we need one of the maintainers approval to merge. |
@oyilmaz-nvidia I have write access. PRs need to be up to date with main to merge. |
|
/ok to test 204d261 |
2 participants
Description
This PR moves the ray data executor out of the experimental folder.
Usage
# Add snippet demonstrating usageChecklist