Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make input dir in DataProcessor required #18910

Merged
merged 7 commits into from
Nov 18, 2023
Merged

Conversation

awaelchli
Copy link
Contributor

@awaelchli awaelchli commented Nov 1, 2023

What does this PR do?

The input dir in DataProcessor is required to read the files in the user's directory. It shouldn't default to None. Found in #18907.

Blocked by some work needed by Thomas on map()


📚 Documentation preview 📚: https://pytorch-lightning--18910.org.readthedocs.build/en/18910/

cc @Borda

Copy link
Contributor

github-actions bot commented Nov 1, 2023

⚡ Required checks status: All passing 🟢

Groups summary

🟢 lightning_data: CPU workflow
Check ID Status
data-cpu (macOS-11, lightning, 3.10, 2.1) success
data-cpu (ubuntu-20.04, lightning, 3.10, 2.1) success
data-cpu (windows-2022, lightning, 3.10, 2.1) success

These checks are required after the changes to src/lightning/data/streaming/data_processor.py.

🟢 mypy
Check ID Status
mypy success

These checks are required after the changes to src/lightning/data/streaming/data_processor.py.

🟢 install
Check ID Status
install-pkg (ubuntu-22.04, app, 3.8) success
install-pkg (ubuntu-22.04, app, 3.11) success
install-pkg (ubuntu-22.04, fabric, 3.8) success
install-pkg (ubuntu-22.04, fabric, 3.11) success
install-pkg (ubuntu-22.04, pytorch, 3.8) success
install-pkg (ubuntu-22.04, pytorch, 3.11) success
install-pkg (ubuntu-22.04, lightning, 3.8) success
install-pkg (ubuntu-22.04, lightning, 3.11) success
install-pkg (ubuntu-22.04, notset, 3.8) success
install-pkg (ubuntu-22.04, notset, 3.11) success
install-pkg (macOS-12, app, 3.8) success
install-pkg (macOS-12, app, 3.11) success
install-pkg (macOS-12, fabric, 3.8) success
install-pkg (macOS-12, fabric, 3.11) success
install-pkg (macOS-12, pytorch, 3.8) success
install-pkg (macOS-12, pytorch, 3.11) success
install-pkg (macOS-12, lightning, 3.8) success
install-pkg (macOS-12, lightning, 3.11) success
install-pkg (macOS-12, notset, 3.8) success
install-pkg (macOS-12, notset, 3.11) success
install-pkg (windows-2022, app, 3.8) success
install-pkg (windows-2022, app, 3.11) success
install-pkg (windows-2022, fabric, 3.8) success
install-pkg (windows-2022, fabric, 3.11) success
install-pkg (windows-2022, pytorch, 3.8) success
install-pkg (windows-2022, pytorch, 3.11) success
install-pkg (windows-2022, lightning, 3.8) success
install-pkg (windows-2022, lightning, 3.11) success
install-pkg (windows-2022, notset, 3.8) success
install-pkg (windows-2022, notset, 3.11) success

These checks are required after the changes to src/lightning/data/streaming/data_processor.py.


Thank you for your contribution! 💜

Note
This comment is automatically generated and updates for 60 minutes every 180 seconds. If you have any other questions, contact carmocca for help.

Copy link
Contributor

@tchaton tchaton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM !

@mergify mergify bot added the has conflicts label Nov 3, 2023
@mergify mergify bot removed the has conflicts label Nov 6, 2023
@github-actions github-actions bot added the data (external) litdata package label Nov 16, 2023
@mergify mergify bot added the ready PRs ready to be merged label Nov 18, 2023
@awaelchli awaelchli merged commit 61a286d into master Nov 18, 2023
54 checks passed
@awaelchli awaelchli deleted the data/input-dir-required branch November 18, 2023 17:24
@awaelchli awaelchli added this to the 2.1.x milestone Nov 18, 2023
Borda pushed a commit that referenced this pull request Dec 19, 2023
lantiga pushed a commit that referenced this pull request Dec 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data (external) litdata package ready PRs ready to be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants