[SPARK-31748][PYTHON] Document resource module in PySpark doc and rename/move classes #28569
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
Author
|
cc @holdenk too - seems I can't add you as a reviewer. |
HyukjinKwon
commented
May 18, 2020
| :param name: the name of the resource | ||
| :param addresses: an array of strings describing the addresses of the resource | ||
|
|
||
| .. versionadded:: 3.0.0 |
Member
Author
There was a problem hiding this comment.
Will open a PR to backport for this change specifically for branch-3.0. This PR itself should go to master only.
Member
There was a problem hiding this comment.
Please make a PR to branch-3.0 for this part.
This comment has been minimized.
This comment has been minimized.
|
Test build #122798 has finished for PR 28569 at commit
|
BryanCutler
approved these changes
May 18, 2020
Member
BryanCutler
left a comment
BryanCutler
left a comment
There was a problem hiding this comment.
+1 on the new module layout and the doc changes LGTM
viirya
approved these changes
May 18, 2020
Member
viirya
left a comment
viirya
left a comment
There was a problem hiding this comment.
Looks good too. It looks clearer.
viirya
reviewed
May 18, 2020
viirya
reviewed
May 18, 2020
|
Test build #122824 has finished for PR 28569 at commit
|
Member
|
retest this please |
|
Test build #122825 has finished for PR 28569 at commit
|
dongjoon-hyun
changed the title
dongjoon-hyun
approved these changes
May 20, 2020
Member
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM. Merged to master. Thank you.
Member
Author
|
Thank you all! |
HyukjinKwon
added a commit
that referenced
this pull request
May 20, 2020
…d rename/move classes ### What changes were proposed in this pull request? This PR partially backports #28569. 1.. Rename ``` pyspark └── resourceinformation.py └── class ResourceInformation ``` to ``` pyspark └── resource.py └── class ResourceInformation ``` So, the `ResourceInformation` is consistently imported via `pyspark.resource.ResourceInformation`. 2.. Document the new `pyspark.resource` module 3.. Minor docstring fix e.g.: ```diff - param name the name of the resource - param addresses an array of strings describing the addresses of the resource + :param name: the name of the resource + :param addresses: an array of strings describing the addresses of the resource + + .. versionadded:: 3.0.0 ``` ### Why are the changes needed? To document APIs, and move Python modules to fewer and simpler modules. ### Does this PR introduce _any_ user-facing change? No, the changes are in unreleased branches. ### How was this patch tested? Manually tested via: ```bash cd python ./run-tests --python-executables=python3 --modules=pyspark-core ``` Closes #28586 from HyukjinKwon/SPARK-31748. Authored-by: HyukjinKwon <gurwls223@apache.org> Signed-off-by: HyukjinKwon <gurwls223@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR is kind of a followup for SPARK-29641 and SPARK-28234. This PR proposes:
1.. Document the new
pyspark.resourcemodule introduced at 95aec09, in PySpark API docs.2.. Move classes into fewer and simpler modules
Before:
After:
3.. Minor docstring fix e.g.:
Why are the changes needed?
To document APIs, and move Python modules to fewer and simpler modules.
Does this PR introduce any user-facing change?
No, the changes are in unreleased branches.
How was this patch tested?
Manually tested via:
cd python ./run-tests --python-executables=python3 --modules=pyspark-core ./run-tests --python-executables=python3 --modules=pyspark-resource