[SPARK-40993][CONNECT][PYTHON][DOCS] Migrate markdown style README to PySpark Development Documentation #38477
Closed
amaliujia wants to merge 2 commits intoapache:masterfrom
Closed
[SPARK-40993][CONNECT][PYTHON][DOCS] Migrate markdown style README to PySpark Development Documentation #38477amaliujia wants to merge 2 commits intoapache:masterfrom
amaliujia wants to merge 2 commits intoapache:masterfrom
Conversation
…ython/docs/development/testing.rst.
Contributor
Author
HyukjinKwon
previously approved these changes
Nov 2, 2022
Member
HyukjinKwon
left a comment
HyukjinKwon
left a comment
There was a problem hiding this comment.
LGTM from a cursory look. Thanks for doing this.
cc @itholic mind double checking if anything is broken or inconsistent?
HyukjinKwon
changed the title
Contributor
|
Can we maybe hold this off until we have a decision if the Python client should be directly embedded in the PySpark code or extracted to its own module / artifact? |
Member
|
Yeah, I am fine with that. |
Contributor
|
Looks good, and I'm also fine with. |
|
Can one of the admins verify this patch? |
Member
|
Fixed in #38470 for now. Per #38477 (comment), let me close this for now. |
Member
|
Sicen we're going to package together, let's migrate this docs too. cc @zhengruifeng too |
HyukjinKwon
added a commit
that referenced
this pull request
Jan 2, 2023
…and script to dev/ and Python documentation ### What changes were proposed in this pull request? This PR takes over #39211 and #38477 that proposes: - Move `connector/connect/dev/generate_protos.sh` → `dev/generate_protos.sh` to be consistent with other places - Move Python-specific development guides into `python/docs/source/development/testing.rst` ### Why are the changes needed? To keep the project structure and documentation consistent. ### Does this PR introduce _any_ user-facing change? Python-specific development guides for Spark Connect will be added in https://spark.apache.org/docs/latest/api/python/development/testing.html. ### How was this patch tested? I manually tested: ``` ./dev/generate_protos.sh ./dev/check-codegen-python.py ``` I also manually verified the Python documentation. Closes #39338 from HyukjinKwon/SPARK-41705. Lead-authored-by: Hyukjin Kwon <gurwls223@apache.org> Co-authored-by: Ted Yu <yuzhihong@gmail.com> Co-authored-by: Rui Wang <rui.wang@databricks.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This PR consolidates the development facing documentation of Spark Connect Python client into existing PySpark development doc (more specifically:
python/docs/developmement/testing.rst).Why are the changes needed?
Improve development documentation.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Documentation check script.