Add Google Sheets data source #6
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Owner
allisonwang-db
left a comment
allisonwang-db
left a comment
There was a problem hiding this comment.
Looks great! Left some comments
allisonwang-db
approved these changes
Jan 22, 2025
|
|
||
| class GoogleSheetsDataSource(DataSource): | ||
| """ | ||
| A DataSource for reading table from public Google Sheets. |
Owner
There was a problem hiding this comment.
It would be great to also support private ones :)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Note: Description below is partially generated by Copilot.
This pull request introduces a new data source for Google Sheets, adds it to the documentation, and includes tests to ensure its functionality. The most important changes include the creation of the
GoogleSheetsDataSourceclass, updates to the documentation, and the addition of new tests.New Data Source Implementation:
pyspark_datasources/googlesheets.py: Implemented theGoogleSheetsDataSourceclass to read tables from public Google Sheets documents. See docstring for more detail.pyspark_datasources/__init__.py: Added the import statement for the newGoogleSheetsDataSourceclass.Documentation Updates:
docs/datasources/googlesheets.md: Created a new documentation file for theGoogleSheetsDataSource.docs/index.md: Updated the data source table to include the newGoogleSheetsDataSource.mkdocs.yml: Added the new documentation file to the navigation.Testing:
tests/test_google_sheets.py: Added tests for theGoogleSheetsDataSourceto verify its functionality, including tests for URL handling, spreadsheet ID handling, missing options, mutual exclusive options, and custom schema.