Skip to content

Conversation

@zhengruifeng
Copy link
Contributor

What changes were proposed in this pull request?

Make createDataFrame support empty dataframe:

In [24]: spark.createDataFrame([], schema="x STRING, y INTEGER")
Out[24]: DataFrame[x: string, y: int]

Why are the changes needed?

to be consistent with PySpark

Does this PR introduce any user-facing change?

yes

How was this patch tested?

added UT and enabled doctests

try {
parser.parseTableSchema(sqlText)
} catch {
case _: ParseException =>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

refer to #38979 (comment)

@zhengruifeng
Copy link
Contributor Author

cc @HyukjinKwon @grundprinzip

@HyukjinKwon
Copy link
Member

Merged to master.

@zhengruifeng zhengruifeng deleted the connect_fix_41828 branch January 4, 2023 09:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants