Skip to content

[SPARK-40733][SQL] Make the contents of SERDEPROPERTIES in the result of ShowCreateTableAsSerdeCommand have a fixed order #38238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
LuciferYang wants to merge 1 commit into from
+5 −5
Closed

[SPARK-40733][SQL] Make the contents of SERDEPROPERTIES in the result of ShowCreateTableAsSerdeCommand have a fixed order #38238

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1342,7 +1342,7 @@ case class ShowCreateTableAsSerdeCommand(
storage.serde.foreach { serde =>
builder ++= s"ROW FORMAT SERDE '$serde'\n"

val serdeProps = conf.redactOptions(metadata.storage.properties).map {
val serdeProps = conf.redactOptions(metadata.storage.properties).toSeq.sortBy(_._1).map {
Copy link
Contributor Author

@LuciferYang LuciferYang Oct 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spark/sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala

Lines 1202 to 1210 in 1cb8250

private def showDataSourceTableOptions(metadata: CatalogTable, builder: StringBuilder): Unit = {
// For datasource table, there is a provider there in the metadata.
// If it is a Hive table, we already convert its metadata and fill in a provider.
builder ++= s"USING ${metadata.provider.get}\n"
val dataSourceOptions = conf.redactOptions(metadata.storage.properties).toSeq.sortBy(_._1).map {
case (key, value) =>
s"'${escapeSingleQuotedString(key)}' = '${escapeSingleQuotedString(value)}'"
}

showDataSourceTableOptions already do the similar work in #34753

case (key, value) =>
s"'${escapeSingleQuotedString(key)}' = '${escapeSingleQuotedString(value)}'"
}
Expand Down
8 changes: 4 additions & 4 deletions ...ive/src/test/scala/org/apache/spark/sql/hive/execution/command/ShowCreateTableSuite.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,9 +128,9 @@ class ShowCreateTableSuite extends v1.ShowCreateTableSuiteBase with CommandSuite
" ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe'" +
" WITH SERDEPROPERTIES (" +
" 'colelction.delim' = '@'," +
" 'field.delim' = ','," +
" 'mapkey.delim' = '#'," +
" 'serialization.format' = ','," +
" 'field.delim' = ',')" +
" 'serialization.format' = ',')" +
" STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat'" +
" OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat'" +
" TBLPROPERTIES ("
Expand Down Expand Up @@ -178,9 +178,9 @@ class ShowCreateTableSuite extends v1.ShowCreateTableSuiteBase with CommandSuite
val expected = s"CREATE TABLE $fullName ( c1 INT COMMENT 'bla', c2 STRING)" +
" ROW FORMAT SERDE 'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe'" +
" WITH SERDEPROPERTIES (" +
" 'field.delim' = ','," +
" 'mapkey.delim' = ','," +
" 'serialization.format' = '1'," +
" 'field.delim' = ',')" +
" 'serialization.format' = '1')" +
" STORED AS INPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat'" +
" OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat'" +
" TBLPROPERTIES ("
Expand Down