Skip to content

Commit df766a4

Browse files
committed
[SPARK-21089][SQL] Fix DESC EXTENDED/FORMATTED to Show Table Properties
### What changes were proposed in this pull request? Since both table properties and storage properties share the same key values, table properties are not shown in the output of DESC EXTENDED/FORMATTED when the storage properties are not empty. This PR is to fix the above issue by renaming them to different keys. ### How was this patch tested? Added test cases. Author: Xiao Li <[email protected]> Closes apache#18294 from gatorsmile/tableProperties.
1 parent 0c88e8d commit df766a4

File tree

4 files changed

+105
-87
lines changed

4 files changed

+105
-87
lines changed

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ case class CatalogStorageFormat(
7575
CatalogUtils.maskCredentials(properties) match {
7676
case props if props.isEmpty => // No-op
7777
case props =>
78-
map.put("Properties", props.map(p => p._1 + "=" + p._2).mkString("[", ", ", "]"))
78+
map.put("Storage Properties", props.map(p => p._1 + "=" + p._2).mkString("[", ", ", "]"))
7979
}
8080
map
8181
}
@@ -316,7 +316,7 @@ case class CatalogTable(
316316
}
317317
}
318318

319-
if (properties.nonEmpty) map.put("Properties", tableProperties)
319+
if (properties.nonEmpty) map.put("Table Properties", tableProperties)
320320
stats.foreach(s => map.put("Statistics", s.simpleString))
321321
map ++= storage.toLinkedHashMap
322322
if (tracksPartitionsInCatalog) map.put("Partition Provider", "Catalog")

sql/core/src/test/resources/sql-tests/inputs/describe.sql

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
CREATE TABLE t (a STRING, b INT, c STRING, d STRING) USING parquet
2+
OPTIONS (a '1', b '2')
23
PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS
34
COMMENT 'table_comment';
45

@@ -13,6 +14,8 @@ CREATE TEMPORARY VIEW temp_Data_Source_View
1314

1415
CREATE VIEW v AS SELECT * FROM t;
1516

17+
ALTER TABLE t SET TBLPROPERTIES (e = '3');
18+
1619
ALTER TABLE t ADD PARTITION (c='Us', d=1);
1720

1821
DESCRIBE t;

sql/core/src/test/resources/sql-tests/results/describe-table-after-alter-table.sql.out

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ Last Access [not included in comparison]
5757
Type MANAGED
5858
Provider parquet
5959
Comment modified comment
60-
Properties [type=parquet]
60+
Table Properties [type=parquet]
6161
Location [not included in comparison]sql/core/spark-warehouse/table_with_comment
6262

6363

0 commit comments

Comments
 (0)