Skip to content

[SPARK-28809][DOC][SQL]Document SHOW TABLE in SQL Reference #25632

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
shivusondur wants to merge 4 commits into from
Closed

[SPARK-28809][DOC][SQL]Document SHOW TABLE in SQL Reference #25632

Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8d16dc1
added doc for "show table extended" sql command.
shivusondur Aug 30, 2019
b81aaf2
handled the comments
shivusondur Oct 29, 2019
90467ac
handled the comments
shivusondur Oct 31, 2019
79db8ca
handled the comments
shivusondur Nov 4, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
154 changes: 151 additions & 3 deletions docs/sql-ref-syntax-aux-show-table.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: global
title: SHOW TABLE
displayTitle: SHOW TABLE
title: SHOW TABLE EXTENDED
displayTitle: SHOW TABLE EXTENDED
license: |
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
Expand All @@ -18,5 +18,153 @@ license: |
See the License for the specific language governing permissions and
limitations under the License.
---
### Description

**This page is under construction**
`SHOW TABLE EXTENDED` will show information for all tables matching the given regular expression.
Output includes basic table information and file system information like `Last Access`,
`Created By`, `Type`, `Provider`, `Table Properties`, `Location`, `Serde Library`, `InputFormat`,
`OutputFormat`, `Storage Properties`, `Partition Provider`, `Partition Columns` and `Schema`.

Users cannot use regular expression for the table name if a partition specification is present. It will output
Copy link
Member

@srowen srowen Oct 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Last nit: a regular expression. But how about:
"If a partition specification is present, it outputs the given partition's ... Note that a table regex cannot be used with a partition specification."

Copy link
Contributor Author

@shivusondur shivusondur Oct 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@srowen
If the PARTITION key present in the query, the pattern mentioned in LIKE will not be treated as a regex. It will be treated as normal name.

for example. If we mention the regex with LIKE and PARTITION keyword, it fails like below.
image

But if mention the full name like below it works
image

Is it better to mention this exception scenario in the examples section?

Copy link
Member

@srowen srowen Oct 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think that agrees with the text I suggested? there's no problem documenting this in the docs here.

Copy link
Contributor Author

@shivusondur shivusondur Oct 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@srowen
Documented Exception scenario.

Copy link
Member

@srowen srowen Oct 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but I don't think you changed this sentence. I'm suggesting a typo fix and rewrite right here.

Copy link
Member

@srowen srowen Nov 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shivusondur still at least one more change here.

Copy link
Contributor Author

@shivusondur shivusondur Nov 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@srowen
Sorry for late, I updated it according to your suggestion.

the given partition's file-system-specific information such as `Partition Parameters` and `Partition Statistics`

### Syntax
{% highlight sql %}
SHOW TABLE EXTENDED [IN|FROM database_name] LIKE 'identifier_with_wildcards' [PARTITION(partition_spec)];
{% endhighlight %}

### Parameters
<dl>
<dt><code><em>IN|FROM database_name</em></code></dt>
<dd>
Specifies database name. If not provided, will use the current database.
</dd>
<dt><code><em>LIKE string_pattern</em></code></dt>
<dd>
Specifies the regular expression pattern that is used to filter out unwanted tables.
<ul>
<li> Except for `*` and `|` character, the pattern works like a regex.</li>
<li> `*` alone matches 0 or more characters and `|` is used to separate multiple different regexes,
any of which can match. </li>
<li> The leading and trailing blanks are trimmed in the input pattern before processing.</li>
</ul>
</dd>
<dt><code><em>PARTITION(partition_spec)</em></code></dt>
<dd>
Specifies partition column and its value which is exists in the table. With this user cannot use regular expression for table name.
</dd>
</dl>
### Examples
{% highlight sql %}
-- Assumes `employee` table created with partitioned by column `grade`
-- +-------+--------+--+
-- | name | grade |
-- +-------+--------+--+
-- | sam | 1 |
-- | suj | 2 |
-- +-------+--------+--+

-- Show the details of the table
SHOW TABLE EXTENDED LIKE `employee`;
+--------+---------+-----------+---------------------------------------------------------------
|database|tableName|isTemporary| information
+--------+---------+-----------+---------------------------------------------------------------
|default |employee |false |Database: default
Table: employee
Owner: root
Created Time: Fri Aug 30 15:10:21 IST 2019
Last Access: Thu Jan 01 05:30:00 IST 1970
Created By: Spark 3.0.0-SNAPSHOT
Type: MANAGED
Provider: hive
Table Properties: [transient_lastDdlTime=1567158021]
Location: file:/opt/spark1/spark/spark-warehouse/employee
Serde Library: org.apache.hadoop.hive.serde2.lazy
.LazySimpleSerDe
InputFormat: org.apache.hadoop.mapred.TextInputFormat
OutputFormat: org.apache.hadoop.hive.ql.io
.HiveIgnoreKeyTextOutputFormat
Storage Properties: [serialization.format=1]
Partition Provider: Catalog
Partition Columns: [`grade`]
Schema: root
|-- name: string (nullable = true)
|-- grade: integer (nullable = true)

+--------+---------+-----------+---------------------------------------------------------------

-- showing the multiple table details with pattern matching
SHOW TABLE EXTENDED LIKE `employe*`;
+--------+---------+-----------+---------------------------------------------------------------
|database|tableName|isTemporary| information
+--------+---------+-----------+---------------------------------------------------------------
|default |employee |false |Database: default
Table: employee
Owner: root
Created Time: Fri Aug 30 15:10:21 IST 2019
Last Access: Thu Jan 01 05:30:00 IST 1970
Created By: Spark 3.0.0-SNAPSHOT
Type: MANAGED
Provider: hive
Table Properties: [transient_lastDdlTime=1567158021]
Location: file:/opt/spark1/spark/spark-warehouse/employee
Serde Library: org.apache.hadoop.hive.serde2.lazy
.LazySimpleSerDe
InputFormat: org.apache.hadoop.mapred.TextInputFormat
OutputFormat: org.apache.hadoop.hive.ql.io
.HiveIgnoreKeyTextOutputFormat
Storage Properties: [serialization.format=1]
Partition Provider: Catalog
Partition Columns: [`grade`]
Schema: root
|-- name: string (nullable = true)
|-- grade: integer (nullable = true)

|default |employee1|false |Database: default
Table: employee1
Owner: root
Created Time: Fri Aug 30 15:22:33 IST 2019
Last Access: Thu Jan 01 05:30:00 IST 1970
Created By: Spark 3.0.0-SNAPSHOT
Type: MANAGED
Provider: hive
Table Properties: [transient_lastDdlTime=1567158753]
Location: file:/opt/spark1/spark/spark-warehouse/employee1
Serde Library: org.apache.hadoop.hive.serde2.lazy
.LazySimpleSerDe
InputFormat: org.apache.hadoop.mapred.TextInputFormat
OutputFormat: org.apache.hadoop.hive.ql.io
.HiveIgnoreKeyTextOutputFormat
Storage Properties: [serialization.format=1]
Partition Provider: Catalog
Schema: root
|-- name: string (nullable = true)

+--------+---------+----------+----------------------------------------------------------------

-- show partition file system details
SHOW TABLE EXTENDED IN `default` LIKE `employee` PARTITION (`grade=1`);
+--------+---------+-----------+---------------------------------------------------------------
|database|tableName|isTemporary| information information |
+--------+---------+-----------+---------------------------------------------------------------
|default |employee |false | Partition Values: [grade=1]
Location: file:/opt/spark1/spark/spark-warehouse/employee
/grade=1
Serde Library: org.apache.hadoop.hive.serde2.lazy
.LazySimpleSerDe
InputFormat: org.apache.hadoop.mapred.TextInputFormat
OutputFormat: org.apache.hadoop.hive.ql.io
.HiveIgnoreKeyTextOutputFormat
Storage Properties: [serialization.format=1]
Partition Parameters: {rawDataSize=-1, numFiles=1,
transient_lastDdlTime=1567158221, totalSize=4,
COLUMN_STATS_ACCURATE=false, numRows=-1}
Created Time: Fri Aug 30 15:13:41 IST 2019
Last Access: Thu Jan 01 05:30:00 IST 1970
Partition Statistics: 4 bytes
|
+--------+---------+-----------+---------------------------------------------------------------
{% endhighlight %}
### Related Statements
- [CREATE TABLE](sql-ref-syntax-ddl-create-table.html)
- [DESCRIBE TABLE](sql-ref-syntax-aux-describe-table.html)