[SPARK-28786][DOC][SQL]Document INSERT statement in SQL Reference #25525

huaxingao · 2019-08-21T06:05:15Z

What changes were proposed in this pull request?

Document INSERT statement in SQL Reference

Why are the changes needed?

To complete SQL reference.

Does this PR introduce any user-facing change?

Yes.

How was this patch tested?

Tested using jykyll build --serve

Here are the screen shots:

SparkQA · 2019-08-21T06:16:43Z

Test build #109468 has finished for PR 25525 at commit b1f189e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2019-08-21T18:46:20Z

Test build #109511 has finished for PR 25525 at commit d80bf4d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

huaxingao · 2019-08-21T19:12:39Z

@srowen Could you please review this one too? Thanks a lot in advance!

srowen · 2019-08-21T19:34:06Z

docs/sql-ref-syntax-dml-dynamic-partition-insert.md

+
+### Description
+
+When the partition value is not provided, such inserts are called as the dynamic partition inserts, also called as multi-partition inserts. In partition spec, the partition column values are optional. When the values are not given, these columns are referred to as dynamic partition columns; otherwise, they are static partition columns. For example, the partition spec (p1 = 3, p2, p3) has a static partition column (p1) and two dynamic partition columns (p2 and p3).


This looks like it's copied from the Databricks documentation: https://docs.databricks.com/spark/latest/spark-sql/language-manual/insert.html In general we can't just copy content from third party sources.

I think we'd have to check whether this is OK to contribute as OSS docs, and it might be. However there are references below to Databricks runtime that would not be appropriate or relevant.

Before going further, were any other of the sections here or in other PRs copied from elsewhere?

I refer to Databricks documentation but only directly copied this dynamic partition insert because I am not familiar with this part. I will check and rewrite.

@srowen I removed dynamic partition insert. I also looked closely to make sure other insert files are OK. Could you please review one more time?

SparkQA · 2019-08-21T22:02:08Z

Test build #109527 has finished for PR 25525 at commit 6966262.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

srowen

OK, if the text here is written from scratch, that's OK. The form looks fine and looks reasonable to me. I don't know the semantics of these commands enough to evaluate if it's 100% correct, but looks good. As long as the info is drawn from what Spark supports rather than Hive, it looks directionally correct.

SparkQA · 2019-08-22T06:22:21Z

Test build #109557 has finished for PR 25525 at commit 4dc6caa.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dilipbiswal · 2019-08-23T03:40:02Z

docs/sql-ref-syntax-dml-insert-into.md

+
+### Description
+
+The `INSERT INTO` statement inserts new rows into a table. The inserted rows can be specified by value expressions, or resulted from a query.


@huaxingao Wondering if this would sound better ?

The input rows for the INSERT statement can be produced by one of two ways :

By value expressions

By a query

dilipbiswal · 2019-08-23T05:30:56Z

docs/sql-ref-syntax-dml-insert-into.md

+
+### Syntax
+{% highlight sql %}
+INSERT INTO [TABLE] [db_name.]table_name [partition_spec] value_clause | query


@huaxingao
Should it be { value_clause | query }
or
( value_clause | query ) ?

dilipbiswal · 2019-08-23T06:08:02Z

docs/sql-ref-syntax-dml-insert-into.md

+
+### Syntax
+{% highlight sql %}
+INSERT INTO [TABLE] [db_name.]table_name [partition_spec] value_clause | query


@huaxingao Can you please check the grammar. I think we allow more cases ..

dilipbiswal · 2019-08-23T06:08:52Z

docs/sql-ref-syntax-dml-insert-into.md

+
+ INSERT INTO employees PARTITION (age = 35)
+   SELECT * FROM candidates WHERE name = "Bob Doe"
+{% endhighlight %}


@huaxingao If we determine that we allow more syntax flavors .. could we please add one test for each ?

dilipbiswal · 2019-08-23T06:10:00Z

docs/sql-ref-syntax-dml-insert-into.md

+Specify the values to be inserted.
+
+#### ***query***:
+A `SELECT` statement that provides the rows to be inserted.


@huaxingao would produces sound better ?

SparkQA · 2019-08-27T07:25:26Z

Test build #109792 has finished for PR 25525 at commit bd7677b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dilipbiswal · 2019-08-27T20:06:42Z

docs/sql-ref-syntax-dml-insert-into.md

+### Examples
+#### Single Row Insert Using a VALUES Clause
+{% highlight sql %}
+ CREATE TABLE students (Name VARCHAR(64), Address VARCHAR(64), StudentID INT)


@huaxingao The SQLs in the example, should we terminate with a semi colon ? In case users would like to cut-paste into the shell ? What do you think ?

Yes, we should terminate with a semi colon. I checked several other SQL reference, all of the example SQLs end with a semi colon.

dilipbiswal · 2019-08-27T20:17:31Z

docs/sql-ref-syntax-dml-insert-overwrite-directory-hive.md

+Specifies the destination directory.
+
+#### ***row_format***:
+Specifies the row format for this insert. `SERDE` clause can be used to specify a custom `SERDE` for this insert. Alternatively, `DELIMITED` clause can be used to specify the native `SERDE` and state the delimiter, escape character, null character, and so on.


@huaxingao In here we specify the SERDE clause. How do we correlate this to the syntax diagram ? Is "ROW FORMAT" same as the "SERDE" clause ?

Will add Valid options are `SERDE` clause and `DELIMITED` clause.

dilipbiswal · 2019-08-27T20:31:05Z

docs/sql-ref-syntax-dml-insert-into.md

+
+The `INSERT INTO` statement inserts new rows into a table. The inserted rows can be specified by value expressions or result from a query.
+
+See also:


@huaxingao Do you think a "see also" or "Related statements (i have used this)" will be better at the end of the page ? After reading the description, personally i would prefer to see the syntax. What do you think ?

I will move to the end and also use "Related Statements". It sounds better than "see also".

dilipbiswal · 2019-08-27T20:32:23Z

docs/sql-ref-syntax-dml-insert-into.md

+#### ***query***:
+A query that produces the rows to be inserted. It can be in one of following formats:
+- a `SELECT` statement
+- a table


TABLE statement ?

dilipbiswal · 2019-08-27T20:33:50Z

@huaxingao Thanks.. this looks much better. I will follow your format :-)

dilipbiswal · 2019-08-27T20:38:24Z

@huaxingao Is there a way we can delete the old screen shots ? We could also move the new screen-shots to the description part of the PR to make it easier for reviewers.. just a thought..

SparkQA · 2019-08-28T01:02:02Z

Test build #109837 has finished for PR 25525 at commit 5724edd.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dilipbiswal · 2019-08-28T04:03:30Z

LGTM

cc @gatorsmile @srowen for final sign off.

…BASE command(jira-28789)

huaxingao · 2019-08-29T01:19:21Z

@dilipbiswal I rebased and changed the code to make the parameters have the same format as your alter database cmd doc.

SparkQA · 2019-08-29T01:38:13Z

Test build #109892 has finished for PR 25525 at commit caacd58.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

dilipbiswal · 2019-08-29T02:51:31Z

@huaxingao Thanks.. looks good to me ..

gatorsmile · 2019-08-29T04:57:53Z

docs/sql-ref-syntax-dml-insert-into.md

+### Examples
+#### Single Row Insert Using a VALUES Clause
+{% highlight sql %}
+ CREATE TABLE students (Name VARCHAR(64), Address VARCHAR(64), StudentID INT)


Could you change all the column names to lower case?

Name -> name.

@gatorsmile
Changed. Thanks!

SparkQA · 2019-08-29T06:03:55Z

Test build #109898 has finished for PR 25525 at commit 43de251.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

gatorsmile · 2019-08-29T16:00:51Z

LGTM

Thanks! Merged to master.

huaxingao · 2019-08-29T16:08:01Z

Thank you @gatorsmile @srowen @dilipbiswal for the help!

dongjoon-hyun added DOCUMENTATION SQL labels Aug 21, 2019

srowen reviewed Aug 21, 2019

View reviewed changes

srowen reviewed Aug 22, 2019

View reviewed changes

dilipbiswal reviewed Aug 23, 2019

View reviewed changes

dilipbiswal reviewed Aug 27, 2019

View reviewed changes

huaxingao added 7 commits August 28, 2019 16:39

[SPARK-28786][DOC][SQL]Document INSERT statement in SQL Reference

37b2685

fix a few problems

a5733a2

remove dynamic-partition-insert and fix a few issues

6fda06a

fix a few issues

0b3fa02

address comments

e57c1a5

address comments

71778b7

make the paramters the same format as the ones in Document ALTER DATA…

caacd58

…BASE command(jira-28789)

huaxingao force-pushed the spark-28786 branch from 5724edd to caacd58 Compare August 29, 2019 01:17

gatorsmile reviewed Aug 29, 2019

View reviewed changes

change column names to lower case

43de251

gatorsmile closed this in 3e09a0f Aug 29, 2019

dilipbiswal mentioned this pull request Aug 29, 2019

[SPARK-28798][DOC][SQL]Document DROP TABLE/VIEW statement in SQL Reference. #25533

Closed

This was referenced Sep 2, 2019

[SPARK-28810][DOC][SQL] Document SHOW TABLES in SQL Reference. #25561

Closed

[SPARK-28822][DOC][SQL]Document USE DATABASE in SQL Reference. #25572

Closed

This was referenced Sep 12, 2019

[SPARK-28796][DOC]Document DROP DATABASE statement in SQL Reference #25554

Closed

[SPARK-28797][DOC] Document DROP FUNCTION statement in SQL Reference. #25553

Closed

[SPARK-28799][DOC]Documentation for Truncate command #25557

Closed


		### Description

		When the partition value is not provided, such inserts are called as the dynamic partition inserts, also called as multi-partition inserts. In partition spec, the partition column values are optional. When the values are not given, these columns are referred to as dynamic partition columns; otherwise, they are static partition columns. For example, the partition spec (p1 = 3, p2, p3) has a static partition column (p1) and two dynamic partition columns (p2 and p3).


		### Description

		The `INSERT INTO` statement inserts new rows into a table. The inserted rows can be specified by value expressions, or resulted from a query.


		The `INSERT INTO` statement inserts new rows into a table. The inserted rows can be specified by value expressions or result from a query.

		See also:

[SPARK-28786][DOC][SQL]Document INSERT statement in SQL Reference #25525

[SPARK-28786][DOC][SQL]Document INSERT statement in SQL Reference #25525

Uh oh!

Conversation

huaxingao commented Aug 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

SparkQA commented Aug 21, 2019

Uh oh!

SparkQA commented Aug 21, 2019

Uh oh!

huaxingao commented Aug 21, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 21, 2019

Uh oh!

srowen left a comment

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 22, 2019

Uh oh!

dilipbiswal Aug 23, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dilipbiswal Aug 23, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 27, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dilipbiswal commented Aug 27, 2019

Uh oh!

dilipbiswal commented Aug 27, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SparkQA commented Aug 28, 2019

Uh oh!

dilipbiswal commented Aug 28, 2019

Uh oh!

huaxingao commented Aug 29, 2019

Uh oh!

SparkQA commented Aug 29, 2019

Uh oh!

dilipbiswal commented Aug 29, 2019

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Aug 29, 2019

huaxingao commented Aug 21, 2019 •

edited

Loading

dilipbiswal Aug 23, 2019 •

edited

Loading

dilipbiswal Aug 23, 2019 •

edited

Loading

dilipbiswal commented Aug 27, 2019 •

edited

Loading

gatorsmile commented Aug 29, 2019 •

edited

Loading