Skip to content

[SPARK-23619][DOCS] Add output description for some generator expressions / functions #23748

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jashgala wants to merge 8 commits into from
Closed

[SPARK-23619][DOCS] Add output description for some generator expressions / functions #23748

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
82e217b
[SPARK-23619][SQL] Added default column names for explode & posexplod…
jashgala Feb 8, 2019
e7a34eb
Added more comments for explode_outer and posexplode_outer
jashgala Feb 9, 2019
4ed035b
Added default column names in ExpressionDescription for explode and p…
jashgala Feb 9, 2019
e5751ed
Added default column names for inline and stack in comments
jashgala Feb 22, 2019
80c0649
Added scala style line-size exception to prevent failure due to comment
jashgala Feb 22, 2019
d029d3c
Fixed typo in scala style exceptions in comments
jashgala Feb 22, 2019
0313b59
Added default column names in comments for explode, etc. functions in R
jashgala Feb 22, 2019
d0dea9c
Added default column names in comments for explode, etc. functions in…
jashgala Feb 22, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/generators.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -352,7 +352,7 @@ abstract class ExplodeBase extends UnaryExpression with CollectionGenerator with
*/
// scalastyle:off line.size.limit
@ExpressionDescription(
usage = "_FUNC_(expr) - Separates the elements of array `expr` into multiple rows, or the elements of map `expr` into multiple rows and columns.",
usage = "_FUNC_(expr) - Separates the elements of array `expr` into multiple rows, or the elements of map `expr` into multiple rows and columns. Unless specified otherwise, uses the default column name `col` for elements of the array or `key` and `value` for the elements of the map.",
examples = """
Examples:
> SELECT _FUNC_(array(10, 20));
Expand All @@ -375,7 +375,7 @@ case class Explode(child: Expression) extends ExplodeBase {
*/
// scalastyle:off line.size.limit
@ExpressionDescription(
usage = "_FUNC_(expr) - Separates the elements of array `expr` into multiple rows with positions, or the elements of map `expr` into multiple rows and columns with positions.",
usage = "_FUNC_(expr) - Separates the elements of array `expr` into multiple rows with positions, or the elements of map `expr` into multiple rows and columns with positions. Unless specified otherwise, uses the column name `pos` for position, `col` for elements of the array or `key` and `value` for elements of the map.",
examples = """
Examples:
> SELECT _FUNC_(array(10,20));
Expand Down
8 changes: 8 additions & 0 deletions sql/core/src/main/scala/org/apache/spark/sql/functions.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3305,6 +3305,8 @@ object functions {

/**
* Creates a new row for each element in the given array or map column.
* Uses the default column name `col` for elements in the array and
* `key` and `value` for elements in the map unless specified otherwise.

@HyukjinKwon HyukjinKwon Feb 11, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also update Python doc and R doc while we're here. Take a look for functions.py and functions.R

@maropu maropu Feb 12, 2019

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh... I forgot the viewpoint...thanks.

*
* @group collection_funcs
* @since 1.3.0
Expand All @@ -3313,6 +3315,8 @@ object functions {

/**
* Creates a new row for each element in the given array or map column.
* Uses the default column name `col` for elements in the array and
* `key` and `value` for elements in the map unless specified otherwise.
* Unlike explode, if the array/map is null or empty then null is produced.
*
* @group collection_funcs
Expand All @@ -3322,6 +3326,8 @@ object functions {

/**
* Creates a new row for each element with position in the given array or map column.
* Uses the default column name `pos` for position, and `col` for elements in the array
* and `key` and `value` for elements in the map unless specified otherwise.
*
* @group collection_funcs
* @since 2.1.0
Expand All @@ -3330,6 +3336,8 @@ object functions {

/**
* Creates a new row for each element with position in the given array or map column.
* Uses the default column name `pos` for position, and `col` for elements in the array
* and `key` and `value` for elements in the map unless specified otherwise.
* Unlike posexplode, if the array/map is null or empty then the row (null, null) is produced.
*
* @group collection_funcs
Expand Down