Skip to content

[SPARK-20889][SparkR] Grouped documentation for MISC column methods #18448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
actuaryzhang wants to merge 4 commits into from
Closed

[SPARK-20889][SparkR] Grouped documentation for MISC column methods #18448

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
00d8bd8
update doc for column misc functions
actuaryzhang Jun 28, 2017
203be11
Merge branch 'master' into sparkRDocMisc
actuaryzhang Jun 29, 2017
ff27f18
address comments
actuaryzhang Jun 29, 2017
35c5c4a
Merge branch 'master' into sparkRDocMisc
actuaryzhang Jun 29, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 45 additions & 53 deletions R/pkg/R/functions.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,27 @@ NULL
#' df <- createDataFrame(cbind(model = rownames(mtcars), mtcars))}
NULL

#' Miscellaneous functions for Column operations
#'
#' Miscellaneous functions defined for \code{Column}.
#'
#' @param x Column to compute on. In \code{sha2}, it is one of 224, 256, 384, or 512.
#' @param y Column to compute on.
Copy link
Copy Markdown
Member

@felixcheung felixcheung Jun 29, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably not only in this PR ... since y always go first, should we flip this order I think? list y first?

Copy link
Copy Markdown
Contributor Author

@actuaryzhang actuaryzhang Jun 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think roxygen automatically chooses the order of the arguments based on the order they appear in the file, and ignores the order we specify. So even if I move y before x here, in the generated doc, x will still appear before y. Indeed, as you can see from the screenshot, ... appears before y.

Copy link
Copy Markdown
Member

@felixcheung felixcheung Jun 29, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm

#' @param ... additional Columns.
#' @name column_misc_functions
#' @rdname column_misc_functions
#' @family misc functions
#' @examples
#' \dontrun{
#' # Dataframe used throughout this doc
#' df <- createDataFrame(cbind(model = rownames(mtcars), mtcars)[, 1:2])
#' tmp <- mutate(df, v1 = crc32(df$model), v2 = hash(df$model),
#' v3 = hash(df$model, df$mpg), v4 = md5(df$model),
#' v5 = sha1(df$model), v6 = sha2(df$model, 256))
#' head(tmp)
#' }
NULL

#' @details
#' \code{lit}: A new Column is created to represent the literal value.
#' If the parameter is a Column, it is returned unchanged.
Expand Down Expand Up @@ -569,19 +590,13 @@ setMethod("count",
column(jc)
})

#' crc32
#'
#' Calculates the cyclic redundancy check value (CRC32) of a binary column and
#' returns the value as a bigint.
#'
#' @param x Column to compute on.
#' @details
#' \code{crc32}: Calculates the cyclic redundancy check value (CRC32) of a binary column
#' and returns the value as a bigint.
#'
#' @rdname crc32
#' @name crc32
#' @family misc functions
#' @aliases crc32,Column-method
#' @rdname column_misc_functions
#' @aliases crc32 crc32,Column-method
#' @export
#' @examples \dontrun{crc32(df$c)}
#' @note crc32 since 1.5.0
setMethod("crc32",
signature(x = "Column"),
Expand All @@ -590,19 +605,13 @@ setMethod("crc32",
column(jc)
})

#' hash
#'
#' Calculates the hash code of given columns, and returns the result as a int column.
#'
#' @param x Column to compute on.
#' @param ... additional Column(s) to be included.
#' @details
#' \code{hash}: Calculates the hash code of given columns, and returns the result
#' as an int column.
#'
#' @rdname hash
#' @name hash
#' @family misc functions
#' @aliases hash,Column-method
#' @rdname column_misc_functions
#' @aliases hash hash,Column-method
#' @export
#' @examples \dontrun{hash(df$c)}
#' @note hash since 2.0.0
setMethod("hash",
signature(x = "Column"),
Expand Down Expand Up @@ -1055,19 +1064,13 @@ setMethod("max",
column(jc)
})

#' md5
#'
#' Calculates the MD5 digest of a binary column and returns the value
#' @details
#' \code{md5}: Calculates the MD5 digest of a binary column and returns the value
#' as a 32 character hex string.
#'
#' @param x Column to compute on.
#'
#' @rdname md5
#' @name md5
#' @family misc functions
#' @aliases md5,Column-method
#' @rdname column_misc_functions
#' @aliases md5 md5,Column-method
#' @export
#' @examples \dontrun{md5(df$c)}
#' @note md5 since 1.5.0
setMethod("md5",
signature(x = "Column"),
Expand Down Expand Up @@ -1307,19 +1310,13 @@ setMethod("second",
column(jc)
})

#' sha1
#'
#' Calculates the SHA-1 digest of a binary column and returns the value
#' @details
#' \code{sha1}: Calculates the SHA-1 digest of a binary column and returns the value
#' as a 40 character hex string.
#'
#' @param x Column to compute on.
#'
#' @rdname sha1
#' @name sha1
#' @family misc functions
#' @aliases sha1,Column-method
#' @rdname column_misc_functions
#' @aliases sha1 sha1,Column-method
#' @export
#' @examples \dontrun{sha1(df$c)}
#' @note sha1 since 1.5.0
setMethod("sha1",
signature(x = "Column"),
Expand Down Expand Up @@ -2309,19 +2306,14 @@ setMethod("format_number", signature(y = "Column", x = "numeric"),
column(jc)
})

#' sha2
#'
#' Calculates the SHA-2 family of hash functions of a binary column and
#' returns the value as a hex string.
#' @details
#' \code{sha2}: Calculates the SHA-2 family of hash functions of a binary column and
#' returns the value as a hex string. The second argument \code{x} specifies the number
#' of bits, and is one of 224, 256, 384, or 512.
#'
#' @param y column to compute SHA-2 on.
#' @param x one of 224, 256, 384, or 512.
#' @family misc functions
#' @rdname sha2
#' @name sha2
#' @aliases sha2,Column,numeric-method
#' @rdname column_misc_functions
#' @aliases sha2 sha2,Column,numeric-method
#' @export
#' @examples \dontrun{sha2(df$c, 256)}
#' @note sha2 since 1.5.0
setMethod("sha2", signature(y = "Column", x = "numeric"),
function(y, x) {
Expand Down
15 changes: 10 additions & 5 deletions R/pkg/R/generics.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -992,8 +992,9 @@ setGeneric("conv", function(x, fromBase, toBase) { standardGeneric("conv") })
#' @name NULL
setGeneric("countDistinct", function(x, ...) { standardGeneric("countDistinct") })

#' @rdname crc32
#' @rdname column_misc_functions
#' @export
#' @name NULL
setGeneric("crc32", function(x) { standardGeneric("crc32") })

#' @rdname column_nonaggregate_functions
Expand All @@ -1006,8 +1007,9 @@ setGeneric("create_array", function(x, ...) { standardGeneric("create_array") })
#' @name NULL
setGeneric("create_map", function(x, ...) { standardGeneric("create_map") })

#' @rdname hash
#' @rdname column_misc_functions
#' @export
#' @name NULL
setGeneric("hash", function(x, ...) { standardGeneric("hash") })

#' @param x empty. Should be used with no argument.
Expand Down Expand Up @@ -1205,8 +1207,9 @@ setGeneric("lpad", function(x, len, pad) { standardGeneric("lpad") })
#' @name NULL
setGeneric("ltrim", function(x) { standardGeneric("ltrim") })

#' @rdname md5
#' @rdname column_misc_functions
#' @export
#' @name NULL
setGeneric("md5", function(x) { standardGeneric("md5") })

#' @rdname column_datetime_functions
Expand Down Expand Up @@ -1350,12 +1353,14 @@ setGeneric("sd", function(x, na.rm = FALSE) { standardGeneric("sd") })
#' @name NULL
setGeneric("second", function(x) { standardGeneric("second") })

#' @rdname sha1
#' @rdname column_misc_functions
#' @export
#' @name NULL
setGeneric("sha1", function(x) { standardGeneric("sha1") })

#' @rdname sha2
#' @rdname column_misc_functions
#' @export
#' @name NULL
setGeneric("sha2", function(y, x) { standardGeneric("sha2") })

#' @rdname column_math_functions
Expand Down