Resync from upstream master#307

Merged

dansanduleac merged 60 commits intomasterfrom

ds/sync-upstream

Feb 6, 2018

Commits on Jan 30, 2018

[SPARK-23260][SPARK-23262][SQL] several data source v2 naming cleanup
cloud-fan
committed
[SPARK-23261][PYSPARK] Rename Pandas UDFs

gatorsmile
authored and
HyukjinKwon
committed
[SPARK-23222][SQL] Make DataFrameRangeSuite not flaky

viirya
authored and
cloud-fan
committed
[SPARK-23174][BUILD][PYTHON][FOLLOWUP] Add pycodestyle*.py to .gitignore file.

ueshin
authored and
HyukjinKwon
committed
[SPARK-23267][SQL] Increase spark.sql.codegen.hugeMethodLimit to 65535
gatorsmile
committed
[SPARK-23275][SQL] hive/tests have been failing when run locally on the laptop (Mac) with OOM

dilipbiswal
authored and
gatorsmile
committed
[MINOR] Fix typos in dev/* scripts.

ashashwat
authored and
HyukjinKwon
committed

Commits on Jan 31, 2018

Commits on Feb 1, 2018

[SPARK-23157][SQL][FOLLOW-UP] DataFrame -> SparkDataFrame in R comment

Henry Robinson
authored and
HyukjinKwon
committed
[SPARK-23280][SQL] add map type support to ColumnVector
cloud-fan
committed
[SPARK-23247][SQL] combines Unsafe operations and statistics operations in Scan Data Source

heary-cao
authored and
cloud-fan
committed
[SPARK-23268][SQL] Reorganize packages in data source V2

gengliangwang
authored and
gatorsmile
committed
[SPARK-21396][SQL] Fixes MatchError when UDTs are passed through Hive Thriftserver

atallahhezbor
authored and
gatorsmile
committed
[SPARK-23188][SQL] Make vectorized columar reader batch size configurable

jiangxb1987
authored and
cloud-fan
committed
[SQL][MINOR] Inline SpecifiedWindowFrame.defaultWindowFrame().

jiangxb1987
authored and
gatorsmile
committed
Revert "[SPARK-23200] Reset Kubernetes-specific config on Checkpoint restore"
jerryshao
committed
[SPARK-22274][PYTHON][SQL][FOLLOWUP] Use `assertRaisesRegexp` instead of `assertRaisesRegex`.

ueshin
authored and
gatorsmile
committed
[SPARK-23107][ML] ML 2.3 QA: New Scala APIs, docs.

yanboliang
authored and
Nick Pentreath
committed
[SPARK-23280][SQL][FOLLOWUP] Fix Java style check issues.
ueshin
committed
[SPARK-23280][SQL][FOLLOWUP] Enable `MutableColumnarRow.getMap()`.
ueshin
committed
[SPARK-23202][SQL] Add new API in DataSourceWriter: onDataWriterCommit

gengliangwang
authored and
cloud-fan
committed
[SPARK-23289][CORE] OneForOneBlockFetcher.DownloadCallback.onData should write the buffer fully

zsxwing
authored and
cloud-fan
committed
[SPARK-13983][SQL] Fix HiveThriftServer2 can not get "--hiveconf" and ''--hivevar" variables since 2.0

wangyum
authored and
gatorsmile
committed
[SPARK-23293][SQL] fix data source v2 self join

cloud-fan
authored and
gatorsmile
committed
[INFRA] Close stale PRs.
Marcelo Vanzin
committed
[SPARK-23296][YARN] Include stacktrace in YARN-app diagnostic

gerashegalov
authored and
Marcelo Vanzin
committed

Commits on Feb 2, 2018

[SPARK-23284][SQL] Document the behavior of several ColumnVector's get APIs when accessing null slot

viirya
authored and
cloud-fan
committed
[SPARK-23020][CORE] Fix another race in the in-process launcher test.

Marcelo Vanzin
authored and
cloud-fan
committed
[SPARK-23306] Fix the oom caused by contention

Zhan Zhang
authored and
cloud-fan
committed
[SPARK-23301][SQL] data source column pruning should work for arbitrary expressions

cloud-fan
authored and
gatorsmile
committed
[SPARK-23312][SQL] add a config to turn off vectorized cache reader
cloud-fan
committed
[SPARK-23253][CORE][SHUFFLE] Only write shuffle temporary index file when there is not an existing one

yaooqinn
authored and
squito
committed
[SPARK-23295][BUILD][MINOR] Exclude Waring message when generating versions in make-distribution.sh

yaooqinn
authored and
srowen
committed

Commits on Feb 3, 2018

Commits on Feb 4, 2018

Commits on Feb 5, 2018

[SPARK-23307][WEBUI] Sort jobs/stages/tasks/queries with the completed timestamp before cleaning up them

zsxwing
authored and
cloud-fan
committed
[SPARK-23310][CORE] Turn off read ahead input stream for unshafe shuffle reader

Sital Kedia
authored and
sameeragarwal
committed
[SPARK-23330][WEBUI] Spark UI SQL executions page throws NPE

jiangxb1987
authored and
Marcelo Vanzin
committed

Commits on Feb 6, 2018