SPARK-1429: Debian packaging#2

Closed

ianlcsd wants to merge 1270 commits intomarkhamstra:masterfrom

ianlcsd:mark-csd-2.2

This pull request is big! We're only showing the most recent 250 commits

Commits on Jun 13, 2017

[SPARK-20920][SQL] ForkJoinPool pools are leaked when writing hive tables with many partitions
srowen
committed
[SPARK-20920][SQL] ForkJoinPool pools are leaked when writing hive tables with many partitions
srowen
committed
[SPARK-21060][WEB-UI] Css style about paging function is error in the executor page. Css style about paging function is error in the executor page. It is different of history server ui paging funct…

guoxiaolong
authored and
srowen
committed
[SPARK-21064][CORE][TEST] Fix the default value bug in NettyBlockTransferServiceSuite

DjvuLee
authored and
srowen
committed
[SPARK-21064][CORE][TEST] Fix the default value bug in NettyBlockTransferServiceSuite

DjvuLee
authored and
srowen
committed
[SPARK-20979][SS] Add RateSource to generate values for tests and benchmark
zsxwing
committed

Commits on Jun 14, 2017

Commits on Jun 15, 2017

Commits on Jun 16, 2017

[SPARK-21111][TEST][2.2] Fix the test failure of describe.sql

gatorsmile
authored and
yhuai
committed
[SPARK-21072][SQL] TreeNode.mapChildren should only apply to the children node.

ConeyLiu
authored and
cloud-fan
committed
[SPARK-21072][SQL] TreeNode.mapChildren should only apply to the children node.

ConeyLiu
authored and
cloud-fan
committed
[SPARK-21114][TEST][2.1] Fix test failure in Spark 2.1/2.0 due to name mismatch

gatorsmile
authored and
cloud-fan
committed
[SPARK-12552][FOLLOWUP] Fix flaky test for "o.a.s.deploy.master.MasterSuite.master correctly recover the application"

jerryshao
authored and
cloud-fan
committed
[MINOR][DOCS] Improve Running R Tests docs

wangyum
authored and
srowen
committed

Commits on Jun 18, 2017

Commits on Jun 19, 2017

[SPARK-21090][CORE] Optimize the unified memory manager code

10110346
authored and
cloud-fan
committed
[SPARK-21132][SQL] DISTINCT modifier of function arguments should not be silently ignored

gatorsmile
authored and
cloud-fan
committed
[SPARK-19688][STREAMING] Not to read `spark.yarn.credentials.file` from checkpoint.

saturday_s
authored and
Marcelo Vanzin
committed
[SPARK-19688][STREAMING] Not to read `spark.yarn.credentials.file` from checkpoint.

saturday_s
authored and
Marcelo Vanzin
committed
[SPARK-21123][DOCS][STRUCTURED STREAMING] Options for file stream source are in a wrong table

assafmendelson
authored and
zsxwing
committed
Merge branch 'branch-2.1' of github.com:apache/spark into csd-2.1
markhamstra
committed
[MINOR][BUILD] Fix Java linter errors

dongjoon-hyun
authored and
srowen
committed
[SPARK-21138][YARN] Cannot delete staging dir when the clusters of "spark.yarn.stagingDir" and "spark.hadoop.fs.defaultFS" are different

sharkdtu
authored and
Marcelo Vanzin
committed
[SPARK-21138][YARN] Cannot delete staging dir when the clusters of "spark.yarn.stagingDir" and "spark.hadoop.fs.defaultFS" are different

sharkdtu
authored and
Marcelo Vanzin
committed

Commits on Jun 20, 2017

Commits on Jun 21, 2017

[MINOR][DOCS] Add lost <tr> tag for configuration.md

wangyum
authored and
srowen
committed

Commits on Jun 22, 2017

[SPARK-18016][SQL][CATALYST][BRANCH-2.1] Code Generation: Constant Pool Limit - Class Splitting

ALeksander Eskilson
authored and
cloud-fan
committed
[SPARK-18016][SQL][CATALYST][BRANCH-2.2] Code Generation: Constant Pool Limit - Class Splitting

ALeksander Eskilson
authored and
cloud-fan
committed
[SPARK-21167][SS] Decode the path generated by File sink to handle special characters
zsxwing
committed
[SPARK-21167][SS] Decode the path generated by File sink to handle special characters
zsxwing
committed
[SQL][DOC] Fix documentation of lpad

actuaryzhang
authored and
srowen
committed

Commits on Jun 23, 2017

Revert "[SPARK-18016][SQL][CATALYST][BRANCH-2.2] Code Generation: Constant Pool Limit - Class Splitting"
cloud-fan
committed
[SPARK-21165] [SQL] [2.2] Use executedPlan instead of analyzedPlan in INSERT AS SELECT [WIP]

gatorsmile
authored and
cloud-fan
committed
Merge branch 'branch-2.1' of github.com:apache/spark into csd-2.1
markhamstra
committed
[SPARK-21144][SQL] Print a warning if the data schema and partition schema have the duplicate columns

maropu
authored and
gatorsmile
committed
[SPARK-21181] Release byteBuffers to suppress netty error messages

dhruve
authored and
Marcelo Vanzin
committed
[SPARK-21181] Release byteBuffers to suppress netty error messages

dhruve
authored and
Marcelo Vanzin
committed
[MINOR][DOCS] Docs in DataFrameNaFunctions.scala use wrong method

ongmingyang
authored and
gatorsmile
committed
[MINOR][DOCS] Docs in DataFrameNaFunctions.scala use wrong method

ongmingyang
authored and
gatorsmile
committed

Commits on Jun 24, 2017

[SPARK-20555][SQL] Fix mapping of Oracle DECIMAL types to Spark types in read path

Gabor Feher
authored and
gatorsmile
committed
[SPARK-20555][SQL] Fix mapping of Oracle DECIMAL types to Spark types in read path

Gabor Feher
authored and
gatorsmile
committed
[SPARK-21159][CORE] Don't try to connect to launcher in standalone cluster mode.

Marcelo Vanzin
authored and
cloud-fan
committed
[SPARK-21159][CORE] Don't try to connect to launcher in standalone cluster mode.

Marcelo Vanzin
authored and
cloud-fan
committed
[SPARK-21203][SQL] Fix wrong results of insertion of Array of Struct

gatorsmile
authored and
cloud-fan
committed
[SPARK-21203][SQL] Fix wrong results of insertion of Array of Struct

gatorsmile
authored and
cloud-fan
committed

Commits on Jun 25, 2017

Commits on Jun 26, 2017

Merge branch 'branch-2.1' of github.com:apache/spark into csd-2.1
markhamstra
committed

Commits on Jun 27, 2017

[SPARK-19104][SQL] Lambda variables in ExternalMapToCatalyst should be global

viirya
authored and
cloud-fan
committed
treating empty string as null for csd

ianlcsd
authored and
markhamstra
committed
TableNamePreprocessor support
markhamstra
committed

Commits on Jun 29, 2017

[SPARK-21210][DOC][ML] Javadoc 8 fixes for ML shared param traits

Nick Pentreath
authored and
srowen
committed

Commits on Jun 30, 2017

Commits on Jul 1, 2017

[SPARK-21170][CORE] Utils.tryWithSafeFinallyAndFailureCallbacks throws IllegalArgumentException: Self-suppression not permitted

Devaraj K
authored and
srowen
committed

Commits on Jul 4, 2017

[SPARK-20256][SQL] SessionState should be created more lazily

dongjoon-hyun
authored and
gatorsmile
committed

Commits on Jul 5, 2017

Commits on Jul 6, 2017

[SPARK-21312][SQL] correct offsetInBytes in UnsafeRow.writeToStream

Sumedh Wale
authored and
cloud-fan
committed
[SPARK-21312][SQL] correct offsetInBytes in UnsafeRow.writeToStream

Sumedh Wale
authored and
cloud-fan
committed
[SS][MINOR] Fix flaky test in DatastreamReaderWriterSuite. temp checkpoint dir should be deleted
tdas
committed

Commits on Jul 7, 2017

Commits on Jul 8, 2017

[SPARK-21069][SS][DOCS] Add rate source to programming guide.

ScrapCodes
authored and
zsxwing
committed
[SPARK-21228][SQL][BRANCH-2.2] InSet incorrect handling of structs

bogdanrdc
authored and
cloud-fan
committed
[SPARK-21345][SQL][TEST][TEST-MAVEN] SparkSessionBuilderSuite should clean up stopped sessions.

dongjoon-hyun
authored and
cloud-fan
committed
[SPARK-20342][CORE] Update task accumulators before sending task end event.

Marcelo Vanzin
authored and
cloud-fan
committed
[SPARK-21343] Refine the document for spark.reducer.maxReqSizeShuffleToMem.

jinxing
authored and
cloud-fan
committed

Commits on Jul 9, 2017

[SPARK-21345][SQL][TEST][TEST-MAVEN][BRANCH-2.1] SparkSessionBuilderSuite should clean up stopped sessions.

dongjoon-hyun
authored and
cloud-fan
committed
[SPARK-21083][SQL][BRANCH-2.2] Store zero size and row count when analyzing empty table

Zhenhua Wang
authored and
cloud-fan
committed

Commits on Jul 10, 2017

[SPARK-21083][SQL][BRANCH-2.1] Store zero size and row count when analyzing empty table

wzhfy
authored and
cloud-fan
committed
[SPARK-21342] Fix DownloadCallback to work well with RetryingBlockFetcher.

jinxing
authored and
cloud-fan
committed
[SPARK-21272] SortMergeJoin LeftAnti does not update numOutputRows

juliuszsompolski
authored and
gatorsmile
committed
foo
markhamstra
committed

Commits on Jul 11, 2017

[SPARK-21369][CORE] Don't use Scala Tuple2 in common/network-*

zsxwing
authored and
cloud-fan
committed
[SPARK-21366][SQL][TEST] Add sql test for window functions

jiangxb1987
authored and
cloud-fan
committed

Commits on Jul 12, 2017

[SPARK-21219][CORE] Task retry occurs on same executor due to race co…

Eric Vandenberg
authored and
cloud-fan
committed

Commits on Jul 13, 2017

Commits on Jul 14, 2017

[SPARK-21376][YARN] Fix yarn client token expire issue when cleaning the staging files in long running scenario

jerryshao
authored and
Marcelo Vanzin
committed

Commits on Jul 15, 2017

Commits on Jul 17, 2017

[SPARK-21321][SPARK CORE] Spark very verbose on shutdown

John Lee
authored and
Tom Graves
committed

Commits on Jul 18, 2017

[SPARK-19104][BACKPORT-2.1][SQL] Lambda variables in ExternalMapToCatalyst should be global

kiszk
authored and
cloud-fan
committed
[SPARK-21332][SQL] Incorrect result type inferred for some decimal expressions

aokolnychyi
authored and
gatorsmile
committed
[SPARK-21332][SQL] Incorrect result type inferred for some decimal expressions

aokolnychyi
authored and
gatorsmile
committed
[SPARK-21445] Make IntWrapper and LongWrapper in UTF8String Serializable

brkyvz
authored and
cloud-fan
committed
[SPARK-18631][SQL] Changed ExchangeCoordinator re-partitioning to avoid more data skew
markhamstra
committed
[SPARK-21457][SQL] ExternalCatalog.listPartitions should correctly handle partition values with dot

cloud-fan
authored and
gatorsmile
committed

Commits on Jul 19, 2017

Commits on Jul 21, 2017

[SPARK-21243][CORE] Limit no. of map outputs in a shuffle fetch

dhruve
authored and
Tom Graves
committed
[SPARK-21434][PYTHON][DOCS] Add pyspark pip documentation.
holdenk
committed

Commits on Jul 23, 2017

[SPARK-20904][CORE] Don't report task failures to driver during shutdown.

Marcelo Vanzin
authored and
cloud-fan
committed

Commits on Jul 25, 2017

Commits on Jul 26, 2017

[SPARK-21494][NETWORK] Use correct app id when authenticating to external service.
Marcelo Vanzin
committed

Commits on Jul 27, 2017

Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed
fix mismerge
markhamstra
committed
[SPARK-21538][SQL] Attribute resolution inconsistency in the Dataset API

aokolnychyi
authored and
gatorsmile
committed

Commits on Jul 28, 2017

[SPARK-21306][ML] OneVsRest should support setWeightCol

facaiy
authored and
yanboliang
committed

Commits on Jul 29, 2017

[SPARK-21508][DOC] Fix example code provided in Spark Streaming Documentation

Remis Haroon
authored and
srowen
committed
[SPARK-21555][SQL] RuntimeReplaceable should be compared semantically by its canonicalized child

viirya
authored and
gatorsmile
committed
[SPARK-19451][SQL] rangeBetween method should accept Long value as boundary

jiangxb1987
authored and
gatorsmile
committed

Commits on Jul 30, 2017

Revert "[SPARK-19451][SQL] rangeBetween method should accept Long value as boundary"
gatorsmile
committed

Commits on Aug 1, 2017

[SPARK-21522][CORE] Fix flakiness in LauncherServerSuite.
Marcelo Vanzin
committed
[SPARK-21593][DOCS] Fix 2 rendering errors on configuration page
srowen
committed
[SPARK-21339][CORE] spark-shell --packages option does not add jars to classpath on windows

Devaraj K
authored and
Marcelo Vanzin
committed
Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed

Commits on Aug 2, 2017

Commits on Aug 3, 2017

Commits on Aug 4, 2017

Commits on Aug 5, 2017

[SPARK-21580][SQL] Integers in aggregation expressions are wrongly taken as group-by ordinal

10110346
authored and
gatorsmile
committed

Commits on Aug 6, 2017

[SPARK-21588][SQL] SQLContext.getConf(key, null) should return null

vinodkc
authored and
gatorsmile
committed

Commits on Aug 7, 2017

[SPARK-21621][CORE] Reset numRecordsWritten after DiskBlockObjectWriter.commitAndGet called

ConeyLiu
authored and
cloud-fan
committed
[SPARK-21647][SQL] Fix SortMergeJoin when using CROSS

gatorsmile
authored and
cloud-fan
committed
[SPARK-21374][CORE] Fix reading globbed paths from S3 into DF with disabled FS cache

Andrey Taptunov
authored and
zsxwing
committed
Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed
[SPARK-21565][SS] Propagate metadata in attribute replacement.

Jose Torres
authored and
zsxwing
committed
[SPARK-21648][SQL] Fix confusing assert failure in JDBC source when parallel fetching parameters are not properly provided.
gatorsmile
committed

Commits on Aug 8, 2017

Commits on Aug 9, 2017

Commits on Aug 11, 2017

Commits on Aug 15, 2017

[SPARK-21721][SQL] Clear FileSystem deleteOnExit cache when paths are successfully removed

viirya
authored and
gatorsmile
committed

Commits on Aug 16, 2017

[SPARK-21723][ML] Fix writing LibSVM (key not found: numFeatures)

Jan Vrsovsky
authored and
srowen
committed
[SPARK-21656][CORE] spark dynamic allocation should not idle timeout executors when tasks still to run

John Lee
authored and
Tom Graves
committed
[SPARK-18464][SQL][BACKPORT] support old table which doesn't store schema in table properties

cloud-fan
authored and
gatorsmile
committed
Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed

Commits on Aug 18, 2017

[SPARK-21739][SQL] Cast expression should initialize timezoneId when it is called statically to convert something into TimestampType

DonnyZone
authored and
gatorsmile
committed

Commits on Aug 20, 2017

Commits on Aug 21, 2017

Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed
[SPARK-21617][SQL] Store correct table metadata when altering schema in Hive metastore.

Marcelo Vanzin
authored and
gatorsmile
committed

Commits on Aug 23, 2017

Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed

Commits on Aug 24, 2017

[SPARK-21805][SPARKR] Disable R vignettes code on Windows

felixcheung
authored and
Felix Cheung
committed
[SPARK-21826][SQL] outer broadcast hash join should not throw NPE

cloud-fan
authored and
hvanhovell
committed
[SPARK-21681][ML] fix bug of MLOR do not work correctly when featureStd contains zero (backport PR for 2.2)

WeichenXu123
authored and
jkbradley
committed

Commits on Aug 25, 2017

Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed

Commits on Aug 28, 2017

[SPARK-21818][ML][MLLIB] Fix bug of MultivariateOnlineSummarizer.variance generate negative result

WeichenXu123
authored and
srowen
committed
[SPARK-21798] No config to replace deprecated SPARK_CLASSPATH config for launching daemons like History Server

pgandhi
authored and
Tom Graves
committed
Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed

Commits on Aug 29, 2017

[SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-uploading remote resources in yarn client mode

jerryshao
authored and
Marcelo Vanzin
committed
Revert "[SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-uploading remote resources in yarn client mode"
Marcelo Vanzin
committed

Commits on Aug 30, 2017

[SPARK-21254][WEBUI] History UI performance fixes

2ooom
authored and
srowen
committed
Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed
[SPARK-21714][CORE][BACKPORT-2.2] Avoiding re-uploading remote resources in yarn client mode

jerryshao
authored and
Marcelo Vanzin
committed
[SPARK-21834] Incorrect executor request in case of dynamic allocation

Sital Kedia
authored and
Marcelo Vanzin
committed

Commits on Aug 31, 2017

Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed

Commits on Sep 1, 2017

[SPARK-21884][SPARK-21477][BACKPORT-2.2][SQL] Mark LocalTableScanExec's input data transient
gatorsmile
committed

Commits on Sep 4, 2017

[SPARK-21418][SQL] NoSuchElementException: None.get in DataSourceScanExec with sun.io.serialization.extendedDebugInfo=true

srowen
authored and
hvanhovell
committed

Commits on Sep 5, 2017

Commits on Sep 6, 2017

[SPARK-21924][DOCS] Update structured streaming programming guide doc

Riccardo Corbella
authored and
srowen
committed
Merge branch 'branch-2.2' of github.com:apache/spark into csd-2.2
markhamstra
committed
[SPARK-21901][SS] Define toString for StateOperatorProgress

jaceklaskowski
authored and
zsxwing
committed

Commits on Sep 7, 2017

Fixed pandoc dependency issue in python/setup.py

Tucker Beck
authored and
HyukjinKwon
committed
[SPARK-21890] Credentials not being passed to add the tokens

Sanket Chintapalli
authored and
Marcelo Vanzin
committed

Commits on Sep 8, 2017

[SPARK-21950][SQL][PYTHON][TEST] pyspark.sql.tests.SQLTests2 should stop SparkContext.
ueshin
committed
[SPARK-21915][ML][PYSPARK] Model 1 and Model 2 ParamMaps Missing

marktab
authored and
srowen
committed
[SPARK-21936][SQL][2.2] backward compatibility test framework for HiveExternalCatalog

cloud-fan
authored and
gatorsmile
committed
[SPARK-21946][TEST] fix flaky test: "alter table: rename cached table" in InMemoryCatalogedDDLSuite

kiszk
authored and
gatorsmile
committed
[SPARK-21128][R][BACKPORT-2.2] Remove both "spark-warehouse" and "metastore_db" before listing files in R tests

HyukjinKwon
authored and
Felix Cheung
committed

Commits on Sep 9, 2017

[SPARK-21954][SQL] JacksonUtils should verify MapType's value type instead of key type

viirya
authored and
HyukjinKwon
committed

Commits on Sep 10, 2017

[SPARK-20098][PYSPARK] dataType's typeName fix

szalai1
authored and
HyukjinKwon
committed

Commits on Sep 12, 2017

Commits on Sep 13, 2017

Commits on Sep 14, 2017

[SPARK-18608][ML][FOLLOWUP] Fix double caching for PySpark OneVsRest.
yanboliang
committed

Commits on Sep 17, 2017

[SPARK-21985][PYSPARK] PairDeserializer is broken for double-zipped RDDs

aray
authored and
HyukjinKwon
committed

Commits on Sep 18, 2017

Commits on Sep 19, 2017

Commits on Sep 20, 2017

[SPARK-22076][SQL] Expand.projections should not be a Stream

cloud-fan
authored and
gatorsmile
committed
[SPARK-21384][YARN] Spark + YARN fails with LocalFileSystem as default FS

Devaraj K
authored and
Marcelo Vanzin
committed

Commits on Sep 21, 2017

[SPARK-21928][CORE] Set classloader on SerializerManager's private kryo

squito
authored and
Marcelo Vanzin
committed

Commits on Sep 22, 2017

Commits on Sep 23, 2017

Commits on Sep 25, 2017

Commits on Sep 27, 2017

Commits on Sep 28, 2017

SPY-1429
ianlcsd
committed