Patch/1.2.1 palantir2#3

Closed

mingyukim wants to merge 540 commits intomasterfrom

patch/1.2.1-palantir2

This pull request is big! We're only showing the most recent 250 commits

Commits on Nov 28, 2014

HOTFIX: Rolling back incorrect version change
pwendell
committed

Commits on Nov 29, 2014

[SPARK-4597] Use proper exception and reset variable in Utils.createTempDir()

viirya
authored and
JoshRosen
committed

Commits on Nov 30, 2014

[DOCS][BUILD] Add instruction to use change-version-to-2.11.sh in 'Building for Scala 2.11'.

ueshin
authored and
pwendell
committed
SPARK-2143 [WEB UI] Add Spark version to UI footer

srowen
authored and
JoshRosen
committed

Commits on Dec 1, 2014

[SPARK-4656][Doc] Typo in Programming Guide markdown

Lewuathe
authored and
JoshRosen
committed
[DOC] Fixes formatting typo in SQL programming guide

liancheng
authored and
JoshRosen
committed
SPARK-2192 [BUILD] Examples Data Not in Binary Distribution

srowen
authored and
mengxr
committed
[SPARK-4661][Core] Minor code and docs cleanup

zsxwing
authored and
rxin
committed
Documentation: add description for repartitionAndSortWithinPartitions

msiddalingaiah
authored and
JoshRosen
committed
[SPARK-4258][SQL][DOC] Documents spark.sql.parquet.filterPushdown

liancheng
authored and
marmbrus
committed
[SQL] add @group tab in limit() and count()

Jacky Li
authored and
marmbrus
committed
[SPARK-4358][SQL] Let BigDecimal do checking type compatibility

viirya
authored and
marmbrus
committed
[SPARK-4650][SQL] Supporting multi column support in countDistinct function like count(distinct c1,c2..) in Spark SQL

ravipesala
authored and
marmbrus
committed
[SPARK-4658][SQL] Code documentation issue in DDL of datasource API

ravipesala
authored and
marmbrus
committed
[SQL] Minor fix for doc and comment

scwf
authored and
marmbrus
committed
[SQL][DOC] Date type in SQL programming guide

adrian-wang
authored and
marmbrus
committed

Commits on Dec 2, 2014

[SPARK-4529] [SQL] support view with column alias

adrian-wang
authored and
marmbrus
committed
[SPARK-4611][MLlib] Implement the efficient vector norm

DB Tsai
authored and
mengxr
committed
[SPARK-4686] Link to allowed master URLs is broken
kayousterhout
committed
[SPARK-4536][SQL] Add sqrt and abs to Spark SQL DSL

sarutak
authored and
marmbrus
committed
[SPARK-4663][sql]add finally to avoid resource leak

baishuo
authored and
marmbrus
committed
[SPARK-4676][SQL] JavaSchemaRDD.schema may throw NullType MatchError if sql has null

YanTangZhai
authored and
marmbrus
committed
[SPARK-4593][SQL] Return null when denominator is 0

adrian-wang
authored and
marmbrus
committed
[SPARK-4670] [SQL] wrong symbol for bitwise not

adrian-wang
authored and
marmbrus
committed
[SPARK-4695][SQL] Get result using executeCollect

scwf
authored and
marmbrus
committed

Commits on Dec 3, 2014

[Release] Translate unknown author names automatically
Andrew Or
committed
[SPARK-4672][GraphX]Perform checkpoint() on PartitionsRDD to shorten the lineage

JerryLead
authored and
ankurdave
committed
[SPARK-4672][GraphX]Non-transient PartitionsRDDs will lead to StackOverflow error

JerryLead
authored and
ankurdave
committed
[SPARK-4672][Core]Checkpoint() should clear f to shorten the serialization chain

JerryLead
authored and
ankurdave
committed
[SPARK-4710] [mllib] Eliminate MLlib compilation warnings

jkbradley
authored and
mengxr
committed
[SPARK-4708][MLLib] Make k-mean runs two/three times faster with dense/sparse sample

DB Tsai
authored and
mengxr
committed
[SPARK-4717][MLlib] Optimize BLAS library to avoid de-reference multiple times in loop

DB Tsai
authored and
mengxr
committed
SPARK-2624 add datanucleus jars to the container in yarn-cluster

Jim Lim
authored and
Andrew Or
committed
[SPARK-4701] Typo in sbt/sbt

tsudukim
authored and
Andrew Or
committed
[SPARK-4715][Core] Make sure tryToAcquire won't return a negative value

zsxwing
authored and
Andrew Or
committed
[SPARK-4642] Add description about spark.yarn.queue to running-on-YARN document.

tsudukim
authored and
Andrew Or
committed
[HOT FIX] [YARN] Check whether `/lib` exists before listing its files
Andrew Or
committed
[SPARK-4552][SQL] Avoid exception when reading empty parquet data through Hive
marmbrus
committed
[SPARK-4498][core] Don't transition ExecutorInfo to RUNNING until Driver adds Executor

markhamstra
authored and
JoshRosen
committed

Commits on Dec 4, 2014

[SPARK-4085] Propagate FetchFailedException when Spark fails to read local shuffle file.

rxin
authored and
pwendell
committed
[SPARK-4711] [mllib] [docs] Programming guide advice on choosing optimizer

jkbradley
authored and
mengxr
committed
[SPARK-4580] [SPARK-4610] [mllib] [docs] Documentation for tree ensembles + DecisionTree API fix

jkbradley
authored and
mengxr
committed
[Release] Correctly translate contributors name in release notes
Andrew Or
committed
[SPARK-4685] Include all spark.ml and spark.mllib packages in JavaDoc's MLlib group

Lewuathe
authored and
mengxr
committed
[SQL] Minor: Avoid calling Seq#size in a loop

aarondav
authored and
rxin
committed
[docs] Fix outdated comment in tuning guide

jkbradley
authored and
rxin
committed
[SPARK-4575] [mllib] [docs] spark.ml pipelines doc + bug fixes

jkbradley
authored and
mengxr
committed
[FIX][DOC] Fix broken links in ml-guide.md
mengxr
committed
[SPARK-4683][SQL] Add a beeline.cmd to run on Windows

liancheng
authored and
pwendell
committed
Revert "HOTFIX: Rolling back incorrect version change"
pwendell
committed
Revert "Preparing development version 1.2.1-SNAPSHOT"
pwendell
committed
Revert "Preparing Spark release v1.2.0-rc1"
pwendell
committed
[SPARK-4253] Ignore spark.driver.host in yarn-cluster and standalone-cluster modes

WangTaoTheTonic
authored and
JoshRosen
committed
[HOTFIX] Fixing two issues with the release script.
pwendell
committed
Preparing Spark release v1.2.0-rc2
pwendell
committed
Preparing development version 1.2.1-SNAPSHOT
pwendell
committed
[SPARK-4745] Fix get_existing_cluster() function with multiple security groups

alexdebrie
authored and
JoshRosen
committed
[SPARK-4459] Change groupBy type parameter from K to U

Saldanha
authored and
JoshRosen
committed
[SPARK-4652][DOCS] Add docs about spark-git-repo option

Lewuathe
authored and
JoshRosen
committed

Commits on Dec 5, 2014

[SPARK-4421] Wrong link in spark-standalone.html

tsudukim
authored and
JoshRosen
committed
Fix typo in Spark SQL docs.

andyk
authored and
JoshRosen
committed
[SPARK-4464] Description about configuration options need to be modified in docs.

tsudukim
authored and
JoshRosen
committed
Revert "[HOT FIX] [YARN] Check whether `/lib` exists before listing its files"
Andrew Or
committed
Revert "SPARK-2624 add datanucleus jars to the container in yarn-cluster"
Andrew Or
committed
[SPARK-4753][SQL] Use catalyst for partition pruning in newParquet.

marmbrus
authored and
pwendell
committed
[SPARK-4761][SQL] Enables Kryo by default in Spark SQL Thrift server

liancheng
authored and
pwendell
committed
Streaming doc : do you mean inadvertently?

CrazyJvm
authored and
rxin
committed

Commits on Dec 6, 2014

[SPARK-3623][GraphX] GraphX should support the checkpoint operation

witgo
authored and
ankurdave
committed

Commits on Dec 8, 2014

[SPARK-4646] Replace Scala.util.Sorting.quickSort with Sorter(TimSort) in Spark

maropu
authored and
ankurdave
committed
[SPARK-4620] Add unpersist in Graph and GraphImpl

maropu
authored and
ankurdave
committed
[SPARK-4774] [SQL] Makes HiveFromSpark more portable

Kostas Sakellis
authored and
marmbrus
committed

Commits on Dec 10, 2014

Commits on Dec 11, 2014

[SPARK-4806] Streaming doc update for 1.2
tdas
committed

Commits on Dec 12, 2014

[SPARK-4825] [SQL] CTAS fails to resolve when created using saveAsTable

chenghao-intel
authored and
marmbrus
committed

Commits on Dec 14, 2014

fixed spelling errors in documentation

peterklipfel
authored and
JoshRosen
committed

Commits on Dec 15, 2014

Commits on Dec 16, 2014

Commits on Dec 17, 2014

[Release] Major improvements to generate contributors script
Andrew Or
committed
[Release] Cache known author translations locally
Andrew Or
committed
[Release] Update contributors list format and sort it
Andrew Or
committed
[HOTFIX] Fix RAT exclusion for known_translations file
JoshRosen
committed
[SPARK-4595][Core] Fix MetricsServlet not work issue

jerryshao
authored and
JoshRosen
committed
[SPARK-4764] Ensure that files are fetched atomically

Christophe Préaud
authored and
JoshRosen
committed
[SPARK-4750] Dynamic allocation - synchronize kills

Andrew Or
authored and
JoshRosen
committed
SPARK-3926 [CORE] Reopened: result of JavaRDD collectAsMap() is not serializable

srowen
authored and
JoshRosen
committed
[SPARK-4691][shuffle] Restructure a few lines in shuffle code

maji2014
authored and
JoshRosen
committed
[SPARK-4714] BlockManager.dropFromMemory() should check whether block has been removed after synchronizing on BlockInfo instance.

suyanNone
authored and
JoshRosen
committed
[SPARK-4772] Clear local copies of accumulators as soon as we're done with them

Nathan Kronenfeld
authored and
JoshRosen
committed
SPARK-785 [CORE] ClosureCleaner not invoked on most PairRDDFunctions

srowen
authored and
JoshRosen
committed
[SPARK-4841] fix zip with textFile()

Davies Liu
authored and
JoshRosen
committed
[SPARK-4821] [mllib] [python] [docs] Fix for pyspark.mllib.rand doc

jkbradley
authored and
mengxr
committed

Commits on Dec 18, 2014

Add mesos specific configurations into doc

tnachen
authored and
rxin
committed
HOTFIX: Changing doc color
pwendell
committed
[SPARK-4880] remove spark.locality.wait in Analytics

Earne
authored and
rxin
committed

Commits on Dec 19, 2014

[SPARK-4884]: Improve Partition docs

msiddalingaiah
authored and
JoshRosen
committed
[SPARK-4837] NettyBlockTransferService should use spark.blockManager.port config

aarondav
authored and
JoshRosen
committed
[SPARK-4754] Refactor SparkContext into ExecutorAllocationClient
Andrew Or
committed
SPARK-3428. TaskMetrics for running tasks is missing GC time metrics

sryza
authored and
JoshRosen
committed
[SPARK-4889] update history server example cmds

ryan-williams
authored and
Andrew Or
committed
[SPARK-4896] don’t redundantly overwrite executor JAR deps

ryan-williams
authored and
JoshRosen
committed

Commits on Dec 20, 2014

change signature of example to match released code

eranation
authored and
Andrew Or
committed
SPARK-2641: Passing num executors to spark arguments from properties file

Kanwaljit Singh
authored and
Andrew Or
committed
[SPARK-4140] Document dynamic allocation
Andrew Or
committed
[Minor] Build Failed: value defaultProperties not found

SaintBacchus
authored and
JoshRosen
committed

Commits on Dec 22, 2014

[SPARK-2075][Core] Make the compiler generate same bytes code for Hadoop 1.+ and Hadoop 2.+

zsxwing
authored and
rxin
committed
[SPARK-2075][Core] backport for branch-1.2

zsxwing
authored and
rxin
committed
[SPARK-4915][YARN] Fix classname to be specified for external shuffle service.

Tsuyoshi Ozawa
authored and
Andrew Or
committed
[SPARK-4883][Shuffle] Add a name to the directoryCleaner thread

zsxwing
authored and
Andrew Or
committed
[Minor] Improve some code in BroadcastTest for short

SaintBacchus
authored and
Andrew Or
committed
[SPARK-4864] Add documentation to Netty-based configs

aarondav
authored and
pwendell
committed
[SPARK-4920][UI]:current spark version in UI is not striking.

uncleGen
authored and
Andrew Or
committed
[SPARK-4818][Core] Add 'iterator' to reduce memory consumed by join

zsxwing
authored and
JoshRosen
committed

Commits on Dec 23, 2014

[Docs] Minor typo fixes

nchammas
authored and
pwendell
committed
[SPARK-4931][Yarn][Docs] Fix the format of running-on-yarn.md

zsxwing
authored and
JoshRosen
committed
[SPARK-4834] [standalone] Clean up application files after app finishes.

Marcelo Vanzin
authored and
JoshRosen
committed
[SPARK-4932] Add help comments in Analytics

maropu
authored and
JoshRosen
committed
[SPARK-4914][Build] Cleans lib_managed before compiling with Hive 0.13.1

liancheng
authored and
JoshRosen
committed
[SPARK-4730][YARN] Warn against deprecated YARN settings

Andrew Or
authored and
JoshRosen
committed
[SPARK-4802] [streaming] Remove receiverInfo once receiver is de-registered

ilayaperumalg
authored and
tdas
committed
[SPARK-4671][Streaming]Do not replicate streaming block when WAL is enabled

jerryshao
authored and
tdas
committed

Commits on Dec 24, 2014

[SPARK-4606] Send EOF to child JVM when there's no more data to read.

Marcelo Vanzin
authored and
JoshRosen
committed

Commits on Dec 25, 2014

[SPARK-4873][Streaming] Use `Future.zip` instead of `Future.flatMap`(for-loop) in WriteAheadLogBasedBlockHandler

zsxwing
authored and
tdas
committed
Fix "Building Spark With Maven" link in README.md

dennyglee
authored and
JoshRosen
committed

Commits on Dec 26, 2014

[SPARK-4537][Streaming] Expand StreamingSource to add more metrics

jerryshao
authored and
tdas
committed
SPARK-4971: Fix typo in BlockGenerator comment

CodingCat
authored and
JoshRosen
committed

Commits on Dec 27, 2014

Commits on Dec 29, 2014

[SPARK-4966][YARN]The MemoryOverhead value is setted not correctly

XuTingjun
authored and
tgravescs
committed
[SPARK-4982][DOC] `spark.ui.retainedJobs` description is wrong in Spark UI configuration guide

wangxiaojing
authored and
JoshRosen
committed
SPARK-4968: takeOrdered to skip reduce step in case mappers return no partitions

Yash Datta
authored and
rxin
committed

Commits on Dec 30, 2014

Commits on Jan 1, 2015

Commits on Jan 2, 2015

Fixed typos in streaming-kafka-integration.md

Akhil Das
authored and
tdas
committed

Commits on Jan 4, 2015

[SPARK-5058] Updated broken links

sigmoidanalytics
authored and
tdas
committed
[SPARK-4787] Stop SparkContext if a DAGScheduler init error occurs

tigerquoll
authored and
JoshRosen
committed

Commits on Jan 5, 2015

[SPARK-4631] unit test for MQTT

bilna
authored and
tdas
committed
[SPARK-4835] Disable validateOutputSpecs for Spark Streaming jobs

JoshRosen
authored and
tdas
committed
[SPARK-4465] runAsSparkUser doesn't affect TaskRunner in Mesos environme...

jongyoul
authored and
JoshRosen
committed
[SPARK-5089][PYSPARK][MLLIB] Fix vector convert

freeman-lab
authored and
mengxr
committed

Commits on Jan 6, 2015

[HOTFIX] Add missing SparkContext._ import to fix 1.2 build.
JoshRosen
committed

Commits on Jan 7, 2015

[YARN][SPARK-4929] Bug fix: fix the yarn-client code to support HA

SaintBacchus
authored and
tgravescs
committed
[SPARK-5132][Core]Correct stage Attempt Id key in stageInfofromJson

suyanNone
authored and
JoshRosen
committed

Commits on Jan 8, 2015

[SPARK-5130][Deploy]Take yarn-cluster as cluster mode in spark-submit

WangTaoTheTonic
authored and
Andrew Or
committed
[SPARK-4973][CORE] Local directory in the driver of client-mode continues remaining even if application finished when external shuffle is enabled

sarutak
authored and
Andrew Or
committed

Commits on Jan 9, 2015

SPARK-5136 [DOCS] Improve documentation around setting up Spark IntelliJ project

srowen
authored and
pwendell
committed

Commits on Jan 10, 2015

Commits on Jan 13, 2015

[SPARK-5131][Streaming][DOC]: There is a discrepancy in WAL implementation and configuration doc.

uncleGen
authored and
Andrew Or
committed
[SPARK-5223] [MLlib] [PySpark] fix MapConverter and ListConverter in MLlib

Davies Liu
authored and
mengxr
committed

Commits on Jan 14, 2015

[SPARK-5234][ml]examples for ml don't have sparkContext.stop

Yuhao Yang
authored and
mengxr
committed

Commits on Jan 15, 2015

Commits on Jan 16, 2015

[SPARK-4033][Examples]Input of the SparkPi too big causes the emption exception

SaintBacchus
authored and
Andrew Or
committed
[SPARK-5201][CORE] deal with int overflow in the ParallelCollectionRDD.slice method

advancedxy
authored and
Andrew Or
committed
[DOCS] Fix typo in return type of cogroup

srowen
authored and
Andrew Or
committed
[SPARK-733] Add documentation on use of accumulators in lazy transformation

Ilya Ganelin
authored and
squito
committed

Commits on Jan 18, 2015

[SPARK-5289]: Backport publishing of repl, yarn into branch-1.2.
pwendell
committed

Commits on Jan 19, 2015

[SPARK-5282][mllib]: RowMatrix easily gets int overflow in the memory size warning

hhbyyh
authored and
mengxr
committed
[SPARK-4504][Examples] fix run-example failure if multiple assembly jars exist

gvramana
authored and
JoshRosen
committed

Commits on Jan 20, 2015

[SPARK-4803] [streaming] Remove duplicate RegisterReceiver message

ilayaperumalg
authored and
tdas
committed
SPARK-4660: Use correct class loader in JavaSerializer (copy of PR #3840...

jacek-lewandowski
authored and
pwendell
committed

Commits on Jan 21, 2015

[SPARK-4959][SQL] Attributes are case sensitive when using a select query from a projection(Backport to Spark-1.2)

chenghao-intel
authored and
rxin
committed
[SPARK-5275] [Streaming] include python source code

Davies Liu
authored and
pwendell
committed
[HOTFIX] Update pom.xml to pull MapR's Hadoop version 2.4.1.

rkannan82
authored and
pwendell
committed
[SPARK-4759] Fix driver hanging from coalescing partitions
Andrew Or
committed
[SPARK-4569] Rename 'externalSorting' in Aggregator

Ilya Ganelin
authored and
Andrew Or
committed
[SPARK-4161]Spark shell class path is not correctly set if "spark.driver.extraClassPath" is set in defaults.conf

witgo
authored and
Andrew Or
committed
[SPARK-5064][GraphX] Add numEdges upperbound validation for R-MAT graph generator to prevent infinite loop

Kenji Kikushima
authored and
ankurdave
committed
[SPARK-5006][Deploy]spark.port.maxRetries doesn't work

WangTaoTheTonic
authored and
Andrew Or
committed
Make sure only owner can read / write to directories created for the job.

Marcelo Vanzin
authored and
JoshRosen
committed

Commits on Jan 22, 2015

Commits on Jan 23, 2015

[SPARK-5233][Streaming] Fix error replaying of WAL introduced bug

jerryshao
authored and
tdas
committed

Commits on Jan 24, 2015

Commits on Jan 25, 2015

[SPARK-5401] set executor ID before creating MetricsSystem

ryan-williams
authored and
Andrew Or
committed
[SPARK-5402] log executor ID at executor-construction time

ryan-williams
authored and
Andrew Or
committed
SPARK-3852 [DOCS] Document spark.driver.extra* configs

srowen
authored and
Andrew Or
committed
SPARK-5382: Use SPARK_CONF_DIR in spark-class if it is defined

jacek-lewandowski
authored and
Andrew Or
committed
SPARK-5382: Use SPARK_CONF_DIR in spark-class and spark-submit, spark-su...

jacek-lewandowski
authored and
Andrew Or
committed
SPARK-4506 [DOCS] Addendum: Update more docs to reflect that standalone works in cluster mode

srowen
authored and
Andrew Or
committed
[SPARK-5344][WebUI] HistoryServer cannot recognize that inprogress file was renamed to completed file

sarutak
authored and
Andrew Or
committed
Revert "[SPARK-5344][WebUI] HistoryServer cannot recognize that inprogress file was renamed to completed file"
Andrew Or
committed

Commits on Jan 26, 2015

SPARK-4430 [STREAMING] [TEST] Apache RAT Checks fail spuriously on test files

srowen
authored and
Andrew Or
committed
[SPARK-5355] use j.u.c.ConcurrentHashMap instead of TrieMap

Davies Liu
authored and
JoshRosen
committed
SPARK-4147 [CORE] Reduce log4j dependency

srowen
authored and
pwendell
committed

Commits on Jan 27, 2015

Updating versions for Spark 1.2.1
pwendell
committed
Preparing Spark release v1.2.1-rc1
Ubuntu
committed
Preparing development version 1.2.2-SNAPSHOT
Ubuntu
committed
Revert "Preparing development version 1.2.2-SNAPSHOT"
pwendell
committed
Revert "Preparing Spark release v1.2.1-rc1"
pwendell
committed
Preparing Spark release v1.2.1-rc1
pwendell
committed
Preparing development version 1.2.2-SNAPSHOT
pwendell
committed
SPARK-5308 [BUILD] MD5 / SHA1 hash format doesn't match standard Maven output

srowen
authored and
pwendell
committed
[MLlib] fix python example of ALS in guide

Davies Liu
authored and
mengxr
committed

Commits on Feb 2, 2015

[Docs] Fix Building Spark link text

nchammas
authored and
Andrew Or
committed
Disabling Utils.chmod700 for Windows

MartinWeindel
authored and
JoshRosen
committed

Commits on Mar 6, 2015

[SPARK-5691] Fixing wrong data structure lookup for dupe app registratio...

mccheah
authored and
mingyukim
committed
[SPARK-4808] Removing minimum number of elements read before spill check

mccheah
authored and
mingyukim
committed

Commits on Mar 9, 2015

Preparing Palantir Spark release v1.2.1-palantir1
mingyukim
committed

Commits on Mar 24, 2015

Preparing Palantir Spark release v1.2.1-palantir2
mingyukim
committed