External Group By to handle huge keys#18

Closed

mccheah wants to merge 342 commits intofeature/external-group-by-basefrom

feature/external-group-by-wip

This pull request is big! We're only showing the most recent 250 commits

Commits on Aug 3, 2015

[SPARK-9404][SPARK-9542][SQL] unsafe array data and map data

cloud-fan
authored and
rxin
committed
[SPARK-9549][SQL] fix bugs in expressions

yjshen
authored and
rxin
committed
[SPARK-9240] [SQL] Hybrid aggregate operator using unsafe row

yhuai
authored and
rxin
committed
[SPARK-8873] [MESOS] Clean up shuffle files if external shuffle service is used

tnachen
authored and
Andrew Or
committed
[SPARK-9551][SQL] add a cheap version of copy for UnsafeRow to reuse a copy buffer

cloud-fan
authored and
rxin
committed
[SPARK-9518] [SQL] cleanup generated UnsafeRowJoiner and fix bug

Davies Liu
authored and
rxin
committed
Two minor comments from code review on 191bf2689.
rxin
committed
[SPARK-9528] [ML] Changed RandomForestClassifier to extend ProbabilisticClassifier
jkbradley
committed
[SPARK-1855] Local checkpointing

Andrew Or
authored and
tdas
committed
[SPARK-9511] [SQL] Fixed Table Name Parsing

Joseph Batchik
authored and
marmbrus
committed
[SQL][minor] Simplify UnsafeRow.calculateBitSetWidthInBytes.
rxin
committed
[SPARK-9554] [SQL] Enables in-memory partition pruning by default

liancheng
authored and
rxin
committed
[SPARK-5133] [ML] Added featureImportance to RandomForestClassifier and Regressor
jkbradley
committed
[SPARK-9558][DOCS]Update docs to follow the increase of memory defaults.

sarutak
authored and
rxin
committed
[SPARK-9191] [ML] [Doc] Add ml.PCA user guide and code examples

yanboliang
authored and
jkbradley
committed
[SPARK-9544] [MLLIB] add Python API for RFormula
mengxr
committed
[SPARK-8735] [SQL] Expose memory usage for shuffles, joins and aggregations

Andrew Or
authored and
JoshRosen
committed
Revert "[SPARK-9372] [SQL] Filter nulls in join keys"
rxin
committed
[SPARK-8064] [SQL] Build against Hive 1.2.1

steveloughran
authored and
marmbrus
committed
[SPARK-8874] [ML] Add missing methods in Word2Vec

MechCoder
authored and
jkbradley
committed

Commits on Aug 4, 2015

Add a prerequisites section for building docs

shivaram
authored and
rxin
committed
[SPARK-9483] Fix UTF8String.getPrefix for big-endian.

mtbrandy
authored and
davies
committed
[SPARK-9263] Added flags to exclude dependencies when using --packages

brkyvz
authored and
Marcelo Vanzin
committed
[SPARK-8416] highlight and topping the executor threads in thread dumping page

CodingCat
authored and
JoshRosen
committed
[SPARK-9577][SQL] Surface concrete iterator types in various sort classes.
rxin
committed
[SPARK-9521] [DOCS] Addendum. Require Maven 3.3.3+ in the build

srowen
authored and
sarutak
committed
[SPARK-3190] [GRAPHX] Fix VertexRDD.count() overflow regression

ankurdave
authored and
rxin
committed
[SPARK-9534] [BUILD] Enable javac lint for scalac parity; fix a lot of build warnings, 1.5.0 edition
srowen
committed
[SPARK-8064] [BUILD] Follow-up. Undo change from SPARK-9507 that was accidentally reverted

tedyu
authored and
srowen
committed
[SPARK-2016] [WEBUI] RDD partition table pagination for the RDD Page

carsonwang
authored and
sarutak
committed
[SPARK-9583] [BUILD] Do not print mvn debug messages to stdout.

Marcelo Vanzin
authored and
sarutak
committed
[SPARK-8244] [SQL] string function: find in set

tarekbecker
authored and
davies
committed
[SPARK-8246] [SQL] Implement get_json_object

Davies Liu
authored and
davies
committed
[SPARK-9541] [SQL] DataTimeUtils cleanup

yjshen
authored and
davies
committed
[SPARK-9562] Change reference to amplab/spark-ec2 from mesos/
shivaram
committed
[SPARK-9512][SQL] Revert SPARK-9251, Allow evaluation while sorting
marmbrus
committed
[SPARK-8069] [ML] Add multiclass thresholds for ProbabilisticClassifier

holdenk
authored and
jkbradley
committed
[SPARK-9606] [SQL] Ignore flaky thrift server tests
marmbrus
committed
[SPARK-9553][SQL] remove the no-longer-necessary createCode and createStructCode, and replace the usage

cloud-fan
authored and
rxin
committed
[SPARK-9452] [SQL] Support records larger than page size in UnsafeExternalSorter

JoshRosen
authored and
rxin
committed
[SPARK-9602] remove "Akka/Actor" words from comments

CodingCat
authored and
rxin
committed
[SPARK-9447] [ML] [PYTHON] Added HasRawPredictionCol, HasProbabilityCol to RandomForestClassifier
jkbradley
committed
Adding a comment
mccheah
committed
[SPARK-9582] [ML] LDA cleanups
jkbradley
committed
[SPARK-6485] [MLLIB] [PYTHON] Add CoordinateMatrix/RowMatrix/IndexedRowMatrix to PySpark.

dusenberrymw
authored and
mengxr
committed
[SPARK-9586] [ML] Update BinaryClassificationEvaluator to use setRawPredictionCol

jkbradley
authored and
mengxr
committed

Commits on Aug 5, 2015

[SPARK-9598][SQL] do not expose generic getter in internal row

cloud-fan
authored and
rxin
committed
[SPARK-9609] [MLLIB] Fix spelling of Strategy.defaultStrategy

Feynman Liang
authored and
jkbradley
committed
[SPARK-8601] [ML] Add an option to disable standardization for linear regression

holdenk
authored and
jkbradley
committed
[SPARK-9432][SQL] Audit expression unit tests to make sure we pass the proper numeric ranges

yjshen
authored and
rxin
committed
[SPARK-8313] R Spark packages support

brkyvz
authored and
shivaram
committed
[SPARK-7119] [SQL] Give script a default serde with the user specific types

zhichao-li
authored and
marmbrus
committed
[SPARK-9513] [SQL] [PySpark] Add python API for DataFrame functions

Davies Liu
authored and
rxin
committed
[SPARK-9504] [STREAMING] [TESTS] Fix o.a.s.streaming.StreamingContextSuite.stop gracefully again

zsxwing
authored and
tdas
committed
Update docs/README.md to put all prereqs together.
rxin
committed
[SPARK-9540] [MLLIB] optimize PrefixSpan implementation
mengxr
committed
[SPARK-8231] [SQL] Add array_contains

Pedro Rodriguez
authored and
davies
committed
[SPARK-9119] [SPARK-8359] [SQL] match Decimal.precision/scale with DecimalType

Davies Liu
authored and
davies
committed
[SPARK-9217] [STREAMING] Make the kinesis receiver reliable by recording sequence numbers
tdas
committed
[SPARK-9581][SQL] Add unit test for JSON UDT

drubbo
authored and
rxin
committed
Closes #7917
rxin
committed
[SPARK-9360] [SQL] Support BinaryType in PrefixComparators for UnsafeExternalSort

maropu
authored and
davies
committed
[SPARK-9601] [DOCS] Fix JavaPairDStream signature for stream-stream and windowed join in streaming guide doc

namitk
authored and
tdas
committed
[SPARK-8861][SPARK-8862][SQL] Add basic instrumentation to each SparkPlan operator and add a new SQL tab

zsxwing
authored and
rxin
committed
[SPARK-9628][SQL]Rename int to SQLDate, long to SQLTimestamp for better readability

yjshen
authored and
rxin
committed
[HOTFIX] Add static import to fix build break from #7676.
JoshRosen
committed
[SPARK-9607] [SPARK-9608] fix zinc-port handling in build/mvn

ryan-williams
authored and
srowen
committed
[SPARK-9593] [SQL] Fixes Hadoop shims loading
liancheng
committed
[SPARK-9618] [SQL] Use the specified schema when reading Parquet files

Nathan Howell
authored and
liancheng
committed
[SPARK-9381] [SQL] Migrate JSON data source to the new partitioning data source

chenghao-intel
authored and
liancheng
committed
[SPARK-6486] [MLLIB] [PYTHON] Add BlockMatrix to PySpark.

dusenberrymw
authored and
mengxr
committed
[SPARK-9141] [SQL] Remove project collapsing from DataFrame API

marmbrus
authored and
yhuai
committed
[SPARK-9519] [YARN] Confirm stop sc successfully when application was killed

Sephiroth-Lin
authored and
Marcelo Vanzin
committed
[SPARK-9141] [SQL] [MINOR] Fix comments of PR #7920
yhuai
committed
[SPARK-9403] [SQL] Add codegen support in In and InSet

viirya
authored and
davies
committed
Closes #7778 since it is done as #7893.
rxin
committed
[SPARK-9649] Fix flaky test MasterSuite - randomize ports

Andrew Or
authored and
yhuai
committed
Closes #7474 since it's marked as won't fix.
rxin
committed
[SPARK-9657] Fix return type of getMaxPatternLength

Feynman Liang
authored and
mengxr
committed
[SPARK-9054] [SQL] Rename RowOrdering to InterpretedOrdering; use newOrdering in SMJ
JoshRosen
committed

Commits on Aug 6, 2015

[SPARK-5895] [ML] Add VectorSlicer - updated

yinxusen
authored and
mengxr
committed
[SPARK-6591] [SQL] Python data source load options should auto convert common types into strings

yjshen
authored and
davies
committed
[SPARK-9651] Fix UnsafeExternalSorterSuite.

Marcelo Vanzin
authored and
rxin
committed
[SPARK-9611] [SQL] Fixes a few corner cases when we spill a UnsafeFixedWidthAggregationMap
yhuai
committed
[SPARK-6923] [SPARK-7550] [SQL] Persists data source relations in Hive compatible format when possible

chenghao-intel
authored and
liancheng
committed
[SPARK-9674][SQL] Remove GeneratedAggregate.
rxin
committed
[SPARK-9664] [SQL] Remove UDAFRegistration and add apply to UserDefinedAggregateFunction.

yhuai
authored and
rxin
committed
[SPARK-8266] [SQL] add function translate

zhichao-li
authored and
davies
committed
[SPARK-9644] [SQL] Support update DecimalType with precision > 18 in UnsafeRow

Davies Liu
authored and
davies
committed
[SPARK-9482] [SQL] Fix thread-safey issue of using UnsafeProjection in join

Davies Liu
authored and
davies
committed
[SPARK-9593] [SQL] [HOTFIX] Makes the Hadoop shims loading fix more robust

liancheng
authored and
yhuai
committed
[SPARK-9112] [ML] Implement Stats for LogisticRegression

MechCoder
authored and
jkbradley
committed
[SPARK-9533] [PYSPARK] [ML] Add missing methods in Word2Vec ML

MechCoder
authored and
jkbradley
committed
[SPARK-9615] [SPARK-9616] [SQL] [MLLIB] Bugs related to FrequentItems when merging and with Tungsten

brkyvz
authored and
mengxr
committed
[SPARK-9659][SQL] Rename inSet to isin to match Pandas function.
rxin
committed
[SPARK-9632][SQL] update InternalRow.toSeq to make it accept data type info

cloud-fan
authored and
rxin
committed
Revert "[SPARK-9632][SQL] update InternalRow.toSeq to make it accept data type info"
davies
committed
[SPARK-9632] [SQL] [HOT-FIX] Fix build.
yhuai
committed
[SPARK-9641] [DOCS] spark.shuffle.service.port is not documented
srowen
committed
[SPARK-8978] [STREAMING] Implements the DirectKafkaRateController

nraychaudhuri
authored and
tdas
committed
[SPARK-9632][SQL] update InternalRow.toSeq to make it accept data type info

cloud-fan
authored and
rxin
committed
[SPARK-9493] [ML] add featureIndex to handle vector features in IsotonicRegression

mengxr
authored and
jkbradley
committed
[SPARK-9211] [SQL] [TEST] normalize line separators before generating MD5 hash

ckadner
authored and
marmbrus
committed
[SPARK-9548][SQL] Add a destructive iterator for BytesToBytesMap

viirya
authored and
rxin
committed
[SPARK-9556] [SPARK-9619] [SPARK-9624] [STREAMING] Make BlockGenerator more robust and make all BlockGenerators subscribe to rate limit updates
tdas
committed
[DOCS] [STREAMING] make the existing parameter docs for OffsetRange ac…

koeninger
authored and
tdas
committed
[SPARK-9639] [STREAMING] Fix a potential NPE in Streaming JobScheduler

zsxwing
authored and
tdas
committed
[SPARK-9630] [SQL] Clean up new aggregate operators (SPARK-9240 follow up)

yhuai
authored and
rxin
committed
[SPARK-9645] [YARN] [CORE] Allow shuffle service to read shuffle files.
Marcelo Vanzin
committed
[SPARK-9633] [BUILD] SBT download locations outdated; need an update
srowen
committed

Commits on Aug 7, 2015

[SPARK-9691] [SQL] PySpark SQL rand function treats seed 0 as no seed

yhuai
authored and
rxin
committed
[SPARK-9228] [SQL] use tungsten.enabled in public for both of codegen/unsafe

Davies Liu
authored and
davies
committed
[SPARK-9650][SQL] Fix quoting behavior on interpolated column names

marmbrus
authored and
rxin
committed
Revert "[SPARK-9228] [SQL] use tungsten.enabled in public for both of codegen/unsafe"
davies
committed
[SPARK-9692] Remove SqlNewHadoopRDD's generated Tuple2 and InterruptibleIterator.
rxin
committed
[SPARK-9709] [SQL] Avoid starving unsafe operators that use sort

Andrew Or
authored and
rxin
committed
[SPARK-9228] [SQL] use tungsten.enabled in public for both of codegen/unsafe

Davies Liu
authored and
rxin
committed
Fix doc typo

zjffdu
authored and
rxin
committed
[SPARK-8057][Core]Call TaskAttemptContext.getTaskAttemptID using Reflection

zsxwing
authored and
rxin
committed
[SPARK-7550] [SQL] [MINOR] Fixes logs when persisting DataFrames

liancheng
authored and
rxin
committed
[SPARK-8862][SQL]Support multiple SQLContexts in Web UI

zsxwing
authored and
rxin
committed
[SPARK-9700] Pick default page size more intelligently.
rxin
committed
[SPARK-9453] [SQL] support records larger than page size in UnsafeShuffleExternalSorter

Davies Liu
authored and
davies
committed
[SPARK-9683] [SQL] copy UTF8String when convert unsafe array/map to safe

cloud-fan
authored and
davies
committed
[SPARK-9467][SQL]Add SQLMetric to specialize accumulators to avoid boxing

zsxwing
authored and
rxin
committed
[SPARK-9674][SPARK-9667] Remove SparkSqlSerializer2
rxin
committed
[SPARK-8382] [SQL] Improve Analysis Unit test framework

cloud-fan
authored and
marmbrus
committed
[SPARK-9736] [SQL] JoinedRow.anyNull should delegate to the underlying rows.

rxin
authored and
yhuai
committed
[SPARK-9667][SQL] followup: Use GenerateUnsafeProjection.canSupport to test Exchange supported data types.
rxin
committed
[SPARK-9733][SQL] Improve physical plan explain for data sources
rxin
committed
[SPARK-9674] Re-enable ignored test in SQLQuerySuite

Andrew Or
authored and
yhuai
committed
[SPARK-8481] [MLLIB] GaussianMixtureModel predict accepting single vector

dkobylarz
authored and
jkbradley
committed
[SPARK-9748] [MLLIB] Centriod typo in KMeansModel

BertrandDechoux
authored and
jkbradley
committed
[SPARK-8890] [SQL] Fallback on sorting when writing many dynamic partitions
marmbrus
committed

Commits on Aug 8, 2015

[SPARK-9756] [ML] Make constructors in ML decision trees private

Feynman Liang
authored and
jkbradley
committed
[SPARK-9719] [ML] Clean up Naive Bayes doc

Feynman Liang
authored and
jkbradley
committed
[SPARK-9754][SQL] Remove TypeCheck in debug package.
rxin
committed
[SPARK-9753] [SQL] TungstenAggregate should also accept InternalRow instead of just UnsafeRow

yhuai
authored and
rxin
committed
[SPARK-9731] Standalone scheduling incorrect cores if spark.executor.cores is not set

carsonwang
authored and
Andrew Or
committed
[SPARK-4176] [SQL] [MINOR] Should use unscaled Long to write decimals for precision <= 18 rather than 8
liancheng
committed
[SPARK-9738] [SQL] remove FromUnsafe and add its codegen version to GenerateSafe

cloud-fan
authored and
davies
committed
[SPARK-4561] [PYSPARK] [SQL] turn Row into dict recursively

Davies Liu
authored and
davies
committed
[SPARK-6902] [SQL] [PYSPARK] Row should be read-only

Davies Liu
authored and
davies
committed
[SPARK-9728][SQL]Support CalendarIntervalType in HiveQL

yjshen
authored and
rxin
committed
[SPARK-9486][SQL] Add data source aliasing for external packages

Joseph Batchik
authored and
rxin
committed

Commits on Aug 9, 2015

Commits on Aug 10, 2015

[SPARK-9777] [SQL] Window operator can accept UnsafeRows

yhuai
authored and
rxin
committed
[SPARK-9743] [SQL] Fixes JSONRelation refreshing

liancheng
authored and
yhuai
committed
[SPARK-9710] [TEST] Fix RPackageUtilsSuite when R is not available.

Marcelo Vanzin
authored and
shivaram
committed
[SPARK-9755] [MLLIB] Add docs to MultivariateOnlineSummarizer methods

Feynman Liang
authored and
jkbradley
committed
Fixed AtmoicReference<> Example

lababidi
authored and
rxin
committed
[SPARK-9784] [SQL] Exchange.isUnsafe should check whether codegen and unsafe are enabled
JoshRosen
committed
[SPARK-9763][SQL] Minimize exposure of internal SQL classes.
rxin
committed
[SPARK-9620] [SQL] generated UnsafeProjection should support many columns or large exressions

Davies Liu
authored and
rxin
committed
[SPARK-9759] [SQL] improve decimal.times() and cast(int, decimalType)

Davies Liu
authored and
rxin
committed
[SPARK-5155] [PYSPARK] [STREAMING] Mqtt streaming support in Python

prabeesh
authored and
tdas
committed

Commits on Aug 11, 2015

[SPARK-9801] [STREAMING] Check if file exists before deleting temporary files.

viadea
authored and
tdas
committed
[SPARK-9340] [SQL] Fixes converting unannotated Parquet lists

dguy
authored and
liancheng
committed
[SPARK-9729] [SPARK-9363] [SQL] Use sort merge join for left and right outer join

JoshRosen
authored and
rxin
committed
[SPARK-9640] [STREAMING] [TEST] Do not run Python Kinesis tests when the Kinesis assembly JAR has not been generated
tdas
committed
[SPARK-9810] [BUILD] Remove individual commit messages from the squash commit message
rxin
committed
[SPARK-9727] [STREAMING] [BUILD] Updated streaming kinesis SBT project name to be more consistent
tdas
committed
[SPARK-9815] Rename PlatformDependent.UNSAFE -> Platform.

rxin
authored and
davies
committed
[SPARK-9785] [SQL] HashPartitioning compatibility should consider expression ordering

JoshRosen
authored and
yhuai
committed
Fix comment error

zjffdu
authored and
rxin
committed
[SPARK-8345] [ML] Add an SQL node as a feature transformer

yanboliang
authored and
mengxr
committed
[SPARK-8764] [ML] string indexer should take option to handle unseen values

holdenk
authored and
jkbradley
committed
[SPARK-9572] [STREAMING] [PYSPARK] Added StreamingContext.getActiveOrCreate() in Python
tdas
committed
[SPARK-9646] [SQL] Add metrics for all join and aggregate operators

zsxwing
authored and
yhuai
committed
[SPARK-9750] [MLLIB] Improve equals on SparseMatrix and DenseMatrix

Feynman Liang
authored and
jkbradley
committed
[SPARK-7726] Add import so Scaladoc doesn't fail.

pwendell
authored and
rxin
committed
[SPARK-9814] [SQL] EqualNotNull not passing to data sources

HyukjinKwon
authored and
rxin
committed
[SPARK-9824] [CORE] Fix the issue that InternalAccumulator leaks WeakReference

zsxwing
authored and
rxin
committed
Closes #1290
mengxr
committed
[SPARK-9788] [MLLIB] Fix LDA Binary Compatibility

Feynman Liang
authored and
jkbradley
committed
[SPARK-8925] [MLLIB] Add @since tags to mllib.util

sthota2014
authored and
mengxr
committed
[HOTFIX] Fix style error caused by 017b5de
Andrew Or
committed
[SPARK-9074] [LAUNCHER] Allow arbitrary Spark args to be set.
Marcelo Vanzin
committed

Commits on Aug 12, 2015

[SPARK-9849] [SQL] DirectParquetOutputCommitter qualified name should be backward compatible

rxin
authored and
yhuai
committed
[SPARK-9649] Fix flaky test MasterSuite again - disable REST

Andrew Or
authored and
JoshRosen
committed
[SPARK-1517] Refactor release scripts to facilitate nightly publishing
pwendell
committed
[SPARK-9713] [ML] Document SparkR MLlib glm() integration in Spark 1.5

ericl
authored and
mengxr
committed
[SPARK-9831] [SQL] fix serialization with empty broadcast

Davies Liu
authored and
davies
committed
[SPARK-9854] [SQL] RuleExecutor.timeMap should be thread-safe
JoshRosen
committed
[SPARK-8366] maxNumExecutorsNeeded should properly handle failed tasks

XuTingjun
authored and
Andrew Or
committed
[SPARK-9806] [WEB UI] Don't share ReplayListenerBus between multiple applications

Rohit Agarwal
authored and
Andrew Or
committed
[SPARK-9829] [WEBUI] Display the update value for peak execution memory

zsxwing
authored and
Andrew Or
committed
[SPARK-9426] [WEBUI] Job page DAG visualization is not shown

carsonwang
authored and
Andrew Or
committed
[SPARK-8798] [MESOS] Allow additional uris to be fetched with mesos

tnachen
authored and
Andrew Or
committed
[SPARK-9575] [MESOS] Add docuemntation around Mesos shuffle service.

tnachen
authored and
Andrew Or
committed
[SPARK-9182] [SQL] Filters are not passed through to jdbc source

yjshen
authored and
liancheng
committed
[SPARK-9407] [SQL] Relaxes Parquet ValidTypeMap to allow ENUM predicates to be pushed down
liancheng
committed
[SPARK-8625] [CORE] Propagate user exceptions in tasks back to driver

tomwhite
authored and
squito
committed
[SPARK-9795] Dynamic allocation: avoid double counting when killing same executor twice
Andrew Or
committed
[SPARK-7583] [MLLIB] User guide update for RegexTokenizer

hhbyyh
authored and
mengxr
committed
[SPARK-9747] [SQL] Avoid starving an unsafe operator in aggregation

Andrew Or
authored and
rxin
committed
[SPARK-9804] [HIVE] Use correct value for isSrcLocal parameter.

Marcelo Vanzin
authored and
marmbrus
committed
[SPARK-9847] [ML] Modified copyValues to distinguish between default, explicit param values

jkbradley
authored and
mengxr
committed
[SPARK-9726] [PYTHON] PySpark DF join no longer accepts on=None

btashton
authored and
marmbrus
committed
[SPARK-9766] [ML] [PySpark] check and add miss docs for PySpark ML

yanboliang
authored and
jkbradley
committed
[SPARK-9789] [ML] Added logreg threshold param back
jkbradley
committed
[SPARK-8967] [DOC] add Since annotation
mengxr
committed
[SPARK-9907] [SQL] Python crc32 is mistakenly calling md5
rxin
committed
[SPARK-9092] Fixed incompatibility when both num-executors and dynamic...

Niranjan Padmanabhan
authored and
Marcelo Vanzin
committed
[SPARK-9826] [CORE] Fix cannot use custom classes in log4j.properties

michellemay
authored and
Marcelo Vanzin
committed
[SPARK-9894] [SQL] Json writer should handle MapData.

yhuai
authored and
marmbrus
committed
[SPARK-9913] [MLLIB] LDAUtils should be private
mengxr
committed
[SPARK-9909] [ML] [TRIVIAL] move weightCol to shared params

holdenk
authored and
mengxr
committed

Commits on Aug 13, 2015

[SPARK-9912] [MLLIB] QRDecomposition should use QType and RType for type names instead of UType and VType
mengxr
committed
[SPARK-9915] [ML] stopWords should use StringArrayParam
mengxr
committed
[SPARK-9449] [SQL] Include MetastoreRelation's inputFiles
marmbrus
committed
[SPARK-9780] [STREAMING] [KAFKA] prevent NPE if KafkaRDD instantiation …

koeninger
authored and
tdas
committed
[SPARK-9724] [WEB UI] Avoid unnecessary redirects in the Spark Web UI.

Rohit Agarwal
authored and
Marcelo Vanzin
committed
[SPARK-9855] [SPARKR] Add expression functions into SparkR whose params are simple

yu-iskw
authored and
shivaram
committed
[SPARK-9870] Disable driver UI and Master REST server in SparkSubmitSuite

JoshRosen
authored and
Andrew Or
committed
[SPARK-9827] [SQL] fix fd leak in UnsafeRowSerializer

Davies Liu
authored and
rxin
committed
[SPARK-9908] [SQL] When spark.sql.tungsten.enabled is false, broadcast join does not work

yhuai
authored and
rxin
committed
[SPARK-9704] [ML] Made ProbabilisticClassifier, Identifiable, VectorUDT public APIs

jkbradley
authored and
mengxr
committed
[SPARK-9903] [MLLIB] skip local processing in PrefixSpan if there are no small prefixes
mengxr
committed
[SPARK-9916] [BUILD] [SPARKR] removed left-over sparkr.zip copy/create commands from codebase

brkyvz
authored and
shivaram
committed
[SPARK-9920] [SQL] The simpleString of TungstenAggregate does not show its output

yhuai
authored and
rxin
committed
[SPARK-9832] [SQL] add a thread-safe lookup for BytesToBytseMap

Davies Liu
authored and
rxin
committed
[SPARK-9917] [ML] add getMin/getMax and doc for originalMin/origianlMax in MinMaxScaler
mengxr
committed
[SPARK-8922] [DOCUMENTATION, MLLIB] Add @since tags to mllib.evaluation

mosessky
authored and
mengxr
committed
[SPARK-9914] [ML] define setters explicitly for Java and use setParam group in RFormula
mengxr
committed
[SPARK-9927] [SQL] Revert 8049 since it's pushing wrong filter down

yjshen
authored and
liancheng
committed
[SPARK-9918] [MLLIB] remove runs from k-means and rename epsilon to tol
mengxr
committed
[SPARK-9885] [SQL] Also pass barrierPrefixes and sharedPrefixes to IsolatedClientLoader when hiveMetastoreJars is set to maven.

yhuai
authored and
liancheng
committed
[SPARK-9757] [SQL] Fixes persistence of Parquet relation with decimal column
liancheng
committed
[SPARK-9073] [ML] spark.ml Models copy() should call setParent when there is a parent

Lewuathe
authored and
jkbradley
committed
[SPARK-8965] [DOCS] Add ml-guide Python Example: Estimator, Transformer, and Param

Rosstin
authored and
jkbradley
committed
[MINOR] [ML] change MultilayerPerceptronClassifierModel to MultilayerPerceptronClassificationModel

yanboliang
authored and
mengxr
committed
[MINOR] [DOC] fix mllib pydoc warnings
mengxr
committed
[SPARK-9649] Fix MasterSuite, third time's a charm
Andrew Or
committed
[SPARK-9661] [MLLIB] [ML] Java compatibility

MechCoder
authored and
jkbradley
committed
[SPARK-9942] [PYSPARK] [SQL] ignore exceptions while try to import pandas

Davies Liu
authored and
rxin
committed
[SPARK-9935] [SQL] EqualNotNull not processed in ORC

HyukjinKwon
authored and
rxin
committed
[SPARK-9922] [ML] rename StringIndexerReverse to IndexToString
mengxr
committed

Commits on Aug 17, 2015

Using ExternalList[_] in KryoSerializer. Clean up SpillableCollection.next
mccheah
committed

Commits on Aug 18, 2015

Fixing unit test
mccheah
committed

Commits on Aug 25, 2015