Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1948 commits
Select commit Hold shift + click to select a range
bab88c4
[SPARK-28622][SQL][PYTHON] Rename PullOutPythonUDFInJoinCondition to …
HyukjinKwon Aug 6, 2019
c17fa13
[SPARK-28514][ML] Remove the redundant transformImpl method in RF & GBT
zhengruifeng Aug 6, 2019
9e3aab8
[SPARK-28623][SQL] Support `dow`, `isodow` and `doy` by `extract()`
MaxGekk Aug 6, 2019
9e931e7
[SPARK-27905][SQL] Add higher order function 'forall'
nvander1 Aug 6, 2019
a133175
[SPARK-28615][SQL][DOCS] Add a guide line for dataframe functions to …
WeichenXu123 Aug 7, 2019
44e607e
[SPARK-28238][SQL] Implement DESCRIBE TABLE for Data Source V2 Tables
mccheah Aug 7, 2019
99de6a4
[SPARK-27924][SQL][TEST][FOLLOW-UP] Enable Boolean-Predicate syntax t…
beliefer Aug 7, 2019
469423f
[SPARK-28595][SQL] explain should not trigger partition listing
cloud-fan Aug 7, 2019
a59fdc4
[SPARK-28472][SQL][TEST] Add test for thriftserver protocol versions
wangyum Aug 7, 2019
325bc8e
[SPARK-28583][SQL] Subqueries should not call `onUpdatePlan` in Adapt…
maryannxue Aug 7, 2019
8617bf6
[SPARK-28470][SQL] Cast to decimal throws ArithmeticException on over…
mgaido91 Aug 7, 2019
c88df2c
[SPARK-28331][SQL] Catalogs.load() should be able to load built-in ca…
gengliangwang Aug 7, 2019
eeaf185
[SPARK-28617][SQL][TEST] Fix misplacement when comment is at the end …
wangyum Aug 7, 2019
6ea4a73
[SPARK-28649][INFRA] Add Python .eggs to .gitignore
rvesse Aug 7, 2019
3586cdd
[SPARK-28395][FOLLOW-UP][SQL] Make spark.sql.function.preferIntegralD…
wangyum Aug 8, 2019
e58dd4a
[MINOR][DOC] Fix a typo 'lister' -> 'listener'
lys0716 Aug 8, 2019
bda5b51
[SPARK-28454][PYTHON] Validate LongType in `createDataFrame(verifySch…
simplylizz Aug 8, 2019
c4acfe7
[SPARK-28474][SQL] Hive 0.12 JDBC client can not handle binary type
wangyum Aug 8, 2019
1941d35
[SPARK-28644][SQL] Port HIVE-10646: ColumnValue does not handle NULL_…
wangyum Aug 8, 2019
8c0dc38
[SPARK-28654][SQL] Move "Extract Python UDFs" to the last in optimizer
HyukjinKwon Aug 8, 2019
d19a56f
[SPARK-28642][SQL] Hide credentials in SHOW CREATE TABLE
wangyum Aug 8, 2019
2580c1b
[SPARK-28660][SQL][TEST] Port AGGREGATES.sql [Part 4]
wangyum Aug 8, 2019
c80430f
[SPARK-28572][SQL] Simple analyzer checks for v2 table creation code …
brkyvz Aug 9, 2019
997d153
[SPARK-28017][SQL] Support additional levels of truncations by DATE_T…
MaxGekk Aug 9, 2019
5368eaa
[SPARK-28565][SQL] DataFrameWriter saveAsTable support for V2 catalogs
brkyvz Aug 9, 2019
5663386
[SPARK-28163][SS] Use CaseInsensitiveMap for KafkaOffsetReader
gaborgsomogyi Aug 9, 2019
5bb6994
[SPARK-28651][SS] Force the schema of Streaming file source to be nul…
zsxwing Aug 9, 2019
f21bc18
[SPARK-27889][INFRA] Make development scripts under dev/ support Pyth…
WeichenXu123 Aug 9, 2019
5159876
[SPARK-28077][SQL][TEST][FOLLOW-UP] Enable Overlay function tests
beliefer Aug 9, 2019
cbad616
[SPARK-27371][CORE] Support GPU-aware resources scheduling in Standalone
Ngone51 Aug 9, 2019
8b08e14
[SPARK-21481][ML][FOLLOWUP] HashingTF Cleanup
zhengruifeng Aug 9, 2019
924d794
[SPARK-28656][SQL] Support `millennium`, `century` and `decade` at `e…
MaxGekk Aug 9, 2019
ef80c32
[SPARK-28676][CORE] Avoid Excessive logging from ContextCleaner
ajithme Aug 9, 2019
8535df7
[MINOR] Fix typos in comments and replace an explicit type with <>
Aug 10, 2019
dd5599e
[SPARK-28677][WEBUI] "Select All" checkbox in StagePage doesn't work …
sarutak Aug 10, 2019
47af892
[SPARK-28675][SQL] Remove maskCredentials and use redactOptions
wangyum Aug 10, 2019
58cc0df
[SPARK-28685][SQL][TEST] Test HMS 2.0.0+ in VersionsSuite/HiveClientS…
wangyum Aug 11, 2019
31ef268
[SPARK-28639][CORE][DOC] Configuration doc for Barrier Execution Mode
sarutak Aug 11, 2019
37eedf6
[SPARK-28652][TESTS][K8S] Add python version check for executor
viirya Aug 12, 2019
6c06eea
[SPARK-28686][SQL][TEST] Move udf_radians from HiveCompatibilitySuite…
wangyum Aug 12, 2019
e5f4a10
[SPARK-28688][SQL][TEST] Skip `VersionsSuite.read hive materialized v…
wangyum Aug 12, 2019
0f2efe6
[SPARK-28366][CORE][FOLLOW-UP] Refine logging in driver when loading …
WeichenXu123 Aug 12, 2019
ae4edd5
[SPARK-28538][UI] Document SQL page
zhengruifeng Aug 12, 2019
6964128
[SPARK-28017][SPARK-28656][SQL][FOLLOW-UP] Restore comments in date.sql
MaxGekk Aug 12, 2019
48d04f7
[SPARK-28638][WEBUI] Task summary should only contain successful task…
gengliangwang Aug 12, 2019
163f4a4
[SPARK-26969][SQL] Using ODBC client not able to see the query data w…
sujith71955 Aug 12, 2019
25857c6
[SPARK-28647][WEBUI] Recover additional metric feature and remove add…
sarutak Aug 13, 2019
ec84415
[SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group…
skonto Aug 13, 2019
016e1b4
[SPARK-28703][SQL][TEST] Skip HiveExternalCatalogVersionsSuite and 3 …
wangyum Aug 13, 2019
9a7f290
[SPARK-28383][SQL] SHOW CREATE TABLE is not supported on a temporary …
wangyum Aug 13, 2019
247bebc
[SPARK-28561][WEBUI] DAG viz for barrier-execution mode
sarutak Aug 13, 2019
3249c7a
[SPARK-28706][SQL] Allow cast null type to any types
jiangxb1987 Aug 13, 2019
48adc91
[SPARK-28698][SQL] Support user-specified output schema in `to_avro`
gengliangwang Aug 13, 2019
e6a0385
[SPARK-28422][SQL][PYTHON] GROUPED_AGG pandas_udf should work without…
viirya Aug 13, 2019
c81da27
[SPARK-28714][SQL][TEST] Add `hive.aux.jars.path` test for spark-sql …
wangyum Aug 13, 2019
d8dd571
[SPARK-28713][BUILD] Bump checkstyle from 8.14 to 8.23
Fokko Aug 13, 2019
13b62f3
[SPARK-28708][SQL] IsolatedClientLoader will not load hive classes fr…
wangyum Aug 13, 2019
f04a766
[SPARK-28718][SQL] Support `field` synonyms at `extract`
MaxGekk Aug 13, 2019
3d85c54
[SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()`
MaxGekk Aug 13, 2019
601fd45
[SPARK-28719][BUILD] Enable Github Actions for master
dbtsai Aug 13, 2019
f1d6b19
[SPARK-28720][BUILD][R] Update AppVeyor R version to 3.6.1
dongjoon-hyun Aug 13, 2019
331f265
[SPARK-27768][SQL] Support Infinity/NaN-related float/double literals…
dilipbiswal Aug 13, 2019
0343854
[SPARK-28487][K8S] More responsive dynamic allocation with K8S
Aug 14, 2019
3302042
[SPARK-28719][BUILD] [FOLLOWUP] Add JDK11 for Github Actions
dbtsai Aug 14, 2019
598fcbe
[SPARK-28265][SQL] Add renameTable to TableCatalog API
edgarRd Aug 14, 2019
391c7e8
[SPARK-27739][SQL] df.persist should save stats from optimized plan
jzhuge Aug 14, 2019
2eeb25e
[SPARK-28351][SQL] Support DELETE in DataSource V2
Aug 14, 2019
3a4afce
[SPARK-28687][SQL] Support `epoch`, `isoyear`, `milliseconds` and `mi…
MaxGekk Aug 14, 2019
3ec24fd
[SPARK-28203][CORE][PYTHON] PythonRDD should respect SparkContext's h…
advancedxy Aug 15, 2019
0526529
[SPARK-28666] Support saveAsTable for V2 tables through Session Catalog
brkyvz Aug 15, 2019
a493031
[SPARK-28695][SS] Use CaseInsensitiveMap in KafkaSourceProvider to ma…
gaborgsomogyi Aug 15, 2019
1b416a0
[SPARK-27592][SQL] Set the bucketed data source table SerDe correctly
wangyum Aug 15, 2019
3f35440
[SPARK-28543][DOCS][WEBUI] Document Spark Jobs page
planga82 Aug 15, 2019
ba5ee27
[SPARK-28243][PYSPARK][ML][FOLLOW-UP] Move Python DecisionTreeParams …
huaxingao Aug 15, 2019
2ac6163
[SPARK-23977][SQL] Support High Performance S3A committers [test-hado…
steveloughran Aug 15, 2019
123eb58
[MINOR][DOC] Use `Java 8` instead of `Java 8+` as a running environment
dongjoon-hyun Aug 15, 2019
96ca734
[SPARK-28745][SQL][TEST] Add benchmarks for `extract()`
MaxGekk Aug 15, 2019
c2b40a7
[SPARK-28719][BUILD][FOLLOWUP] Make Github Actions log quieter
dbtsai Aug 15, 2019
0ea8db9
[SPARK-28578][INFRA] Improve Github pull request template
HyukjinKwon Aug 16, 2019
babdba0
[SPARK-28728][BUILD] Bump Jackson Databind to 2.9.9.3
Fokko Aug 16, 2019
ef14237
[SPARK-28736][SPARK-28735][PYTHON][ML] Fix PySpark ML tests to pass i…
HyukjinKwon Aug 16, 2019
c48e381
[SPARK-28671][SQL] Throw NoSuchPermanentFunctionException for a non-e…
PavithraRamachandran Aug 16, 2019
92bfd9a
[SPARK-28757][SQL] File table location should include both values of …
gengliangwang Aug 16, 2019
036fd39
[SPARK-27637][SHUFFLE][FOLLOW-UP] For nettyBlockTransferService, if I…
AngersZhuuuu Aug 16, 2019
2f04152
[SPARK-28756][R] Fix checkJavaVersion to accept JDK8+
dongjoon-hyun Aug 16, 2019
7f44a6e
[SPARK-28755][R][TESTS] Increase tolerance in 'spark.mlp' SparkR test…
HyukjinKwon Aug 16, 2019
0094b5f
[SPARK-28722][ML] Change sequential label sorting in StringIndexer fi…
viirya Aug 16, 2019
43101c7
[SPARK-28758][BUILD][SQL] Upgrade Janino to 3.0.15
dongjoon-hyun Aug 16, 2019
c9b49f3
[SPARK-28737][CORE] Update Jersey to 2.29
srowen Aug 16, 2019
1819a6f
[SPARK-28759][BUILD] Upgrade scala-maven-plugin to 4.1.1
dongjoon-hyun Aug 17, 2019
f7c9de9
[SPARK-28765][BUILD] Add explict exclusions to avoid JDK11 dependency…
dongjoon-hyun Aug 17, 2019
5756a47
[SPARK-28766][R][DOC] Fix CRAN incoming feasibility warning on invali…
dongjoon-hyun Aug 17, 2019
efbb035
[SPARK-28527][SQL][TEST] Re-run all the tests in SQLQueryTestSuite vi…
wangyum Aug 18, 2019
c308ab5
[MINOR][SQL] Make analysis error msg more meaningful on DISTINCT queries
wangyum Aug 18, 2019
c097c55
[SPARK-21067][DOC] Fix Thrift Server - CTAS fail with Unable to move …
Deegue Aug 18, 2019
f0834d3
Revert "[SPARK-28527][SQL][TEST] Re-run all the tests in SQLQueryTest…
dongjoon-hyun Aug 18, 2019
4ddad79
[SPARK-28598][SQL] Few date time manipulation functions does not prov…
WeichenXu123 Aug 19, 2019
c96b615
[SPARK-28390][SQL][PYTHON][TESTS][FOLLOW-UP] Update the TODO with act…
shivusondur Aug 19, 2019
d75a11d
[SPARK-27330][SS] support task abort in foreach writer
Aug 19, 2019
a5df5ff
[SPARK-28734][DOC] Initial table of content in the left hand side bar…
dilipbiswal Aug 19, 2019
f999e00
[SPARK-28356][SHUFFLE][FOLLOWUP] Fix case with different pre-shuffle …
peter-toth Aug 19, 2019
97dc4c0
[SPARK-28744][SQL][TEST] rename SharedSQLContext to SharedSparkSession
cloud-fan Aug 19, 2019
ec14b6e
[SPARK-28393][SQL][PYTHON][TESTS] Convert and port 'pgSQL/join.sql' i…
huaxingao Aug 19, 2019
2fd83c2
[SPARK-28756][R][FOLLOW-UP] Specify minimum and maximum Java versions
HyukjinKwon Aug 19, 2019
1de4a22
Revert "[SPARK-28759][BUILD] Upgrade scala-maven-plugin to 4.1.1"
HyukjinKwon Aug 19, 2019
5f6eb5d
[SPARK-28634][YARN] Ignore kerberos login config in client mode AM
Aug 19, 2019
fa7fd8f
[SPARK-28434][TESTS][ML] Fix values in dummy tree in DecisionTreeSuite
srowen Aug 19, 2019
26f3443
[SPARK-27905][SQL][FOLLOW-UP] Add prettyNames
ueshin Aug 19, 2019
b79cf0d
[SPARK-28224][SQL] Check overflow in decimal Sum aggregate
mickjermsurawong-stripe Aug 20, 2019
3b4e345
[SPARK-28775][CORE][TESTS] Skip date 8633 in Kwajalein due to changes…
srowen Aug 20, 2019
a787bc2
[SPARK-28777][PYTHON][DOCS] Fix format_string doc string with the cor…
darrentirto Aug 20, 2019
0d3a783
[SPARK-28699][CORE] Fix a corner case for aborting indeterminate stage
xuanyuanking Aug 20, 2019
bc75ed6
[SPARK-28483][CORE] Fix canceling a spark job using barrier mode but …
WeichenXu123 Aug 20, 2019
79464be
[SPARK-28662][SQL] Create Hive Partitioned Table DDL should fail when…
lidinghao Aug 20, 2019
b37c8d5
[SPARK-28650][SS][DOC] Correct explanation of guarantee for ForeachWr…
HeartSaVioR Aug 20, 2019
d045221
[SPARK-28635][SQL] create CatalogManager to track registered v2 catalogs
cloud-fan Aug 20, 2019
39c1127
[SPARK-28753][SQL] Dynamically reuse subqueries in AQE
maryannxue Aug 20, 2019
a50959a
[SPARK-27937][CORE] Revert partial logic for auto namespace discovery
dhruve Aug 20, 2019
0bfcf9c
[SPARK-28322][SQL] Add support to Decimal type for integral divide
mgaido91 Aug 20, 2019
c4257b1
[SPARK-28541][WEBUI] Document Storage page
zhengruifeng Aug 21, 2019
4855bfe
[SPARK-28554][SQL] Adds a v1 fallback writer implementation for v2 da…
brkyvz Aug 21, 2019
fac469e
[SPARK-28774][SQL] Fix exchange reuse for columnar data
revans2 Aug 21, 2019
9779a82
[SPARK-28483][CORE][FOLLOW-UP] Dealing with interrupted exception in …
WeichenXu123 Aug 21, 2019
49ffbff
[SPARK-28780][ML] Delete the incorrect setWeightCol method in LinearS…
zhengruifeng Aug 21, 2019
bdef712
[SPARK-28540][WEBUI] Document Environment page
zhengruifeng Aug 21, 2019
4dc3093
[SPARK-28715][SQL] Introduce collectInPlanAndSubqueries and subquerie…
dbaliafroozeh Aug 21, 2019
2d9cc42
[SPARK-28699][SQL] Disable using radix sort for ShuffleExchangeExec i…
xuanyuanking Aug 21, 2019
97b046f
[SPARK-28635][SQL][FOLLOWUP] CatalogManager should reflect the change…
cloud-fan Aug 21, 2019
ed3ea67
[SPARK-28837][SQL] CTAS/RTAS should use nullable schema
cloud-fan Aug 22, 2019
aefb2e7
[SPARK-28739][SQL] Add a simple cost check for Adaptive Query Execution
maryannxue Aug 22, 2019
9ea37b0
[SPARK-17875][CORE][BUILD] Remove dependency on Netty 3
srowen Aug 22, 2019
48578a4
[SPARK-28844][SQL] Fix typo in SQLConf FILE_COMRESSION_FACTOR
triplesheep Aug 22, 2019
36da2e3
[SPARK-28847][TEST] Annotate HiveExternalCatalogVersionsSuite with Ex…
dongjoon-hyun Aug 22, 2019
fb1f868
[SPARK-28776][ML] SparkML Writer gets hadoop conf from session state
helenyuyu Aug 22, 2019
defb65e
[SPARK-13677][ML] Implement Tree-Based Feature Transformation for ML
zhengruifeng Aug 22, 2019
33e45ec
[SPARK-28769][CORE] Improve warning message of BarrierExecutionMode w…
sarutak Aug 22, 2019
23bed0d
[SPARK-28702][SQL] Display useful error message (instead of NPE) for …
shivusondur Aug 23, 2019
895c90b
[SPARK-28730][SQL] Configurable type coercion policy for table insertion
gengliangwang Aug 23, 2019
406c533
[SPARK-28025][SS] Fix FileContextBasedCheckpointFileManager leaking c…
HeartSaVioR Aug 23, 2019
9976b87
[SPARK-28835][SQL][TEST] Add TPCDSSchema trait
dbaliafroozeh Aug 23, 2019
98e1a4c
[SPARK-28319][SQL] Implement SHOW TABLES for Data Source V2 Tables
imback82 Aug 23, 2019
1fd7f29
[SPARK-28857][INFRA] Clean up the comments of PR template during merging
dongjoon-hyun Aug 23, 2019
aef7ca1
[SPARK-28836][SQL] Remove the canonicalize(attributes) method from Pl…
dbaliafroozeh Aug 23, 2019
1472e66
[SPARK-28716][SQL] Add id to Exchange and Subquery's stringArgs metho…
dbaliafroozeh Aug 23, 2019
8258660
[SPARK-28741][SQL] Optional mode: throw exceptions when casting to in…
gengliangwang Aug 23, 2019
07c4b9b
Revert "[SPARK-25474][SQL] Support `spark.sql.statistics.fallBackToHd…
gatorsmile Aug 23, 2019
d25cbd4
[SPARK-28839][CORE] Avoids NPE in context cleaner when dynamic alloca…
HyukjinKwon Aug 23, 2019
573b1cb
[SPARK-28858][ML][PYSPARK] add tree-based transformation in the py side
zhengruifeng Aug 23, 2019
f17f1d0
[SPARK-28778][MESOS] Fixed executors advertised address when running …
Aug 24, 2019
02a0cde
[SPARK-28723][SQL] Upgrade to Hive 2.3.6 for HiveMetastore Client and…
wangyum Aug 24, 2019
4b16cf1
[SPARK-27988][SQL][TEST] Port AGGREGATES.sql [Part 3]
wangyum Aug 25, 2019
6214b6a
[SPARK-28868][INFRA] Specify Jekyll version to 3.8.6 in release docke…
dongjoon-hyun Aug 25, 2019
a3328cd
[SPARK-28238][SQL][FOLLOW-UP] Clean up attributes for Datasource v2 D…
imback82 Aug 26, 2019
adb506a
[SPARK-28852][SQL] Implement SparkGetCatalogsOperation for Thrift Server
wangyum Aug 26, 2019
c353a84
[SPARK-28642][SQL][TEST][FOLLOW-UP] Test spark.sql.redaction.options.…
wangyum Aug 26, 2019
c61270f
[SPARK-27395][SQL] Improve EXPLAIN command
dilipbiswal Aug 26, 2019
6e12b58
[SPARK-28527][SQL][TEST] Re-run all the tests in SQLQueryTestSuite vi…
wangyum Aug 26, 2019
9f8c7a2
[SPARK-28709][DSTREAMS] Fix StreamingContext leak through Streaming
Aug 26, 2019
13fd32c
[SPARK-28701][TEST-HADOOP3.2][TEST-JAVA11][K8S] adding java11 support…
shaneknapp Aug 26, 2019
2efa6f5
[SPARK-28607][CORE][SHUFFLE] Don't store partition lengths twice
mccheah Aug 26, 2019
dd0725d
[SPARK-28679][YARN] changes to setResourceInformation to handle empty…
abellina Aug 26, 2019
84d4f94
[SPARK-28701][INFRA][FOLLOWUP] Fix the key error when looking in os.e…
shaneknapp Aug 26, 2019
b205269
[SPARK-28875][DSTREAMS][SS][TESTS] Add Task retry tests to make sure …
gaborgsomogyi Aug 26, 2019
64032cb
[MINOR][SS] Reuse KafkaSourceInitialOffsetWriter to deduplicate
HeartSaVioR Aug 27, 2019
13b1eb6
[SPARK-22955][DSTREAMS] - graceful shutdown shouldn't lead to job gen…
Aug 27, 2019
e31aec9
[SPARK-28667][SQL] Support InsertInto through the V2SessionCatalog
brkyvz Aug 27, 2019
7701d29
[SPARK-28877][PYSPARK][test-hadoop3.2][test-java11] Make jaxb-runtime…
dongjoon-hyun Aug 27, 2019
ab1819d
[SPARK-28527][SQL][TEST][FOLLOW-UP] Ignores Thrift server ThriftServe…
wangyum Aug 27, 2019
00cb2f9
[SPARK-28881][PYTHON][TESTS] Add a test to make sure toPandas with Ar…
HyukjinKwon Aug 27, 2019
c02c86e
[SPARK-28691][EXAMPLES] Add Java/Scala DirectKerberizedKafkaWordCount…
hddong Aug 27, 2019
cb06209
[SPARK-28747][SQL] merge the two data source v2 fallback configs
cloud-fan Aug 27, 2019
9617973
[SPARK-27592][SQL][TEST][FOLLOW-UP] Test set the partitioned bucketed…
wangyum Aug 27, 2019
4cf8128
[SPARK-28871][MINOR][DOCS] WaterMark doc fix
Aug 27, 2019
e12da8b
[SPARK-28876][SQL] fallBackToHdfs should not support Hive partitioned…
wangyum Aug 27, 2019
7f605f5
[SPARK-28621][SQL] Make spark.sql.crossJoin.enabled default value true
WeichenXu123 Aug 27, 2019
7fe7506
[SPARK-11215][ML][FOLLOWUP] update the examples and suites using new api
zhengruifeng Aug 27, 2019
70f4bbc
[SPARK-28414][WEBUI] UI updates to show resource info in Standalone
Ngone51 Aug 27, 2019
2b24a71
[SPARK-28495][SQL] Introduce ANSI store assignment policy for table i…
gengliangwang Aug 27, 2019
90b10b4
[HOT-FIX] fix compilation
cloud-fan Aug 27, 2019
8848af2
[SPARK-28881][PYTHON][TESTS][FOLLOW-UP] Use SparkSession(SparkContext…
HyukjinKwon Aug 28, 2019
6252c54
[SPARK-23519][SQL] create view should work from query with duplicate …
hemanthmeka Aug 28, 2019
e3b32da
[SPARK-25474][SQL][DOCS] Update the docs for spark.sql.statistics.fal…
wangyum Aug 28, 2019
9d6bec1
[SPARK-28730][SPARK-28495][SQL][FOLLOW-UP] Revise the doc of option s…
gengliangwang Aug 28, 2019
3e7b0e1
[SPARK-28539][WEBUI][DOC] Document Executors page
zhengruifeng Aug 28, 2019
1b404b9
[SPARK-28890][SQL] Upgrade Hive Metastore Client to the 3.1.2 for Hiv…
wangyum Aug 28, 2019
7452786
[SPARK-28789][DOCS][SQL] Document ALTER DATABASE command
dilipbiswal Aug 28, 2019
137b20b
[SPARK-28818][SQL] Respect source column nullability in the arrays cr…
Aug 29, 2019
2465558
[SPARK-28495][SQL][FOLLOW-UP] Disallow conversions between timestamp …
gengliangwang Aug 29, 2019
3e09a0f
[SPARK-28786][DOC][SQL] Document INSERT statement in SQL Reference
huaxingao Aug 29, 2019
fb1053d
[SPARK-28807][DOCS][SQL] Document SHOW DATABASES in SQL Reference
dilipbiswal Aug 29, 2019
7d72c07
[SPARK-28760][SS][TESTS] Add Kafka delegation token end-to-end test w…
gaborgsomogyi Aug 29, 2019
f8f7c52
[SPARK-28899][SQL][TEST] merge the testing in-memory v2 catalogs from…
cloud-fan Aug 29, 2019
31b59bd
[SPARK-28843][PYTHON] Set OMP_NUM_THREADS to executor cores for pytho…
rdblue Aug 30, 2019
d502c80
[SPARK-28922][SS] Safe Kafka parameter redaction
gaborgsomogyi Aug 30, 2019
780aa71
[SPARK-28919][INFRA] Add more profiles for JDK8/11 build test for Git…
dongjoon-hyun Aug 30, 2019
2db45cb
[SPARK-28920][INFRA] Set up java version for github workflow
viirya Aug 30, 2019
8279693
[SPARK-28668][SQL] Support V2SessionCatalog for ALTER TABLE
brkyvz Aug 30, 2019
2bd02e2
[SPARK-28866][ML] Persist item factors RDD when checkpointing in ALS
viirya Aug 30, 2019
92cabf6
[SPARK-28759][BUILD] Upgrade scala-maven-plugin to 4.2.0 and fix buil…
HyukjinKwon Aug 30, 2019
ea90ea6
[SPARK-28571][CORE][SHUFFLE] Use the shuffle writer plugin for the So…
mccheah Aug 30, 2019
3b07a4e
[SPARK-27931][SQL] Accept "true", "yes", "1", "false", "no", "0", and…
Aug 30, 2019
7cc0f0e
[SPARK-28894][SQL][TESTS] Add a clue to make it easier to debug via J…
HyukjinKwon Aug 30, 2019
a08f33b
[SPARK-28804][DOCS][SQL] Document DESCRIBE QUERY in SQL Reference
dilipbiswal Aug 30, 2019
d5b7eed
[SPARK-28903][STREAMING][PYSPARK][TESTS] Fix AWS JDK version conflict…
srowen Aug 31, 2019
1f96ce5
[SPARK-28932][BUILD] Add scala-library test dependency to network-com…
dongjoon-hyun Aug 31, 2019
d573e4c
[SPARK-28542][DOCS][WEBUI] Stages Tab
planga82 Aug 31, 2019
b4d7b30
[SPARK-28803][DOCS][SQL] Document DESCRIBE TABLE in SQL Reference
dilipbiswal Aug 31, 2019
b85a554
[SPARK-28786][DOC][SQL][FOLLOW-UP] Change "Related Statements" to bold
huaxingao Aug 31, 2019
3821d75
[SPARK-28612][SQL] Add DataFrameWriterV2 API
rdblue Sep 1, 2019
eb037a8
[SPARK-28855][CORE][ML][SQL][STREAMING] Remove outdated usages of Exp…
srowen Sep 1, 2019
585954d
[SPARK-28790][DOC][SQL] Document CACHE TABLE statement in SQL Reference
huaxingao Sep 2, 2019
19f882c
[SPARK-28933][ML] Reduce unnecessary shuffle in ALS when initializing…
viirya Sep 2, 2019
bd3915e
Revert "[SPARK-28612][SQL] Add DataFrameWriterV2 API"
HyukjinKwon Sep 2, 2019
e1946a5
[SPARK-28705][SQL][TEST] Drop tables after being used in AnalysisExte…
sandeep-katta Sep 2, 2019
560df0e
[SPARK-28951][INFRA] Add release announce template
dongjoon-hyun Sep 2, 2019
35d4edf
[SPARK-28921][BUILD][K8S] Upgrade kubernetes client to 4.4.2
andygrove Sep 2, 2019
d5688dc
[SPARK-28573][SQL] Convert InsertIntoTable(HiveTableRelation) to Data…
advancedxy Sep 3, 2019
5cf2602
[SPARK-28946][R][DOCS] Add some more information about building Spark…
HyukjinKwon Sep 3, 2019
92ae271
[SPARK-28806][DOCS][SQL] Document SHOW COLUMNS in SQL Reference
dilipbiswal Sep 3, 2019
94e6674
[SPARK-28805][DOCS][SQL] Document DESCRIBE FUNCTION in SQL Reference
dilipbiswal Sep 3, 2019
2856398
[SPARK-28961][HOT-FIX][BUILD] Upgrade Maven from 3.6.1 to 3.6.2
gatorsmile Sep 3, 2019
5ea134c
[SPARK-28628][SQL] Implement SupportsNamespaces in V2SessionCatalog
rdblue Sep 3, 2019
8980093
[SPARK-3137][CORE] Replace the global TorrentBroadcast lock with fine…
zsxwing Sep 3, 2019
56f2887
[SPARK-28788][DOC][SQL] Document ANALYZE TABLE statement in SQL Refer…
huaxingao Sep 3, 2019
4c8f114
[SPARK-27489][WEBUI] UI updates to show executor resource information
tgravescs Sep 4, 2019
a838699
[SPARK-28694][EXAMPLES] Add Java/Scala StructuredKerberizedKafkaWordC…
hddong Sep 4, 2019
df39855
[SPARK-28963][BUILD] Fall back to archive.apache.org in build/mvn for…
srowen Sep 4, 2019
a07f795
[SPARK-28577][YARN] Resource capability requested for each executor a…
LuciferYang Sep 4, 2019
9f478a6
[SPARK-28901][SQL] SparkThriftServer's Cancel SQL Operation show it i…
AngersZhuuuu Sep 4, 2019
712874f
[SPARK-28931][CORE][TESTS] Fix couple of bugs in FsHistoryProviderSuite
HeartSaVioR Sep 4, 2019
594c9c5
[SPARK-25151][SS] Apply Apache Commons Pool to KafkaDataConsumer
HeartSaVioR Sep 4, 2019
b992160
[SPARK-28811][DOCS][SQL] Document SHOW TBLPROPERTIES in SQL Reference
dilipbiswal Sep 4, 2019
f96486b
[SPARK-28808][DOCS][SQL] Document SHOW FUNCTIONS in SQL Reference
dilipbiswal Sep 4, 2019
a7a3935
[SPARK-11150][SQL] Dynamic Partition Pruning
maryannxue Sep 4, 2019
ded23f8
[SPARK-28921][K8S][FOLLOWUP] Also bump K8S client version in integrat…
srowen Sep 5, 2019
ca71177
[SPARK-28907][CORE] Review invalid usage of new Configuration()
advancedxy Sep 5, 2019
5adaa2e
[SPARK-28979][SQL] Rename UnresovledTable to V1Table
rdblue Sep 5, 2019
e4f7002
[SPARK-28830][DOC][SQL] Document UNCACHE TABLE statement in SQL Refer…
huaxingao Sep 5, 2019
84a4d3a
[SPARK-28976][CORE] Use KeyLock to simplify MapOutputTracker.getStatuses
zsxwing Sep 5, 2019
b9edd44
[SPARK-28964] Add the provider information to the table properties in…
brkyvz Sep 5, 2019
dde3931
[SPARK-28878][SQL] Remove extra project for DSv2 reads with columnar …
rdblue Sep 5, 2019
f8bc91f
[SPARK-28782][SQL] Generator support in aggregate expressions
WeichenXu123 Sep 5, 2019
36559b6
[SPARK-28977][DOCS][SQL] Fix DataFrameReader.json docs to doc that pa…
srowen Sep 5, 2019
be04c97
[SPARK-28971][SQL][PYTHON][TESTS] Convert and port 'pgSQL/aggregates_…
HyukjinKwon Sep 5, 2019
103d50b
[SPARK-28272][SQL][PYTHON][TESTS] Convert and port 'pgSQL/aggregates_…
HyukjinKwon Sep 5, 2019
c81fd0c
[SPARK-28974][SQL] centralize the Data Source V2 table capability checks
cloud-fan Sep 5, 2019
0647906
[SPARK-28910][SQL] Prevent schema verification when connecting to in …
bogdanghit Sep 5, 2019
151b954
[SPARK-28770][CORE][TEST] Fix ReplayListenerSuite tests that sometime…
wypoon Sep 5, 2019
3929d16
[SPARK-26046][SS] Add StreamingQueryManager.listListeners()
mukulmurthy Sep 5, 2019
36f8e53
[SPARK-28802][DOC][SQL] Document DESCRIBE DATABASE statement in SQL R…
kevinyu98 Sep 5, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
44 changes: 38 additions & 6 deletions .github/PULL_REQUEST_TEMPLATE
Original file line number Diff line number Diff line change
@@ -1,10 +1,42 @@
## What changes were proposed in this pull request?
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
-->

(Please fill in changes proposed in this fix)
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->

## How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->

Please review http://spark.apache.org/contributing.html before opening a pull request.

### Does this PR introduce any user-facing change?
<!--
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If no, write 'No'.
-->


### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
27 changes: 27 additions & 0 deletions .github/workflows/master.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
name: master

on:
push:
branches:
- master

jobs:
build:

runs-on: ubuntu-latest
strategy:
matrix:
java: [ '1.8', '11' ]
name: Build Spark with JDK ${{ matrix.java }}

steps:
- uses: actions/checkout@master
- name: Set up JDK ${{ matrix.java }}
uses: actions/setup-java@v1
with:
version: ${{ matrix.java }}
- name: Build with Maven
run: |
export MAVEN_OPTS="-Xmx2g -XX:ReservedCodeCacheSize=512m -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
export MAVEN_CLI_OPTS="--no-transfer-progress"
./build/mvn $MAVEN_CLI_OPTS -DskipTests -Pyarn -Pmesos -Pkubernetes -Phive -Phive-thriftserver -Phadoop-3.2 -Phadoop-cloud -Djava.version=${{ matrix.java }} package
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ project/plugins/project/build.properties
project/plugins/src_managed/
project/plugins/target/
python/lib/pyspark.zip
python/.eggs/
python/deps
python/test_coverage/coverage_data
python/test_coverage/htmlcov
Expand All @@ -70,14 +71,14 @@ scalastyle-on-compile.generated.xml
scalastyle-output.xml
scalastyle.txt
spark-*-bin-*.tgz
spark-resources/
spark-tests.log
src_managed/
streaming-tests.log
target/
unit-tests.log
work/
docs/.jekyll-metadata
*.crc

# For Hive
TempStatsStore/
Expand All @@ -95,3 +96,6 @@ spark-warehouse/
*.Rproj.*

.Rproj.user

# For SBT
.jvmopts
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
## Contributing to Spark

*Before opening a pull request*, review the
[Contributing to Spark guide](http://spark.apache.org/contributing.html).
[Contributing to Spark guide](https://spark.apache.org/contributing.html).
It lists steps that are required before creating a PR. In particular, consider:

- Is the change important and ready enough to ask the community to spend time reviewing?
- Have you searched for existing, related JIRAs and pull requests?
- Is this a new feature that can stand alone as a [third party project](http://spark.apache.org/third-party-projects.html) ?
- Is this a new feature that can stand alone as a [third party project](https://spark.apache.org/third-party-projects.html) ?
- Is the change being proposed clearly explained and motivated?

When you contribute code, you affirm that the contribution is your original work and that you
Expand Down
4 changes: 2 additions & 2 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -222,7 +222,7 @@ Python Software Foundation License
----------------------------------

pyspark/heapq3.py

python/docs/_static/copybutton.js

BSD 3-Clause
------------
Expand Down Expand Up @@ -258,4 +258,4 @@ data/mllib/images/kittens/29.5.a_b_EGDP022204.jpg
data/mllib/images/kittens/54893.jpg
data/mllib/images/kittens/DP153539.jpg
data/mllib/images/kittens/DP802813.jpg
data/mllib/images/multi-channel/chr30.4.184.jpg
data/mllib/images/multi-channel/chr30.4.184.jpg
73 changes: 41 additions & 32 deletions LICENSE-binary
Original file line number Diff line number Diff line change
Expand Up @@ -209,34 +209,34 @@ org.apache.zookeeper:zookeeper
oro:oro
commons-configuration:commons-configuration
commons-digester:commons-digester
com.chuusai:shapeless_2.11
com.chuusai:shapeless_2.12
com.googlecode.javaewah:JavaEWAH
com.twitter:chill-java
com.twitter:chill_2.11
com.twitter:chill_2.12
com.univocity:univocity-parsers
javax.jdo:jdo-api
joda-time:joda-time
net.sf.opencsv:opencsv
org.apache.derby:derby
org.objenesis:objenesis
org.roaringbitmap:RoaringBitmap
org.scalanlp:breeze-macros_2.11
org.scalanlp:breeze_2.11
org.typelevel:macro-compat_2.11
org.scalanlp:breeze-macros_2.12
org.scalanlp:breeze_2.12
org.typelevel:macro-compat_2.12
org.yaml:snakeyaml
org.apache.xbean:xbean-asm5-shaded
com.squareup.okhttp3:logging-interceptor
com.squareup.okhttp3:okhttp
com.squareup.okio:okio
org.apache.spark:spark-catalyst_2.11
org.apache.spark:spark-kvstore_2.11
org.apache.spark:spark-launcher_2.11
org.apache.spark:spark-mllib-local_2.11
org.apache.spark:spark-network-common_2.11
org.apache.spark:spark-network-shuffle_2.11
org.apache.spark:spark-sketch_2.11
org.apache.spark:spark-tags_2.11
org.apache.spark:spark-unsafe_2.11
org.apache.spark:spark-catalyst_2.12
org.apache.spark:spark-kvstore_2.12
org.apache.spark:spark-launcher_2.12
org.apache.spark:spark-mllib-local_2.12
org.apache.spark:spark-network-common_2.12
org.apache.spark:spark-network-shuffle_2.12
org.apache.spark:spark-sketch_2.12
org.apache.spark:spark-tags_2.12
org.apache.spark:spark-unsafe_2.12
commons-httpclient:commons-httpclient
com.vlkan:flatbuffers
com.ning:compress-lzf
Expand All @@ -253,16 +253,12 @@ commons-codec:commons-codec
commons-collections:commons-collections
io.fabric8:kubernetes-client
io.fabric8:kubernetes-model
io.netty:netty
io.netty:netty-all
net.hydromatic:eigenbase-properties
net.sf.supercsv:super-csv
org.apache.arrow:arrow-format
org.apache.arrow:arrow-memory
org.apache.arrow:arrow-vector
org.apache.calcite:calcite-avatica
org.apache.calcite:calcite-core
org.apache.calcite:calcite-linq4j
org.apache.commons:commons-crypto
org.apache.commons:commons-lang3
org.apache.hadoop:hadoop-annotations
Expand All @@ -287,25 +283,24 @@ org.apache.orc:orc-mapreduce
org.mortbay.jetty:jetty
org.mortbay.jetty:jetty-util
com.jolbox:bonecp
org.json4s:json4s-ast_2.11
org.json4s:json4s-core_2.11
org.json4s:json4s-jackson_2.11
org.json4s:json4s-scalap_2.11
org.json4s:json4s-ast_2.12
org.json4s:json4s-core_2.12
org.json4s:json4s-jackson_2.12
org.json4s:json4s-scalap_2.12
com.carrotsearch:hppc
com.fasterxml.jackson.core:jackson-annotations
com.fasterxml.jackson.core:jackson-core
com.fasterxml.jackson.core:jackson-databind
com.fasterxml.jackson.dataformat:jackson-dataformat-yaml
com.fasterxml.jackson.module:jackson-module-jaxb-annotations
com.fasterxml.jackson.module:jackson-module-paranamer
com.fasterxml.jackson.module:jackson-module-scala_2.11
com.fasterxml.jackson.module:jackson-module-scala_2.12
com.github.mifmif:generex
com.google.code.findbugs:jsr305
com.google.code.gson:gson
com.google.inject:guice
com.google.inject.extensions:guice-servlet
com.twitter:parquet-hadoop-bundle
commons-beanutils:commons-beanutils-core
commons-cli:commons-cli
commons-dbcp:commons-dbcp
commons-io:commons-io
Expand Down Expand Up @@ -372,6 +367,8 @@ org.eclipse.jetty:jetty-servlets
org.eclipse.jetty:jetty-util
org.eclipse.jetty:jetty-webapp
org.eclipse.jetty:jetty-xml
org.scala-lang.modules:scala-xml_2.12
org.opencypher:okapi-shade

core/src/main/java/org/apache/spark/util/collection/TimSort.java
core/src/main/resources/org/apache/spark/ui/static/bootstrap*
Expand Down Expand Up @@ -415,8 +412,7 @@ com.thoughtworks.paranamer:paranamer
org.scala-lang:scala-compiler
org.scala-lang:scala-library
org.scala-lang:scala-reflect
org.scala-lang.modules:scala-parser-combinators_2.11
org.scala-lang.modules:scala-xml_2.11
org.scala-lang.modules:scala-parser-combinators_2.12
org.fusesource.leveldbjni:leveldbjni-all
net.sourceforge.f2j:arpack_combined_all
xmlenc:xmlenc
Expand All @@ -437,15 +433,15 @@ is distributed under the 3-Clause BSD license.
MIT License
-----------

org.spire-math:spire-macros_2.11
org.spire-math:spire_2.11
org.typelevel:machinist_2.11
org.spire-math:spire-macros_2.12
org.spire-math:spire_2.12
org.typelevel:machinist_2.12
net.razorvine:pyrolite
org.slf4j:jcl-over-slf4j
org.slf4j:jul-to-slf4j
org.slf4j:slf4j-api
org.slf4j:slf4j-log4j12
com.github.scopt:scopt_2.11
com.github.scopt:scopt_2.12

core/src/main/resources/org/apache/spark/ui/static/dagre-d3.min.js
core/src/main/resources/org/apache/spark/ui/static/*dataTables*
Expand All @@ -467,10 +463,9 @@ javax.xml.stream:stax-api https://jcp.org/en/jsr/detail?id=173
Common Development and Distribution License (CDDL) 1.1
------------------------------------------------------

javax.annotation:javax.annotation-api https://jcp.org/en/jsr/detail?id=250
javax.el:javax.el-api https://javaee.github.io/uel-ri/
javax.servlet:javax.servlet-api https://javaee.github.io/servlet-spec/
javax.transaction:jta http://www.oracle.com/technetwork/java/index.html
javax.ws.rs:javax.ws.rs-api https://github.com/jax-rs
javax.xml.bind:jaxb-api https://github.com/javaee/jaxb-v2
org.glassfish.hk2:hk2-api https://github.com/javaee/glassfish
org.glassfish.hk2:hk2-locator (same)
Expand All @@ -487,6 +482,20 @@ org.glassfish.jersey.core:jersey-server
org.glassfish.jersey.media:jersey-media-jaxb


Eclipse Distribution License (EDL) 1.0
--------------------------------------

org.glassfish.jaxb:jaxb-runtime
jakarta.xml.bind:jakarta.xml.bind-api
com.sun.istack:istack-commons-runtime


Eclipse Public License (EPL) 2.0
--------------------------------

jakarta.annotation:jakarta-annotation-api https://projects.eclipse.org/projects/ee4j.ca
jakarta.ws.rs:jakarta.ws.rs-api https://github.com/eclipse-ee4j/jaxrs-api

Mozilla Public License (MPL) 1.1
--------------------------------

Expand Down
Loading