Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7069 commits
Select commit Hold shift + click to select a range
e2c7bfc
[SPARK-33407][PYTHON] Simplify the exception message from Python UDFs…
HyukjinKwon Nov 17, 2020
09bb9be
[SPARK-33416][SQL] Avoid Hive metastore stack overflow when InSet pre…
wangyum Nov 17, 2020
9283484
[SPARK-33427][SQL] Add subexpression elimination for interpreted expr…
viirya Nov 17, 2020
2a8e253
[SPARK-32222][K8S][TESTS] Add K8s IT for conf propagation
ScrapCodes Nov 17, 2020
5e85499
[SPARK-33471][K8S][BUILD] Upgrade kubernetes-client to 4.12.0
ramesh-muthusamy Nov 17, 2020
7f3d99a
[MINOR][SQL][DOCS] Update schema_of_csv and schema_of_json doc
viirya Nov 18, 2020
dd32f45
[SPARK-31069][CORE] Avoid repeat compute `chunksBeingTransferred` cau…
AngersZhuuuu Nov 18, 2020
8e2a0bd
[SPARK-24554][PYTHON][SQL] Add MapType support for PySpark with Arrow
BryanCutler Nov 18, 2020
74bd046
[SPARK-33475][BUILD] Bump ANTLR runtime version to 4.8-1
maropu Nov 18, 2020
a180e02
[SPARK-32852][SQL][DOC][FOLLOWUP] Revise the documentation of spark.s…
gengliangwang Nov 18, 2020
689c294
[SPARK-32907][ML][PYTHON] Adaptively blockify instances - AFT,LiR,LoR
zhengruifeng Nov 18, 2020
594c7c6
[SPARK-33476][CORE] Generalize ExecutorSource to expose user-given fi…
dongjoon-hyun Nov 18, 2020
dcac78e
[SPARK-27936][K8S] Support python deps
Nov 18, 2020
27cd945
[SPARK-32381][CORE][SQL][FOLLOWUP] More cleanup on HadoopFSUtils
sunchao Nov 18, 2020
1df69f7
[SPARK-31255][SQL] Add SupportsMetadataColumns to DSv2
rdblue Nov 18, 2020
fbfc0bf
[SPARK-33464][INFRA] Add/remove (un)necessary cache and restructure G…
HyukjinKwon Nov 18, 2020
9a4c790
[SPARK-33354][SQL] New explicit cast syntax rules in ANSI mode
gengliangwang Nov 19, 2020
56a8510
[SPARK-33304][R][SQL] Add from_avro and to_avro functions to SparkR
zero323 Nov 19, 2020
4267ca9
[SPARK-33479][DOC] Make the API Key of DocSearch configurable
gengliangwang Nov 19, 2020
97d2cee
[SPARK-33427][SQL][FOLLOWUP] Prevent test flakyness in SubExprEvaluat…
viirya Nov 19, 2020
e518008
[SPARK-33473][SQL] Extend interpreted subexpression elimination to ot…
viirya Nov 19, 2020
66a7637
[SPARK-31255][SQL][FOLLOWUP] Add missing license headers
rdblue Nov 19, 2020
e3058ba
[SPARK-33441][BUILD] Add unused-imports compilation check and remove …
LuciferYang Nov 19, 2020
014e1fb
[SPARK-27421][SQL] Fix filter for int column and value class java.lan…
wangyum Nov 19, 2020
0b0fb70
[SPARK-33400][SQL] Normalize sameOrderExpressions in SortOrder to avo…
prakharjain09 Nov 19, 2020
d5e7bd0
[SPARK-33483][INFRA][TESTS] Fix rat exclusion patterns and add a LICENSE
dongjoon-hyun Nov 19, 2020
ef2638c
[SPARK-33183][SQL][FOLLOW-UP] Update rule RemoveRedundantSorts config…
allisonwang-db Nov 19, 2020
a03c540
[SPARK-33472][SQL] Adjust RemoveRedundantSorts rule order
allisonwang-db Nov 19, 2020
21b1350
[SPARK-33442][SQL] Change Combine Limit to Eliminate limit using max row
ulysses-you Nov 19, 2020
3695e99
[SPARK-33045][SQL] Support build-in function like_all and fix StackOv…
beliefer Nov 19, 2020
6da8ade
[SPARK-33045][SQL][FOLLOWUP] Fix build failure with Scala 2.13
sunchao Nov 19, 2020
883a213
[MINOR] Structured Streaming statistics page indent fix
gaborgsomogyi Nov 19, 2020
02d410a
[MINOR][DOCS] Document 'without' value for HADOOP_VERSION in pip inst…
HyukjinKwon Nov 20, 2020
8218b48
[SPARK-32919][SHUFFLE][TEST-MAVEN][TEST-HADOOP2.7] Driver side change…
venkata91 Nov 20, 2020
2289389
[SPARK-33441][BUILD][FOLLOWUP] Make unused-imports check for SBT spec…
LuciferYang Nov 20, 2020
870d409
[SPARK-32512][SQL][TESTS][FOLLOWUP] Remove duplicate tests for ALTER …
MaxGekk Nov 20, 2020
cbc8be2
[SPARK-33422][DOC] Fix the correct display of left menu item
liucht-inspur Nov 20, 2020
3384bda
[SPARK-33468][SQL] ParseUrl in ANSI mode should fail if input string…
ulysses-you Nov 20, 2020
47326ac
[SPARK-28704][SQL][TEST] Add back Skiped HiveExternalCatalogVersionsS…
AngersZhuuuu Nov 20, 2020
116b7b7
[SPARK-33466][ML][PYTHON] Imputer support mode(most_frequent) strategy
zhengruifeng Nov 20, 2020
a1a3d5c
[MINOR][TESTS][DOCS] Use fully-qualified class name in docker integra…
huaxingao Nov 20, 2020
2479778
[SPARK-33492][SQL] DSv2: Append/Overwrite/ReplaceTable should invalid…
sunchao Nov 20, 2020
de0f50a
[SPARK-32670][SQL] Group exception messages in Catalyst Analyzer in o…
anchovYu Nov 20, 2020
67c6ed9
[SPARK-33223][SS][FOLLOWUP] Clarify the meaning of "number of rows dr…
HeartSaVioR Nov 21, 2020
530c0a8
[SPARK-33505][SQL][TESTS] Fix adding new partitions by INSERT INTO `I…
MaxGekk Nov 21, 2020
b623c03
[SPARK-32381][CORE][FOLLOWUP][TEST-HADOOP2.7] Don't remove Serializab…
sunchao Nov 21, 2020
cf74901
Revert "[SPARK-28704][SQL][TEST] Add back Skiped HiveExternalCatalogV…
dongjoon-hyun Nov 21, 2020
517b810
[SPARK-33463][SQL] Keep Job Id during incremental collect in Spark Th…
gumartinm Nov 21, 2020
d7f4b2a
[SPARK-28704][SQL][TEST] Add back Skiped HiveExternalCatalogVersionsS…
AngersZhuuuu Nov 22, 2020
d338af3
[SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options w…
cchighman Nov 22, 2020
6d625cc
[SPARK-33469][SQL] Add current_timezone function
ulysses-you Nov 22, 2020
df4a1c2
[SPARK-33512][BUILD] Upgrade test libraries
dongjoon-hyun Nov 23, 2020
a459238
[MINOR][INFRA] Suppress warning in check-license
williamhyun Nov 23, 2020
aa78c05
[SPARK-33427][SQL][FOLLOWUP] Put key and value into IdentityHashMap s…
viirya Nov 23, 2020
0bb911d
[SPARK-33143][PYTHON] Add configurable timeout to python server and c…
gaborgsomogyi Nov 23, 2020
84e7036
[SPARK-33510][BUILD] Update SBT to 1.4.4
williamhyun Nov 23, 2020
c891e02
Revert "[SPARK-32481][CORE][SQL] Support truncate table to move data …
gatorsmile Nov 23, 2020
60f3a73
[SPARK-33515][SQL] Improve exception messages while handling Unresolv…
imback82 Nov 23, 2020
23e9920
[SPARK-33511][SQL] Respect case sensitivity while resolving V2 partit…
MaxGekk Nov 23, 2020
f83fcb1
[SPARK-33278][SQL][FOLLOWUP] Improve OptimizeWindowFunctions to avoid…
beliefer Nov 23, 2020
1bd897c
[SPARK-32918][SHUFFLE] RPC implementation to support control plane co…
zhouyejoe Nov 23, 2020
0592181
[SPARK-33479][DOC][FOLLOWUP] DocSearch: Support filtering search resu…
gengliangwang Nov 24, 2020
3ce4ab5
[SPARK-33513][BUILD] Upgrade to Scala 2.13.4 to improve exhaustivity
dongjoon-hyun Nov 24, 2020
8380e00
[SPARK-33524][SQL][TESTS] Change `InMemoryTable` not to use Tuple.has…
dongjoon-hyun Nov 24, 2020
f35e28f
[SPARK-33523][SQL][TEST] Add predicate related benchmark to SubExprEl…
viirya Nov 24, 2020
a6555ee
[SPARK-33521][SQL] Universal type conversion in resolving V2 partitio…
MaxGekk Nov 24, 2020
fdd6c73
[SPARK-33514][SQL] Migrate TRUNCATE TABLE command to use UnresolvedTa…
imback82 Nov 24, 2020
048a982
[SPARK-33535][INFRA][TESTS] Export LANG to en_US.UTF-8 in run-tests-j…
LuciferYang Nov 24, 2020
95b6dab
[SPARK-33287][SS][UI] Expose state custom metrics information on SS UI
gaborgsomogyi Nov 24, 2020
665817b
[SPARK-33457][PYTHON] Adjust mypy configuration
zero323 Nov 25, 2020
01321bc
[SPARK-33252][PYTHON][DOCS] Migration to NumPy documentation style in…
zero323 Nov 25, 2020
d1b4f06
[SPARK-33494][SQL][AQE] Do not use local shuffle reader for repartition
cloud-fan Nov 25, 2020
b7f034d
[SPARK-33543][SQL] Migrate SHOW COLUMNS command to use UnresolvedTabl…
imback82 Nov 25, 2020
edab094
[SPARK-33224][SS][WEBUI] Add watermark gap information into SS UI page
HeartSaVioR Nov 25, 2020
c3ce970
[SPARK-33533][SQL] Fix the regression bug that ConnectionProviders do…
sarutak Nov 25, 2020
781e19c
[SPARK-33477][SQL] Hive Metastore support filter by date type
wangyum Nov 25, 2020
19f3b89
[SPARK-33549][SQL] Remove configuration spark.sql.legacy.allowCastNum…
gengliangwang Nov 25, 2020
2c5cc36
[SPARK-33509][SQL] List partition by names from a V2 table which supp…
MaxGekk Nov 25, 2020
7c59aee
[SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic parti…
WinkerDu Nov 25, 2020
6f68ccf
[SPARK-31257][SPARK-33561][SQL] Unify create table syntax
rdblue Nov 25, 2020
d691d85
[SPARK-33496][SQL] Improve error message of ANSI explicit cast
gengliangwang Nov 25, 2020
9643eab
[SPARK-33540][SQL] Subexpression elimination for interpreted predicate
viirya Nov 25, 2020
7cf6a6f
[SPARK-31257][SPARK-33561][SQL][FOLLOWUP] Fix Scala 2.13 compilation
dongjoon-hyun Nov 25, 2020
1de3fc4
[SPARK-33525][SQL] Update hive-service-rpc to 3.1.2
wangyum Nov 25, 2020
c529426
[SPARK-33565][BUILD][PYTHON] remove python3.8 and fix breakage
shaneknapp Nov 25, 2020
fb7b870
[SPARK-33523][SQL][TEST][FOLLOWUP] Fix benchmark case name in SubExpr…
viirya Nov 25, 2020
919ea45
[SPARK-33562][UI] Improve the style of the checkbox in executor page
gengliangwang Nov 26, 2020
ed9e6fc
[SPARK-33565][INFRA][FOLLOW-UP] Keep the test coverage with Python 3.…
HyukjinKwon Nov 26, 2020
dfa3978
[SPARK-33551][SQL] Do not use custom shuffle reader for repartition
maryannxue Nov 26, 2020
d082ad0
[SPARK-33563][PYTHON][R][SQL] Expose inverse hyperbolic trig function…
zero323 Nov 27, 2020
433ae90
[SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling opti…
LuciferYang Nov 27, 2020
8792280
[SPARK-33575][SQL] Fix misleading exception for "ANALYZE TABLE ... FO…
imback82 Nov 27, 2020
2c41d9d
[SPARK-33522][SQL] Improve exception messages while handling Unresolv…
imback82 Nov 27, 2020
e432550
[SPARK-28645][SQL] ParseException is thrown when the window is redefined
beliefer Nov 27, 2020
b9f2f78
[SPARK-33498][SQL] Datetime parsing should fail if the input string c…
leanken-zz Nov 27, 2020
35ded12
[SPARK-33141][SQL] Capture SQL configs when creating permanent views
luluorta Nov 27, 2020
13fd272
Spelling r common dev mlib external project streaming resource manage…
jsoref Nov 27, 2020
cf98a76
[SPARK-33570][SQL][TESTS] Set the proper version of gssapi plugin aut…
sarutak Nov 28, 2020
3650a6b
[SPARK-33580][CORE] resolveDependencyPaths should use classifier attr…
viirya Nov 28, 2020
bfe9380
[MINOR][SQL] Remove `getTables()` from `r.SQLUtils`
MaxGekk Nov 29, 2020
ba178f8
[SPARK-33581][SQL][TEST] Refactor HivePartitionFilteringSuite
wangyum Nov 29, 2020
b94ff1e
[SPARK-33590][DOCS][SQL] Add missing sub-bullets in Spark SQL Guide
kiszk Nov 29, 2020
c8286ec
[SPARK-33587][CORE] Kill the executor on nested fatal errors
zsxwing Nov 29, 2020
0054fc9
[SPARK-33588][SQL] Respect the `spark.sql.caseSensitive` config while…
MaxGekk Nov 29, 2020
a088a80
[SPARK-33585][SQL][DOCS] Fix the comment for `SQLContext.tables()` an…
MaxGekk Nov 29, 2020
3d54774
[SPARK-33517][SQL][DOCS] Fix the correct menu items and page links in…
liucht-inspur Nov 30, 2020
f93d439
[SPARK-33589][SQL] Close opened session if the initialization fails
wangyum Nov 30, 2020
a5e13ac
[SPARK-33582][SQL] Hive Metastore support filter by not-equals
wangyum Nov 30, 2020
feda729
[SPARK-33567][SQL] DSv2: Use callback instead of passing Spark sessio…
sunchao Nov 30, 2020
4851453
[MINOR] Spelling bin core docs external mllib repl
jsoref Nov 30, 2020
2da7259
[SPARK-32976][SQL] Support column list in INSERT statement
yaooqinn Nov 30, 2020
0fd9f57
[SPARK-33448][SQL] Support CACHE/UNCACHE TABLE commands for v2 tables
imback82 Nov 30, 2020
225c2e2
[SPARK-33498][SQL][FOLLOW-UP] Deduplicate the unittest by using check…
leanken-zz Nov 30, 2020
b665d58
[SPARK-28646][SQL] Fix bug of Count so as consistent with mainstream …
beliefer Nov 30, 2020
5cfbddd
[SPARK-33480][SQL] Support char/varchar type
cloud-fan Nov 30, 2020
6e5446e
[SPARK-33579][UI] Fix executor blank page behind proxy
Nov 30, 2020
0a612b6
[SPARK-33452][SQL] Support v2 SHOW PARTITIONS
MaxGekk Nov 30, 2020
6fd148f
[SPARK-33569][SQL] Remove getting partitions by an identifier prefix
MaxGekk Nov 30, 2020
030b313
[SPARK-33569][SPARK-33452][SQL][FOLLOWUP] Fix a build error in `ShowP…
MaxGekk Nov 30, 2020
f3c2583
[SPARK-33185][YARN][FOLLOW-ON] Leverage RM's RPC API instead of REST …
xkrogen Nov 30, 2020
c699435
[SPARK-33545][CORE] Support Fallback Storage during Worker decommission
dongjoon-hyun Nov 30, 2020
f5d2165
[SPARK-33440][CORE] Use current timestamp with warning log in HadoopF…
HeartSaVioR Nov 30, 2020
596fbc1
[SPARK-33556][ML] Add array_to_vector function for dataframe column
WeichenXu123 Dec 1, 2020
aeb3649
[SPARK-33613][PYTHON][TESTS] Replace deprecated APIs in pyspark tests
BryanCutler Dec 1, 2020
8016123
[SPARK-33592] Fix: Pyspark ML Validator params in estimatorParamMaps …
WeichenXu123 Dec 1, 2020
c50fcac
[SPARK-33607][SS][WEBUI] Input Rate timeline/histogram aren't rendere…
sarutak Dec 1, 2020
2af2da5
[SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata lo…
HeartSaVioR Dec 1, 2020
1a042cc
[SPARK-33530][CORE] Support --archives and spark.archives option nati…
HyukjinKwon Dec 1, 2020
52e5cc4
[SPARK-27188][SS] FileStreamSink: provide a new option to have retent…
HeartSaVioR Dec 1, 2020
1034815
[SPARK-33572][SQL] Datetime building should fail if the year, month, …
waitinfuture Dec 1, 2020
e5bb293
[SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsu…
gaborgsomogyi Dec 1, 2020
d38883c
[SPARK-32405][SQL][FOLLOWUP] Throw Exception if provider is specified…
huaxingao Dec 1, 2020
9273d42
[SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and f…
beliefer Dec 1, 2020
cf4ad21
[SPARK-33503][SQL] Refactor SortOrder class to allow multiple childrens
prakharjain09 Dec 1, 2020
478fb7f
[SPARK-33608][SQL] Handle DELETE/UPDATE/MERGE in PullupCorrelatedPred…
aokolnychyi Dec 1, 2020
c24f2b2
[SPARK-33612][SQL] Add dataSourceRewriteRules batch to Optimizer
aokolnychyi Dec 1, 2020
5d0045e
[SPARK-33611][UI] Avoid encoding twice on the query parameter of rewr…
gengliangwang Dec 1, 2020
5a1c5ac
[SPARK-33622][R][ML] Add array_to_vector to SparkR
zero323 Dec 1, 2020
f71f345
[SPARK-33544][SQL] Optimize size of CreateArray/CreateMap to be the s…
tgravescs Dec 2, 2020
51ebcd9
[SPARK-32863][SS] Full outer stream-stream join
c21 Dec 2, 2020
a4788ee
[MINOR][SS] Rename auxiliary protected methods in StreamingJoinSuite
c21 Dec 2, 2020
290aa02
[SPARK-33618][CORE] Use hadoop-client instead of hadoop-client-api to…
dongjoon-hyun Dec 2, 2020
084d38b
[SPARK-33557][CORE][MESOS][TEST] Ensure the relationship between STOR…
LuciferYang Dec 2, 2020
28dad1b
[SPARK-33504][CORE] The application log in the Spark history server c…
echohlne Dec 2, 2020
df8d3f1
[SPARK-33544][SQL][FOLLOW-UP] Rename NoSideEffect to NoThrow and clar…
HyukjinKwon Dec 2, 2020
58583f7
[SPARK-33619][SQL] Fix GetMapValueUtil code generation error
leanken-zz Dec 2, 2020
91182d6
[SPARK-33626][K8S][TEST] Allow k8s integration tests to assert both d…
ScrapCodes Dec 2, 2020
a082f46
[SPARK-33071][SPARK-33536][SQL] Avoid changing dataset_id of LogicalP…
Ngone51 Dec 2, 2020
b76c6b7
[SPARK-33627][SQL] Add new function UNIX_SECONDS, UNIX_MILLIS and UNI…
gengliangwang Dec 2, 2020
92bfbcb
[SPARK-33631][DOCS][TEST] Clean up spark.core.connection.ack.wait.tim…
LuciferYang Dec 2, 2020
f94cb53
[MINOR][INFRA] Use the latest image for GitHub Action jobs
dongjoon-hyun Dec 3, 2020
4f96670
[SPARK-31953][SS] Add Spark Structured Streaming History Server Support
uncleGen Dec 3, 2020
90d4d7d
[SPARK-33610][ML] Imputer transform skip duplicate head() job
zhengruifeng Dec 3, 2020
878cc0e
[SPARK-32896][SS][FOLLOW-UP] Rename the API to `toTable`
xuanyuanking Dec 3, 2020
0880989
[SPARK-22798][PYTHON][ML][FOLLOWUP] Add labelsArray to PySpark String…
viirya Dec 3, 2020
3b2ff16
[SPARK-33636][PYTHON][ML][FOLLOWUP] Update since tag of labelsArray i…
viirya Dec 3, 2020
ff13f57
[SPARK-20044][SQL] Add new function DATE_FROM_UNIX_DATE and UNIX_DATE
gengliangwang Dec 3, 2020
512fb32
[SPARK-26218][SQL][FOLLOW UP] Fix the corner case of codegen when cas…
luluorta Dec 3, 2020
0706e64
[SPARK-30098][SQL] Add a configuration to use default datasource as p…
cloud-fan Dec 3, 2020
bd71186
[SPARK-33629][PYTHON] Make spark.buffer.size configuration visible on…
gaborgsomogyi Dec 3, 2020
aa13e20
[SPARK-33623][SQL] Add canDeleteWhere to SupportsDelete
aokolnychyi Dec 3, 2020
63f9d47
[SPARK-33634][SQL][TESTS] Use Analyzer in PlanResolutionSuite
cloud-fan Dec 3, 2020
7e759b2
[SPARK-33520][ML][PYSPARK] make CrossValidator/TrainValidateSplit/One…
WeichenXu123 Dec 4, 2020
8594958
[SPARK-33650][SQL] Fix the error from ALTER TABLE .. ADD/DROP PARTITI…
MaxGekk Dec 4, 2020
29e415d
[SPARK-33649][SQL][DOC] Improve the doc of spark.sql.ansi.enabled
gengliangwang Dec 4, 2020
e22ddb6
[SPARK-32405][SQL][FOLLOWUP] Remove USING _ in CREATE TABLE in JDBCTa…
huaxingao Dec 4, 2020
e02324f
[SPARK-33142][SPARK-33647][SQL] Store SQL text for SQL temp view
linhongliu-db Dec 4, 2020
15579ba
[SPARK-33430][SQL] Support namespaces in JDBC v2 Table Catalog
huaxingao Dec 4, 2020
e838066
[SPARK-33658][SQL] Suggest using Datetime conversion functions for in…
gengliangwang Dec 4, 2020
94c144b
[SPARK-33571][SQL][DOCS] Add a ref to INT96 config from the doc for `…
MaxGekk Dec 4, 2020
325abf7
[SPARK-33577][SS] Add support for V1Table in stream writer table API …
xuanyuanking Dec 4, 2020
91baab7
[SPARK-33656][TESTS] Add option to keep container after tests finish …
sarutak Dec 4, 2020
976e897
[SPARK-33640][TESTS] Extend connection timeout to DB server for DB2In…
sarutak Dec 4, 2020
233a849
[SPARK-27237][SS] Introduce State schema validation among query restart
HeartSaVioR Dec 4, 2020
990bee9
[SPARK-33615][K8S] Make 'spark.archives' working in Kubernates
HyukjinKwon Dec 4, 2020
acc211d
[SPARK-33141][SQL][FOLLOW-UP] Store the max nested view depth in Anal…
cloud-fan Dec 4, 2020
d671e05
[SPARK-33660][DOCS][SS] Fix Kafka Headers Documentation
Gschiavon Dec 4, 2020
de9818f
[SPARK-33662][BUILD] Setting version to 3.2.0-SNAPSHOT
dongjoon-hyun Dec 4, 2020
b6b45bc
[SPARK-33141][SQL][FOLLOW-UP] Fix Scala 2.13 compilation
dongjoon-hyun Dec 4, 2020
960d6af
[SPARK-33472][SQL][FOLLOW-UP] Update RemoveRedundantSorts comment
allisonwang-db Dec 4, 2020
1b4e35d
[SPARK-33651][SQL] Allow CREATE EXTERNAL TABLE with LOCATION for data…
cloud-fan Dec 5, 2020
154f604
[MINOR] Fix string interpolation in CommandUtils.scala and KafkaDataC…
imback82 Dec 6, 2020
6317ba2
[SPARK-33668][K8S][TEST] Fix flaky test "Verify logging configuration…
ScrapCodes Dec 6, 2020
e857e06
[SPARK-33652][SQL] DSv2: DeleteFrom should refresh cache
sunchao Dec 6, 2020
5250841
[SPARK-33256][PYTHON][DOCS] Clarify PySpark follows NumPy documentati…
HyukjinKwon Dec 6, 2020
4829781
[SPARK-33667][SQL] Respect the `spark.sql.caseSensitive` config while…
MaxGekk Dec 6, 2020
b94ecf0
[SPARK-33674][TEST] Show Slowpoke notifications in SBT tests
gatorsmile Dec 6, 2020
119539f
[SPARK-33663][SQL] Uncaching should not be called on non-existing tem…
imback82 Dec 7, 2020
e32de29
[SPARK-33675][INFRA] Add GitHub Action job to publish snapshot
dongjoon-hyun Dec 7, 2020
29096a8
[SPARK-33670][SQL] Verify the partition provider is Hive in v1 SHOW T…
MaxGekk Dec 7, 2020
e88f0d4
[SPARK-33683][INFRA] Remove -Djava.version=11 from Scala 2.13 build i…
sarutak Dec 7, 2020
73412ff
[SPARK-33680][SQL][TESTS] Fix PrunePartitionSuiteBase/BucketedReadWit…
dongjoon-hyun Dec 7, 2020
d48ef34
[SPARK-33684][BUILD] Upgrade httpclient from 4.5.6 to 4.5.13
sarutak Dec 7, 2020
87c0560
[SPARK-33671][SQL] Remove VIEW checks from V1 table commands
MaxGekk Dec 7, 2020
26c0493
[SPARK-33676][SQL] Require exact matching of partition spec to the sc…
MaxGekk Dec 7, 2020
1e0c006
[SPARK-33617][SQL] Add default parallelism configuration for Spark SQ…
wangyum Dec 7, 2020
d730b6b
[SPARK-32680][SQL] Don't Preprocess V2 CTAS with Unresolved Query
linhongliu-db Dec 7, 2020
da72b87
[SPARK-33641][SQL] Invalidate new char/varchar types in public APIs t…
yaooqinn Dec 7, 2020
c62b84a
[MINOR] Spelling sql not core
jsoref Dec 7, 2020
6aff215
[SPARK-33693][SQL] deprecate spark.sql.hive.convertCTAS
cloud-fan Dec 7, 2020
c0874ba
[SPARK-33480][SQL][FOLLOWUP] do not expose user data in error message
cloud-fan Dec 7, 2020
02508b6
[SPARK-33621][SQL] Add a way to inject data source rewrite rules
aokolnychyi Dec 7, 2020
e4d1c10
[SPARK-32320][PYSPARK] Remove mutable default arguments
Fokko Dec 8, 2020
b2a7930
[SPARK-33680][SQL][TESTS][FOLLOWUP] Fix more test suites to have expl…
dongjoon-hyun Dec 8, 2020
ebd8b93
[SPARK-33609][ML] word2vec reduce broadcast size
zhengruifeng Dec 8, 2020
8bcebfa
[SPARK-33698][BUILD][TESTS] Fix the build error of OracleIntegrationS…
sarutak Dec 8, 2020
5aefc49
[SPARK-33664][SQL] Migrate ALTER TABLE ... RENAME TO to use Unresolve…
imback82 Dec 8, 2020
3a6546d
[MINOR][INFRA] Add -Pdocker-integration-tests to GitHub Action Scala …
dongjoon-hyun Dec 8, 2020
031c5ef
[SPARK-33679][SQL] Enable spark.sql.adaptive.enabled by default
dongjoon-hyun Dec 8, 2020
99613cd
[SPARK-33677][SQL] Skip LikeSimplification rule if pattern contains a…
luluorta Dec 8, 2020
2b30dde
[SPARK-33688][SQL] Migrate SHOW TABLE EXTENDED to new resolution fram…
MaxGekk Dec 8, 2020
c05ee06
[SPARK-33685][SQL] Migrate DROP VIEW command to use UnresolvedView to…
imback82 Dec 8, 2020
a093d6f
[MINOR] Spelling sql/core
jsoref Dec 8, 2020
c001dd4
[SPARK-33675][INFRA][FOLLOWUP] Schedule branch-3.1 snapshot at master…
dongjoon-hyun Dec 8, 2020
6fd2345
[SPARK-32110][SQL] normalize special floating numbers in HyperLogLog++
cloud-fan Dec 8, 2020
3ac70f1
[SPARK-33695][BUILD] Upgrade to jackson to 2.10.5 and jackson-databin…
n-marion Dec 8, 2020
f021f6d
[MINOR][ML] Increase Bounded MLOR (without regularization) test error…
WeichenXu123 Dec 9, 2020
29fed23
[SPARK-33703][SQL] Migrate MSCK REPAIR TABLE to use UnresolvedTable t…
imback82 Dec 9, 2020
c88edda
[SPARK-33641][SQL][DOC][FOLLOW-UP] Add migration guide for CHAR VARCH…
yaooqinn Dec 9, 2020
48f93af
[SPARK-33669] Wrong error message from YARN application state monitor…
Dec 9, 2020
a713a7e
[SPARK-33655][SQL] Improve performance of processing FETCH_PRIOR
Dec 9, 2020
9959d49
[SPARK-33719][DOC] Add make_date/make_timestamp/make_interval into th…
gengliangwang Dec 9, 2020
b5399d4
[SPARK-33071][SPARK-33536][SQL][FOLLOW-UP] Rename deniedMetadataKeys …
HyukjinKwon Dec 9, 2020
fa9ce1d
[SPARK-33722][SQL] Handle DELETE in ReplaceNullWithFalseInPredicate
aokolnychyi Dec 9, 2020
667f64f
[SPARK-33725][BUILD] Upgrade snappy-java to 1.1.8.2
viirya Dec 9, 2020
991b797
[SPARK-33727][K8S] Fall back from gnupg.net to openpgp.org
holdenk Dec 10, 2020
1c7f5f1
[SPARK-33724][K8S] Add decom script as a configuration param
holdenk Dec 10, 2020
af37c7f
[SPARK-33558][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. ADD PARTITIO…
MaxGekk Dec 10, 2020
b112e2b
[SPARK-33714][SQL] Migrate ALTER VIEW ... SET/UNSET TBLPROPERTIES com…
imback82 Dec 10, 2020
795db05
[SPARK-33732][K8S][TESTS] Kubernetes integration tests doesn't work w…
sarutak Dec 10, 2020
cef28c2
[SPARK-32670][SQL][FOLLOWUP] Group exception messages in Catalyst Ana…
beliefer Dec 10, 2020
1554977
[SPARK-33692][SQL] View should use captured catalog and namespace to …
linhongliu-db Dec 10, 2020
31e0bac
[SPARK-33740][SQL] hadoop configs in hive-site.xml can overrides pre-…
yaooqinn Dec 11, 2020
fab2995
[SPARK-33742][SQL] Throw PartitionsAlreadyExistException from HiveExt…
MaxGekk Dec 11, 2020
1ba1732
[SPARK-33295][BUILD] Upgrade ORC to 1.6.6
dongjoon-hyun Dec 11, 2020
cd7a306
[SPARK-33749][BUILD][PYTHON] Exclude target directory in pycodestyle …
HyukjinKwon Dec 11, 2020
7895ea1
[SPARK-32910][SS] Remove UninterruptibleThread usage from KafkaOffset…
gaborgsomogyi Dec 11, 2020
24d7e45
[SPARK-33527][SQL] Extend the function of decode so as consistent wit…
beliefer Dec 11, 2020
8ac86a4
[SPARK-33750][SQL][TESTS] Use `hadoop-3.2` distribution in HiveExtern…
dongjoon-hyun Dec 11, 2020
c05f6f9
[MINOR][SQL] Spelling: enabled - legacy_setops_precedence_enbled
jsoref Dec 11, 2020
d662b95
[SPARK-33754][K8S][DOCS] Update kubernetes/integration-tests/README.m…
sarutak Dec 11, 2020
8377aca
[SPARK-33527][SQL][FOLLOWUP] Fix the scala 2.13 build failure
sarutak Dec 11, 2020
8f5db71
[SPARK-33654][SQL] Migrate CACHE TABLE to use UnresolvedRelation to r…
imback82 Dec 11, 2020
8b97b19
[SPARK-33706][SQL] Require fully specified partition identifier in pa…
MaxGekk Dec 11, 2020
5bab27e
[SPARK-33526][SQL] Add config to control if cancel invoke interrupt t…
ulysses-you Dec 11, 2020
29cc5b3
[MINOR][INFRA] Add kubernetes-integration-tests to GitHub Actions for…
sarutak Dec 11, 2020
fb2e3af
[SPARK-33757][INFRA][R] Fix the R dependencies build error on GitHub …
sarutak Dec 11, 2020
950def6
[SPARK-22256] - Introduce spark.mesos.driver.memoryOverhead
Apr 6, 2018
8a5661c
[SPARK-22256] add test case when spark.mesos.driver.memoryOverhead is…
dmcwhorter Oct 8, 2019
8149c34
[SPARK-22256] Update comments for driverContainerMemory to accruately…
dmcwhorter Dec 15, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
The diff you're trying to view is too large. We only load the first 3000 changed files.
29 changes: 29 additions & 0 deletions .asf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# https://cwiki.apache.org/confluence/display/INFRA/.asf.yaml+features+for+git+repositories
---
github:
description: "Apache Spark - A unified analytics engine for large-scale data processing"
homepage: https://spark.apache.org/
labels:
- python
- scala
- r
- java
- big-data
- jdbc
- sql
- spark
5 changes: 5 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1,2 +1,7 @@
*.bat text eol=crlf
*.cmd text eol=crlf
*.java text eol=lf
*.scala text eol=lf
*.xml text eol=lf
*.py text eol=lf
*.R text eol=lf
48 changes: 42 additions & 6 deletions .github/PULL_REQUEST_TEMPLATE
Original file line number Diff line number Diff line change
@@ -1,10 +1,46 @@
## What changes were proposed in this pull request?
<!--
Thanks for sending a pull request! Here are some tips for you:
1. If this is your first time, please read our contributor guidelines: https://spark.apache.org/contributing.html
2. Ensure you have added or run the appropriate tests for your PR: https://spark.apache.org/developer-tools.html
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP][SPARK-XXXX] Your PR title ...'.
4. Be sure to keep the PR description updated to reflect all changes.
5. Please write your PR title to summarize what this PR proposes.
6. If possible, provide a concise example to reproduce the issue for a faster review.
7. If you want to add a new configuration, please read the guideline first for naming configurations in
'core/src/main/scala/org/apache/spark/internal/config/ConfigEntry.scala'.
-->

(Please fill in changes proposed in this fix)
### What changes were proposed in this pull request?
<!--
Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue.
If possible, please consider writing useful notes for better and faster reviews in your PR. See the examples below.
1. If you refactor some codes with changing classes, showing the class hierarchy will help reviewers.
2. If you fix some SQL features, you can provide some references of other DBMSes.
3. If there is design documentation, please add the link.
4. If there is a discussion in the mailing list, please add the link.
-->

## How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
### Why are the changes needed?
<!--
Please clarify why the changes are needed. For instance,
1. If you propose a new API, clarify the use case for a new API.
2. If you fix a bug, you can clarify why it is a bug.
-->

Please review http://spark.apache.org/contributing.html before opening a pull request.

### Does this PR introduce _any_ user-facing change?
<!--
Note that it means *any* user-facing change including all aspects such as the documentation fix.
If yes, please clarify the previous behavior and the change this PR proposes - provide the console output, description and/or an example to show the behavior difference if possible.
If possible, please also clarify if this is a user-facing change compared to the released Spark versions or within the unreleased branches such as master.
If no, write 'No'.
-->


### How was this patch tested?
<!--
If tests were added, say they were added here. Please make sure to add some test cases that check the changes thoroughly including negative and positive cases if possible.
If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future.
If tests were not added, please describe why they were not added and/or why it was difficult to add.
-->
152 changes: 152 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#

#
# Pull Request Labeler Github Action Configuration: https://github.com/marketplace/actions/labeler
#
# Note that we currently cannot use the negatioon operator (i.e. `!`) for miniglob matches as they
# would match any file that doesn't touch them. What's needed is the concept of `any `, which takes a
# list of constraints / globs and then matches all of the constraints for either `any` of the files or
# `all` of the files in the change set.
#
# However, `any`/`all` are not supported in a released version and testing off of the `main` branch
# resulted in some other errors when testing.
#
# An issue has been opened upstream requesting that a release be cut that has support for all/any:
# - https://github.com/actions/labeler/issues/111
#
# While we wait for this issue to be handled upstream, we can remove
# the negated / `!` matches for now and at least have labels again.
#
INFRA:
- ".github/**/*"
- "appveyor.yml"
- "tools/**/*"
- "dev/create-release/**/*"
- ".asf.yaml"
- ".gitattributes"
- ".gitignore"
- "dev/github_jira_sync.py"
- "dev/merge_spark_pr.py"
- "dev/run-tests-jenkins*"
BUILD:
# Can be supported when a stable release with correct all/any is released
#- any: ['dev/**/*', '!dev/github_jira_sync.py', '!dev/merge_spark_pr.py', '!dev/.rat-excludes']
- "dev/**/*"
- "build/**/*"
- "project/**/*"
- "assembly/**/*"
- "**/*pom.xml"
- "bin/docker-image-tool.sh"
- "bin/find-spark-home*"
- "scalastyle-config.xml"
# These can be added in the above `any` clause (and the /dev/**/* glob removed) when
# `any`/`all` support is released
# - "!dev/github_jira_sync.py"
# - "!dev/merge_spark_pr.py"
# - "!dev/run-tests-jenkins*"
# - "!dev/.rat-excludes"
DOCS:
- "docs/**/*"
- "**/README.md"
- "**/CONTRIBUTING.md"
EXAMPLES:
- "examples/**/*"
- "bin/run-example*"
# CORE needs to be updated when all/any are released upstream.
CORE:
# - any: ["core/**/*", "!**/*UI.scala", "!**/ui/**/*"] # If any file matches all of the globs defined in the list started by `any`, label is applied.
- "core/**/*"
- "common/kvstore/**/*"
- "common/network-common/**/*"
- "common/network-shuffle/**/*"
- "python/pyspark/**/*.py"
- "python/pyspark/tests/**/*.py"
SPARK SUBMIT:
- "bin/spark-submit*"
SPARK SHELL:
- "repl/**/*"
- "bin/spark-shell*"
SQL:
#- any: ["**/sql/**/*", "!python/pyspark/sql/avro/**/*", "!python/pyspark/sql/streaming.py", "!python/pyspark/sql/tests/test_streaming.py"]
- "**/sql/**/*"
- "common/unsafe/**/*"
#- "!python/pyspark/sql/avro/**/*"
#- "!python/pyspark/sql/streaming.py"
#- "!python/pyspark/sql/tests/test_streaming.py"
- "bin/spark-sql*"
- "bin/beeline*"
- "sbin/*thriftserver*.sh"
- "**/*SQL*.R"
- "**/DataFrame.R"
- "**/*WindowSpec.R"
- "**/*catalog.R"
- "**/*column.R"
- "**/*functions.R"
- "**/*group.R"
- "**/*schema.R"
- "**/*types.R"
AVRO:
- "external/avro/**/*"
- "python/pyspark/sql/avro/**/*"
DSTREAM:
- "streaming/**/*"
- "data/streaming/**/*"
- "external/kinesis*"
- "external/kafka*"
- "python/pyspark/streaming/**/*"
GRAPHX:
- "graphx/**/*"
- "data/graphx/**/*"
ML:
- "**/ml/**/*"
- "**/*mllib_*.R"
MLLIB:
- "**/spark/mllib/**/*"
- "mllib-local/**/*"
- "python/pyspark/mllib/**/*"
STRUCTURED STREAMING:
- "**/sql/**/streaming/**/*"
- "external/kafka-0-10-sql/**/*"
- "python/pyspark/sql/streaming.py"
- "python/pyspark/sql/tests/test_streaming.py"
- "**/*streaming.R"
PYTHON:
- "bin/pyspark*"
- "**/python/**/*"
R:
- "**/r/**/*"
- "**/R/**/*"
- "bin/sparkR*"
YARN:
- "resource-managers/yarn/**/*"
MESOS:
- "resource-managers/mesos/**/*"
- "sbin/*mesos*.sh"
KUBERNETES:
- "resource-managers/kubernetes/**/*"
WINDOWS:
- "**/*.cmd"
- "R/pkg/tests/fulltests/test_Windows.R"
WEB UI:
- "**/ui/**/*"
- "**/*UI.scala"
DEPLOY:
- "sbin/**/*"

Loading