Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1009 commits
Select commit Hold shift + click to select a range
82aca7e
[SPARK-33779][SQL] DataSource V2: API to request distribution and ord…
aokolnychyi Dec 14, 2020
bb60fb1
[SPARK-33779][SQL][FOLLOW-UP] Fix Java Linter error
aokolnychyi Dec 14, 2020
5885cc1
[SPARK-33261][K8S] Add a developer API for custom feature steps
holdenk Dec 14, 2020
412d86e
[SPARK-33771][SQL][TESTS] Fix Invalid value for HourOfAmPm when testi…
wangyum Dec 14, 2020
f156718
[SPARK-33777][SQL] Sort output of V2 SHOW PARTITIONS
MaxGekk Dec 14, 2020
49d3256
[SPARK-33653][SQL] DSv2: REFRESH TABLE should recache the table itself
sunchao Dec 14, 2020
a99a47c
[SPARK-33748][K8S] Respect environment variables and configurations f…
HyukjinKwon Dec 14, 2020
366beda
[SPARK-33785][SQL] Migrate ALTER TABLE ... RECOVER PARTITIONS to use …
imback82 Dec 15, 2020
141e26d
[SPARK-33767][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. DROP PARTITI…
MaxGekk Dec 15, 2020
0304252
[SPARK-33273][SQL] Fix a race condition in subquery execution
cloud-fan Dec 15, 2020
20f6d63
[SPARK-33769][SQL] Improve the next-day function of the sql component…
Dec 15, 2020
58cb2ba
[SPARK-33752][SQL] Avoid the getSimpleMessage of AnalysisException ad…
beliefer Dec 15, 2020
23083aa
[SPARK-33758][SQL] Prune unrequired partitionings from AliasAwareOutp…
prakharjain09 Dec 15, 2020
40c37d6
[SPARK-33617][SQL][FOLLOWUP] refine the default parallelism SQL config
cloud-fan Dec 15, 2020
4d56d43
[SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate
aokolnychyi Dec 15, 2020
87c5836
[SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead
dmcwhorter Dec 15, 2020
3dfdcf4
[SPARK-33788][SQL] Throw NoSuchPartitionsException from HiveExternalC…
MaxGekk Dec 16, 2020
dd042f5
[SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc
gengliangwang Dec 16, 2020
ddff94f
[SPARK-33793][TESTS] Introduce withExecutor to ensure proper cleanup …
sander-goos Dec 16, 2020
62be248
[SPARK-33765][SQL] Migrate UNCACHE TABLE to use UnresolvedRelation to…
imback82 Dec 16, 2020
ef7f690
[SPARK-33786][SQL] The storage level for a cache should be respected …
imback82 Dec 16, 2020
12f3715
[MINOR][DOCS] Fix Jenkins job badge image and link in README.md
HyukjinKwon Dec 16, 2020
888a274
[SPARK-33802][INFRA] Override name and email address explicitly when …
HyukjinKwon Dec 16, 2020
7845865
[SPARK-33803][SQL] Sort table properties by key in DESCRIBE TABLE com…
HyukjinKwon Dec 16, 2020
9d9d4a8
[SPARK-33789][SQL][TESTS] Refactor unified V1 and V2 datasource tests
MaxGekk Dec 16, 2020
205d8e4
[SPARK-32991][SQL] [FOLLOWUP] Reset command relies on session initial…
yaooqinn Dec 16, 2020
ddda32b
[SPARK-33802][INFRA][FOLLOW-UP] Separate arguments properly for -c op…
HyukjinKwon Dec 16, 2020
8666d1c
[SPARK-33800][SQL] Remove command name in AnalysisException message w…
imback82 Dec 16, 2020
3d03234
[SPARK-33810][TESTS] Reenable test cases disabled in SPARK-31732
HyukjinKwon Dec 16, 2020
728a129
[SPARK-33806][SQL] limit partition num to 1 when distributing by fold…
yaooqinn Dec 16, 2020
e7e29fd
[SPARK-33514][SQL][FOLLOW-UP] Remove unused TruncateTableStatement ca…
imback82 Dec 16, 2020
477046c
[SPARK-33775][BUILD] Suppress sbt compilation warnings in Scala 2.13
LuciferYang Dec 16, 2020
0c12900
[SPARK-33790][CORE] Reduce the rpc call of getFileStatus in SingleFil…
cxzl25 Dec 17, 2020
0c19497
[SPARK-33815][SQL] Migrate ALTER TABLE ... SET [SERDE|SERDEPROPERTIES…
imback82 Dec 17, 2020
1e85707
[SPARK-33697][SQL] RemoveRedundantProjects should require column orde…
allisonwang-db Dec 17, 2020
b1950cc
[SPARK-33821][BUILD] Upgrade SBT to 1.4.5
dongjoon-hyun Dec 17, 2020
ed09673
[SPARK-33819][CORE] SingleFileEventLogFileReader/RollingEventLogFiles…
dongjoon-hyun Dec 17, 2020
12b69cc
[SPARK-26199][SPARK-31517][R] Fix strategy for handling ... names in …
Dec 17, 2020
34e4d87
[SPARK-33774][UI][CORE] Back to Master" returns 500 error in Standalo…
Ngone51 Dec 17, 2020
8c81cf7
[SPARK-22769] Do not log rpc post message error when sparkEnv is alre…
Dec 17, 2020
15616f4
[SPARK-33173][CORE][TESTS][FOLLOWUP] Use `local[2]` and AtomicInteger
Ngone51 Dec 17, 2020
51ef443
[SPARK-33822][SQL] Use the `CastSupport.cast` method in HashJoin
maropu Dec 18, 2020
6315118
[SPARK-33824][PYTHON][DOCS] Restructure and improve Python package ma…
HyukjinKwon Dec 18, 2020
42e1831
[SPARK-33797][SS][DOCS] Update SS doc about State Store and task loca…
viirya Dec 18, 2020
131a23d
[SPARK-33831][UI] Update to jetty 9.4.34
srowen Dec 18, 2020
0f1a183
[SPARK-33817][SQL] CACHE TABLE uses a logical plan when caching a que…
imback82 Dec 18, 2020
25c6cc2
[SPARK-26341][WEBUI] Expose executor memory metrics at the stage leve…
AngersZhuuuu Dec 18, 2020
b0da2bc
[MINOR][INFRA] Add -Pspark-ganglia-lgpl to the build definition with …
sarutak Dec 18, 2020
0603913
[SPARK-33593][SQL] Vector reader got incorrect data with binary parti…
AngersZhuuuu Dec 18, 2020
bc46d27
[SPARK-33840][DOCS] Add spark.sql.files.minPartitionNum to performenc…
ulysses-you Dec 18, 2020
06b1bbb
[SPARK-33798][SQL] Add new rule to push down the foldable expressions…
wangyum Dec 18, 2020
f239128
[SPARK-33597][SQL] Support REGEXP_LIKE for consistent with mainstream…
beliefer Dec 18, 2020
6dca2e5
[SPARK-33599][SQL] Group exception messages in catalyst/analysis
beliefer Dec 18, 2020
c17c76d
[SPARK-33599][SQL][FOLLOWUP] FIX Github Action with unidoc
yaooqinn Dec 18, 2020
554600c
[SPARK-33841][CORE] Fix issue with jobs disappearing intermittently f…
vhlinskyi Dec 18, 2020
de234ee
[SPARK-33812][SQL] Split the histogram column stats when saving to hi…
cloud-fan Dec 19, 2020
44563a0
[SPARK-33518][ML] Improve performance of ML ALS recommendForAll by GEMV
zhengruifeng Dec 19, 2020
00642ee
[SPARK-33843][BUILD] Upgrade to Zstd 1.4.8
dongjoon-hyun Dec 19, 2020
dd44ba5
[SPARK-32976][SQL][FOLLOWUP] SET and RESTORE hive.exec.dynamic.partit…
yaooqinn Dec 19, 2020
06075d8
[SPARK-33829][SQL] Renaming v2 tables should recreate the cache
imback82 Dec 19, 2020
37c4cd8
[MINOR][DOCS] Fix typos in ScalaDocs for DataStreamWriter#foreachBatch
ammar1x Dec 19, 2020
70da86a
[SPARK-33850][SQL] EXPLAIN FORMATTED doesn't show the plan for subque…
sarutak Dec 19, 2020
2b6ef56
[SPARK-33854][BUILD] Use ListBuffer instead of Stack in SparkBuild.scala
williamhyun Dec 19, 2020
df2314b
[SPARK-33852][SQL][TESTS] Use assertAnalysisError in HiveDDLSuite.scala
imback82 Dec 19, 2020
1339168
[SPARK-33756][SQL] Make BytesToBytesMap's MapIterator idempotent
advancedxy Dec 20, 2020
3c8be39
[SPARK-33850][SQL][FOLLOWUP] Improve and cleanup the test code
sarutak Dec 21, 2020
8e26339
[SPARK-26341][WEBUI][FOLLOWUP] Update stage memory metrics on stage end
sarutak Dec 21, 2020
1c7b79c
[SPARK-33856][SQL] Migrate ALTER TABLE ... RENAME TO PARTITION to use…
imback82 Dec 21, 2020
b313a1e
[SPARK-33849][SQL][TESTS] Unify v1 and v2 DROP TABLE tests
MaxGekk Dec 21, 2020
8d4d433
[SPARK-33836][SS][PYTHON] Expose DataStreamReader.table and DataStrea…
HeartSaVioR Dec 21, 2020
f4e1069
[SPARK-33853][SQL] EXPLAIN CODEGEN and BenchmarkQueryTest don't show …
sarutak Dec 21, 2020
cdd1752
[SPARK-33862][SQL] Throw `PartitionAlreadyExistsException` if the tar…
MaxGekk Dec 21, 2020
b4bea1a
[SPARK-28863][SQL][FOLLOWUP] Make sure optimized plan will not be re-…
cloud-fan Dec 21, 2020
4b19f49
[SPARK-33845][SQL] Remove unnecessary if when trueValue and falseValu…
wangyum Dec 21, 2020
69aa727
[SPARK-33124][SQL] Fills missing group tags and re-categorizes all th…
maropu Dec 21, 2020
661ac10
[SPARK-33838][SQL][DOCS] Comment the `PURGE` option in the DropTable …
MaxGekk Dec 21, 2020
1c77605
[SPARK-33848][SQL] Push the UnaryExpression into (if / case) branches
wangyum Dec 21, 2020
38bbcca
[SPARK-33869][PYTHON][SQL][TESTS] Have a separate metastore directory…
HyukjinKwon Dec 21, 2020
4106731
[SPARK-33836][SS][PYTHON][FOLLOW-UP] Use test utils and clean up doct…
HyukjinKwon Dec 21, 2020
0bf3828
[MINOR] update dstream.py with more accurate exceptions
kykrueger Dec 21, 2020
f62e957
[SPARK-33873][CORE][TESTS] Test all compression codecs with encrypted…
dongjoon-hyun Dec 22, 2020
7466031
[SPARK-32106][SQL] Implement script transform in sql/core
AngersZhuuuu Dec 22, 2020
f5fd10b
[SPARK-33834][SQL] Verify ALTER TABLE CHANGE COLUMN with Char and Var…
yaooqinn Dec 22, 2020
16ae3a5
[MINOR][CORE] Remove unused variable CompressionCodec.DEFAULT_COMPRES…
dongjoon-hyun Dec 22, 2020
b887455
[SPARK-33700][SQL] Avoid file meta reading when enableFilterPushDown …
LuciferYang Dec 22, 2020
1dd63dc
[SPARK-33860][SQL] Make CatalystTypeConverters.convertToCatalyst matc…
ulysses-you Dec 22, 2020
2562183
[SPARK-33808][SQL] DataSource V2: Build logical writes in the optimizer
aokolnychyi Dec 22, 2020
7bbcbb8
[SPARK-33784][SQL] Rename dataSourceRewriteRules batch
aokolnychyi Dec 22, 2020
43a5620
[SPARK-33846][SQL] Include Comments for a nested schema in StructType…
Dec 22, 2020
84bf07b
[SPARK-33878][SQL][TESTS] Fix resolving of `spark_catalog` in v1 Hive…
MaxGekk Dec 22, 2020
6da5cdf
[SPARK-33876][SQL] Add length-check for reading char/varchar from tab…
yaooqinn Dec 22, 2020
1d45025
[BUILD][MINOR] Do not publish snapshots from forks
EnricoMi Dec 22, 2020
303b8c8
[SPARK-23862][SQL] Support Java enums from Scala Dataset API
xkrogen Dec 22, 2020
ec1560a
[SPARK-33364][SQL][FOLLOWUP] Refine the catalog v2 API to purge a table
cloud-fan Dec 23, 2020
a3dd8da
[SPARK-33877][SQL] SQL reference documents for INSERT w/ a column list
yaooqinn Dec 23, 2020
ea37717
[SPARK-32106][SQL][FOLLOWUP] Fix flaky tests in transform.sql
maropu Dec 23, 2020
90d6f86
[SPARK-33870][CORE] Enable spark.storage.replication.proactive by def…
dongjoon-hyun Dec 23, 2020
e853f06
[SPARK-33526][SQL][FOLLOWUP] Fix flaky test due to timeout and fix docs
ulysses-you Dec 23, 2020
2287f56
[SPARK-33879][SQL] Char Varchar values fails w/ match error as partit…
yaooqinn Dec 23, 2020
d98c216
[SPARK-31960][YARN][DOCS][FOLLOW-UP] Document the behaviour change of…
HyukjinKwon Dec 23, 2020
34bfb3a
[SPARK-33787][SQL] Allow partition purge for v2 tables
MaxGekk Dec 23, 2020
f421c17
[SPARK-33497][SQL] Override maxRows in some LogicalPlan
ulysses-you Dec 23, 2020
cc23581
[SPARK-33858][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. RENAME PARTI…
MaxGekk Dec 23, 2020
303df64
[SPARK-33889][SQL] Fix NPE from `SHOW PARTITIONS` on V2 tables
MaxGekk Dec 23, 2020
7ffcfcf
[SPARK-33847][SQL] Simplify CaseWhen if elseValue is None
wangyum Dec 23, 2020
47d1aa4
[SPARK-33891][DOCS][CORE] Update dynamic allocation related documents
dongjoon-hyun Dec 23, 2020
0677c39
[SPARK-32916][SHUFFLE][TEST-MAVEN][TEST-HADOOP2.7] Ensure the number …
otterc Dec 23, 2020
5c9b421
[SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consumin…
ueshin Dec 23, 2020
d467d81
[SPARK-33893][CORE] Exclude fallback block manager from executorList
dongjoon-hyun Dec 23, 2020
368a2c3
[SPARK-33877][SQL][FOLLOWUP] SQL reference documents for INSERT w/ a …
yaooqinn Dec 23, 2020
61881bb
[SPARK-33835][CORE] Refector AbstractCommandBuilder.buildJavaCommand:…
Dec 24, 2020
86c1cfc
[SPARK-33659][SS] Document the current behavior for DataStreamWriter.…
xuanyuanking Dec 24, 2020
f1d3797
[SPARK-33886][SQL] UnresolvedTable should retain SQL text position fo…
imback82 Dec 24, 2020
d7dc42d
[SPARK-33895][SQL] Char and Varchar fail in MetaOperation of ThriftSe…
yaooqinn Dec 24, 2020
32d4a2b
[SPARK-33861][SQL] Simplify conditional in predicate
wangyum Dec 24, 2020
3e9821e
[SPARK-33443][SQL] LEAD/LAG should support [ IGNORE NULLS | RESPECT N…
beliefer Dec 24, 2020
54a6784
[SPARK-33881][SQL][TESTS] Check null and empty string as partition va…
MaxGekk Dec 24, 2020
29cca68
[SPARK-33892][SQL] Display char/varchar in DESC and SHOW CREATE TABLE
yaooqinn Dec 24, 2020
700f5ab
[SPARK-33900][WEBUI] Show shuffle read size / records correctly when …
cxzl25 Dec 24, 2020
9c30116
[SPARK-33857][SQL] Unify the default seed of random functions
ulysses-you Dec 24, 2020
65a9ac2
[SPARK-30027][SQL] Support codegen for aggregate filters in HashAggre…
maropu Dec 24, 2020
10b6466
[SPARK-33084][CORE][SQL] Add jar support ivy path
AngersZhuuuu Dec 25, 2020
2553d53
[SPARK-33897][SQL] Can't set option 'cross' in join method
kozakana Dec 26, 2020
37ae0a6
[SPARK-33560][TEST-MAVEN][BUILD] Add "unused-import" check to Maven c…
LuciferYang Dec 26, 2020
b2dfeae
[SPARK-33911][SQL][DOCS] Update the SQL migration guide about changes…
MaxGekk Dec 27, 2020
678294d
[SPARK-33824][PYTHON][DOCS][FOLLOW-UP] Clarify about PYSPARK_DRIVER_P…
gaborgsomogyi Dec 28, 2020
e6f0198
[SPARK-33532][SQL] Add comments to a unreachable branch in SpecificPa…
LuciferYang Dec 28, 2020
fe33262
[SPARK-33918][SQL] UnresolvedView should retain SQL text position for…
imback82 Dec 28, 2020
1be9e7e
[SPAKR-33801][CORE][SQL] Fix compilation warnings about 'Unicode esca…
LuciferYang Dec 28, 2020
3fdbc48
[SPARK-33901][SQL] Fix Char and Varchar display error after DDLs
yaooqinn Dec 28, 2020
0a3f3d6
[SPARK-33908][CORE] Refactor SparkSubmitUtils.resolveMavenCoordinates…
AngersZhuuuu Dec 28, 2020
4a61fc1
[SPARK-33914][SQL][DOCS] Describe the structure of unified DS v1 and …
MaxGekk Dec 28, 2020
c75f779
[SPARK-33827][SS] Unload inactive state store as soon as possible
viirya Dec 28, 2020
00fa49a
[SPARK-33923][SQL][TESTS] Fix some tests with AQE enabled
Ngone51 Dec 28, 2020
fc508d1
[SPARK-32685][SQL] When specify serde, default filed.delim is '\t'
AngersZhuuuu Dec 28, 2020
0617dfc
[SPARK-33899][SQL] Fix assert failure in v1 SHOW TABLES/VIEWS on `spa…
MaxGekk Dec 28, 2020
6497ccb
[SPARK-33916][CORE] Fix fallback storage offset and improve compressi…
dongjoon-hyun Dec 29, 2020
c2eac1d
[SPARK-33845][SQL][FOLLOWUP] fix SimplifyConditionals
cloud-fan Dec 29, 2020
b33fa53
[SPARK-33925][CORE] Remove unused SecurityManager in Utils.fetchFile
HyukjinKwon Dec 29, 2020
379afcd
[SPARK-33924][SQL][TESTS] Preserve partition metadata by INSERT INTO …
MaxGekk Dec 29, 2020
f7bdea3
[SPARK-33884][SQL] Simplify CaseWhenclauses with (true and false) and…
wangyum Dec 29, 2020
1ef7ddd
[SPARK-33928][SPARK-23365][TEST][CORE] Fix flaky o.a.s.ExecutorAlloca…
Ngone51 Dec 29, 2020
2627825
[SPARK-33931][INFRA] Recover GitHub Action `build_and_test` job
dongjoon-hyun Dec 29, 2020
f9fe742
[SPARK-32968][SQL] Prune unnecessary columns from CsvToStructs
viirya Dec 29, 2020
9d6dbe0
[SPARK-33775][FOLLOWUP][TEST-MAVEN][BUILD] Suppress maven compilation…
LuciferYang Dec 29, 2020
e0d2ffe
[SPARK-33859][SQL] Support V2 ALTER TABLE .. RENAME PARTITION
MaxGekk Dec 29, 2020
3b1b209
[SPARK-33909][SQL] Check rand functions seed is legal at analyer side
ulysses-you Dec 29, 2020
872107f
[SPARK-33848][SQL][FOLLOWUP] Introduce allowList for push into (if / …
wangyum Dec 29, 2020
aadda4b
[SPARK-33930][SQL] Script Transform default FIELD DELIMIT should be \…
AngersZhuuuu Dec 29, 2020
16c594d
[SPARK-33859][SQL][FOLLOWUP] Add version to `SupportsPartitionManagem…
MaxGekk Dec 29, 2020
c425024
[SPARK-33847][SQL][FOLLOWUP] Remove the CaseWhen should consider dete…
wangyum Dec 29, 2020
2b6836c
[SPARK-33936][SQL] Add the version when connector's methods and inter…
MaxGekk Dec 29, 2020
951afc3
[SPARK-33932][SS] Clean up KafkaOffsetReader API document
viirya Dec 30, 2020
448494e
[SPARK-33874][K8S] Handle long lived sidecars
holdenk Dec 30, 2020
49aa6eb
[SPARK-32684][SQL][TESTS] Add a test case to check if null value is s…
AngersZhuuuu Dec 30, 2020
687f465
[SPARK-33890][SQL] Improve the implement of trim/trimleft/trimright
beliefer Dec 30, 2020
4a669f5
[MINOR][SS] Call fetchEarliestOffsets when it is necessary
viirya Dec 30, 2020
403bf55
[SPARK-33927][BUILD] Fix Dockerfile for Spark release to work
HyukjinKwon Dec 30, 2020
0eb4961
[SPARK-33926][SQL] Improve the error message from resolving of v1 dat…
MaxGekk Dec 30, 2020
2afd1fb
[SPARK-33904][SQL] Recognize `spark_catalog` in `saveAsTable()` and `…
MaxGekk Dec 30, 2020
ba974ea
[SPARK-30789][SQL] Support (IGNORE | RESPECT) NULLS for LEAD/LAG/NTH_…
beliefer Dec 30, 2020
f38265d
[SPARK-33907][SQL] Only prune columns of from_json if parsing options…
viirya Dec 30, 2020
85de644
[SPARK-33804][CORE] Fix compilation warnings about 'view bounds are d…
LuciferYang Dec 30, 2020
13e8c28
[SPARK-33942][DOCS] Remove `hiveClientCalls.count` in `CodeGenerator`…
Dec 31, 2020
3fe5614
[SPARK-31946][CORE] Make worker/executor decommission signal configur…
Ngone51 Dec 31, 2020
771c538
[SPARK-33084][SQL][TESTS][FOLLOW-UP] Fix Scala 2.13 UT failure
AngersZhuuuu Dec 31, 2020
ed9f728
[SPARK-33944][SQL] Incorrect logging for warehouse keys in SharedStat…
yaooqinn Dec 31, 2020
45df6db
[SPARK-33906][WEBUI] Fix the bug of UI Executor page stuck due to und…
Dec 31, 2020
bd346f4
[SPARK-33957][BUILD] Update commons-lang3 to 3.11
williamhyun Jan 2, 2021
4cd6805
[SPARK-33956][SQL] Add rowCount for Range operator
wangyum Jan 2, 2021
1c25bea
[SPARK-33961][BUILD] Upgrade SBT to 1.4.6
dongjoon-hyun Jan 2, 2021
6c5ba81
[SPARK-33959][SQL] Improve the statistics estimation of the Tail
wangyum Jan 3, 2021
fc7d016
[SPARK-33963][SQL] Canonicalize `HiveTableRelation` w/o table stats
MaxGekk Jan 3, 2021
cfd4a08
[SPARK-33962][SS] Fix incorrect min partition condition
viirya Jan 3, 2021
963c60f
[SPARK-33955][SS] Add latest offsets to source progress
viirya Jan 3, 2021
6b7527e
[SPARK-33398] Fix loading tree models prior to Spark 3.0
zhengruifeng Jan 3, 2021
67195d0
[SPARK-33950][SQL] Refresh cache in v1 `ALTER TABLE .. DROP PARTITION`
MaxGekk Jan 4, 2021
b037930
[SPARK-33951][SQL] Distinguish the error between filter and distinct
beliefer Jan 4, 2021
2a68ed7
[SPARK-33954][SQL] Some operator missing rowCount when enable CBO
wangyum Jan 4, 2021
adac633
[SPARK-33934][SQL] Add SparkFile's root dir to env property PATH
AngersZhuuuu Jan 4, 2021
0b647fe
[SPARK-33888][SQL] JDBC SQL TIME type represents incorrectly as Times…
saikocat Jan 4, 2021
8b3fb43
[SPARK-33965][SQL][TESTS] Recognize `spark_catalog` by `CACHE TABLE` …
MaxGekk Jan 4, 2021
271c4f6
[SPARK-33978][SQL] Support ZSTD compression in ORC data source
dongjoon-hyun Jan 4, 2021
8583a46
[SPARK-33844][SQL] InsertIntoHiveDir command should check col name too
AngersZhuuuu Jan 4, 2021
ddc0d51
[SPARK-33875][SQL] Implement DESCRIBE COLUMN for v2 tables
imback82 Jan 4, 2021
6b86aa0
[SPARK-33984][PYTHON] Upgrade to Py4J 0.10.9.1
HyukjinKwon Jan 4, 2021
fc3f226
[SPARK-33990][SQL][TESTS] Remove partition data by v2 `ALTER TABLE ..…
MaxGekk Jan 4, 2021
414d323
[SPARK-33988][SQL][TEST] Add an option to enable CBO in TPCDSQueryBen…
maropu Jan 4, 2021
d6322bf
[SPARK-33983][PYTHON] Update cloudpickle to v1.6.0
HyukjinKwon Jan 4, 2021
ac4651a
[SPARK-33980][SS] Invalidate char/varchar in spark.readStream.schema
yaooqinn Jan 4, 2021
90f4ecf
[SPARK-33996][BUILD] Upgrade checkstyle plugins
williamhyun Jan 4, 2021
84c1f43
[SPARK-33987][SQL] Refresh cache in v2 `ALTER TABLE .. DROP PARTITION`
MaxGekk Jan 4, 2021
9b4173f
[SPARK-33894][SQL] Change visibility of private case classes in mllib…
koertkuipers Jan 4, 2021
559f411
[SPARK-33908][CORE][FOLLOWUP] Correct Scaladoc of resolveDependencyPa…
AngersZhuuuu Jan 4, 2021
bb6d6b5
[SPARK-33964][SQL] Combine distinct unions in more cases
tanelk Jan 5, 2021
976e97a
[SPARK-33794][SQL] NextDay expression throw runtime IllegalArgumentEx…
Jan 5, 2021
6b00fdc
[SPARK-33998][SQL] Provide an API to create an InternalRow in V2Comma…
imback82 Jan 5, 2021
15a863f
[SPARK-34001][SQL][TESTS] Remove unused runShowTablesSql() in DataSou…
imback82 Jan 5, 2021
f0ffe0c
[SPARK-33992][SQL] override transformUpWithNewOutput to add allowInvo…
yaooqinn Jan 5, 2021
a7d3fcd
[SPARK-34000][CORE] Fix stageAttemptToNumSpeculativeTasks java.util.N…
LantaoJin Jan 5, 2021
a071826
[SPARK-33100][SQL] Ignore a semicolon inside a bracketed comment in s…
turboFei Jan 5, 2021
f252a93
[SPARK-33935][SQL] Fix CBO cost function
tanelk Jan 5, 2021
122f8f0
[SPARK-33919][SQL][TESTS] Unify v1 and v2 SHOW NAMESPACES tests
MaxGekk Jan 5, 2021
356fdc9
[SPARK-34007][BUILD] Downgrade scala-maven-plugin to 4.3.0
HyukjinKwon Jan 5, 2021
329850c
[SPARK-32017][PYTHON][FOLLOW-UP] Rename HADOOP_VERSION to PYSPARK_HAD…
HyukjinKwon Jan 5, 2021
acf0a4f
[SPARK-33999][BUILD] Make sbt unidoc success with JDK11
sarutak Jan 5, 2021
8d09f96
[SPARK-34010][SQL][DODCS] Use python3 instead of python in SQL docume…
HyukjinKwon Jan 5, 2021
14c2eda
[SPARK-34009][BUILD] To activate profile 'aarch64' based on OS settings
huangtianhua Jan 5, 2021
cc1d9d2
[SPARK-33542][SQL] Group exception messages in catalyst/catalog
beliefer Jan 5, 2021
171db85
[SPARK-33874][K8S][FOLLOWUP] Handle long lived sidecars - clean up lo…
holdenk Jan 5, 2021
e279ed3
[SPARK-34012][SQL] Keep behavior consistent when conf `spark.sql.lega…
AngersZhuuuu Jan 5, 2021
b77d11d
[SPARK-34011][SQL] Refresh cache in `ALTER TABLE .. RENAME TO PARTITION`
MaxGekk Jan 6, 2021
3d8ee49
[SPARK-34015][R] Fixing input timing in gapply
Jan 6, 2021
2951082
[SPARK-33029][CORE][WEBUI] Fix the UI executor page incorrectly marki…
Jan 6, 2021
2ab77d6
[SPARK-34004][SQL] Change FrameLessOffsetWindowFunction as sealed abs…
beliefer Jan 6, 2021
b1c4fc7
[SPARK-34008][BUILD] Upgrade derby to 10.14.2.0
sarutak Jan 6, 2021
fa93090
[SPARK-33635][SS] Adjust the order of check in KafkaTokenUtil.needTok…
HeartSaVioR Jan 6, 2021
c0d0dba
[SPARK-33934][SQL][FOLLOW-UP] Use SubProcessor's exit code as assert …
AngersZhuuuu Jan 6, 2021
45a4ff8
[SPARK-33948][SQL] Fix CodeGen error of MapObjects.doGenCode method i…
LuciferYang Jan 6, 2021
26d8df3
[SPARK-33938][SQL] Optimize Like Any/All by LikeSimplification
beliefer Jan 6, 2021
f64dfa8
[SPARK-32221][K8S] Avoid possible errors due to incorrect file size o…
ScrapCodes Jan 6, 2021
ff284fb
[SPARK-30681][PYTHON][FOLLOW-UP] Keep the name similar with Scala sid…
HyukjinKwon Jan 6, 2021
0d86a02
[SPARK-34022][DOCS] Support latest mkdocs in SQL built-in function docs
HyukjinKwon Jan 6, 2021
6788304
[SPARK-33977][SQL][DOCS] Add doc for "'like any' and 'like all' opera…
beliefer Jan 6, 2021
3cdc4ef
[SPARK-32685][SQL][FOLLOW-UP] Update migration guide about change def…
AngersZhuuuu Jan 6, 2021
a0269bb
[SPARK-34022][DOCS][FOLLOW-UP] Fix typo in SQL built-in function docs
kiszk Jan 6, 2021
8bb70bf
[SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyProvider
dongjoon-hyun Jan 6, 2021
f9daf03
[SPARK-33806][SQL][FOLLOWUP] Fold RepartitionExpression num partition…
ulysses-you Jan 7, 2021
9b5df2a
[SPARK-34036][DOCS] Update ORC data source documentation
dongjoon-hyun Jan 7, 2021
0ba3ab4
[SPARK-34021][R] Fix hyper links in SparkR documentation for CRAN sub…
HyukjinKwon Jan 7, 2021
26b6039
[SPARK-34028][SQL] Cleanup "unreachable code" compilation warning
LuciferYang Jan 7, 2021
3aa4e11
[SPARK-33861][SQL][FOLLOWUP] Simplify conditional in predicate should…
wangyum Jan 7, 2021
aa509c1
[SPARK-34031][SQL] Union operator missing rowCount when CBO enabled
wangyum Jan 7, 2021
194edc8
Revert "[SPARK-34029][SQL][TESTS] Add OrcEncryptionSuite and FakeKeyP…
dongjoon-hyun Jan 7, 2021
d36cdd5
[SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid bro…
Jan 7, 2021
7b06acc
[SPARK-33100][SQL][FOLLOWUP] Find correct bound of bracketed comment …
turboFei Jan 7, 2021
aa388cf
[SPARK-34041][PYTHON][DOCS] Miscellaneous cleanup for new PySpark doc…
HyukjinKwon Jan 8, 2021
5b16d70
[SPARK-34044][DOCS] Add spark.sql.hive.metastore.jars.path to sql-dat…
dongjoon-hyun Jan 8, 2021
8e11ce5
[SPARK-34018][K8S] NPE in ExecutorPodsSnapshot
holdenk Jan 8, 2021
9b54da4
[SPARK-33818][SQL][DOC] Add descriptions about `spark.sql.parser.quot…
AngersZhuuuu Jan 8, 2021
0de7f2f
[SPARK-34039][SQL] ReplaceTable should invalidate cache
sunchao Jan 8, 2021
cc20154
[SPARK-34005][CORE] Update peak memory metrics for each Executor on t…
sarutak Jan 8, 2021
b95a847
[SPARK-34046][SQL][TESTS] Use join hint for constructing joins in Joi…
gengliangwang Jan 8, 2021
0f8e5dd
[SPARK-34003][SQL] Fix Rule conflicts between PaddingAndLengthCheckFo…
yaooqinn Jan 8, 2021
71d261a
[SPARK-34032][SS] Add truststore and keystore type config possibility…
gaborgsomogyi Jan 8, 2021
157b72a
[SPARK-33591][SQL] Recognize `null` in partition spec values
MaxGekk Jan 8, 2021
023eba2
[SPARK-33796][DOCS][FOLLOWUP] Tweak the width of left-menu of Spark S…
sarutak Jan 8, 2021
0781ed4
[MINOR][SQL][TESTS] Fix the incorrect unicode escape test in ParserUt…
sarutak Jan 8, 2021
d00f069
[SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuff…
otterc Jan 8, 2021
6b34745
[SPARK-34049][SS] DataSource V2: Use Write abstraction in StreamExecu…
aokolnychyi Jan 9, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
133 changes: 0 additions & 133 deletions .github/autolabeler.yml

This file was deleted.

152 changes: 152 additions & 0 deletions .github/labeler.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#

#
# Pull Request Labeler Github Action Configuration: https://github.com/marketplace/actions/labeler
#
# Note that we currently cannot use the negatioon operator (i.e. `!`) for miniglob matches as they
# would match any file that doesn't touch them. What's needed is the concept of `any `, which takes a
# list of constraints / globs and then matches all of the constraints for either `any` of the files or
# `all` of the files in the change set.
#
# However, `any`/`all` are not supported in a released version and testing off of the `main` branch
# resulted in some other errors when testing.
#
# An issue has been opened upstream requesting that a release be cut that has support for all/any:
# - https://github.com/actions/labeler/issues/111
#
# While we wait for this issue to be handled upstream, we can remove
# the negated / `!` matches for now and at least have labels again.
#
INFRA:
- ".github/**/*"
- "appveyor.yml"
- "tools/**/*"
- "dev/create-release/**/*"
- ".asf.yaml"
- ".gitattributes"
- ".gitignore"
- "dev/github_jira_sync.py"
- "dev/merge_spark_pr.py"
- "dev/run-tests-jenkins*"
BUILD:
# Can be supported when a stable release with correct all/any is released
#- any: ['dev/**/*', '!dev/github_jira_sync.py', '!dev/merge_spark_pr.py', '!dev/.rat-excludes']
- "dev/**/*"
- "build/**/*"
- "project/**/*"
- "assembly/**/*"
- "**/*pom.xml"
- "bin/docker-image-tool.sh"
- "bin/find-spark-home*"
- "scalastyle-config.xml"
# These can be added in the above `any` clause (and the /dev/**/* glob removed) when
# `any`/`all` support is released
# - "!dev/github_jira_sync.py"
# - "!dev/merge_spark_pr.py"
# - "!dev/run-tests-jenkins*"
# - "!dev/.rat-excludes"
DOCS:
- "docs/**/*"
- "**/README.md"
- "**/CONTRIBUTING.md"
EXAMPLES:
- "examples/**/*"
- "bin/run-example*"
# CORE needs to be updated when all/any are released upstream.
CORE:
# - any: ["core/**/*", "!**/*UI.scala", "!**/ui/**/*"] # If any file matches all of the globs defined in the list started by `any`, label is applied.
- "core/**/*"
- "common/kvstore/**/*"
- "common/network-common/**/*"
- "common/network-shuffle/**/*"
- "python/pyspark/**/*.py"
- "python/pyspark/tests/**/*.py"
SPARK SUBMIT:
- "bin/spark-submit*"
SPARK SHELL:
- "repl/**/*"
- "bin/spark-shell*"
SQL:
#- any: ["**/sql/**/*", "!python/pyspark/sql/avro/**/*", "!python/pyspark/sql/streaming.py", "!python/pyspark/sql/tests/test_streaming.py"]
- "**/sql/**/*"
- "common/unsafe/**/*"
#- "!python/pyspark/sql/avro/**/*"
#- "!python/pyspark/sql/streaming.py"
#- "!python/pyspark/sql/tests/test_streaming.py"
- "bin/spark-sql*"
- "bin/beeline*"
- "sbin/*thriftserver*.sh"
- "**/*SQL*.R"
- "**/DataFrame.R"
- "**/*WindowSpec.R"
- "**/*catalog.R"
- "**/*column.R"
- "**/*functions.R"
- "**/*group.R"
- "**/*schema.R"
- "**/*types.R"
AVRO:
- "external/avro/**/*"
- "python/pyspark/sql/avro/**/*"
DSTREAM:
- "streaming/**/*"
- "data/streaming/**/*"
- "external/kinesis*"
- "external/kafka*"
- "python/pyspark/streaming/**/*"
GRAPHX:
- "graphx/**/*"
- "data/graphx/**/*"
ML:
- "**/ml/**/*"
- "**/*mllib_*.R"
MLLIB:
- "**/spark/mllib/**/*"
- "mllib-local/**/*"
- "python/pyspark/mllib/**/*"
STRUCTURED STREAMING:
- "**/sql/**/streaming/**/*"
- "external/kafka-0-10-sql/**/*"
- "python/pyspark/sql/streaming.py"
- "python/pyspark/sql/tests/test_streaming.py"
- "**/*streaming.R"
PYTHON:
- "bin/pyspark*"
- "**/python/**/*"
R:
- "**/r/**/*"
- "**/R/**/*"
- "bin/sparkR*"
YARN:
- "resource-managers/yarn/**/*"
MESOS:
- "resource-managers/mesos/**/*"
- "sbin/*mesos*.sh"
KUBERNETES:
- "resource-managers/kubernetes/**/*"
WINDOWS:
- "**/*.cmd"
- "R/pkg/tests/fulltests/test_Windows.R"
WEB UI:
- "**/ui/**/*"
- "**/*UI.scala"
DEPLOY:
- "sbin/**/*"

Loading