Skip to content
Closed
Show file tree
Hide file tree
Changes from 11 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,14 @@ object SQLConf {
.booleanConf
.createWithDefault(true)

val PARQUET_FILTER_PUSHDOWN_DECIMAL_ENABLED =
buildConf("spark.sql.parquet.filterPushdown.decimal")
.doc("If true, enables Parquet filter push-down optimization for Decimal. " +
"This configuration only has an effect when 'spark.sql.parquet.filterPushdown' is enabled.")
.internal()
.booleanConf
.createWithDefault(true)

val PARQUET_FILTER_PUSHDOWN_STRING_STARTSWITH_ENABLED =
buildConf("spark.sql.parquet.filterPushdown.string.startsWith")
.doc("If true, enables Parquet filter push-down optimization for string startsWith function. " +
Expand Down Expand Up @@ -1482,6 +1490,8 @@ class SQLConf extends Serializable with Logging {

def parquetFilterPushDownDate: Boolean = getConf(PARQUET_FILTER_PUSHDOWN_DATE_ENABLED)

def parquetFilterPushDownDecimal: Boolean = getConf(PARQUET_FILTER_PUSHDOWN_DECIMAL_ENABLED)

def parquetFilterPushDownStringStartWith: Boolean =
getConf(PARQUET_FILTER_PUSHDOWN_STRING_STARTSWITH_ENABLED)

Expand Down
96 changes: 48 additions & 48 deletions sql/core/benchmarks/FilterPushdownBenchmark-results.txt
Original file line number Diff line number Diff line change
Expand Up @@ -292,120 +292,120 @@ Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 1 decimal(9, 2) row (value = 7864320): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 3785 / 3867 4.2 240.6 1.0X
Parquet Vectorized (Pushdown) 3820 / 3928 4.1 242.9 1.0X
Native ORC Vectorized 3981 / 4049 4.0 253.1 1.0X
Native ORC Vectorized (Pushdown) 702 / 735 22.4 44.6 5.4X
Parquet Vectorized 4407 / 4852 3.6 280.2 1.0X
Parquet Vectorized (Pushdown) 1602 / 1634 9.8 101.8 2.8X

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any thoughts on why this is slower than the other tests with decimal(18, 2) and decimal(38, 2)? This seems very strange to me.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it is that the data is more dense, so we need to read more values in the row group that contains the one we're looking for?

@wangyum wangyum Jul 12, 2018

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because 1024 * 1024 * 15 is out of decimal(9, 2) range. so no stats for that column. I will update benchmark.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understand. That's less than 2^24, so it should fit in an int. It should also fit in 8 base-ten digits so decimal(9,2) should work. And last, if the values don't fit in an int, I'm not sure how we would be able to store them in the first place, regardless of how stats are handled.

Did you verify that there are no stats for the file produced here? If that's the case, it would make sense with these numbers. I think we just need to look for a different reason why stats are missing.

@wangyum wangyum Jul 13, 2018

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is a test:

// decimal(9, 2) max values is 9999999.99
// 1024 * 1024 * 15 =          15728640
val path = "/tmp/spark/parquet"
spark.range(1024 * 1024 * 15).selectExpr("cast((id) as decimal(9, 2)) as id").orderBy("id").write.mode("overwrite").parquet(path)

The generated parquet metadata:

$ java -jar ./parquet-tools/target/parquet-tools-1.10.1-SNAPSHOT.jar meta  /tmp/spark/parquet
file:        file:/tmp/spark/parquet/part-00000-26b38556-494a-4b89-923e-69ea73365488-c000.snappy.parquet 
creator:     parquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a) 
extra:       org.apache.spark.sql.parquet.row.metadata = {"type":"struct","fields":[{"name":"id","type":"decimal(9,2)","nullable":true,"metadata":{}}]} 

file schema: spark_schema 
--------------------------------------------------------------------------------
id:          OPTIONAL INT32 O:DECIMAL R:0 D:1

row group 1: RC:5728640 TS:36 OFFSET:4 
--------------------------------------------------------------------------------
id:           INT32 SNAPPY DO:0 FPO:4 SZ:38/36/0.95 VC:5728640 ENC:PLAIN,BIT_PACKED,RLE ST:[no stats for this column]
file:        file:/tmp/spark/parquet/part-00001-26b38556-494a-4b89-923e-69ea73365488-c000.snappy.parquet 
creator:     parquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a) 
extra:       org.apache.spark.sql.parquet.row.metadata = {"type":"struct","fields":[{"name":"id","type":"decimal(9,2)","nullable":true,"metadata":{}}]} 

file schema: spark_schema 
--------------------------------------------------------------------------------
id:          OPTIONAL INT32 O:DECIMAL R:0 D:1

row group 1: RC:651016 TS:2604209 OFFSET:4 
--------------------------------------------------------------------------------
id:           INT32 SNAPPY DO:0 FPO:4 SZ:2604325/2604209/1.00 VC:651016 ENC:PLAIN,BIT_PACKED,RLE ST:[min: 0.00, max: 651015.00, num_nulls: 0]
file:        file:/tmp/spark/parquet/part-00002-26b38556-494a-4b89-923e-69ea73365488-c000.snappy.parquet 
creator:     parquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a) 
extra:       org.apache.spark.sql.parquet.row.metadata = {"type":"struct","fields":[{"name":"id","type":"decimal(9,2)","nullable":true,"metadata":{}}]} 

file schema: spark_schema 
--------------------------------------------------------------------------------
id:          OPTIONAL INT32 O:DECIMAL R:0 D:1

row group 1: RC:3231146 TS:12925219 OFFSET:4 
--------------------------------------------------------------------------------
id:           INT32 SNAPPY DO:0 FPO:4 SZ:12925864/12925219/1.00 VC:3231146 ENC:PLAIN,BIT_PACKED,RLE ST:[min: 651016.00, max: 3882161.00, num_nulls: 0]
file:        file:/tmp/spark/parquet/part-00003-26b38556-494a-4b89-923e-69ea73365488-c000.snappy.parquet 
creator:     parquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a) 
extra:       org.apache.spark.sql.parquet.row.metadata = {"type":"struct","fields":[{"name":"id","type":"decimal(9,2)","nullable":true,"metadata":{}}]} 

file schema: spark_schema 
--------------------------------------------------------------------------------
id:          OPTIONAL INT32 O:DECIMAL R:0 D:1

row group 1: RC:2887956 TS:11552408 OFFSET:4 
--------------------------------------------------------------------------------
id:           INT32 SNAPPY DO:0 FPO:4 SZ:11552986/11552408/1.00 VC:2887956 ENC:PLAIN,BIT_PACKED,RLE ST:[min: 3882162.00, max: 6770117.00, num_nulls: 0]
file:        file:/tmp/spark/parquet/part-00004-26b38556-494a-4b89-923e-69ea73365488-c000.snappy.parquet 
creator:     parquet-mr version 1.10.0 (build 031a6654009e3b82020012a18434c582bd74c73a) 
extra:       org.apache.spark.sql.parquet.row.metadata = {"type":"struct","fields":[{"name":"id","type":"decimal(9,2)","nullable":true,"metadata":{}}]} 

file schema: spark_schema 
--------------------------------------------------------------------------------
id:          OPTIONAL INT32 O:DECIMAL R:0 D:1

row group 1: RC:3229882 TS:12920163 OFFSET:4 
--------------------------------------------------------------------------------
id:           INT32 SNAPPY DO:0 FPO:4 SZ:12920808/12920163/1.00 VC:3229882 ENC:PLAIN,BIT_PACKED,RLE ST:[min: 6770118.00, max: 9999999.00, num_nulls: 0]

As you can see file:/tmp/spark/parquet/part-00000-26b38556-494a-4b89-923e-69ea73365488-c000.snappy.parquet have not generated stats for that column.

scala> spark.read.parquet(path).filter("id is null").count
res0: Long = 5728640 

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I see. The tenths and hundredths are always 0, which makes the precision-8 numbers actually precision-10. It is still odd that this is causing Parquet to have no stats, but I'm happy with the fix. Thanks for explaining.

Native ORC Vectorized 4814 / 5094 3.3 306.1 0.9X
Native ORC Vectorized (Pushdown) 836 / 984 18.8 53.1 5.3X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 10% decimal(9, 2) rows (value < 1572864): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 4694 / 4813 3.4 298.4 1.0X
Parquet Vectorized (Pushdown) 4839 / 4907 3.3 307.6 1.0X
Native ORC Vectorized 4943 / 5032 3.2 314.2 0.9X
Native ORC Vectorized (Pushdown) 2043 / 2085 7.7 129.9 2.3X
Parquet Vectorized 5310 / 5626 3.0 337.6 1.0X
Parquet Vectorized (Pushdown) 2843 / 3043 5.5 180.7 1.9X
Native ORC Vectorized 5739 / 5907 2.7 364.9 0.9X
Native ORC Vectorized (Pushdown) 2316 / 2383 6.8 147.2 2.3X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 50% decimal(9, 2) rows (value < 7864320): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 8321 / 8472 1.9 529.0 1.0X
Parquet Vectorized (Pushdown) 8125 / 8471 1.9 516.6 1.0X
Native ORC Vectorized 8524 / 8616 1.8 541.9 1.0X
Native ORC Vectorized (Pushdown) 7961 / 8383 2.0 506.1 1.0X
Parquet Vectorized 8719 / 8877 1.8 554.4 1.0X
Parquet Vectorized (Pushdown) 8206 / 8573 1.9 521.7 1.1X
Native ORC Vectorized 9628 / 10197 1.6 612.2 0.9X
Native ORC Vectorized (Pushdown) 8168 / 8301 1.9 519.3 1.1X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 90% decimal(9, 2) rows (value < 14155776): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 9587 / 10112 1.6 609.5 1.0X
Parquet Vectorized (Pushdown) 9726 / 10370 1.6 618.3 1.0X
Native ORC Vectorized 10119 / 11147 1.6 643.4 0.9X
Native ORC Vectorized (Pushdown) 9366 / 9497 1.7 595.5 1.0X
Parquet Vectorized 10223 / 10371 1.5 649.9 1.0X
Parquet Vectorized (Pushdown) 10348 / 10485 1.5 657.9 1.0X
Native ORC Vectorized 11283 / 11355 1.4 717.4 0.9X
Native ORC Vectorized (Pushdown) 10541 / 10669 1.5 670.2 1.0X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 1 decimal(18, 2) row (value = 7864320): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 4060 / 4093 3.9 258.1 1.0X
Parquet Vectorized (Pushdown) 4037 / 4125 3.9 256.6 1.0X
Native ORC Vectorized 4756 / 4811 3.3 302.4 0.9X
Native ORC Vectorized (Pushdown) 824 / 889 19.1 52.4 4.9X
Parquet Vectorized 4701 / 6416 3.3 298.9 1.0X
Parquet Vectorized (Pushdown) 128 / 164 122.8 8.1 36.7X
Native ORC Vectorized 5698 / 7904 2.8 362.3 0.8X
Native ORC Vectorized (Pushdown) 913 / 942 17.2 58.0 5.2X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 10% decimal(18, 2) rows (value < 1572864): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 5157 / 5271 3.0 327.9 1.0X
Parquet Vectorized (Pushdown) 5051 / 5141 3.1 321.1 1.0X
Native ORC Vectorized 5723 / 6146 2.7 363.9 0.9X
Native ORC Vectorized (Pushdown) 2198 / 2317 7.2 139.8 2.3X
Parquet Vectorized 5376 / 5461 2.9 341.8 1.0X
Parquet Vectorized (Pushdown) 1479 / 1543 10.6 94.0 3.6X
Native ORC Vectorized 6640 / 6748 2.4 422.2 0.8X
Native ORC Vectorized (Pushdown) 2438 / 2479 6.5 155.0 2.2X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 50% decimal(18, 2) rows (value < 7864320): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 8608 / 8647 1.8 547.3 1.0X
Parquet Vectorized (Pushdown) 8471 / 8584 1.9 538.6 1.0X
Native ORC Vectorized 9249 / 10048 1.7 588.0 0.9X
Native ORC Vectorized (Pushdown) 7645 / 8091 2.1 486.1 1.1X
Parquet Vectorized 9224 / 9356 1.7 586.5 1.0X
Parquet Vectorized (Pushdown) 7172 / 7415 2.2 456.0 1.3X
Native ORC Vectorized 11017 / 11408 1.4 700.4 0.8X
Native ORC Vectorized (Pushdown) 8771 / 10218 1.8 557.7 1.1X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 90% decimal(18, 2) rows (value < 14155776): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 11658 / 11888 1.3 741.2 1.0X
Parquet Vectorized (Pushdown) 11812 / 12098 1.3 751.0 1.0X
Native ORC Vectorized 12943 / 13312 1.2 822.9 0.9X
Native ORC Vectorized (Pushdown) 13139 / 13465 1.2 835.4 0.9X
Parquet Vectorized 13933 / 15990 1.1 885.8 1.0X
Parquet Vectorized (Pushdown) 12683 / 12942 1.2 806.4 1.1X
Native ORC Vectorized 16344 / 20196 1.0 1039.1 0.9X
Native ORC Vectorized (Pushdown) 15162 / 16627 1.0 964.0 0.9X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 1 decimal(38, 2) row (value = 7864320): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 5491 / 5716 2.9 349.1 1.0X
Parquet Vectorized (Pushdown) 5515 / 5615 2.9 350.6 1.0X
Native ORC Vectorized 4582 / 4654 3.4 291.3 1.2X
Native ORC Vectorized (Pushdown) 815 / 861 19.3 51.8 6.7X
Parquet Vectorized 7102 / 8282 2.2 451.5 1.0X
Parquet Vectorized (Pushdown) 124 / 150 126.4 7.9 57.1X
Native ORC Vectorized 5811 / 6883 2.7 369.5 1.2X
Native ORC Vectorized (Pushdown) 1121 / 1502 14.0 71.3 6.3X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 10% decimal(38, 2) rows (value < 1572864): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 6432 / 6527 2.4 409.0 1.0X
Parquet Vectorized (Pushdown) 6513 / 6607 2.4 414.1 1.0X
Native ORC Vectorized 5618 / 6085 2.8 357.2 1.1X
Native ORC Vectorized (Pushdown) 2403 / 2443 6.5 152.8 2.7X
Parquet Vectorized 6894 / 7562 2.3 438.3 1.0X
Parquet Vectorized (Pushdown) 1863 / 1980 8.4 118.4 3.7X
Native ORC Vectorized 6812 / 6848 2.3 433.1 1.0X
Native ORC Vectorized (Pushdown) 2511 / 2598 6.3 159.7 2.7X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 50% decimal(38, 2) rows (value < 7864320): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 11041 / 11467 1.4 701.9 1.0X
Parquet Vectorized (Pushdown) 10909 / 11484 1.4 693.5 1.0X
Native ORC Vectorized 9860 / 10436 1.6 626.9 1.1X
Native ORC Vectorized (Pushdown) 7908 / 8069 2.0 502.8 1.4X
Parquet Vectorized 11732 / 12183 1.3 745.9 1.0X
Parquet Vectorized (Pushdown) 8912 / 9945 1.8 566.6 1.3X
Native ORC Vectorized 11499 / 12387 1.4 731.1 1.0X
Native ORC Vectorized (Pushdown) 9328 / 9382 1.7 593.1 1.3X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Select 90% decimal(38, 2) rows (value < 14155776): Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Parquet Vectorized 14816 / 16877 1.1 942.0 1.0X
Parquet Vectorized (Pushdown) 15383 / 15740 1.0 978.0 1.0X
Native ORC Vectorized 14408 / 14771 1.1 916.0 1.0X
Native ORC Vectorized (Pushdown) 13968 / 14805 1.1 888.1 1.1X
Parquet Vectorized 16272 / 16328 1.0 1034.6 1.0X
Parquet Vectorized (Pushdown) 15714 / 18100 1.0 999.1 1.0X
Native ORC Vectorized 16539 / 18897 1.0 1051.5 1.0X
Native ORC Vectorized (Pushdown) 16328 / 17306 1.0 1038.1 1.0X


================================================================================================
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,7 @@ class ParquetFileFormat
// Whole stage codegen (PhysicalRDD) is able to deal with batches directly
val returningBatch = supportBatch(sparkSession, resultSchema)
val pushDownDate = sqlConf.parquetFilterPushDownDate
val pushDownDecimal = sqlConf.parquetFilterPushDownDecimal
val pushDownStringStartWith = sqlConf.parquetFilterPushDownStringStartWith

(file: PartitionedFile) => {
Expand Down Expand Up @@ -376,7 +377,7 @@ class ParquetFileFormat
// Collects all converted Parquet filter predicates. Notice that not all predicates can be
// converted (`ParquetFilters.createFilter` returns an `Option`). That's why a `flatMap`
// is used here.
.flatMap(new ParquetFilters(pushDownDate, pushDownStringStartWith)
.flatMap(new ParquetFilters(pushDownDate, pushDownDecimal, pushDownStringStartWith)
.createFilter(parquetSchema, _))
.reduceOption(FilterApi.and)
} else {
Expand Down
Loading