Skip to content

[SPARK-31353][SQL] Set a time zone in DateTimeBenchmark and DateTimeRebaseBenchmark #28127

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaxGekk wants to merge 6 commits into from
Closed

[SPARK-31353][SQL] Set a time zone in DateTimeBenchmark and DateTimeRebaseBenchmark #28127

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
39308f4
Set time zone in DateTimeRebaseBenchmark
MaxGekk Apr 5, 2020
0abdff2
Set time zone in DateTimeBenchmark
MaxGekk Apr 5, 2020
1c14975
Re-gen DateTimeBenchmark results on JDK 8
MaxGekk Apr 5, 2020
daea6af
Re-gen DateTimeBenchmark results on JDK 8
MaxGekk Apr 5, 2020
dbe9ad6
Re-gen DateTimeRebaseBenchmark results on JDK 11
MaxGekk Apr 5, 2020
f3ed1d9
Re-gen DateTimeBenchmark results on JDK 11
MaxGekk Apr 5, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
434 changes: 217 additions & 217 deletions sql/core/benchmarks/DateTimeBenchmark-jdk11-results.txt
View file
Open in desktop

Large diffs are not rendered by default.

434 changes: 217 additions & 217 deletions sql/core/benchmarks/DateTimeBenchmark-results.txt
View file
Open in desktop

Large diffs are not rendered by default.

88 changes: 44 additions & 44 deletions sql/core/benchmarks/DateTimeRebaseBenchmark-jdk11-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,49 +6,49 @@ OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save dates to parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 9392 9392 0 10.6 93.9 1.0X
before 1582, noop 9324 9324 0 10.7 93.2 1.0X
after 1582, rebase off 20975 20975 0 4.8 209.7 0.4X
after 1582, rebase on 20016 20016 0 5.0 200.2 0.5X
before 1582, rebase off 20088 20088 0 5.0 200.9 0.5X
before 1582, rebase on 20310 20310 0 4.9 203.1 0.5X
after 1582, noop 18597 18597 0 5.4 186.0 1.0X
Copy link
Member Author

@MaxGekk MaxGekk Apr 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cloud-fan @HyukjinKwon Overhead of preparing input data depends on the system time zone. I opened this PR to have a base line for the optimisation #28119

before 1582, noop 10565 10565 0 9.5 105.7 1.8X
after 1582, rebase off 29811 29811 0 3.4 298.1 0.6X
after 1582, rebase on 31110 31110 0 3.2 311.1 0.6X
before 1582, rebase off 23144 23144 0 4.3 231.4 0.8X
before 1582, rebase on 23689 23689 0 4.2 236.9 0.8X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load dates from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off, rebase off 13371 13463 154 7.5 133.7 1.0X
after 1582, vec off, rebase on 13482 13533 57 7.4 134.8 1.0X
after 1582, vec on, rebase off 3713 3781 96 26.9 37.1 3.6X
after 1582, vec on, rebase on 5153 5173 29 19.4 51.5 2.6X
before 1582, vec off, rebase off 12939 12998 97 7.7 129.4 1.0X
before 1582, vec off, rebase on 14160 14255 85 7.1 141.6 0.9X
before 1582, vec on, rebase off 3748 3776 28 26.7 37.5 3.6X
before 1582, vec on, rebase on 5532 5575 54 18.1 55.3 2.4X
after 1582, vec off, rebase off 12944 13064 104 7.7 129.4 1.0X
after 1582, vec off, rebase on 13223 13255 34 7.6 132.2 1.0X
after 1582, vec on, rebase off 3656 3793 219 27.4 36.6 3.5X
after 1582, vec on, rebase on 5176 5205 38 19.3 51.8 2.5X
before 1582, vec off, rebase off 12926 12970 45 7.7 129.3 1.0X
before 1582, vec off, rebase on 13836 13872 40 7.2 138.4 0.9X
before 1582, vec on, rebase off 3664 3672 7 27.3 36.6 3.5X
before 1582, vec on, rebase on 6049 6078 30 16.5 60.5 2.1X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save timestamps to parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 2795 2795 0 35.8 27.9 1.0X
before 1582, noop 2806 2806 0 35.6 28.1 1.0X
after 1582, rebase off 16113 16113 0 6.2 161.1 0.2X
after 1582, rebase on 70198 70198 0 1.4 702.0 0.0X
before 1582, rebase off 16690 16690 0 6.0 166.9 0.2X
before 1582, rebase on 75706 75706 0 1.3 757.1 0.0X
after 1582, noop 2831 2831 0 35.3 28.3 1.0X
before 1582, noop 2769 2769 0 36.1 27.7 1.0X
after 1582, rebase off 17296 17296 0 5.8 173.0 0.2X
after 1582, rebase on 81434 81434 0 1.2 814.3 0.0X
before 1582, rebase off 17563 17563 0 5.7 175.6 0.2X
before 1582, rebase on 94977 94977 0 1.1 949.8 0.0X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load timestamps from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off, rebase off 15631 15753 111 6.4 156.3 1.0X
after 1582, vec off, rebase on 45834 46027 193 2.2 458.3 0.3X
after 1582, vec on, rebase off 4883 4964 70 20.5 48.8 3.2X
after 1582, vec on, rebase on 34514 34563 63 2.9 345.1 0.5X
before 1582, vec off, rebase off 15253 15354 104 6.6 152.5 1.0X
before 1582, vec off, rebase on 47353 47412 59 2.1 473.5 0.3X
before 1582, vec on, rebase off 4848 4894 69 20.6 48.5 3.2X
before 1582, vec on, rebase on 36125 36143 22 2.8 361.3 0.4X
after 1582, vec off, rebase off 14824 14920 87 6.7 148.2 1.0X
after 1582, vec off, rebase on 54660 54859 210 1.8 546.6 0.3X
after 1582, vec on, rebase off 4876 4954 69 20.5 48.8 3.0X
after 1582, vec on, rebase on 44509 44573 65 2.2 445.1 0.3X
before 1582, vec off, rebase off 14909 14939 30 6.7 149.1 1.0X
before 1582, vec off, rebase on 56092 56346 326 1.8 560.9 0.3X
before 1582, vec on, rebase off 4846 4858 13 20.6 48.5 3.1X
before 1582, vec on, rebase on 46267 46341 64 2.2 462.7 0.3X


================================================================================================
Expand All @@ -59,36 +59,36 @@ OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save dates to ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 9160 9160 0 10.9 91.6 1.0X
before 1582, noop 9235 9235 0 10.8 92.4 1.0X
after 1582 17154 17154 0 5.8 171.5 0.5X
before 1582 17545 17545 0 5.7 175.5 0.5X
after 1582, noop 18035 18035 0 5.5 180.3 1.0X
before 1582, noop 10571 10571 0 9.5 105.7 1.7X
after 1582 26341 26341 0 3.8 263.4 0.7X
before 1582 19589 19589 0 5.1 195.9 0.9X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load dates from ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off 21024 21146 205 4.8 210.2 1.0X
after 1582, vec on 3814 3838 21 26.2 38.1 5.5X
before 1582, vec off 24293 24347 82 4.1 242.9 0.9X
before 1582, vec on 4143 4168 22 24.1 41.4 5.1X
after 1582, vec off 34955 35025 98 2.9 349.6 1.0X
after 1582, vec on 3892 3942 43 25.7 38.9 9.0X
before 1582, vec off 33360 33457 87 3.0 333.6 1.0X
before 1582, vec on 4514 4526 11 22.2 45.1 7.7X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save timestamps to ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 2797 2797 0 35.8 28.0 1.0X
before 1582, noop 2826 2826 0 35.4 28.3 1.0X
after 1582 40021 40021 0 2.5 400.2 0.1X
before 1582 41500 41500 0 2.4 415.0 0.1X
after 1582, noop 2867 2867 0 34.9 28.7 1.0X
before 1582, noop 2812 2812 0 35.6 28.1 1.0X
after 1582 57133 57133 0 1.8 571.3 0.1X
before 1582 53435 53435 0 1.9 534.4 0.1X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load timestamps from ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off 32517 32541 23 3.1 325.2 1.0X
after 1582, vec on 19644 19725 128 5.1 196.4 1.7X
before 1582, vec off 37204 37305 104 2.7 372.0 0.9X
before 1582, vec on 24105 24120 13 4.1 241.1 1.3X
after 1582, vec off 38475 38551 127 2.6 384.7 1.0X
after 1582, vec on 31382 31537 136 3.2 313.8 1.2X
before 1582, vec off 42159 42240 128 2.4 421.6 0.9X
before 1582, vec on 34735 35129 393 2.9 347.3 1.1X


Loading