Skip to content

[SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results #28906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
MaxGekk wants to merge 4 commits into from
Closed

[SPARK-32072][CORE][TESTS] Fix table formatting with benchmark results #28906

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9c1955f
Fix table formatting with benchmark results
MaxGekk Jun 23, 2020
ec4e19e
Re-gen /MakeDateTimeBenchmark-jdk11-results.txt
MaxGekk Jun 23, 2020
f0598b5
Re-gen /MakeDateTimeBenchmark-results.txt
MaxGekk Jun 23, 2020
e1a242e
Merge remote-tracking branch 'origin/master' into benchmark-table-for…
MaxGekk Jun 23, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions core/src/test/scala/org/apache/spark/benchmark/Benchmark.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,11 +112,12 @@ private[spark] class Benchmark(
// The results are going to be processor specific so it is useful to include that.
out.println(Benchmark.getJVMOSInfo())
out.println(Benchmark.getProcessorName())
out.printf("%-40s %14s %14s %11s %12s %13s %10s\n", name + ":", "Best Time(ms)", "Avg Time(ms)", "Stdev(ms)", "Rate(M/s)",
"Per Row(ns)", "Relative")
out.println("-" * 120)
val nameLen = Math.max(40, Math.max(name.length, benchmarks.map(_.name.length).max))
out.printf(s"%-${nameLen}s %14s %14s %11s %12s %13s %10s\n",
name + ":", "Best Time(ms)", "Avg Time(ms)", "Stdev(ms)", "Rate(M/s)", "Per Row(ns)", "Relative")
out.println("-" * (nameLen + 80))
results.zip(benchmarks).foreach { case (result, benchmark) =>
out.printf("%-40s %14s %14s %11s %12s %13s %10s\n",
out.printf(s"%-${nameLen}s %14s %14s %11s %12s %13s %10s\n",
benchmark.name,
"%5.0f" format result.bestMs,
"%4.0f" format result.avgMs,
Expand Down
28 changes: 14 additions & 14 deletions sql/core/benchmarks/MakeDateTimeBenchmark-jdk11-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,21 @@ OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-106
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
make_date(): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
prepare make_date() 3170 3321 230 31.5 31.7 1.0X
make_date(2019, 9, 16) 2359 2566 343 42.4 23.6 1.3X
make_date(*, *, *) 4398 4455 53 22.7 44.0 0.7X
prepare make_date() 3214 3344 209 31.1 32.1 1.0X
make_date(2019, 9, 16) 2342 2348 6 42.7 23.4 1.4X
make_date(*, *, *) 4485 4533 56 22.3 44.8 0.7X

OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
make_timestamp(): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
prepare make_timestamp() 3492 3536 70 0.3 3491.7 1.0X
make_timestamp(2019, 1, 2, 3, 4, 50.123456) 94 110 16 10.7 93.6 37.3X
make_timestamp(2019, 1, 2, 3, 4, 60.000000) 78 83 8 12.8 78.0 44.7X
make_timestamp(2019, 12, 31, 23, 59, 60.00) 66 72 6 15.1 66.2 52.7X
make_timestamp(*, *, *, 3, 4, 50.123456) 306 313 12 3.3 305.9 11.4X
make_timestamp(*, *, *, *, *, 0) 282 284 2 3.5 281.9 12.4X
make_timestamp(*, *, *, *, *, 60.0) 281 288 6 3.6 281.0 12.4X
make_timestamp(2019, 1, 2, *, *, *) 3637 3641 3 0.3 3636.7 1.0X
make_timestamp(*, *, *, *, *, *) 3635 3648 12 0.3 3634.9 1.0X
make_timestamp(): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
prepare make_timestamp() 3744 3775 35 0.3 3744.1 1.0X
make_timestamp(2019, 1, 2, 3, 4, 50.123456) 82 91 9 12.2 82.3 45.5X
make_timestamp(2019, 1, 2, 3, 4, 60.000000) 81 89 7 12.4 81.0 46.2X
make_timestamp(2019, 12, 31, 23, 59, 60.00) 70 80 9 14.3 69.9 53.5X
make_timestamp(*, *, *, 3, 4, 50.123456) 308 314 7 3.2 308.1 12.2X
make_timestamp(*, *, *, *, *, 0) 302 316 14 3.3 301.9 12.4X
make_timestamp(*, *, *, *, *, 60.0) 290 296 6 3.4 290.4 12.9X
make_timestamp(2019, 1, 2, *, *, *) 3888 3902 15 0.3 3888.1 1.0X
make_timestamp(*, *, *, *, *, *) 3902 3908 8 0.3 3901.6 1.0X

28 changes: 14 additions & 14 deletions sql/core/benchmarks/MakeDateTimeBenchmark-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,21 +2,21 @@ OpenJDK 64-Bit Server VM 1.8.0_252-8u252-b09-1~18.04-b09 on Linux 4.15.0-1063-aw
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
make_date(): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
prepare make_date() 2951 3182 355 33.9 29.5 1.0X
make_date(2019, 9, 16) 2325 2415 101 43.0 23.2 1.3X
make_date(*, *, *) 4556 4573 17 21.9 45.6 0.6X
prepare make_date() 3309 3429 110 30.2 33.1 1.0X
Copy link
Member

@srowen srowen Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The formatting changes are positive of course. Most of the numbers didn't change much at all; these kinda of did, some much more than a few stdevs. Just checking, it was on the same hardware as in the heading?

Copy link
Contributor

@cloud-fan cloud-fan Jun 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, as the head is written by the benchmark framework automatically.

make_date(2019, 9, 16) 2336 2359 23 42.8 23.4 1.4X
make_date(*, *, *) 4588 4618 27 21.8 45.9 0.7X

OpenJDK 64-Bit Server VM 1.8.0_252-8u252-b09-1~18.04-b09 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
make_timestamp(): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
prepare make_timestamp() 3677 3730 87 0.3 3677.0 1.0X
make_timestamp(2019, 1, 2, 3, 4, 50.123456) 76 92 15 13.1 76.5 48.1X
make_timestamp(2019, 1, 2, 3, 4, 60.000000) 70 72 3 14.4 69.7 52.8X
make_timestamp(2019, 12, 31, 23, 59, 60.00) 68 75 12 14.7 68.0 54.1X
make_timestamp(*, *, *, 3, 4, 50.123456) 266 277 12 3.8 265.8 13.8X
make_timestamp(*, *, *, *, *, 0) 258 286 46 3.9 258.3 14.2X
make_timestamp(*, *, *, *, *, 60.0) 261 269 11 3.8 260.7 14.1X
make_timestamp(2019, 1, 2, *, *, *) 3748 3762 22 0.3 3748.3 1.0X
make_timestamp(*, *, *, *, *, *) 3761 3762 2 0.3 3760.9 1.0X
make_timestamp(): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
---------------------------------------------------------------------------------------------------------------------------
prepare make_timestamp() 3651 3697 58 0.3 3651.4 1.0X
make_timestamp(2019, 1, 2, 3, 4, 50.123456) 89 99 10 11.3 88.6 41.2X
make_timestamp(2019, 1, 2, 3, 4, 60.000000) 72 73 1 13.9 72.1 50.6X
make_timestamp(2019, 12, 31, 23, 59, 60.00) 66 68 3 15.2 65.8 55.5X
make_timestamp(*, *, *, 3, 4, 50.123456) 265 272 6 3.8 265.1 13.8X
make_timestamp(*, *, *, *, *, 0) 259 266 6 3.9 259.1 14.1X
make_timestamp(*, *, *, *, *, 60.0) 271 278 9 3.7 271.2 13.5X
make_timestamp(2019, 1, 2, *, *, *) 3838 3850 12 0.3 3837.7 1.0X
make_timestamp(*, *, *, *, *, *) 3854 3877 20 0.3 3853.8 0.9X