Skip to content

[SPARK-25658][SQL][TEST] Refactor HashByteArrayBenchmark to use main method #22652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wangyum wants to merge 6 commits into from
Closed

[SPARK-25658][SQL][TEST] Refactor HashByteArrayBenchmark to use main method #22652

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3e6a058
refactor HashByteArrayBenchmark
wangyum Oct 6, 2018
bdb0549
Fix scala doc
wangyum Oct 6, 2018
b5190d4
Update result (#17)
dongjoon-hyun Oct 6, 2018
cc268ca
Fix scala doc
wangyum Oct 6, 2018
11f2bbe
Fix scala doc
wangyum Oct 7, 2018
0a7741a
Fix scala doc
wangyum Oct 7, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 86 additions & 0 deletions sql/catalyst/benchmarks/HashByteArrayBenchmark-results.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
================================================================================================
Benchmark for MurMurHash 3 and xxHash64
================================================================================================

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Hash byte arrays with length 8: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 12 / 15 181.7 5.5 1.0X
xxHash 64-bit 14 / 18 146.0 6.8 0.8X
HiveHasher 11 / 13 183.0 5.5 1.0X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Hash byte arrays with length 16: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 18 / 22 113.4 8.8 1.0X
xxHash 64-bit 17 / 24 120.3 8.3 1.1X
HiveHasher 21 / 31 97.9 10.2 0.9X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Hash byte arrays with length 24: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 25 / 27 84.8 11.8 1.0X
xxHash 64-bit 20 / 22 103.9 9.6 1.2X
HiveHasher 32 / 34 65.0 15.4 0.8X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Hash byte arrays with length 31: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 36 / 38 58.5 17.1 1.0X
xxHash 64-bit 29 / 31 73.2 13.7 1.3X
HiveHasher 43 / 46 48.7 20.5 0.8X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Hash byte arrays with length 95: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 87 / 92 24.2 41.3 1.0X
xxHash 64-bit 57 / 59 37.1 26.9 1.5X
HiveHasher 163 / 167 12.9 77.6 0.5X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Hash byte arrays with length 287: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 263 / 271 8.0 125.3 1.0X
xxHash 64-bit 95 / 100 22.1 45.3 2.8X
HiveHasher 537 / 540 3.9 256.0 0.5X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Hash byte arrays with length 1055: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 976 / 1065 2.1 465.2 1.0X
xxHash 64-bit 233 / 248 9.0 110.9 4.2X
HiveHasher 2054 / 2054 1.0 979.3 0.5X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Hash byte arrays with length 2079: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 1874 / 1878 1.1 893.8 1.0X
xxHash 64-bit 412 / 428 5.1 196.6 4.5X
HiveHasher 4044 / 4049 0.5 1928.5 0.5X

Java HotSpot(TM) 64-Bit Server VM 1.8.0_151-b12 on Mac OS X 10.12.6
Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz

Hash byte arrays with length 8223: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 7744 / 7938 0.3 3692.8 1.0X
xxHash 64-bit 1871 / 1893 1.1 892.3 4.1X
HiveHasher 16043 / 16073 0.1 7650.1 0.5X


118 changes: 23 additions & 95 deletions sql/catalyst/src/test/scala/org/apache/spark/sql/HashByteArrayBenchmark.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,22 @@ package org.apache.spark.sql

import java.util.Random

import org.apache.spark.benchmark.Benchmark
import org.apache.spark.benchmark.{Benchmark, BenchmarkBase}
import org.apache.spark.sql.catalyst.expressions.{HiveHasher, XXH64}
import org.apache.spark.unsafe.Platform
import org.apache.spark.unsafe.hash.Murmur3_x86_32

/**
* Synthetic benchmark for MurMurHash 3 and xxHash64.
* To run this benchmark:
* {{{
* 1. without sbt: bin/spark-submit --class <this class> <spark sql test jar>
* 2. build/sbt "sql/test:runMain <this class>"
* 3. generate result: SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain <this class>"

@dongjoon-hyun dongjoon-hyun Oct 6, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sql/test -> catalyst/test.
If we use sql/test, the result will be generated in sql module instead of catalyst module.

@dongjoon-hyun dongjoon-hyun Oct 6, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's for both line 32 and 33.

@wangyum wangyum Oct 6, 2018

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. Thanks

@dongjoon-hyun dongjoon-hyun Oct 6, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, please check line 31, too.

* Results will be written to "benchmarks/HashByteArrayBenchmark-results.txt".
* }}}
*/
object HashByteArrayBenchmark {
object HashByteArrayBenchmark extends BenchmarkBase {
def test(length: Int, seed: Long, numArrays: Int, iters: Int): Unit = {
val random = new Random(seed)
val arrays = Array.fill[Array[Byte]](numArrays) {
Expand All @@ -36,8 +43,8 @@ object HashByteArrayBenchmark {
bytes
}

val benchmark =
new Benchmark("Hash byte arrays with length " + length, iters * numArrays.toLong)
val benchmark = new Benchmark(
"Hash byte arrays with length " + length, iters * numArrays.toLong, output = output)
benchmark.addCase("Murmur3_x86_32") { _: Int =>
var sum = 0L
for (_ <- 0L until iters) {
Expand Down Expand Up @@ -74,96 +81,17 @@ object HashByteArrayBenchmark {
benchmark.run()
}

def main(args: Array[String]): Unit = {
/*
Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
Hash byte arrays with length 8: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 12 / 16 174.3 5.7 1.0X
xxHash 64-bit 17 / 22 120.0 8.3 0.7X
HiveHasher 13 / 15 162.1 6.2 0.9X
*/
test(8, 42L, 1 << 10, 1 << 11)

/*
Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
Hash byte arrays with length 16: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 19 / 22 107.6 9.3 1.0X
xxHash 64-bit 20 / 24 104.6 9.6 1.0X
HiveHasher 24 / 28 87.0 11.5 0.8X
*/
test(16, 42L, 1 << 10, 1 << 11)

/*
Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
Hash byte arrays with length 24: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 28 / 32 74.8 13.4 1.0X
xxHash 64-bit 24 / 29 87.3 11.5 1.2X
HiveHasher 36 / 41 57.7 17.3 0.8X
*/
test(24, 42L, 1 << 10, 1 << 11)

// Add 31 to all arrays to create worse case alignment for xxHash.
/*
Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
Hash byte arrays with length 31: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 41 / 45 51.1 19.6 1.0X
xxHash 64-bit 36 / 44 58.8 17.0 1.2X
HiveHasher 49 / 54 42.6 23.5 0.8X
*/
test(31, 42L, 1 << 10, 1 << 11)

/*
Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
Hash byte arrays with length 95: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 100 / 110 21.0 47.7 1.0X
xxHash 64-bit 74 / 78 28.2 35.5 1.3X
HiveHasher 189 / 196 11.1 90.3 0.5X
*/
test(64 + 31, 42L, 1 << 10, 1 << 11)

/*
Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
Hash byte arrays with length 287: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 299 / 311 7.0 142.4 1.0X
xxHash 64-bit 113 / 122 18.5 54.1 2.6X
HiveHasher 620 / 624 3.4 295.5 0.5X
*/
test(256 + 31, 42L, 1 << 10, 1 << 11)

/*
Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
Hash byte arrays with length 1055: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 1068 / 1070 2.0 509.1 1.0X
xxHash 64-bit 306 / 315 6.9 145.9 3.5X
HiveHasher 2316 / 2369 0.9 1104.3 0.5X
*/
test(1024 + 31, 42L, 1 << 10, 1 << 11)

/*
Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
Hash byte arrays with length 2079: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 2252 / 2274 0.9 1074.1 1.0X
xxHash 64-bit 534 / 580 3.9 254.6 4.2X
HiveHasher 4739 / 4786 0.4 2259.8 0.5X
*/
test(2048 + 31, 42L, 1 << 10, 1 << 11)

/*
Intel(R) Core(TM) i7-4558U CPU @ 2.80GHz
Hash byte arrays with length 8223: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------
Murmur3_x86_32 9249 / 9586 0.2 4410.5 1.0X
xxHash 64-bit 2897 / 3241 0.7 1381.6 3.2X
HiveHasher 19392 / 20211 0.1 9246.6 0.5X
*/
test(8192 + 31, 42L, 1 << 10, 1 << 11)
override def runBenchmarkSuite(): Unit = {
runBenchmark("Benchmark for MurMurHash 3 and xxHash64") {
test(8, 42L, 1 << 10, 1 << 11)
test(16, 42L, 1 << 10, 1 << 11)
test(24, 42L, 1 << 10, 1 << 11)
test(31, 42L, 1 << 10, 1 << 11)
test(64 + 31, 42L, 1 << 10, 1 << 11)
test(256 + 31, 42L, 1 << 10, 1 << 11)
test(1024 + 31, 42L, 1 << 10, 1 << 11)
test(2048 + 31, 42L, 1 << 10, 1 << 11)
test(8192 + 31, 42L, 1 << 10, 1 << 11)
}
}
}