Use XXH3 hash instead of SIP128 for stable hashing #97952

Kobzol · 2022-06-10T16:51:01Z

This is an experiment to if XXH3 could be faster than SIP128 for incremental stable hashing.

r? @michaelwoerister

Kobzol · 2022-06-10T16:51:14Z

@bors try @rust-timer queue

rust-timer · 2022-06-10T16:51:15Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2022-06-10T16:51:21Z

⌛ Trying commit f12b0ed7d4798058c0f5dbf07d9ad6d8f2ebc185 with merge 2d3d00c03a8a9f87a5384ed886d9996e17316b96...

Kobzol · 2022-06-10T16:57:44Z

@bors try @rust-timer queue

rust-timer · 2022-06-10T16:57:46Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2022-06-10T16:57:52Z

⌛ Trying commit cb2c47a94cc845ebf5d017e5901c85fd6cc7218b with merge 393f79804e7cd37b9b054bd261630dba7c969ded...

bors · 2022-06-10T17:11:00Z

💔 Test failed - checks-actions

Kobzol · 2022-06-10T17:11:55Z

@bors try @rust-timer queue

rust-timer · 2022-06-10T17:11:57Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2022-06-10T17:12:03Z

⌛ Trying commit fa77e8931eb3811d7ab3eeed9b8ae300f86628f0 with merge 88727198ddd5bd9f02c6d7ba34fc0b3e4b89b3ca...

bors · 2022-06-10T18:43:08Z

☀️ Try build successful - checks-actions
Build commit: 88727198ddd5bd9f02c6d7ba34fc0b3e4b89b3ca (88727198ddd5bd9f02c6d7ba34fc0b3e4b89b3ca)

rust-timer · 2022-06-10T18:43:10Z

Queued 88727198ddd5bd9f02c6d7ba34fc0b3e4b89b3ca with parent ec55c61, future comparison URL.

rust-timer · 2022-06-10T19:59:28Z

Finished benchmarking commit (88727198ddd5bd9f02c6d7ba34fc0b3e4b89b3ca): comparison url.

Instruction count

Primary benchmarks: 🎉 relevant improvements found
Secondary benchmarks: mixed results

	mean¹	max	count²
Regressions 😿 (primary)	0.4%	0.4%	6
Regressions 😿 (secondary)	1.0%	1.9%	23
Improvements 🎉 (primary)	-0.9%	-2.8%	142
Improvements 🎉 (secondary)	-2.5%	-46.2%	114
All 😿🎉 (primary)	-0.9%	-2.8%	148

Max RSS (memory usage)

Results

Primary benchmarks: mixed results
Secondary benchmarks: 😿 relevant regressions found

	mean¹	max	count²
Regressions 😿 (primary)	3.5%	3.5%	1
Regressions 😿 (secondary)	2.4%	2.5%	2
Improvements 🎉 (primary)	-0.1%	-0.1%	1
Improvements 🎉 (secondary)	N/A	N/A	0
All 😿🎉 (primary)	1.7%	3.5%	2

Cycles

Results

Primary benchmarks: 🎉 relevant improvements found
Secondary benchmarks: 🎉 relevant improvements found

	mean¹	max	count²
Regressions 😿 (primary)	N/A	N/A	0
Regressions 😿 (secondary)	N/A	N/A	0
Improvements 🎉 (primary)	-2.3%	-3.2%	10
Improvements 🎉 (secondary)	-8.5%	-38.9%	21
All 😿🎉 (primary)	-2.3%	-3.2%	10

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf -perf-regression

the arithmetic mean of the percent change ↩ ↩² ↩³
number of relevant changes ↩ ↩² ↩³

bjorn3 · 2022-06-10T22:46:35Z

How do hashcollision chances between xxh3 and siphash compare? We need extremely low collision rates as any collision can cause a miscompilation.

bors · 2022-10-01T09:38:44Z

💔 Test failed - checks-actions

Kobzol · 2022-10-01T15:36:56Z

@bors try @rust-timer queue

rust-timer · 2022-10-01T15:36:57Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2022-10-01T15:37:05Z

⌛ Trying commit ff1489a with merge 5e83f34b1fffab8706f81070e81f017f7f5b8dbe...

rust-log-analyzer · 2022-10-01T16:33:09Z

The job mingw-check failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

########################################################                  78.4%
######################################################################## 100.0%
extracting /checkout/obj/build/cache/2022-09-20/rustfmt-nightly-x86_64-unknown-linux-gnu.tar.xz to /checkout/obj/build/x86_64-unknown-linux-gnu/stage0
Checking stage0 std artifacts (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu)
    Updating git repository `https://github.com/michaelwoerister/xxh3-port`
---
    Checking crossbeam-channel v0.5.4
    Checking stable_deref_trait v1.2.0
    Checking arrayvec v0.7.0
    Checking remove_dir_all v0.5.3
    Checking xxh3-port v0.1.0 (https://github.com/michaelwoerister/xxh3-port#f2071121)
    Checking cpufeatures v0.2.1
   Compiling quote v1.0.18
    Checking rustc_arena v0.0.0 (/checkout/compiler/rustc_arena)
    Checking scoped-tls v1.0.0
---
configure: rust.debug-assertions := True
configure: rust.overflow-checks := True
configure: llvm.assertions      := True
configure: dist.missing-tools   := True
configure: build.configure-args := ['--enable-sccache', '--disable-manage-submodu ...
configure: writing `config.toml` in current directory
configure: 
configure: run `python /checkout/x.py --help`
Attempting with retry: make prepare
---
    Checking ppv-lite86 v0.2.8
    Checking ena v0.14.0
    Checking stable_deref_trait v1.2.0
    Checking arrayvec v0.7.0
    Checking xxh3-port v0.1.0 (https://github.com/michaelwoerister/xxh3-port#f2071121)
    Checking jobserver v0.1.24
    Checking memmap2 v0.2.1
    Checking rand_chacha v0.3.0
    Checking cpufeatures v0.2.1
---
   Compiling tidy v0.1.0 (/checkout/src/tools/tidy)
    Finished release [optimized] target(s) in 6.66s
tidy check
tidy: Skipping binary file check, read-only filesystem
tidy error: invalid source: "git+https://github.com/michaelwoerister/xxh3-port#f2071121875c9b0a474dcdbbd4c4411011b0c266"
* 632 error codes
* highest error code: E0790
Found 506 error codes
Found 0 error(s) in error codes
Found 0 error(s) in error codes
Done!
tidy error: dependency `xxh3-port 0.1.0 (git+https://github.com/michaelwoerister/xxh3-port#f2071121875c9b0a474dcdbbd4c4411011b0c266)` does not define a license expression
tidy error: Dependencies for rustc not explicitly permitted:
* xxh3-port 0.1.0 (git+https://github.com/michaelwoerister/xxh3-port#f2071121875c9b0a474dcdbbd4c4411011b0c266)
some tidy checks failed
Build completed unsuccessfully in 0:00:09

bors · 2022-10-01T17:27:25Z

☀️ Try build successful - checks-actions
Build commit: 5e83f34b1fffab8706f81070e81f017f7f5b8dbe (5e83f34b1fffab8706f81070e81f017f7f5b8dbe)

rust-timer · 2022-10-01T17:27:27Z

Queued 5e83f34b1fffab8706f81070e81f017f7f5b8dbe with parent 744e397, future comparison URL.

rust-timer · 2022-10-01T19:37:51Z

Finished benchmarking commit (5e83f34b1fffab8706f81070e81f017f7f5b8dbe): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean¹	range	count²
Regressions ❌ (primary)	0.8%	[0.2%, 1.7%]	51
Regressions ❌ (secondary)	1.3%	[0.4%, 2.9%]	35
Improvements ✅ (primary)	-0.8%	[-2.0%, -0.2%]	67
Improvements ✅ (secondary)	-3.1%	[-50.1%, -0.2%]	65
All ❌✅ (primary)	-0.1%	[-2.0%, 1.7%]	118

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean¹	range	count²
Regressions ❌ (primary)	7.9%	[3.9%, 13.8%]	3
Regressions ❌ (secondary)	2.5%	[1.7%, 2.9%]	4
Improvements ✅ (primary)	-9.1%	[-19.5%, -3.9%]	19
Improvements ✅ (secondary)	-3.2%	[-3.5%, -2.8%]	2
All ❌✅ (primary)	-6.8%	[-19.5%, 13.8%]	22

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean¹	range	count²
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-6.2%	[-29.4%, -1.6%]	225
Improvements ✅ (secondary)	-4.8%	[-46.6%, -1.7%]	141
All ❌✅ (primary)	-6.2%	[-29.4%, -1.6%]	225

the arithmetic mean of the percent change ↩ ↩² ↩³
number of relevant changes ↩ ↩² ↩³

Kobzol · 2022-10-01T19:41:52Z

I wonder if this is worth it. There is one big win on incr-unchanged for ctfe-stress-5, but other than that there are a lot of interleaved regressions and wins in instruction counts. Let's see how do cycles looks like once perf stabilises.

nnethercote · 2022-10-02T23:01:46Z

The instruction count results are shrug, the cycle count results are amazing, the wall-time (and bootstrap) results are terrible. That's a very strange set of results.

Mark-Simulacrum · 2022-10-02T23:07:19Z

The parent commit here is the last one with the old machine configuration (per rust-lang/rustc-perf#1450), so everything other than instruction counts is probably misleading.

@bors try @rust-timer queue

rust-timer · 2022-10-02T23:07:21Z

Awaiting bors try build completion.

@rustbot label: +S-waiting-on-perf

bors · 2022-10-02T23:07:28Z

⌛ Trying commit ff1489a with merge e1b19ca349a8408a331893f190193ad37686f845...

bors · 2022-10-03T00:30:20Z

☀️ Try build successful - checks-actions
Build commit: e1b19ca349a8408a331893f190193ad37686f845 (e1b19ca349a8408a331893f190193ad37686f845)

rust-timer · 2022-10-03T00:30:22Z

Queued e1b19ca349a8408a331893f190193ad37686f845 with parent a8a847e, future comparison URL.

rust-timer · 2022-10-03T02:30:39Z

Finished benchmarking commit (e1b19ca349a8408a331893f190193ad37686f845): comparison URL.

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: +S-waiting-on-review -S-waiting-on-perf +perf-regression

Instruction count

This is a highly reliable metric that was used to determine the overall result at the top of this comment.

	mean¹	range	count²
Regressions ❌ (primary)	0.9%	[0.2%, 1.7%]	55
Regressions ❌ (secondary)	1.4%	[0.3%, 3.1%]	37
Improvements ✅ (primary)	-0.8%	[-2.0%, -0.2%]	67
Improvements ✅ (secondary)	-3.2%	[-50.1%, -0.2%]	65
All ❌✅ (primary)	-0.0%	[-2.0%, 1.7%]	122

Max RSS (memory usage)

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean¹	range	count²
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	4.5%	[3.4%, 5.6%]	2
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-2.5%	[-3.7%, -0.8%]	7
All ❌✅ (primary)	-	-	0

Cycles

Results

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

	mean¹	range	count²
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-2.3%	[-2.6%, -2.1%]	4
Improvements ✅ (secondary)	-12.1%	[-46.7%, -1.9%]	11
All ❌✅ (primary)	-2.3%	[-2.6%, -2.1%]	4

the arithmetic mean of the percent change ↩ ↩² ↩³
number of relevant changes ↩ ↩² ↩³

michaelwoerister · 2023-01-19T10:03:13Z

I see that this is still assigned to me -- but I won't have time to help with this. The initial XXH3 port was a spare time project of mine, but others have to push it over the finish line, if there's interest to do so. From the performance results, it looks like nice but only modest wins (except for cases where large chunks of data are hashed in bulk).

Kobzol · 2023-01-19T10:33:28Z

We have discussed it with @lqd and @nnethercote and I think that the decision was that it is not worth it to pursue this further. There are indeed some nice performance wins, but the code complexity is too high to justify switching to it, as the biggest wins are localized to niche benchmarks and there are also some regressions.

michaelwoerister · 2023-01-19T10:46:23Z

Sounds good. I hope it was still an interesting investigation for everyone 🙂

Kobzol · 2023-01-19T10:56:44Z

Definitely! Now we know that the performance of hashing large blobs can be improved significantly (although it's probably not a bottleneck for most crates).

rust-highfive assigned michaelwoerister Jun 10, 2022

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Jun 10, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jun 10, 2022

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jun 10, 2022

This comment has been minimized.

Sign in to view

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 10, 2022

This comment has been minimized.

Sign in to view

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Jun 10, 2022

This comment has been minimized.

Sign in to view

Fix doc comment

ff1489a

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Oct 1, 2022

rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Oct 2, 2022

rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Oct 3, 2022

Kobzol closed this Jan 19, 2023

Use XXH3 hash instead of SIP128 for stable hashing #97952

Use XXH3 hash instead of SIP128 for stable hashing #97952

Conversation

Kobzol commented Jun 10, 2022

Kobzol commented Jun 10, 2022

rust-timer commented Jun 10, 2022

bors commented Jun 10, 2022

This comment has been minimized.

Kobzol commented Jun 10, 2022

rust-timer commented Jun 10, 2022

bors commented Jun 10, 2022

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

bors commented Jun 10, 2022

Kobzol commented Jun 10, 2022

rust-timer commented Jun 10, 2022

bors commented Jun 10, 2022

This comment has been minimized.

bors commented Jun 10, 2022

rust-timer commented Jun 10, 2022

rust-timer commented Jun 10, 2022

Footnotes

bjorn3 commented Jun 10, 2022

This comment has been minimized.

bors commented Oct 1, 2022

This comment has been minimized.

Kobzol commented Oct 1, 2022

rust-timer commented Oct 1, 2022

bors commented Oct 1, 2022

rust-log-analyzer commented Oct 1, 2022

bors commented Oct 1, 2022

rust-timer commented Oct 1, 2022

rust-timer commented Oct 1, 2022

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Footnotes

Kobzol commented Oct 1, 2022

nnethercote commented Oct 2, 2022

Mark-Simulacrum commented Oct 2, 2022

rust-timer commented Oct 2, 2022

bors commented Oct 2, 2022

bors commented Oct 3, 2022

rust-timer commented Oct 3, 2022

rust-timer commented Oct 3, 2022

Overall result: ❌✅ regressions and improvements - ACTION NEEDED

Footnotes

michaelwoerister commented Jan 19, 2023

Kobzol commented Jan 19, 2023

michaelwoerister commented Jan 19, 2023

Kobzol commented Jan 19, 2023