Faster alternative to BLAKE2bp

[jedisct1]

Something for you:

https://docs.rs/sthash - https://github.com/jedisct1/rust-sthash

Note that you will need to generate and store a secret seed (once for the whole cache). This is critical in order to protect against forgery, especially with large files.

[lachlansneff]

How/where do you recommend we store the secret seed?

[jedisct1]

Is the cache directory something guest webassembly modules are supposed to ever have access to?
I guess not, especially if the cache is shared by multiple customers.

In this context, simply storing the key as a file with a fixed name in the cache directory should be acceptable.

[jedisct1]

The last versions have AVX2 support, and should be about 5x faster than BLAKE2bp on Nehalem.

[Hywan]

Thanks for the suggestion. It seems to be a solid candidate!

[repi]

We've started using wasmer here at Embark and for our game runtime use cases I would be hesitant of any hashes (or other code) that use AVX2. Because using those instructions unfortunately slows down the rest of the CPU so often is an overall net loss in performance due to frequency throttling.

Could be available as an option though if one already is running with AVX2/512 and is throttled. Or simply wants the fastest possible hashing for a use case.

[jedisct1]

The current code already heavily depends on AVX2. And STHash is faster than the current hash even without vectorization.

AVX512 quickly gets CPUs to reduce their frequency. Throttling due to AVX2 is more reasonable, but more importantly, STHash only uses AVX2 (if available) for very short amounts of time. NH evaluations alternate with Poly1305 evaluations that don't use vector extensions. On a 8 cores Core i7, I couldn't get the CPU to throttle during STHash benchmarks, so this is unlikely to happen on Xeons either.

[syrusakbary]

We just started using blake3 as it has great performance compared to blake2b, I believe in a similar scale as sthash. (here's the related PR: #1140)

Closing the issue :)