Go implementation by AlexanderYastrebov

[AlexanderYastrebov]

This is a proof-of-concept to demonstrate non-java submission.
It requires Docker with BuildKit plugin to build and export binary.

Updates

• 1BRC in Go #67
• Evaluation Environment Update #253

Check List:

• Tests pass (./test.sh <username> shows no differences between expected and actual outputs)
• All formatting changes by the build are committed
• Your launch script is named calculate_average_<username>.sh (make sure to match casing of your GH user name) and is executable
• Output matches that of calculate_average_baseline.sh

$ ./evaluate2.sh baseline AlexanderYastrebov royvanrijn
...
Benchmark 1: ./calculate_average_AlexanderYastrebov.sh 2>&1
  Time (mean ± σ):     16.786 s ±  0.545 s    [User: 56.030 s, System: 10.068 s]
  Range (min … max):   15.918 s … 17.309 s    5 runs
...
Benchmark 1: ./calculate_average_royvanrijn.sh 2>&1
  Time (mean ± σ):     16.731 s ±  0.190 s    [User: 56.485 s, System: 10.279 s]
  Range (min … max):   16.490 s … 16.951 s    5 runs

[gunnarmorling]

Ah, very interesting. I'll get back to this eventually, for now I'm gonna focus on the backlog of Java entries, as per the guidelines of the original challenge.

[tivrfoa]

Very nice solution and great runtime!

[gunnarmorling]

Finally getting back to this. Great job, everything works flawlessly. I didn't want to install Docker on the official eval machine, so I ran on the previously used CCX33 instance. Here's the comparison to merykittyunsafe from that machine:

Summary
  merykittyunsafe: trimmed mean 2.5619433625399997, raw times 2.59049238354,2.54720515154,2.55371644854,2.5830397135400003,2.54907392554
  AlexanderYastrebov: trimmed mean 3.91984175856, raw times 3.9012023965599996,3.9188414475599997,3.92909370756,3.93988131456,3.9115901205599997

You wanna do any more changes to this PR or is it good to be merged?

[tivrfoa]

Finally getting back to this. Great job, everything works flawlessly. I didn't want to install Docker on the official eval machine, so I ran on the previously used CCX33 instance. Here's the comparison to merykittyunsafe from that machine:

Summary
  merykittyunsafe: trimmed mean 2.5619433625399997, raw times 2.59049238354,2.54720515154,2.55371644854,2.5830397135400003,2.54907392554
  AlexanderYastrebov: trimmed mean 3.91984175856, raw times 3.9012023965599996,3.9188414475599997,3.92909370756,3.93988131456,3.9115901205599997

You wanna do any more changes to this PR or is it good to be merged?

Are going to build his solution in the new machine outside Docker?

Maybe compiling in the machine this is going to run would be better than compiling in Docker, for performance @AlexanderYastrebov ?

go compilation is incredibly fast.

[AlexanderYastrebov]

I didn't want to install Docker on the official eval machine, so I ran on the previously used CCX33 instance.
You wanna do any more changes to this PR or is it good to be merged?

It can be merged. I hope it won't stand in the way due to missing Docker. The main value here is to unlock non-java submissions to the challenge.

I also would like to thank you for organizing 1brc, it was fun and I learned a lot 👍 and also brushed up my java which I did not touch since couple of years.

[tivrfoa]

@AlexanderYastrebov your go solution assume no collision in the hash function, right? Or are you handling collisions?

// use uint64 FNV-1a hash of id value as buckets key and keep mapping to the id value.
// This assumes no collisions of id hashes.

I think that's why it's so fast for the 10k.

But it seems this is not allowed for this challenge:
#186

@gunnarmorling

[AlexanderYastrebov]

@tivrfoa I have a version with collision detection https://github.com/AlexanderYastrebov/1brc/tree/AlexanderYastrebov-go-collision-free which of course is slower :)

[gunnarmorling]

Ah yeah, for merging it, it must handle collisions (maybe it's time to set up some tests with randomized data, so as to increase odds a bit to capture this at least in some cases).

…

Message ID: ***@***.***>

[AlexanderYastrebov]

Ah yeah, for merging it, it must handle collisions

I've updated the PR

[gunnarmorling]

Very cool, also passing the tests on CI (set this up earlier this week). Gonna merge it, but not adding it to the leaderboard, given it's a non-Java solution and as such not within the rules. But it's a very nice example for doing this in a portable way, which for instance may be useful for future challenges. Thanks a lot for creating it!

I also would like to thank you for organizing 1brc, it was fun and I learned a lot 👍 and also brushed up my java which I did not touch since couple of years.

Thanks, really appreciate it! Also thank you for all your work around tests and more!

[gunnarmorling]

Ah, if you wanted to add something to the README for likeminded folks interested in creating non-Java solutions and running them in an easy-to-compare way as you've explored it here, that'd be welcome, too. Thx!

[AlexanderYastrebov]

add something to the README for likeminded folks interested in creating non-Java solutions

Oh, they'll figure it out, I'm sure :)

Thanks!