Data race in consumer

[tymeshifter]

Version: 0.6.14

==================
WARNING: DATA RACE
Write at 0x00c0007e50b0 by goroutine 148:
  runtime.mapassign_fast64()
      runtime/map_fast64.go:92 +0x0
  github.com/twmb/franz-go/pkg/kgo.(*usedCursors).use()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:134 +0x186
  github.com/twmb/franz-go/pkg/kgo.(*consumerSession).handleListOrEpochResults.func2()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:1135 +0xef
  github.com/twmb/franz-go/pkg/kgo.(*consumerSession).handleListOrEpochResults()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:1144 +0x429
  github.com/twmb/franz-go/pkg/kgo.(*consumerSession).listOrEpoch()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:1070 +0xa44

Previous write at 0x00c0007e50b0 by goroutine 70:
  runtime.mapassign_fast64()
      runtime/map_fast64.go:92 +0x0
  github.com/twmb/franz-go/pkg/kgo.(*usedCursors).use()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:134 +0x186
  github.com/twmb/franz-go/pkg/kgo.(*consume  github.com/twmb/franz-go/pkg/kgo.(*consumerSession).handleListOrEpochResults()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:1144 +0x429
  github.com/twmb/franz-go/pkg/kgo.(*consumerSession).listOrEpoch()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:1070 +0xa44

Goroutine 148 (running) created at:
  github.com/twmb/franz-go/pkg/kgo.listOrEpochLoads.loadWithSession()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:748 +0xca
  github.com/twmb/franz-go/pkg/kgo.(*consumerSession).handleListOrEpochResults.func1.1()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:1124 +0x259

Goroutine 70 (finished) created at:
  github.com/twmb/franz-go/pkg/kgo.listOrEpochLoads.loadWithSession()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:748 +0xca
  github.com/twmb/franz-go/pkg/kgo.(*consumerSession).handleListOrEpochResults.func1.1()
      github.com/twmb/[email protected]/pkg/kgo/consumer.go:1124 +0x259
==================

[twmb]

Interesting, thank you. I see that this can be triggered if two handleListOrEpochResults are running. I've checked and the only other place that the field is modified is when the consumer session is stopped.

I forgot about the fact that the function could run multiple times concurrently. :(

There's a few ways to go about guard running that segment of code. I'll have a fix out imminently once I figure out which approach I like best.

[twmb]

Pushed a fix, I'll tag a release likely later today.

Just out of curiosity, what were the conditions that triggered this? I mention one scenario in my commit message, but the more likely scenario I think is that the first few ListOffsets requests failed, which independently triggered backoffs, which then trigger this scenario (I'm going to push a make a small change here as well, unrelated and not necessary for this bugfix)

[twmb]

Tag likely tomorrow actually, I'm close to finishing the documentation I've been trying to finish for a few weeks now.

[tymeshifter]

Just out of curiosity, what were the conditions that triggered this?

Just stress tested an application that uses the library. Use case is normal consumption loop with PollFetches.

[twmb]

The tag took a but, but 0.7.0 is now released that contains this bugfix.