reconnect consumer automatically to other node without waiting

[adambezecny]

Description

we are using managed kafka service (AWS MKS) in our mission critical app. During scheduled maintenance windows of MSK our broker nodes go down one by one for considerably long time (couple of minutes). We know that existing consumers will reconnect once broker is up but because we are high performance critical service we would prefer if consumer transparently switched to other broker node which is up. Java client is capable of this. Any way how to achieve it? Is this something that could be added into confluent-kafka-go library?

How to reproduce

scheduled update of broker nodes or restart manually particular node (the one where consumer is attached) from AWS console. Consumer will not reconnect to other available node but wait until restarting node is up & running. We are getting some errors (like ALlBrokersDown) in the meantime.

Checklist

Please provide the following information:

• confluent-kafka-go and librdkafka version (LibraryVersion()): v2.4.0
• Apache Kafka broker version: Apache Kafka 3.6.0 (managed service AWS MKS)
• Client configuration: ConfigMap{...}
• Operating system:
• Provide client logs (with "debug": ".." as necessary)
• Provide broker log excerpts
• Critical issue