UNKNOWN_TOPIC_ID: This server does not host this topic ID

[jrkinley]

I'm unable to consume from a topic created in Confluent Platform 7.2.1 due to UNKNOWN_TOPIC_ID errors. I'm using the latest version of franz-go (v.1.10.4).

Topic ID is a fairly recent change to the fetch protocol (KIP-516) and the latest version of franz-go includes a fix for handling unknown topic IDs, but it doesn't help in this case.

[jrkinley]

After further investigation, this was actually caused by the setting inter.broker.protocol.version = 2.6, which resulted in the topic IDs not being reported by the API:

kcl metadata -t telemetry1
NAME        ID                                PARTITIONS  REPLICAS
telemetry1  00000000000000000000000000000000  1           1

Since topic IDs were added to the fetch request in Kafka v3.1.0 (KAFKA-10580) the workaround is to set the max version to v3.0.0:

kgo.NewClient(kgo.MaxVersions(kversion.V3_0_0()))

[twmb]

I'll add a behavior improvement to not use v13 fetch requests if any topic does not have an ID. I'm not sure why Kafka would advertise v13 fetch requests for IBP 2.6, though.