order-dependent configuration failure with plugin.library.paths monitoring-interceptor

[dtheodor]

Any plugin-specific configuration properties fail to be set if the plugin has not been registered with the property "plugin.library.paths", with an error that looks like this

panic: No such configuration property: "confluent.monitoring.interceptor.icdebug"

However, there's no way to specify ordering of configuration properties through a ConfigMap, which is just a map with keys that are returned in random order. When instantiating a client, there's a loop over the map's keys, and each of them is set through C.rd_kafka_conf_set which will fail immediately for any unrecognized property

confluent-kafka-go/kafka/config.go

Lines 130 to 132 in 5e4d04e

	func configConvertAnyconf(m ConfigMap, anyconf rdkAnyconf) (err error) {
	for k, v := range m {

That means that about 50% of the time I am starting my program and I set any confluent.monitoring.interceptor.* property the program crashes with panic: No such configuration property because the confluent.monitoring.interceptor.* came before the "plugin.library.paths": "monitoring-interceptor" property during the iteration over the config keys.

Running librdkafka 0.11.5 and confluent-kafka-go version v0.11.4

[rnpridgeon]

Thanks for reporting this @dtheodor, we actually ran into this issue with the python client recently as well.

To fix this the property plugin.library.paths is intercepted and processed prior to handling any additional properties. I can get a fix in for this sometime this week or if you'd like you are more than welcome to submit a PR yourself.

Let me know,
Ryan

[dtheodor]

I can do a PR. Thinking of extending the following snippet with something like this

confluent-kafka-go/kafka/config.go

Lines 130 to 133 in 5e4d04e

	func configConvertAnyconf(m ConfigMap, anyconf rdkAnyconf) (err error) {
	for k, v := range m {
	switch v.(type) {

func configConvertAnyconf(m ConfigMap, anyconf rdkAnyconf) (err error) {
  // set the plugin.library.paths first
  paths, exist := m["plugin.library.paths"]
  if exist {
    anyconfSet(anyconf, "plugin.library.paths", paths)
    delete(m, "plugin.library.paths") // not sure if this is needed
  } 
  for k, v := range m {
    // .. the config property loop unchanged

[dtheodor]

Also an idea is not to impose ordering in the client libraries, but fix this in librdkakfa, by exposing some "finalize configuration" function that will collect all config values set through rd_kafka_conf_set, validate them, and apply any side effects in the right order

[rnpridgeon]

I had a similar discussion with @edenhill while fixing the Python client. In the end it was decided that the client bindings should handle this.

That said @edenhill is a very reasonable man so you may be able to convince him otherwise. I recommend opening an issue for discussion within the librdkafka repo. Go ahead and submit a PR for the client binding though so we can get this fixed sooner rather than later. We can always back the fix out later if you can convince Magnus to solve this in librdkafka.