[chore] kafkareceiver: embed configkafka types

[axw]

Description

Use configkafka.ClientConfig and configkafka.ConsumerConfig, and the newly extracted consumer client. Redundant validation tests have been removed, as they are now covered by configkafka.

This is a non-functional change, as the config defaults do not change for this component. Nevertheless I have manually tested -- see Testing section below.

Link to tracking issue

Fixes #38411

Testing

Updated unit tests.

Manually tested against Redpanda:

$ docker run --publish=9093:9093 --health-cmd "rpk cluster health | grep 'Healthy:.*true'" docker.redpanda.com/redpandadata/redpanda:v23.1.11 redpanda start --kafka-addr=internal://0.0.0.0:9092,external://0.0.0.0:9093 --smp=1 --memory=1G --mode=dev-container

Ran the collector with kafkametrics -> kafkaexporter -> Redpanda -> kafkareceiver -> debugexporter:

receivers:
  kafka:
    brokers: [localhost:9093]
  kafkametrics:
    brokers: [localhost:9093]
    collection_interval: 10s
    scrapers: [topics, consumers]

exporters:
  debug:
    verbosity: detailed
  kafka:
    brokers: [localhost:9093]

service:
  pipelines:
    metrics/kafka:
      receivers: [kafka]
      exporters: [debug]
    metrics:
      receivers: [kafkametrics]
      exporters: [kafka]

Documentation

Updated README to clarify that protocol_version is not required.

[pavolloffay]

LGTM

[pavolloffay]

could we keep this to make sure the API does not change?

[axw]

Do you mean auth and metadata? I removed them because we have coverage for the ClientConfig type in the internal/kafka/configkafka package. We're still verifying client_id and group_id below, as a minimal subset of the CientConfig and ConsumerConfig types - to verify that we're embedding them correctly.

[MovieStoreGuy]

Yeah, it would be great to know which API you mean here @pavolloffay

[MovieStoreGuy]

These changes look great to me and really simplifies the components and maintaining them.