Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KafkaAdminTopicConfigProvider.configure() should report original exception(s) raised by describeClusterConfigs #2087

Open
marcelloromani opened this issue Dec 7, 2023 · 1 comment · May be fixed by #2131
Open

KafkaAdminTopicConfigProvider.configure() should report original exception(s) raised by describeClusterConfigs #2087

marcelloromani opened this issue Dec 7, 2023 · 1 comment · May be fixed by #2131
Labels
question A code or meta question about the project.

Comments

@marcelloromani
Copy link

I am using Cruise Control 2.5.99 with AWS MSK.
I reached the point where Curise Control is able to connect to the Kafka brokers, but terminates with the following exception:

[2023-12-07 11:49:14,062] ERROR Uncaught exception on thread Thread[main,5,main] (com.linkedin.kafka.cruisecontrol.KafkaCruiseControlMain)
java.lang.RuntimeException: Failed to describe Kafka cluster configs.
	at com.linkedin.kafka.cruisecontrol.config.KafkaAdminTopicConfigProvider.configure(KafkaAdminTopicConfigProvider.java:174) ~[cruise-control-2.5.99.jar:?]
	at com.linkedin.kafka.cruisecontrol.config.KafkaCruiseControlConfigUtils.getConfiguredInstance(KafkaCruiseControlConfigUtils.java:49) ~[cruise-control-2.5.99.jar:?]
	at com.linkedin.kafka.cruisecontrol.config.KafkaCruiseControlConfig.getConfiguredInstance(KafkaCruiseControlConfig.java:98) ~[cruise-control-2.5.99.jar:?]
[...]

As the stacktrace indicates, that error message comes from:

cruise-control/cruise-control/src/main/java/com/linkedin/kafka/cruisecontrol/config/KafkaAdminTopicConfigProvider.java

Line 172 in 9ccbb9e

clusterConfigs = describeClusterConfigs(_adminClient, DESCRIBE_CLUSTER_CONFIGS_TIMEOUT); 

  public void configure(Map<String, ?> configs) {
    _adminClient = (AdminClient) validateNotNull(
            configs.get(LoadMonitor.KAFKA_ADMIN_CLIENT_OBJECT_CONFIG),
            () -> String.format("Missing %s when creating Kafka Admin Client based Topic Config Provider",
                    LoadMonitor.KAFKA_ADMIN_CLIENT_OBJECT_CONFIG));
    Config clusterConfigs;
    try {
      clusterConfigs = describeClusterConfigs(_adminClient, DESCRIBE_CLUSTER_CONFIGS_TIMEOUT);
    } catch (InterruptedException | ExecutionException e) {
      throw new RuntimeException("Failed to describe Kafka cluster configs.");
    }
[...]

The catch statement simply swallows any ExecutionException and replaces any useful error with the generic message "Failed to describe Kafka cluster configs."

It would be useful if it instead logged the original exception or included it in the generic RunTimeException message.
@marcelloromani
Copy link
Author

In my setup Cruise Control is running as a pod on an EKS cluster which resides in the same AWS Account as the MSK cluster.

I double checked the Pod Service Account -> IAM Role mapping and verified that the IAM Role has a Policy which allows all of the Kafka operations on the relevant MSK cluster.

Happy to provide more details about this if deemed useful.

@CCisGG CCisGG added the question A code or meta question about the project. label Feb 21, 2024
@marcelloromani marcelloromani linked a pull request Mar 3, 2024 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question A code or meta question about the project.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Log exceptions when unable to list topics or describe cluster config marcelloromani/cruise-control
2 participants
@marcelloromani @CCisGG