Skip to content

confluentinc/confluent-kubernetes-examples

Repository files navigation

Confluent for Kubernetes Scenario Examples

This GitHub repository accompanies the official Confluent for Kubernetes documentation.

This repository contains scenario workflows to deploy and manage Confluent on Kubernetes for various use cases.

Prerequisites

The following prerequisites are assumed for each scenario workflow:

  • A Kubernetes cluster - any CNCF conformant version
  • Helm 3 installed on your local machine
  • Kubectl installed on your local machine
  • A namespace created in the Kubernetes cluster - confluent
  • Kubectl configured to target the confluent namespace:
    kubectl config set-context --current --namespace=confluent
    
  • This repo cloned to your workstation:
    git clone [email protected]:confluentinc/confluent-kubernetes-examples.git
    

Next Steps

You can browse through our curated example scenarios and try out the ones that matter to you.

If you are looking for inspiration, below are some of the popular scenarios.

Quickstart

In this workflow scenario, you'll set up a simple non-secure (no authn, authz or encryption) Confluent Platform, consisting of all components. You can also setup up a simple KRaft based deployment as well.

The goal for this scenario is for you to:

Quickly set up the complete Confluent Platform on the Kubernetes. Configure a producer to generate sample data. Head to CFK Quickstart.

We are adopting KRaft in place of Zookeeper. Give it a quick try here.

Security

We have curated a comprehensive list of examples of commonly used security configurations. Some of the notable ones are:

  • SASL/Plain with LDAP for authentication & authorization, TLS for encryption using CFK auto-generated component certificates. You'll need to provide a certificate authority certificate for CFK to auto-generate the component certificates. Example.
  • TLS encryption using user provided certificates, mTLS authentication, Confluent RBAC authorization Example.

For more security use cases, click here.

Networking

External Access Load Balancer Deployment in CFK

This example provides a way to enable external access to Confluent Platform components using load balancers, enhancing the accessibility and flexibility of your Confluent Platform deployment. It demonstrates

  • How to configure load balancers for external access to Confluent Platform components in a Kubernetes environment.
  • Setting the domain name of your Kubernetes cluster and configuring session affinity.

Advanced Configurations

Blueprints

Blueprint is a configuration template for Confluent Platform deployments using Confluent for Kubernetes. You can use it to enforce infrastructure and deployment standards within your organization. To try out Blueprints, click here.

Advanced Configuration with Pod Overlay

This example provides a way to leverage additional Kubernetes features not currently supported in the Confluent for Kubernetes (CFK) API, enhancing the flexibility and control over your Confluent Platform deployments.

  • The Pod Overlay feature allows you to use ConfigMap and configure a StatefulSet PodTemplate for Confluent Platform components like Zookeeper, Kafka, Connect, Schema Registry, Kafka Rest Proxy, and Control Center.
  • The valid Pod Overlay configuration is strategically merged with the pod spec inside the StatefulSet generated by CFK to form the final pod configuration for the application.

Configure Workloads Scheduling

You can control how the component pods are scheduled on Kubernetes nodes to get optimal performance out of Confluent components.

For example, you can configure pods not to be scheduled on the same node as other resource intensive applications, pods to be scheduled on dedicated nodes, or pods to be scheduled on the nodes with the most suitable hardware.

You can try it out here.

Index

Following index (not exhaustive) provides a list of scenarios available in this repository.

items Tags
autogenerated-tls_only
blueprints #control-plane/data-plane, #multiple-k8s-cluster
ccloud-connect-confluent-hub #confluent-cloud-kafka
ccloud-integration
external-access-load-balancer-deploy #loadbalancer
external-access-nodeport-deploy #nodePort
external-access-static-host-based #static-host
external-access-static-port-based #static-port
hashicorp vault #security #3rd-party
kafka-additional-listeners #custom-listener
kraft_sasl_ssl_autogenerated
kubernetes-rbac #k8s-rbac,#k8s-namespace
monitoring #grafana
mtls-without-rbac
mtls-without-rbac #mtls
multi-Region-Clusters (MRC) #multiple-k8s-cluster
openshift-security
plaintext-basic-auth-Connect #basic-auth
pod-overlay #advanced-configuration
quickstart-deploy #Beginner
replicator
schemalink
separate-listener-tls-rbac #tls, #rbac
separate/mutiple kafka listener, #tls
userprovided-tls_mtls-sasl_confluent-rbac
using-cert-manager #tls, #3rd-party, #cert-manager

Troubleshooting

If you run into issues, don't forget to check the troubleshooting document.