Kubernetes Configuration Provider for Apache Kafka®

Apache Kafka® supports pluggable configuration providers which can load configuration data from external sources. The configuration providers in this repo can be used to load data from Kubernetes Secrets and Config Maps. It can be used in all Kafka components and does not depend on the other Strimzi components. So you could, for example, use it with your producer or consumer applications even if you don't use the Strimzi operators to provide your Kafka cluster. One of the example use-cases is to load certificates or JAAS configuration from Kubernetes Secrets.

Using it with Strimzi

From Strimzi Kafka Operators release 0.24.0, the Kubernetes Configuration Provider is included in all the Kafka deployments. You can use it for example with Kafka Connect, Kafka Mirror Maker 1 or 2 and with Kafka Connect connectors. The following example shows how to use it with Kafka Connect and Connectors:

1. Deploy Kafka Connect and enable the Kubernetes Configuration Provider:

   apiVersion: kafka.strimzi.io/v1beta2
   kind: KafkaConnect
   metadata:
     name: my-connect
     annotations:
       strimzi.io/use-connector-resources: "true"
   spec:
     # ...
     config:
       # ...
       config.providers: secrets,configmaps
       config.providers.secrets.class: io.strimzi.kafka.KubernetesSecretConfigProvider
       config.providers.configmaps.class: io.strimzi.kafka.KubernetesConfigMapConfigProvider
     # ...

2. Create a configuration Config Map

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: connector-configuration
   data:
     option1: value1
     option2: value2

3. Create the Role and RoleBinding:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: Role
   metadata:
     name: connector-configuration-role
   rules:
   - apiGroups: [""]
     resources: ["configmaps"]
     resourceNames: ["connector-configuration"]
     verbs: ["get"]
   ---
   
   apiVersion: rbac.authorization.k8s.io/v1
   kind: RoleBinding
   metadata:
     name: connector-configuration-role-binding
   subjects:
   - kind: ServiceAccount
     name: my-connect-connect
     namespace: myproject
   roleRef:
     kind: Role
     name: connector-configuration-role
     apiGroup: rbac.authorization.k8s.io

   Use the Service Account already used by your Kafka Connect deployment, which is named <CLUSTER_NAME>-connect where <CLUSTER_NAME> is the name of your KafkaConnect custom resource.

4. Create the connector:

   apiVersion: kafka.strimzi.io/v1beta2
   kind: KafkaConnector
   metadata:
     name: my-connector
     labels:
       strimzi.io/cluster: my-connect
   spec:
     # ...
     config:
       option: ${configmaps:myproject/connector-configuration:option1}
       # ...

Adding the Kubernetes Configuration Provider to Apache Kafka clients

You can add Kubernetes Configuration Provider as any other Java dependency using Maven or any other build tool. For example:

<dependency>
    <groupId>io.strimzi</groupId>
    <artifactId>kafka-kubernetes-config-provider</artifactId>
    <version>0.1.0</version>
</dependency>

Adding the Kubernetes Configuration Provider to Apache Kafka server components

You can also use the Kubernetes Configuration Provider with your own Apache Kafka deployments not managed by Strimzi. To add the Kubernetes Configuration Provider to your own Apache Kafka server distribution, you can download the ZIP or TAR.GZ files frm the GitHub release page and unpack it into Kafka's libs directory.

Using the configuration provider

First, you need to initializer the configuration provider. For use with Kubernetes Secrets, use:

config.providers=secrets
config.providers.secrets.class=io.strimzi.kafka.KubernetesSecretConfigProvider

For use with Kubernetes Config Maps, use:

config.providers=configmaps
config.providers.configmaps.class=io.strimzi.kafka.KubernetesConfigMapConfigProvider

You can also use both together:

config.providers=secrets,configmaps
config.providers.secrets.class=io.strimzi.kafka.KubernetesSecretConfigProvider
config.providers.configmaps.class=io.strimzi.kafka.KubernetesConfigMapConfigProvider

Once you initialize it, you can use it to load data from Kubernetes. The config provider configuration is in the form <NAMESPACE>/<RESOURCE-NAME>:<KEY>. Where:

• The <NAMESPACE> is the namespace where the Secret or Config Map exists
• The <RESOURCE-NAME> is the name of the Secret or Config Map
• The <KEY> is the key under which the configuration value is stored in the Secret or Config Map

For example:

option=${secrets:my-namespace/my-secret:my-key}

Alternatively, you can also use the form <RESOURCE-NAME>:<KEY>. Where:

• The <RESOURCE-NAME> is the name of the Secret or Config Map
• The <KEY> is the key under which the configuration value is stored in the Secret or Config Map

In this case, the current namespace of the Kubernetes client will be used.

The following example shows how to use it in a Kafka Consumer consuming from Apache Kafka cluster on Kubernetes using Strimzi:

1. Deploy your Kafka cluster with TLS authentication enabled
2. Create KafkaUser resource using TLS authentication
3. In your consumer properties, use the following options to load the public key of the CA used by the Apache Kafka cluster and the user certificates for authentication

   config.providers=secrets,configmaps
   config.providers.secrets.class=io.strimzi.kafka.KubernetesSecretConfigProvider
   config.providers.configmaps.class=io.strimzi.kafka.KubernetesConfigMapConfigProvider
   security.protocol=SSL
   ssl.keystore.type=PEM
   ssl.keystore.certificate.chain=${secrets:myproject/my-user:user.crt}
   ssl.keystore.key=${secrets:myproject/my-user:user.key}
   ssl.truststore.type=PEM
   ssl.truststore.certificates=${secrets:myproject/my-cluster-cluster-ca-cert:ca.crt}

Using patterns

You can also use Glob patterns to specify the keys in the Secret or ConfigMap that should be used. The Kubernetes Config Provider will find all fields matching the pattern and join them together into a single value. This is useful for example when you want to load multiple certificates from a Secret into a single configuration option. With an example Kubernetes Secret container multiple certificate files:

apiVersion: v1
kind: Secret
metadata:
  name: certificates
data:
  ca.crt: ...
  ca2.crt: ...
  ca3.crt: ...

You can use the provider to load all of the certificates by using the key pattern *.crt:

config.providers=secrets
config.providers.secrets.class=io.strimzi.kafka.KubernetesSecretConfigProvider
...
ssl.truststore.certificates=${secrets:myproject/certificates:*.crt}

By default, a new line will be used as a separator when joining the different values. But you can configure the separator if you want to use a different one:

config.providers=secrets,configmaps
config.providers.secrets.class=io.strimzi.kafka.KubernetesSecretConfigProvider
config.providers.secrets.param.separator=,

Configuring the Kubernetes client

The Kubernetes Config Provider is using the Fabric8 Kubernetes Client. You can configure it using Java system properties, environment variables, Kube config files or ServiceAccount tokens. All available configuration options are described in the client documentation. By default, it will try to automatically find the available configuration - for example from the Kube config file (~/.kube/config) or from the ServiceAccount if running inside Kubernetes Pod.

RBAC rights

The Kubernetes account used by the Kubernetes Configuration Provider needs to have access to the Config Maps or Secrets. The only RBAC rights it needs is the get access rights on a given resource. For example:

- apiGroups: [""]
  resources: ["secrets"]
  resourceNames: ["my-user", "my-cluster-cluster-ca-cert"]
  verbs: ["get"]

It does not need any other access rights.