quickstart-deploy

Deploy Confluent Platform

In this workflow scenario, you'll set up a simple non-secure (no authn, authz or encryption) Confluent Platform, consisting of all components.

The goal for this scenario is for you to:

• Quickly set up the complete Confluent Platform on the Kubernetes.
• Configure a producer to generate sample data.

Watch the walkthrough: Quickstart Demonstration

Before continuing with the scenario, ensure that you have set up the prerequisites.

To complete this scenario, you'll follow these steps:

1. Set the current tutorial directory.
2. Deploy Confluent For Kubernetes.
3. Deploy Confluent Platform.
4. Deploy the Producer application.
5. Tear down Confluent Platform.

Set the current tutorial directory

Set the tutorial directory for this tutorial under the directory you downloaded the tutorial files:

export TUTORIAL_HOME=<Tutorial directory>/quickstart-deploy

Deploy Confluent for Kubernetes

1. Create the namespace to use.

   kubectl create namespace confluent
   

2. Set this namespace to default for your Kubernetes context.

   kubectl config set-context --current --namespace confluent
   

3. Set up the Helm Chart:

   helm repo add confluentinc https://packages.confluent.io/helm
   

4. Install Confluent For Kubernetes using Helm:

   helm upgrade --install confluent-operator confluentinc/confluent-for-kubernetes --namespace confluent
   

5. Check that the Confluent For Kubernetes pod comes up and is running:

   kubectl get pods
   

Review Confluent Platform configurations

You install Confluent Platform components as custom resources (CRs).

You can configure all Confluent Platform components as custom resources. In this tutorial, you will configure all components in a single file and deploy all components with one kubectl apply command.

The entire Confluent Platform is configured in one configuration file: $TUTORIAL_HOME/confluent-platform.yaml

In this configuration file, there is a custom Resource configuration spec for each Confluent Platform component - replicas, image to use, resource allocations.

For example, the Kafka section of the file is as follows:

---
apiVersion: platform.confluent.io/v1beta1
kind: Kafka
metadata:
  name: kafka
  namespace: confluent
spec:
  replicas: 3
  image:
    application: confluentinc/cp-server:7.8.0
    init: confluentinc/confluent-init-container:2.10.0
  dataVolumeCapacity: 10Gi
  metricReporter:
    enabled: true
  dependencies:
    zookeeper:
      endpoint: zookeeper.confluent.svc.cluster.local:2181
---

Deploy Confluent Platform

1. Deploy Confluent Platform with the above configuration:

   kubectl apply -f $TUTORIAL_HOME/confluent-platform.yaml
   

   Note: If you are deploying a single node dev cluster, then use this yaml file:

   kubectl apply -f $TUTORIAL_HOME/confluent-platform-singlenode.yaml
   

2. Check that all Confluent Platform resources are deployed:

   kubectl get confluent
   

3. Get the status of any component. For example, to check Kafka:

   kubectl describe kafka
   

Validate

Deploy producer application

Now that we've got the infrastructure set up, let's deploy the producer client app.

The producer app is packaged and deployed as a pod on Kubernetes. The required topic is defined as a KafkaTopic custom resource in $TUTORIAL_HOME/producer-app-data.yaml.

The $TUTORIAL_HOME/producer-app-data.yaml defines the elastic-0 topic as follows:

apiVersion: platform.confluent.io/v1beta1
kind: KafkaTopic
metadata:
  name: elastic-0
  namespace: confluent
spec:
  replicas: 3 # change to 1 if using single node
  partitionCount: 1
  configs:
    cleanup.policy: "delete"

Deploy the producer app:

kubectl apply -f $TUTORIAL_HOME/producer-app-data.yaml

Note: If you are deploying a single node dev cluster, then use this yaml file:

kubectl apply -f $TUTORIAL_HOME/producer-app-data-singlenode.yaml

Validate in Control Center

Use Control Center to monitor the Confluent Platform, and see the created topic and data.

1. Set up port forwarding to Control Center web UI from local machine:

   kubectl port-forward controlcenter-0 9021:9021
   

2. Browse to Control Center:

   http://localhost:9021
   

3. Check that the elastic-0 topic was created and that messages are being produced to the topic.

Tear Down

Shut down Confluent Platform and the data:

kubectl delete -f $TUTORIAL_HOME/producer-app-data.yaml

kubectl delete -f $TUTORIAL_HOME/confluent-platform.yaml

helm uninstall confluent-operator