> ## Documentation Index
> Fetch the complete documentation index at: https://docs.zeotap.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Kafka

## Overview

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

By integrating Kafka with Zeotap's Customer Data Platform (CDP), you can design and execute customer journeys based on real-time customer behavior using a centralized data streaming platform.

## Prerequisites

### Following details are required while creating the Kafka source

#### 1. Topic name

* This is the name of the topic in Kafka from which the data needs to be ingested into Zeotap.

#### 2. Kafka Broker Host

* Check the Kafka broker configuration in the Kafka installation directory, typically in the **server.properties** file.
  Look for the **listeners** property, which specifies the broker's hostname or IP address.
* Alternatively, ask the administrator managing the Kafka cluster.

#### 3. Broker API Key and Secret

* If the Kafka broker requires authentication, these credentials are usually provided by the administrator or through the Kafka management interface of the cloud provider (e.g., Confluent Cloud, AWS MSK).
* For Confluent Cloud, you can find these in the **"API Keys"** section under the *"* ***Cluster Settings"*** tab.

#### 4. Consumer Group ID

* This is the identifier for a group of consumers that work together to consume messages from a Kafka topic.

#### 5. Data format

* This is the format of data that is ingested. Currently **AVRO** and **JSON** format are supported.
* There would be additional details related to Schema when **AVRO** format is selected

#### 6. Schema Registry URL

* This field is required only for **AVRO** data format.
* If using a schema registry, the URL is provided during the setup or can be found in the configuration settings of the Kafka deployment.
* For Confluent Schema Registry, the URL is typically listed in the **"Schema Registry"** section of the Confluent Cloud dashboard.

#### 7. Schema API Key and Secret

* This field is required only for **AVRO** data format.
* Like the broker credentials, these are provided by the administrator or through the schema registry's management interface.
* In Confluent Cloud, navigate to the **"Schema Registry"**, and then access the **"API Keys"** section to generate or view the credentials.

## Steps to Create Kafka source

Perform the following steps to create a Kafka Streaming Source:

<Steps>
  <Step>
    Log into the **Zeotap CDP App** and navigate to the **Sources** application under **Integrate**.
  </Step>

  <Step>
    Click **CREATE SOURCE**.
  </Step>

  <Step>
    Choose **Message Queue** as the Category.

    <Frame>
      <img src="https://mintcdn.com/zeotap/CSAMM_Rlvn0jps6x/articles/integrate-customer/Storage/integrate-customer/image-139.png?fit=max&auto=format&n=CSAMM_Rlvn0jps6x&q=85&s=14d2e1ca0f5c751a65c3a802778b6277" width="1588" height="1382" data-path="articles/integrate-customer/Storage/integrate-customer/image-139.png" />
    </Frame>
  </Step>

  <Step>
    Click **Kafka Message Queue** as the Data Source.

    <Frame>
      <img src="https://mintcdn.com/zeotap/CSAMM_Rlvn0jps6x/articles/integrate-customer/Storage/integrate-customer/image-140.png?fit=max&auto=format&n=CSAMM_Rlvn0jps6x&q=85&s=9866e10725d217d9e53493733ef56e54" width="1570" height="1396" data-path="articles/integrate-customer/Storage/integrate-customer/image-140.png" />
    </Frame>
  </Step>

  <Step>
    Enter a short and descriptive name for the source.
  </Step>

  <Step>
    By default, the region is chosen as **Europe (EU).**
  </Step>

  <Step>
    Under **Data Entity**, depending on the type of data that you want to ingest, choose either **Customer Data** or **Non Customer Data** and proceed with the source creation. To know more about Customer Data and Non Customer Data, refer [here](https://docs.zeotap.com/smart/project-unify/supported-data-entities).
  </Step>

  <Step>
    Enter the **Topic name, Kafka Broker Host, Broker API Key, Broker API Secret** and **Consumer group ID** obtained from your Kafka account as mentioned in the prerequisites section.
  </Step>

  <Step>
    Select the format of data that will be sent from Kafka. The current support formats are **AVRO** and **JSON**.

    <Frame>
      <img src="https://mintcdn.com/zeotap/CSAMM_Rlvn0jps6x/articles/integrate-customer/Storage/integrate-customer/image-141.png?fit=max&auto=format&n=CSAMM_Rlvn0jps6x&q=85&s=b1fef50a2b727403f6085ef3b7e02587" width="1526" height="1332" data-path="articles/integrate-customer/Storage/integrate-customer/image-141.png" />
    </Frame>
  </Step>

  <Step>
    If **AVRO** format is selected, enter the **Schema Registry URL, Schema Registry Key** and **Schema Registry Secret** from your Kafka account.
  </Step>

  <Step>
    Click **CREATE SOURCE**. Upon successful creation, all the fields that were entered while creating the source are displayed under the **IMPLEMENTATION DETAILS** tab along with all the important information related to the integration, as shown in the image below.

    <Frame>
      <img src="https://mintcdn.com/zeotap/CSAMM_Rlvn0jps6x/articles/integrate-customer/Storage/integrate-customer/image-142.png?fit=max&auto=format&n=CSAMM_Rlvn0jps6x&q=85&s=083dfddfa8e1d8d01ab4837c89d1747e" width="1600" height="828" data-path="articles/integrate-customer/Storage/integrate-customer/image-142.png" />
    </Frame>
  </Step>

  <Step>
    Once the connection between Zeotap and Kafka is successfully established, you can view the data flowing in into the system under the **[PREVIEW DATA](/articles/integrate-customer/knowing-the-fields-on-the-preview-data-tab)** tab.
  </Step>
</Steps>