Apache Kafka Setup Guide

Follow our setup guide to connect Apache Kafka to Fivetran.

Prerequisiteslink

To connect Apache Kafka to Fivetran, you need an Apache Kafka server.

Setup instructionslink

Run the following commands in your Apache Kafka server.

Generate and Import Client Certificate to Server Truststorelink

1. Create a key and keystore for the consumer client.

   keytool -keystore <client_keystore_name>.jks -alias <alias_name> -validity <VALIDITY> -genkey -keyalg RSA

   You will be asked to enter a few details. For What is your first and last name?, enter your Apache Kafka server hostname. Remember the key and keystore password you provide.

2. Export the client certificate from the keystore.

   keytool -exportcert -rfc -file <client_certificate_name>.pem -alias <alias_name> -keystore <client_keystore_name>.jks

   The <alias_name> must be the same as in step 1.

3. Import the client certificate into the truststore of your Apache Kafka server.

   keytool -keystore <server_truststore_name>.jks -alias <alias_name> -import -file <client_certificate_name>.pem

Get Client Certificate Key from Client Keystorelink

1. Convert the client keystore from JKS to PKCS12 format.

   keytool -v -importkeystore -srckeystore <client_keystore_name>.jks -srcalias <alias_name> -destkeystore <client_keystore_name>.p12 -deststoretype PKCS12

2. Extract client certificate key into a .pem file.

   openssl pkcs12 -in <client_keystore_name>.p12 -nocerts -nodes > <client_certificate_key>.pem

Required Certificates and Keylink

Download the following files from your Apache Kafka server:

• Client certificate(<client_certificate_name>.pem)
• Client certificate key (<client_certificate_key>.pem)
• CA certificate (Trusted Certificate): It is a public-private key pair and certificate in your Apache Kafka server which was used to sign other certificates

Finish Fivetran Configurationlink

1. Enter your chosen destination schema name in the connector setup form.

2. Enter the unique consumer group for Fivetran.

3. Enter your server details in <server_host_name>:<port> format.

4. Select your message type: Json, Avro, Protobuf, or Text.

   • If you choose Json, select a sync type: Packed or Unpacked.

   • If you choose Avro or Protobuf, enter the following schema registry credentials:

     • Schema Registry URLs
     • Schema Registry Key
     • Schema Registry Secret

5. Select a security protocol: PLAINTEXT, TLS, or SASL.

   • If you choose TLS, upload the Trusted Certificate, Client Certificate, and Client Certificate Key files.

   • If you choose SASL, select the SASL Mechanism, and then enter your API Key and API Secret.

6. Click Save & Test. Fivetran will take it from here and sync your Apache Kafka data.

Related Contentlink

description Connector Overview

assignment Release Notes

settings API Connector Configuration

home Documentation Home