Follow our setup guide to connect Apache Kafka to Fivetran.
Prerequisiteslink
To connect Apache Kafka to Fivetran, you need an Apache Kafka server.
Setup instructionslink
Run the following commands in your Apache Kafka server.
Generate and Import Client Certificate to Server Truststorelink
-
Create a key and keystore for the consumer client.
keytool -keystore <client_keystore_name>.jks -alias <alias_name> -validity <VALIDITY> -genkey -keyalg RSA
You will be asked to enter a few details. For
What is your first and last name?
, enter your Apache Kafka server hostname. Remember the key and keystore password you provide. -
Export the client certificate from the keystore.
keytool -exportcert -rfc -file <client_certificate_name>.pem -alias <alias_name> -keystore <client_keystore_name>.jks
The
<alias_name>
must be the same as in step 1. -
Import the client certificate into the truststore of your Apache Kafka server.
keytool -keystore <server_truststore_name>.jks -alias <alias_name> -import -file <client_certificate_name>.pem
Get Client Certificate Key from Client Keystorelink
-
Convert the client keystore from
JKS
toPKCS12
format.keytool -v -importkeystore -srckeystore <client_keystore_name>.jks -srcalias <alias_name> -destkeystore <client_keystore_name>.p12 -deststoretype PKCS12
-
Extract client certificate key into a
.pem
file.openssl pkcs12 -in <client_keystore_name>.p12 -nocerts -nodes > <client_certificate_key>.pem
Required Certificates and Keylink
Download the following files from your Apache Kafka server:
- Client certificate(
<client_certificate_name>.pem
) - Client certificate key (
<client_certificate_key>.pem
) - CA certificate (Trusted Certificate): It is a public-private key pair and certificate in your Apache Kafka server which was used to sign other certificates
Finish Fivetran Configurationlink
-
Enter your chosen destination schema name in the connector setup form.
-
Enter the unique consumer group for Fivetran.
-
Enter your server details in
<server_host_name>:<port>
format. -
Select your message type: Json, Avro, Protobuf, or Text.
-
If you choose Json, select a sync type: Packed or Unpacked.
-
If you choose Avro or Protobuf, enter the following schema registry credentials:
- Schema Registry URLs
- Schema Registry Key
- Schema Registry Secret
-
-
Select a security protocol: PLAINTEXT, TLS, or SASL.
-
If you choose TLS, upload the Trusted Certificate, Client Certificate, and Client Certificate Key files.
-
If you choose SASL, select the SASL Mechanism, and then enter your API Key and API Secret.
-
-
Click Save & Test. Fivetran will take it from here and sync your Apache Kafka data.
Related Contentlink
description Connector Overview
settings API Connector Configuration