Follow our setup guide to connect AWS MSK to Fivetran.
To connect AWS MSK to Fivetran, you need an AWS MSK cluster.
Find ENI IDlink
Open the AWS MSK console and go to your cluster.
In the Brokers summary section, for each broker find the Attached ENI ID. Make a note of the ENI ID(s).
Open the EC2 dashboard and select Network Interfaces.
For every broker ENI ID you found in Step 1, select Actions > Associate address to associate an Elastic IP address.
Make a note of the associated Elastic IPs for each broker. You will need them to configure Fivetran.
Configure security groupslink
On the EC2 dashboard, select Security Groups.
advertised.listeners property for the corresponding port of each broker to the Elastic IP associated with the ENI of the broker. Use dynamic configuration to configure the broker-level properties.
Finish Fivetran configurationlink
In the connector setup form, enter the Destination schema name of your choice.
Enter the dedicated Consumer Group for Fivetran.
Enter your Servers details using the
<host>:<port>format. Use the Elastic IPs you found in Step 2 as
<host>. For example, if the Elastic IP for the broker is
10.100.10.10and the authentication is
SASL/SCRAM, then enter the server name as
NOTE: Click + to add more than one server details.
Select your message type: Json, Avro, Protobuf, or Text.
If you choose Json, select a sync type: Packed or Unpacked.
If you choose Avro or Protobuf, enter the following schema registry credentials:
- Schema Registry URLs
- Schema Registry Key
- Schema Registry Secret
Select a security protocol: PLAINTEXT or SASL/SCRAM.
- If you choose SASL/SCRAM, enter your secret’s Username and Password.
Click Save & Test. Fivetran will take it from here and sync your AWS MSK data.