Follow these instructions to connect your MongoDB replica set to your destination using Fivetran.
Setup instructionslink
Find replica set host identifiers link
You need the MongoDB replica set’s host identifier and optionally the alternative host identifiers to configure Fivetran. The replica set’s host identifier has one of the following formats:
- SRV host identifier. For example,
mongodb+srv://example.mongodb.net/?readPreference=secondary&readPreferenceTags=nodeType:ANALYTICS
. - Connection string. For example,
mongodb://mongodb0.example.com:27017,mongodb1.example.com:27017,mongodb2.example.com:27017/?replicaSet=replA"
. - Domain and port. For example,
your.server.com:27017
. - IP address and port. For example,
1.2.3.4:27017
.
NOTE: For SRV host identifier and Connection string, we support connection tags.
TIP: We recommend that you use secondary or analytics nodes. These nodes isolate queries from your operational workload and improve cluster reliability and performance.
MongoDB Atlas
-
Log in to the MongoDB Atlas dashboard.
-
In the Cluster Overview tab, click Connect.
-
Select Connect your application.
-
Copy the SRV host identifier.
MongoDB shell
IMPORTANT: Your MongoDB server must have replication enabled already. If replication is not enabled, follow MongoDB’s Replica Set Deployment instructions before proceeding.
- Connect to your replica set or primary node using the MongoDB shell as an Admin user.
- Execute the
db.adminCommand( { replSetGetStatus : 1 } ).members
command. - Copy the host identifier and optionally the alternative host identifiers.
Allow database accesslink
Create a database user for Fivetran using either MongoDB Atlas or the MongoDB shell.
MongoDB Atlas
- Log in to the MongoDB Atlas dashboard.
- In the left-hand navigation menu, go to Security > Database Access.
- Click New Database User.
- Choose the password authentication method.
- Enter the username and password for the new Fivetran user.
- In the Database User Privileges drop-down menu, select Grant Specific User Privileges.
- Under Specific Privileges, add the following roles/privileges:
readAnyDatabase
read on the local database
- Click Add User.
For more information, see MongoDB Atlas’ Configure Database Users documentation.
MongoDB shell
-
Open the MongoDB shell.
-
Connect to your replica set or primary node using the MongoDB shell.
-
Execute the following command to create a user for Fivetran. Replace
<username>
and<password>
with a username and password of your choice.use admin db.createUser({ user: "<username>", pwd: "<password>", roles: [ "readAnyDatabase", {role: "read", db: "local"} ] })
TIP : The Mongo shell command to create a user is
db.AddUser()
for versions 2.x.
For more information, see MongoDB’s Built-In Roles documentation.
Choose connection methodlink
Decide whether to connect Fivetran to your MongoDB cluster directly, using an SSH tunnel, or using AWS PrivateLink.
For more information about connection options, see How to Choose the Right Database Connection Option.
Connect directly (TLS required)
Fivetran connects directly to your MongoDB cluster. This is the simplest method.
IMPORTANT: You must have TLS enabled on your replica set to connect directly to Fivetran. Follow MongoDB’s TLS Configuration instructions to enable TLS on your replica set.
To connect directly, configure your firewall and/or other access control systems to allow incoming connections to your MongoDB cluster from Fivetran’s IPs for your cluster region. For more information, see MongoDB’s Security documentation.
MongoDB Atlas
- Log in to the MongoDB Atlas dashboard.
- In the Cluster Overview tab, make a note of MongoDB cluster cloud service provider and region.
- In the left-hand navigation menu, go to Security > Network Access.
- Click Add IP address.
- Allow Fivetran’s IP for your MongoDB cluster’s cloud service provider and region.
- In the Access List Entry field, enter the Fivetran IP and click Confirm.
MongoDB shell
For steps to whitelist Fivetran’s IPs, see MongoDB’s Security Considerations documentation.
Connect using SSH (TLS optional)
Fivetran connects to a separate server in your network that provides an SSH tunnel to your cluster. You must connect through SSH if your database is in an inaccessible subnet.
To connect using SSH, configure your firewall and/or other access control systems to allow incoming connections to your MongoDB port (usually 27017
) from your SSH tunnel server’s IP.
Before you proceed to the next step, you must follow our SSH connection instructions. If you want Fivetran to tunnel SSH over TLS, follow MongoDB’s TLS Configuration instructions to enable TLS on your cluster.
Connect using AWS PrivateLink undefined
IMPORTANT: You must have a Business Critical plan to use AWS PrivateLink.
AWS PrivateLink allows VPCs and AWS-hosted or on-premises services to communicate with one another without exposing traffic to the public internet. PrivateLink is the most secure connection method. Learn more in AWS’ PrivateLink documentation.
Follow our AWS PrivateLink setup guide to configure PrivateLink for your database.
IMPORTANT: Fivetran cannot connect to MongoDB Atlas using Privatelink.
(Optional) Set oplog sizelink
Set the oplog size so that it can retain at least 24 hours’ worth of changes. We recommend increasing the size to retain seven days’ worth of data.
Adjust your oplog size using either MongoDB Atlas or the MongoDB shell:
- MongoDB Atlas: Follow MongoDB Atlas’ Set Oplog Size tutorial.
- MongoDB shell: Follow MongoDB’s Change the Size of the Oplog tutorial.
Choose pack modelink
Choose between packed mode or unpacked mode. By default unpacked
mode is selected, where Fivetran unpacks one layer of nested fields and infer types. In packed mode, we write the data without unpacking.
For more information, see our Pack mode documentation.
Finish Fivetran configurationlink
-
In your connector setup form, enter a Destination schema prefix. This prefix applies to each replicated schema and cannot be changed once your connector is created.
-
In the Host and ports field, enter the host identifiers you found in Step 1.
-
Enter the Fivetran-specific User that you created in Step 2.
-
Enter the Password for the Fivetran-specific user that you created in Step 2.
-
Choose your Connection Method. If you selected Connect via an SSH tunnel, provide the following information:
- SSH hostname (do not use a load balancer’s IP address/hostname)
- SSH port
- SSH user
- If you enabled SSL/TLS on your database in Step 3, set the Require TLS through tunnel toggle to ON.
-
Click Save & Test. Fivetran tests and validates our connection to your MongoDB replica set. Upon successful completion of the setup tests, you can sync your data using Fivetran.
Setup testslink
Fivetran performs the following tests to ensure that we can connect to your MongoDB replica set and that it is properly configured:
- The Connecting to SSH Tunnel Test validates the SSH tunnel details you provided in the setup form. It then checks that we can connect to your database using the SSH Tunnel. (We skip this test if you aren’t connecting using SSH.)
- The Connecting to Host Test validates the database credentials you provided in the setup form. It then verifies that the database host is not private and checks that we can connect to the host.
- The Validating Certificate Test generates a pop-up window where you must choose which certificate you want Fivetran to use. Select the root certificate to trust all the configured hosts. It then validates that certificate and checks that we can connect to your database using TLS. (We skip this test if you aren’t connecting directly.)
- The Connecting to Database Test connects to your database instance and verifies that your database is at least version 2.6. It then checks that we can access the schemas in your database.
- The Checking Query Permissions Test queries your MongoDB database to do the following:
- Fetch all databases and check if we have
listDatabase
permissions - Check if we have permissions to query any database and if there are any non-empty collections present
- Check if we have permissions to query any collection
- Fetch all databases and check if we have
- The Checking Change Streams/ Oplog Access Test verifies if we can access the change streams. If the change streams are not accessible, it then verifies if we can access the oplog and if the oplog contains at least 24 hours’ worth of changes.
NOTE: The tests may take a few minutes to finish running.
Related Contentlink
description Connector Overview
account_tree Schema Information
settings API Connector Configuration