Follow these instructions to connect your MongoDB sharded cluster to your destination using Fivetran.
Setup instructionslink
Find mongos query routerlink
You need the MongoDB sharded cluster’s mongos query router’s host identifier to configure Fivetran.
The sharded cluster’s mongos query router’s host identifier has one of the following formats:
- Domain and port. For example,
your.server.comand27017. - IP address and port. For example,
1.2.3.4and27017.
NOTE: We don’t support SRV host in MongoDB sharded clusters.
MongoDB Atlas
-
Log in to the MongoDB Atlas dashboard.
-
In your database cluster, go to the Metrics tab.
-
Set the VIEW metric to MONGOS.
-
Select any one of the MONGOS query router based on your preference.
MongoDB shell
In the terminal connected to MongoDB, execute the sh.status(true) command and select any query router under active mongoses.
Allow database accesslink
Create a database user for Fivetran using either MongoDB Atlas or the MongoDB shell.
MongoDB Atlas
- Log in to the MongoDB Atlas dashboard.
- In the left-hand navigation menu, go to Security > Database Access.
- Click New Database User.
- Choose the password authentication method.
- Enter the username and password for the new Fivetran user.
- In the Database User Privileges drop-down menu, select Grant Specific User Privileges.
- Under Specific Privileges, add the following roles/privileges:
readAnyDatabaseclusterMonitorread on the local database
- Click Add User.
For more information, see MongoDB Atlas’ Configure Database Users documentation.
MongoDB shell
Create an identical Fivetran user in every primary shard node and the mongos query router. Replace <username> and <password> with a username and password of your choice.
use admin
db.createUser({
user: "<username>",
pwd: "<password>",
roles: [ "readAnyDatabase", "clusterMonitor" ]
})
You can pick any username and password, but it must be consistent across all primary shard nodes and the mongos query router.
NOTE: The Mongo shell command to create a user is
db.AddUser()for versions 2.x.
If you want to find out which shards are connected to the mongos query router, use the command db.adminCommand({ listShards: 1 }).
For more information, see MongoDB’s Built-In Roles documentation.
Choose connection methodlink
Decide whether to connect Fivetran to your MongoDB cluster directly, using an SSH tunnel, or using AWS PrivateLink.
For more information about connection options, see How to Choose the Right Database Connection Option.
Connect directly (TLS required)
Fivetran connects directly to your MongoDB cluster. This is the simplest method.
IMPORTANT: You must have TLS enabled on your replica set to connect directly to Fivetran. Follow MongoDB’s TLS Configuration instructions to enable TLS on your replica set.
To connect directly, configure your firewall and/or other access control systems to allow incoming connections to your MongoDB cluster from Fivetran’s IPs for your cluster region. For more information, see MongoDB’s Security documentation.
MongoDB Atlas
- Log in to the MongoDB Atlas dashboard.
- In the Cluster Overview tab, make a note of MongoDB cluster cloud service provider and region.
- In the left-hand navigation menu, go to Security > Network Access.
- Click Add IP address.
- Allow Fivetran’s IP for your MongoDB cluster’s cloud service provider and region.
- In the Access List Entry field, enter the Fivetran IP and click Confirm.
MongoDB shell
For steps to whitelist Fivetran’s IPs, see MongoDB’s Security Considerations documentation.
Connect using SSH (TLS optional)
Fivetran connects to a separate server in your network that provides an SSH tunnel to your cluster. You must connect through SSH if your database is in an inaccessible subnet.
To connect using SSH, configure your firewall and/or other access control systems to allow incoming connections to your MongoDB port (usually 27017) from your SSH tunnel server’s IP.
Before you proceed to the next step, you must follow our SSH connection instructions. If you want Fivetran to tunnel SSH over TLS, follow MongoDB’s TLS Configuration instructions to enable TLS on your cluster.
Connect using AWS PrivateLink undefined
IMPORTANT: You must have a Business Critical plan to use AWS PrivateLink.
AWS PrivateLink allows VPCs and AWS-hosted or on-premises services to communicate with one another without exposing traffic to the public internet. PrivateLink is the most secure connection method. Learn more in AWS’ PrivateLink documentation.
Follow our AWS PrivateLink setup guide to configure PrivateLink for your database.
IMPORTANT: Fivetran cannot connect to MongoDB Atlas using Privatelink.
(Optional) Set oplog sizelink
Set the oplog size so that it can retain at least 24 hours’ worth of changes. We recommend increasing the size to retain seven days’ worth of data.
Adjust your oplog size using either MongoDB Atlas or the MongoDB shell:
- MongoDB Atlas: Follow MongoDB Atlas’ Set Oplog Size tutorial.
- MongoDB shell: Follow MongoDB’s Change the Size of the Oplog tutorial.
Choose pack modelink
Choose between packed mode or unpacked mode. By default unpacked mode is selected, where Fivetran unpacks one layer of nested fields and infer types. In packed mode, we write the data without unpacking.
For more information, see our Pack mode documentation.
-
In your connector setup form, enter a Destination schema prefix. This prefix cannot be changed once your connector is created.
-
In the Host and port fields, enter the mongos host and port.
-
Enter the Fivetran-specific User that you created in Step 2.
-
Enter the Password for the Fivetran-specific user that you created in Step 2.
-
Choose your Connection Method. If you selected Connect via an SSH tunnel, provide the following information:
- SSH hostname (do not use a load balancer’s IP address/hostname)
- SSH port
- SSH user
- If you enabled SSL/TLS on your database in Step 3, set the Require TLS through tunnel toggle to ON.
-
Click Save & Test. Fivetran tests and validates our connection to your MongoDB sharded cluster. Upon successful completion of the setup tests, you can sync your data using Fivetran.
Setup testslink
Fivetran performs the following tests to ensure that we can connect to your MongoDB sharded cluster and that it is properly configured:
- The Connecting to SSH Tunnel Test validates the SSH tunnel details you provided in the setup form. It then checks that we can connect to your database using the SSH Tunnel. (We skip this test if you aren’t connecting using SSH.)
- The Connecting to Host Test validates the database credentials you provided in the setup form. It then verifies that the database host is not private and checks that we can connect to the host.
- The Validating Certificate Test generates a pop-up window where you must choose which certificate you want Fivetran to use. Select the root certificate to trust all the configured hosts. It then validates that certificate and checks that we can connect to your database using TLS. (We skip this test if you aren’t connecting directly.)
- The Connecting to Database Test connects to your database instance and verifies that your database is at least version 2.6. It then checks that we can access the schemas in your database. Lastly, it verifies that we can access the oplog and that it contains at least 24 hours’ worth of changes.
- The Checking Query Permissions Test queries your MongoDB database to do the following:
- Fetch all databases and check if we have
listDatabasepermissions - Check if we have permissions to query any database and if there are any non-empty collections present
- Check if we have permissions to query any collection
- Fetch all databases and check if we have
NOTE: The tests may take a few minutes to finish running.
Related Contentlink
description Connector Overview
account_tree Schema Information
settings API Connector Configuration