Fivetran integrates with Fishtown Analytics’ data build tool (dbt) to power complex transformations. Fivetran connects directly to your Git provider and runs your dbt transformations according to your schedule.
To see how to orchestrate dbt transformations with Fivetran, watch this short video:
dbt Overviewlink
dbt is an open-source software that enables you to perform sophisticated data transformations in your destination using simple SQL statements. With dbt, you can:
- Write and test SQL transformations
- Use version control with your transformations
- Create and share documentation about your dbt transformations
- View data lineage graphs
How does dbt work?link
Once you have set up dbt, you write SQL SELECT statements (a.k.a “dbt models”) to transform your data. dbt runs these SQL statements in your destination to build tables and views. dbt honors dependencies between your models so everything is built in the correct order.
To work with dbt, you can either use the dbt Cloud application or the dbt Command Line Interface (CLI). Fivetran can run dbt projects created with either dbt Cloud or dbt CLI.
dbt Cloud
dbt Cloud is a hosted service that helps you manage your dbt transformations. dbt Cloud provides support for scheduling and monitoring your dbt project, CI/CD, and generating documentation. It also provides an Integrated Developer Environment where you can develop, run, and version control dbt projects from your web browser. You can either use dbt Cloud’s Developer package for free or pay for additional features. Learn more in dbt’s Pricing documentation.
dbt CLI
dbt CLI is a command line interface for running your dbt project. It is open source and free to use.
How does Fivetran work with dbt?link
Fivetran connects to your Git provider and runs your dbt transformations according to the schedule you created in your dbt project. In your Fivetran dashboard, you can view the current status and historical logs of each dbt job. We sync your dbt transformations from your Git provider every few minutes to ensure that we are up to date.
Supported destinationslink
Fivetran supports dbt transformations for the following destinations:
Fivetran dbt packageslink
IMPORTANT: To use Fivetran’s dbt packages, you must have a BigQuery, Redshift, or Snowflake destination.
We offer dbt packages for our top connectors. Our pre-built packages produce clear, comprehensible tables that power your reports, visualizations, and dashboards.
Fivetran has dbt packages for the following connectors:
- Asana (source / transform)
- Facebook Ads (source / transform / creative history)
- Facebook Pages (source / transform)
- Fivetran Log Connector
- GitHub (source / transform)
- Google Ads (source / transform)
- Greenhouse (source / transform)
- HubSpot (source / transform)
- Intercom (source / transform)
- Instagram Business (source / transform)
- Iterable (source / transform)
- Jira (source / transform)
- Klaviyo (source / transform)
- Lever (source / transform)
- LinkedIn Ad Analytics (source / transform)
- LinkedIn Company Pages (source / transform)
- Mailchimp
- Marketo (source / transform)
- Microsoft Advertising (source / transform)
- Mixpanel
- NetSuite SuiteAnalytics (source / transform)
- Pardot (source / transform)
- Pendo (source / transform)
- Pinterest Ads (source / transform)
- QuickBooks (source / transform)
- Salesforce (source / transform)
- Shopify (source / transform)
- Shopify Holistic
- Snapchat Ads (source / transform)
- Stripe (source / transform)
- TikTok Ads (source / transform)
- Twitter Ads (source / transform)
- Twitter Organic (source / transform)
- Xero (source / transform)
- YouTube Analytics (source / transform)
- Zendesk Support (source / transform)
NOTE: Our older dbt packages have all necessary files combined into a single folder. Our newer dbt packages are split into two packages - source and transform.
If you’d like Fivetran to create a dbt package for a connector that’s not listed here, send your request to solutions@fivetran.com.
Ad reporting packagelink
In addition to our connector-level dbt packages, Fivetran offers a pre-built ad reporting dbt package that aggregates data from our most popular advertising connectors.
Shopify Holistic packagelink
In addition to the base-level Shopify dbt package, Fivetran offers the Shopify Holistic package to combine Shopify data with data from other marketing connectors. Currently, we support combining Shopify data with email and SMS marketing data from Fivetran’s Klaviyo dbt package.
Social media reporting packagelink
In addition to our connector-level dbt packages, Fivetran offers a pre-built social media reporting dbt package that aggregates data from our most popular social media connectors.
How do Fivetran dbt packages work?link
After Fivetran syncs your data to your destination, our dbt packages transform your data and create analytics-ready tables in your destination.
Some dbt packages have all necessary files combined into a single package. However, most Fivetran dbt packages consist of two smaller packages:
- The source packages expose, document and standardize the underlying Fivetran schemas being created in the destination.
- The transform packages produce analytics-ready models for each source.
Source packages
Source packages add column-level tests and perform minor transformations on your destination data (for example, renaming columns). They are designed to help you test and clean your data and to create a dictionary of your data. They also prepare your data for our transform packages, if you choose to use them.
Each source package does the following:
- Adds descriptions to the tables and columns that Fivetran has synced.
- Adds column-level testing where applicable. For example, we test all primary keys for uniqueness and non-null values.
- Models the staging tables that our transform packages require.
Transform packages
Transform packages transform your data into analytics-ready models. They are designed to help you understand and analyze your data. They’re especially helpful if you’re using your connector for the first time or haven’t spent much time modeling your data.
Each transform package does the following:
- Automatically installs and runs the corresponding source package.
- Performs a series of SQL transformations on the source package’s staging tables. The specific transformations vary by connector.
- Produces models that address common analytics scenarios for your connector.
How do I add Fivetran dbt packages to my project?link
To add a Fivetran dbt package to your dbt project, see our dbt setup guide.
Setup guidelink
Follow our step-by-step setup guide to learn how to use dbt transformations in Fivetran.
Notificationslink
You can enable email notifications for each of your dbt transformations on your Fivetran dashboard’s Notifications tab. To learn more about email notifications, see our Notifications documentation.