Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Transformations: Trigger Transformations after One Another (Sequential Triggers)

Answered
Gary Sahota User

Hi, I'd love to be able to trigger transformations after one another. Right now, I have a transformation that needs to complete first (to create a blended table) & then a second transformation (that aggregates that blended table & blends other data). But right now I can't choose when they run.

Given the second transformation needs the first one to complete, I'd love to be able to trigger sequential transformations to ensure it doesn't prematurely run.

My current stop gap was to time the start of the transformations when I first kicked them off. But given I needed them to be done during non-business hours, I had to kick off one at 1am and the other at 1:30am to ensure no overlap.

Please sign in to leave a comment.

Comments

5 comments

Date Votes
  • Official comment
    Fraser User
    • Edited

    We recommend that all Fivetran customers who have advanced data modelling needs, such as dependencies between transformations, try out dbt.

    Fivetran orchestrates your dbt Core project - a paid dbt Cloud license is not required. It's Fivetran who is running the queries against your data warehouse, all within our SOC2 certification.  dbt Core is a structure for creating data models & organizing dependencies between them. As well as other nifty things like data validation tests.

    What is dbt Core?  An open source project that compiles your templatized data models into SQL & manages the order of execution of data model dependencies.

    What is a dbt project? It's a folder of code that follows the conventions of dbt.  You can edit the code using an Integrated Developer Environment (IDE).  In order for Fivetran to orchestrate it, the code must be under version control hosted in a git repository.  Popular git repository options are GitHub and Gitlab.

    What is dbt Cloud? dbt Cloud is a fully featured Web-based IDE that makes it easy to code & manage your dbt project. It's a paid service from Fishtown Analytics, the primary sponsors of the dbt Core project.

    There are many popular desktop IDEs you can use for writing your dbt project code. Our customers & team have used VS Code and DataGrip.

    Read more about it here: https://fivetran.com/docs/transformations/dbt#dbtoverview

  • Amy Peterson Fivetranner

    Thanks for submitting this request! In order to handle this today, we'd recommend using our dbt integration. dbt allows you to set transformation dependencies so that your models are always run in the correct order. 

  • Gary Sahota User

    Hi Amy, I'd love to use dbt, but that's a separate tool that I would have to get approval and budget for (while I'll definitely make the case, it's just hard during these times). My IT team doesn't like us using free tools for sensitive data, so couldn't use their free tier either, which is why I asked for this.

  • Amy Peterson Fivetranner

    Thanks for the additional context, that makes sense and we will keep that in mind!

  • Gary Sahota User

    No problem :)