NOTE: This article applies to legacy transformations with dbt Core.
You want to use the
deployment.yml file to perform advanced orchestrations with your dbt Git repository. You want to do the following:
- Run specific models
- Run specific tasks
- Run other dbt commands
dbt Git repository and your local source code editor
Fivetran provides a sample deployment.yml file that enables us to run your dbt commands on a schedule. You can edit the jobs in the
deployment.yml file to reflect how and when you want Fivetran to run your dbt commands.
deployment.yml file allows you to define multiple jobs, each of which translates to a transformation in your Fivetran dashboard.
Each job include three main components:
- Name: Enter a unique job name. The name is displayed in your Fivetran dashboard once your jobs are imported.
- Schedule: Define when this job should run using the Cron format. It is possible to define a list of Cron schedules using a YAML array. Both the single line and multiline formats are supported.
schedule: ["0 12 * * *", "0 5 * * *"]
schedule: - 0 12 * * * - 0 5 * * *
Steps: A job performs multiple steps sequentially, so define each step here:
- Name: Give each step in your job a name. This enables you to track the steps in the logs.
- Command: Enter the dbt command that should run in this step.
To ensure data integrity, use two steps (run the models and test the models) within a job:
- name: daily schedule: 0 12 * * * # This example will run every day at 12:00pm steps: - name: run models command: dbt run - name: test models command: dbt test
You can also run the models in a specific order by including steps that only run certain models:
- name: daily schedule: 0 12 * * * # This example will run every day at 12:00pm steps: - name: run first sales model command: dbt run --models sales_model_1 - name: run second sales model command: dbt run --models sales_model_2