Skip to main content

Community

Transformations: Allow customization to the generated profiles.yml for Dbt

Completed

Please sign in to leave a comment.

Comments

6 comments

  • Official comment

    Would you please tell me more about what you are trying to accomplish? I'm open to the idea of making the yml customizable. I'm keen to understand your goal in doing so to help us understand the value of offering customization here.

    In the same snowflake account I am having multiple database for dev, QA and prod. I am selecting the database/schema to be used for reading source table or creating model/snapshot on basis of jinja template which looks something like this :-

    |
    {%- if target.name == "dev" -%} ODS_POC
    {%- elif target.name == "qa" -%} ODS_QA
    # For testing in Fivetran reverting to ODS_POC
    {%- elif target.name == "prod" -%} ODS_POC
    {%- else -%} invalid_database
    {%- endif -%}

    I am able to set target.name as dev by running 

    dbt run --target dev

    but since the profiles.yml has only prod as tagert.name, this fails. For the time being I am trying to fix this 

    We have Sandeep's use case mentioned previously, as well as another one.

    We would like to use different roles to run different models. When we use dbt to create different views/tables, we want to customize the owner of those. For more sensitive data we want more locked down roles to create them, and vice-versa. There are other work-arounds we can use but being able to use roles would be the cleanest.

    To add to this, we would like to select the _warehouse_ in Snowflake; potentially addressable by specifying a Role for transformations, and setting the default warehouse for that Role (there's a FR in progress for this, I think?), or through a custom profiles.yml (as in this discussion).

    Awesome discussion, thank you for the explanations! I expect our first version of this to be targeted specifically at managing test and production environments. We will look at allowing further customizations to the yml, too. It's a balance against ease of use.

    Using different roles to run different models is further down the road. I see the utility. For now, I took the liberty of creating a separate feature request so Chris' request doesn't get lost in the comments of this one. Here is that idea - be sure to upvote! https://support.fivetran.com/hc/en-us/community/posts/1500000690262-Transformations-Use-different-roles-to-run-different-models

    There is also this related idea: https://support.fivetran.com/hc/en-us/community/posts/360051261534-Transformations-Allow-DBT-Role-to-be-configurable?input_string=Use%20different%20roles%20to%20run%20different%20models

    I see that this feature is now being supported https://fivetran.com/docs/changelog/march-2021#dbttransformations.

    Thanks for the quick turn around

Didn’t find what you need?

Contact support