Hi Christoph Gmeiner, can you tell me more about how these files are stored in your GCS bucket? What structure is applied to the model and package and project files?

Hi @...,

Christoph and I are in the same team, and this request is related: https://support.fivetran.com/hc/en-us/community/posts/4409011892119-Transformations-Manage-simple-SQL-transformations-from-API

We can focus our discussion here. 

Currently we are: 

- Using simple SQL transformations from Fivetran UI
- Applying dbt code within our own structure (totally bypassing your UI) 

We would like to:

- Be able to define and trigger simple SQL transformations from our system through your API, while still seeing them in the UI
- Be able to define dbt transformations by linking to our dbt files in GCS, because the connection with our Gitlab is problematic. Hence to start using the dbt features provided by your UI

As for your question about the structure. We have a GCS bucket which stores our Airflow files. This bucket has a dbt folder. Every project has its own directory, for example `dbt/projects/fivetran_log`. The fivetran package is recorded in the `packages.yml` file in that folder and installed automatically through `dbt deps`, therefore it is found in `dbt/projects/fivetran_log/dbt_modules/fivetran_log`. Does this answer your question? 

Thanks, 

Vlad

Thanks for elaborating Vlad Gheorghe. We do not have support for this type of setup currently planned.

I'd like to understand the issues you're having with Gitlab better– do they extend from the connection to Gitlab from Fivetran Transformations? Or is there complexity in using Airflow to orchestrate dbt through Gitlab?

In terms of how Fivetran Transformations might be helpful here– is there a layer of processing that happens between Fivetran Connectors and dbt Transformations that must be accounted for?