Reduce BigQuery access requirements for Fivetran/Any Service Account
Which destination?: BigQuery
Additional details: My team manages all datasets in BigQuery as-code via Terraform. We never want Fivetran to create datasets, as when Fivetran creates datasets we have to import them in code to be managed via Terraform.
Fivetran does not allow us to remove all project-level permissions, as it checks to see if it can create a dataset every time a sync runs.
Fivetran's documentation for BigQuery roles and permissions states that the SA needs `user` permission on the project at a minimum. This may be the lowest permission required, but it gives excessive permission to view all datasets within the project where it is granted this access. This is also misleading, as one could create a custom role with just `bigquery.datasets.create` permission, which fulfills the aforementioned dataset creation check on each sync.
Since we manage datasets as-code and would prefer Fivetran's SA (or any SA) to not have any project-level permissions where they can be avoided (`jobUser` would have to be granted), allow customers to opt out of verifying that the SA can create a dataset on each sync.
Please sign in to leave a comment.
Comments
0 comments