Question
Why is there an increase in fail-safe backup storage when using Fivetran to copy data to the Snowflake warehouse?
Environment
Destination: Snowflake
Answer
The solution is to use transient tables.
New Connectors
Registering Fivetran with a transient database will ensure all Fivetran created tables are transient by default. This will help in reducing the additional costs and storage space associated with Fail-safe. However, transient tables cannot be recovered in case of operational or system failures.
Existing Connectors
1. Pause the Fivetran database connector
2. Clone the target table into a transient table. This will result in an identical, transient table (make sure that the Schema names, Table Names, Column names, and the column data type are the same).
3. Swap tables
4. Resume Fivetran DB connector
5. Confirmed the sync is working fine
6. Drop the swapped out, permanent table
See Snowflake's Fail-safe documentation for more information.
Read more about data load costs here
Cause
Snowflake defaults to Fail-safe. Fail-safe is a backup/storage mechanism from Snowflake which ensures historical data is protected in the event of a system failure or other catastrophic event.