Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: HVR Refresh should wrap TRUNCATE and COPY INTO into one transaction

Answered
Scott McKenzie User

Please see Support Case 313426.

We just learned that Fivetran Bulk Refresh (Snowflake target) does not wrap table truncation and COPY INTO into the same transaction. This is resulting in periods where our data consumers are seeing 0 records in tables. This behavior is not acceptable for us - is there a way we can configure this behavior?

I do see there is an option for "Merge into Target (No Delete/Truncate on Target)", but this will not work as it will not factor in delete operations on source. Additionally, I'm worried "row-by-row" refresh will not be performant given the high transactional volume / row count inherent to these tables.

Please sign in to leave a comment.

Comments

1 comment

Date Votes
  • Edwin Fivetranner

    Hi Scott,

    We understand your concern regarding the temporary visibility of zero records during the Fivetran Bulk Refresh for Snowflake targets. The current process does not wrap the TRUNCATE and COPY INTO operations in a single transaction, which can lead to a brief gap in data availability. 

    We’ll take this requirement back to our team for further investigation. It’s not yet clear whether this behavior can be changed easily within HVR.

    Please note that bulk refreshes are typically exceptional operations, and in most customer scenarios, the target tables used for refresh are not directly exposed to end users in real-time. However, we do understand that your use case may differ.

    Best regards,
    Edwin