Description:

I’d like to request a feature enhancement to improve the compatibility between Fivetran and Snowflake's auto-refresh functionality when using Azure Blob Storage following official Snowflake documentation - Refreshing directory tables automatically for Azure Blob Storage | Snowflake Documentation

Fivetran's current implementation writes objects to Azure Storage using the AppendFile API call. However, Snowflake's auto-refresh monitoring only supports specific Microsoft.Storage.BlobCreated events: CopyBlob, PutBlob, PutBlockList, FlushWithClose, and SftpCommit.

"The current JAVA Azure APIs used to upload files to ADLS storage only result in operations such as CreatePathFile, AppendFile, and FlushFile" - as confirmed by Fivetran in a support ticket and our logs in azure.

Since AppendFile events don’t align with these supported types, Snowflake’s auto-refresh is unable to detect Fivetran's new data uploads. This forces users to perform manual refreshes, leading to potential data latency and operational inefficiencies.

Proposed Solution:
Adjust Fivetran’s Azure Blob Storage integration to use one of the supported event types (PutBlob or PutBlockList, etc.) when writing data, or provide an option for users to configure how Fivetran writes to Azure Storage to ensure compatibility with Snowflake auto-refresh triggers.

Impact:
This change would enable seamless auto-refresh for Snowflake pipelines using Fivetran with Azure Blob Storage, reducing the need for manual intervention and ensuring timely data availability with minimal latency.

Thank you for considering this enhancement request.