Connector Improvement: Amazon S3 Connector: Fivetran not Extracting the Partition Field (date)
Answered
Since my folder structure is "sample_data/xyzdata/date=2025-02-15/part-0.parquet", Fivetran should be able to extract the partition field "date" but it is missing in the Snowflake.
The expectation is that Fivetran Connector will parse the Folder Structure, create a column called "date" in the destination, and fill it with the date value.
However, it is missing in the destination table.
-
Official comment
Hi Prashanth, thanks for submitting this request! The date can be extracted from the _file metadata column post-load in Snowflake. _file has the file path for every record.
Thank you,
Parmeet
Please sign in to leave a comment.
Comments
1 comment