Connector Improvement: Ability to do full refresh from S3
Completed
We have a customer which uses Looker scheduling service to write data into S3. Every day they write a full data dump. We want to be able to load the data as a full refresh in our BigQuery. We have tried upsert option with latest file, but it doesn't work as Fivetran is not able to find a primary key.
-
Official comment
Hi Amey,
Thank you for reaching out. Is the goal to write data from BigQuery into S3? Our BigQuery connector supports PK-less tables today and can facilitate data movement into a Fivetran managed S3 destination.
If you would like to chat more about your use case, please also feel free to reach out to me directly.
-
Hi Amey,
The Amazon S3 connector now allows setting a primary key. Please refer to the setup guide for details.
Thank you,
Parmeet
Please sign in to leave a comment.
Comments
2 comments