New Destination: AWS S3 destination
CompletedWe need to load data to S3, as CSV for instance.
This is motivated by things such as:
- cost performance
- architectural requirements
-
Official comment
Hello all,
The Fivetran S3 Destination is now in beta! The destination is accessible to all users through the Fivetran UI. -
Hi Guillaume,
Support for S3 as a destination is on our short-term roadmap. I would need more details to understand your use case further. Could I understand how the data landing in S3 and what the consuming services are?
Thanks,
Meera -
Hi Meera,
Support for S3 as a destination is on our short-term roadmap
Would it be possible to have an approximate date ?
I would need more details to understand your use case further. Could I understand how the data landing in S3 and what the consuming services are?
I'd like to load data to S3 as CSV or JSON for instance, and use AWS Glue or similar services to build my custom ETL.
Regards
-
Thanks Guillaume. We are targeting early next quarter to release support for S3 as a destination.
The supported formats will be parquet. Is there a reason for CSV and JSON as the selected formats?
-
Hi Meera, thanks for the update.
Parquet is even better than CSV or JSON.
-
Hi,
Is there any update on when this is planned to be released?
Thanks!
Danny -
Any update on the release date for this feature S3 as destination?
-
Is there any update available for this feature? I would appreciate an estimated timeline or some way to be included in the beta? This is something we would like to implement as a default option for all of our Fivetran usage.
Right now, many of our teams are adding extra complexity to our system where we land data from connectors such as Google Ads, Facebook Ads, or other connectors drop it into a Redshift then use secondary processes that pull that data into our S3 data lake for long-term storage. Once there, then we load it into our final warehouse. This direct to S3 feature would help reduce our AWS costs and the complexity of our infrastructure. -
Thank you for reaching out. We are currently working on support for S3 as a destination. The target beta release is Q1 2022. We definitely will update this thread as we get closer to the beta and reach out individually with more details on enrolling as well.
-
Hi Meera! Thanks for the update. If you need any beta testers please don’t hesitate to reach out.
-
Could you keep us in the loop for when S3 destination is launched?
-
Can you also add me in loop when this is launched.
-
Any update on this where we load data to AWS S3?
I know there is AWS Redshift Spectrum which gives a layer on AWS S3 data and Redshift Spectrum creates database and tables from data residing on AWS S3 to use as Redshift tables and also allows to connect to AWS Glue Data Catalog with AWS Redshift. Can it load to AWS S3 so that AWS Redshift Spectrum can leveraged to use Redshift tables as well ?
-
Hello, I'm jumping in to follow this thread! Very excited for this to be released.
-
any update on when this will be released?
-
Same here. When is the release?
-
Any updates on the timelines for making S3 available as a destination?
-
any update?
-
-
Thanks for your interest in S3 as a Fivetran destination. We apologize for the delay in update. We're currently in development and the solution is not widely available yet.
We don't have an exact date for when the private preview will be available to a wider audience at this time. To be honest, S3 as a Destination has presented some unique technical challenges that we are still sorting out. But we're committed to getting it right.
We understand that this destination is important to you, and it's important to us too — that's why we want to make sure it works the way you expect it to work, before we release it.
If you're interested in joining the private preview, reach out to your account manager / SE / SDR. Once we have a more definite release date, you'll be the first to know. Thanks!
-
Bump^
-
Any updates?
-
ETA?
-
Hello Team,
Do we have any update on this requirement (S3 as a destination). This feature is very much important on cost saving on snowflake to load the data as a micro batch into snowflake instead of real time.
Regards
Ganesh
-
Hi Guillaume,
Thank you for your continued interest in Fivetran's S3 Destination. I am very excited to announce that our beta release for S3 as a Destination is planned for mid-March. Stay tuned for announcements about our exact go-live date!
Best,Coral Trivedi
-
Hi,
are you going to add the option to compressed files (gzip or any other option) ?
Thanks,
Itzik paz, brightdata -
Please add me to the list for testing out the Beta as well
Please sign in to leave a comment.
Comments
27 comments