Destination Improvement: OCI object bucket support for managed data lake
AnsweredIt would be great if the managed data lake could write to OCI object buckets. Really, support for any S3-compatible storage would be ideal, but OCI support specifically would solve my need. Thanks!
-
Official comment
Hi Chris,
This is a valuable idea, we'll evaluate adding support of OCI Object Bucket as a managed data lake destination and get back to you.
To better understand your needs, could you share more about your use case that you plan to use with OCI as a data lake destination?
Thanks,
Shiva -
We're building a data lake to serve as our central data hub - primarily to support AI workloads and model training, but also to power both internal and customer-facing analytics. We chose OCI mainly for cost reasons: while the raw storage costs aren't drastically different from AWS, the egress and compute costs add up quickly.
We're already a Fivetran customer for other projects, but I don't think we have any flexibility on our storage backend for this data set...which unfortunately means Fivetran is likely out of the running for this one. It looks like HVR does support OCI for destination storage, but its feature set obviously doesn't provide the same value as the managed data lake service.
-
As another feature request (happy to start a new topic if preferable) it would also be great if you supported writing Hive catalogs - and Hadoop-style catalogs would also be nice. Hive metastore support is essential for anyone aiming to maintain provider-agnostic flexibility. Hadoop-style catalogs are a nice-to-have just for the simplicity and portability they provide, especially for smaller or simpler use cases where an external service dependency would be overkill.
Please sign in to leave a comment.
Comments
3 comments