Provide users with more flexibility in the destination schema names
AnsweredThe destination schema name can't be configured by the users. For instance, for the Postgres connector, we have to choose a prefix (that is unique among ALL connectors). Then, the schema in which the data is located is used in the destination schema. For example, if I use my_prefix as the connector's prefix and my data is in the default public schema of my database, the destination schema will be: my_prefix_public. Having "_public" appended to my schema in my destination warehouse is not meaningful. I would much prefer to be able to assign a destination schema name to a source schema name.
A similar problem arises with the MySQL connector. If we have different databases (i.e., different connectors), we can't choose the same prefix. Choosing the same prefix would allow us to have consistency in schema names in the destination. Allowing users to use the same prefix across connectors, or even better, allowing users to choose their own destination schema name would provide users with much more flexibility in terms of schema naming convention.
-
Official comment
Hi Gaëtan - we have a strong philosophy of "one simple default choice" with minimal configuration. Our strategy for avoiding schema & table name collisions in the data warehouse is to require the unique prefix. In practice, this simple rule has been incredibly effective - I can't recall a customer ever having a naming collision.
Our recommendation is a "Enterprise Data Warehouse" architecture, with Fivetran delivering the Raw Schemas, and your users being exposed a transformed set of Schemas. In your case, that transformation is a simple set of VIEWs that organize the tables into the schema name you want. -
Hi Fraser,
Thanks for your reply. I understand your reasoning. The "EDWH" with (materialized) views are what I ended-up implementing.
Regards
-
I see the logic above, but 100% disagree with not letting the customer do what they want. I am willing to risk collision at my own peril!!!!!
Please consider reopening.
Please sign in to leave a comment.
Comments
3 comments