Connector Improvement: Include indexing , sort keys ,distribution keys ,etc on destination tables to improve query/transformation performance
AnsweredCan we have sortkeys / distribution keys on the extracted tables in our destination included during the loading process using Fivetran?
This would greatly improve query performance and aid data transformation activities on the extracted tables.
-
Official comment
Thanks for your post in our feature requests portal!
I looked up your account and it seems you are using the Redshift destination. I read through Redshift docs and unfortunately, it seems you cannot alter a table to use a sort key any time after creating the table. This is quite different from other data warehouses, such as Snowflake which will automatically use an index under the hood and the user does not have to configure a sort key or index.
Today are you re-creating tables to use a sort key, or are you currently using the tables as-is when they are written into your data warehouse?
I've submitted this request to our engineering team and I will be sure to discuss it with them. While I don't have a timeline to share at the moment, we will take this feedback and any others who comment/upvote on this post into consideration!
Please sign in to leave a comment.
Comments
1 comment