Connector Improvement: Honor Default values on updates
Answered
When I add a new field to the database table that is not coming in for the source and I give it a default value (current_date, calculation, identity) when an update occurs it is set to NULL.
The merge statement that is build by five-tran matches column names from the database to the source API and any that are not supplied are set to NULL.
In Snowflake for instance you are using the SHOW COLUMNS command to grab the database column list. this is already returning the default and autoincrement values which could identify the columns to ignore if a value is not supplied.
-
Official comment
Hi Darin,
Thanks for your comment in our portal, apologies for the delay in response!
To clarify this, are you saying when a new column is added to a table (via a DDL statement), you would like Fivetran to populate the entire row with the default values as provided by the source?
If so, this is certainly something we are working on to enhance our schema migration workflows. Could you share which source you are using in this instance?
-
Hi Kevin,
I appreciate you following up.
Basically, I am wanting the fivetran connector to honor any defaults or calculations that might be setup on the destination table if a column value is not supplied.
If null is supplied then yes reset the value but if nothing is supplied then you should let the destination table schema handle the field.
Please sign in to leave a comment.
Comments
2 comments