Connector Improvement: Add empty column Name in snowflake table for sftp connector
We have created an sftp connector, but sometimes the source sftp file does not contain data for a few columns in a record, and we were not able to see empty fields in the destination table.
We need destination table to include empty columns if it does not contains data at source
Sftp connector: svt_d_iclear_24.invoice
Destination: sf_dev_svt
-
Official comment
Hi Venkat,
Thank you for taking the time to provide your feedback. The not syncing of empty columns is a core service of our system and has historically helped customers not end up with confusing and unhelpful extra columns in their Dataware houses (doc link).
I'm interested to understand more about how you would want us to set the datatype for an empty column? In particular how inconvenient it would be if for instance a timestamp column ended up with a STRING type due to us getting the datatype wrong?
I believe many of our customers facing the need to control exact columns send a single file with dummy data in every field to get the initial columns configured - would that work for you?Alison
-
Has this feature been improved now?
I also need the data field, even if currently the data doesn't exist, because eventually that field will start populating data
Please sign in to leave a comment.
Comments
2 comments