Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: Column header with NULL values are not created during data sync.

Answered
Frederick Zacarias User
  • Edited

Hi, 

We do data sync through S3 connector, the source of data is from file (Tab delimited format).  File contains a header, one of the header contains null data, upon data synch those header with purely null values are not created on table.

Example: 

Col1 Col2 Col3 Col4

-----------------------------------------------------------

A      B               C

-----------------------------------------------------------

after initial data synch we expect the the following tables 

Col1 Col2 Col3 Col4 with A B  NULL C values., however only  Col1 Col2 and Col4 was created.

Col3 was omitted due to null values.

How can fivetran include those header (columns) with null values?

 

Thanks.

 

 

 

 

 

Please sign in to leave a comment.

Comments

2 comments

Date Votes
  • Official comment
    Kevin Kim Fivetranner

    Hi Frederick,

    Thank you for the question! The headers are a key part of how Fivetran determines types for columns.

    Is the NULL value required to remain in the original file? It is difficult for our product to determine type if there are NULL values in the header row, so we currently skip those columns.

  • Frederick Zacarias User

    Hi Kevin, 

    Thank you for the response!

    For initial data synch NULL values are required, but sooner there will be a right  values for those null.

    Al though there's an alternative solution for that which is to create first the table and do data synch, but it will be better of Fivetran can automatically include those header with Null values.