Connector Improvement: Google Sheets data types
AnsweredAbsolutely love the Google Sheets connector, it's been a big help for quick wins in the business. We have found, however, that the automated type guessing is a bit unreliable and in practice all our GSheet tables have a view-layer in front of them to fix the problems before they go into the warehouse.
A nice-to-have improvement would be to be able to choose the data type for each column -- a little like the csv uploader -- as part of the setup. That would avoid extra transformation steps downstream.
-
Official comment
Hi Doug,
Thank you for the question. What types are currently being casted incorrectly? One of our key principles is to minimize configuration as much as possible. The ideal scenario would be for Fivetran to correctly parse and determine type, rather than having users select this information for every single column!
Otherwise, a best practice would be to use transformations after the load phase, as you mentioned.
-
@kevin Dates and timestamps are generally parsed as VARCHAR instead of as timestamp. It would be nice if the dates/timestamps were cast correctly if we can't select data types
-
Hi Kevin Kim, really sorry I missed your message. I just got the notification from Zelda's message which prompted me to come back here.
Likewise for us it's dates and times that aren't in ISO format. GSheets has a habit of re-formatting them for you to local formats DD/MM/YYYY in the UK for example.
There are some other times where the type of number needs changing (float vs int), and occasions where columns are misinterpreted as numbers. But that's less of a headache.
-
Long time since that, but have some similar issues, with long numbers (campaign_id) being cast as float, others as varchar. Would like to be able to make a selection when uploading the table.
Please sign in to leave a comment.
Comments
4 comments