Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: Single Table for multiple files within a Google Drive Folder Connector

Completed
Barnett Klane User

Until our team can build proper connectors, exporting CSV to import is a great stepping stone to get the data into our Data Store. The issue is with larger tables, where appending to a CSV every refresh is a painful operation where it'd be easier to add a new file to a folder for the refresh period which then in turn updates a single table in the destination, shared by all files in the folder.

As of today this connector has each file as it's own table in the destination source which isn't how we'd use it. For now we are completing a manual merge prior to upload and hoping our local CPUs can handle it for now. While we can build scripts, hoping to have this operation completable by non-technical users.

Please sign in to leave a comment.

Comments

3 comments

Date Votes
  • Official comment
    Fraser User
    • Edited

    Hi Barnett - the file merging functionality is how our Object Store connectors work, like S3, Azure Blob Storage & Google Cloud Storage. SFTP & FTP also work with this way.  They are designed for the use case where some process is regularly sending files of the same format & we can merge them into a single table.  Could you use one of those to accomplish the workflow you want?

    Fraser

  • Barnett Klane User

    Hey Fraser,

    Unfortunately those options are more difficult for us to spin up internally compared to a Google Drive folder. Eventually we might be able to enable that but for now it's not possible.

  • Annelies Buck User

    Our team would also greatly benefit from this added feature. Company wide, SharePoint folders are used to store multiple files which are consolidated into one table.