Skip to main content
Documentation
Feature Requests Trust Center Status Community

Community

Connector Improvement: Email Connector functionality Improvement

Answered
Mucker Martin User

Hi Team, 

In using the email connector, we felt that a dynamic built-in functionality that would replace existing records based on an ID or date would be of great assistance. 

In Summary, we are looking to see if there is a way to set up the connector to replace existing data with new files. So for example, say I load an initial file with date from 1/1 to 1/30. Then the 2nd file has data from 1/1 to 2/28. I'd like the 2nd file to overwrite what is already loaded in there. ​I want to be able to either replace the first one or replace part of the data loaded based on the date. Right now it's stacking all the data on top of each other creating duplicates in the table each time a new file is sent. 

Let us know your thoughts. 

Marty

Please sign in to leave a comment.

Comments

3 comments

Date Votes
  • Official comment
    Alison Fivetranner
    • Edited

    Hi Martin,

    Thanks for your feedback here. I am curious what files you are using with the email connector and how you would want to pick which column should be used to identify when to treat a row as new or as a duplicate? Basically how would you want us to identify the primary key to use? One difficulty is that it may be hard for us to infer what the primary key is (ie which columns we should use), and selecting it manually may not be the best user experience.

    I will share this request with the team, although we do not have a plan to implement this improvement at this time.

    Thank you again, 

    Alison,

  • Hadi Hamed User
    • Edited

    Hi, is there any update to this feature request? Ideally we could opt to have the new email attachment replace the entire table each time it comes in, rather than stacking each sync.

  • Eric Cholis User

    I also had a use-case for deduping csv/email data. At the moment, we're using Sigma to export workbooks to csv. In order to dedupe data, we have a transformation to delete all data that doesn't match a recent upload.

    I think a good solution would be have upsert, append, or replace options. Append would keep the current functionality, replace would delete (not truncate) the entire data set before insertion, and upsert would replace based on a user-defined primary key (built from a sample of the initial import.)