Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: Send alert when S3 file doesn't exist during sync

Carrie Wang User

After setting up the S3 connector, if the file is expired/deleted in S3, users should get notifications/alerts that 'the file could not be found' or 'the data is not up to date'. 

Now if the file could not be found during setup, it will show an error message, but nothing happens when the same scenario occurs later down the path. 

Including this in the alerts during sync is extremely import for making sure the data pipeline is actually working. 

Please sign in to leave a comment.

Comments

2 comments

Date Votes
  • Alison Fivetranner

    Hi Carrie,

    Thank you for sharing this feature request. Our system is looking to sync all new data frequently if your idea that we alert if we find no new data? How would we know when new data is expected versus a normal lull between new data drops?

    I'd love to hear more about your specific use cases and painpoints to figure out a good way to make our connector even better.

    Best regards

    Alison

  • Carrie Wang User

    Hi Alison,

    Thank you for following up on this. 

    Our use case is that we have scheduled to extract records into S3 every 2 weeks. If the extraction goes well, we would have updated files in S3. Old files would go expired after 14 days. So if the connector can't locate the file specified in the 'pattern' setting, it means something goes wrong in the extraction. Basically we don't expect the connector to identify if there's any new records, we only need it to alert when the file is missing, which is what it does during setup tests. 

    Hope this makes sense. 

    Kind regards,

    Carrie