Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Transformations: Need capability of filtering given events/data before putting it in destination

Prashant Pandey User

Currently when the source has a lot of events which we dont want to process in our destination we want to drop these all together before processing and not let it end up in our destination at all and save some ingestion and compute cost. 
We should a capability of listing all the events from the source and select at will which event we want Fivetran to process and send to our destinaton and which event we want to ignore all togehter and not bump up our Fivetran Bill :) 

Please sign in to leave a comment.

Comments

4 comments

Date Votes
  • Anna User

    Hi Prashant Pandey– can you tell me more about the connector tables and events you're looking to filter out here?

  • Prashant Pandey User

    Hey @..., Here the source is Amplitude and we use Fivetran to dump the data in SF. 
    We do have list of events we need to filter out, its our customers data. 
    At http://toplyne.io/  (b2b startup) -> we allow our customers to connect multiple data sources and use fivetran to dump data to SF. So looking for a generic solution for this. 

  • Anna User

    Got it. We don't have row-level filtering available prior to data load, since one of the principles of ELT is that syncing all relevant data helps you audit your source systems and serve more downstream analytical use cases long-term. Would deselecting certain tables from syncing in your client's Amplitude schemas be a solution here?

  • Kate Wiggin User

    We have a similar need, with Mixpanel data.    We would like to filter out certain events prior to loading.  It would not work to deselect the entire table, we're looking for the ability to add certain filter parameters to a table (events) so that we only load the events that we're interested in.