Connector Improvement: Support imported events from the Mixpanel API
Our particular use case requires periodic imports of older event data, which is being skipped due to the processing time filtering used by the connector.
This is important because in our case, past events are added to Mixpanel with Segment replay, having the data sent to Mixpanel, in order to see content like the website visits with the specifics on the utm source (ex. utm_source = 'google'). Our marketing team uses this information to determine how effective their strategies are.
To do this, Fivetran needs to incorporate a process for Mixpanel events added in the past while still being able to incrementally sync the endpoint for performance reasons.
We previously made a ticket regarding this. After some investigation, Fivetran support and engineering found that events in the past using the Import API have two different timestamp dates, processing timestamp and event timestamp. The variance between these are larger than they expected and causes issues with what is filtered when processing the information.
They recommended making a feature request as a result for a fix, as this would be a pretty big effort. Additionally, could we get an ETA for a resolution on this feature request?
Q&A for additional context:
- What exactly is the issue with two timestamps and how its impacting the import?
The processing and event times are used to check for recent data for incremental syncs. In order to prevent excessive data requests, Fivetran limits the difference between the two to several days, as Mixpanel had initially indicated that events should be processed within a few days of the event (supposed to be 5 days or less), but in actuality is a much longer window than originally expected (Fivetran was seeing processing times appear up to a month later). This caused incremental syncs to fail when checking for recently processed events.
- Are we able to adjust the filtering logic?
This is controlled on Fivetran's end and is not adjustable at this time.
Please sign in to leave a comment.
Comments
0 comments