Connector Improvement: Allow snowplow connector to handle bad data
CompletedOccasionally (maybe once a year for the last few years) a bad event gets passed to our snowplow connector. This breaks Fivetran's processing of all other snowplow events. It's usually just one event out of the 10 of millions you all process for us on a monthly basis. Someone on your side then has to go in and manually delete the event. On average this has taken ~2 weeks to resolve. In the meantime we aren't getting any new snowplow events. There's not any data loss, but the timely delivery of this data is critical for downstream consumers like our internal lead scoring system and A/B tests on our marketing site.
We'd like to see the snowplow connector improved, for instance by dumping bad events into a separate table, or your internal process for removing bad events improved, so that there's minimal downtime from bad events. Thanks
-
Official comment
Hi Chris,
Great news, we have been able to improve how the snowplow connector handles badly formed events and will be rolling these changes out to production in the next few days.
Thank you so much for reporting your experiences and helping us make our Fivetran product the better.
Best - Alison -
Hi Chris,
Thank you for using Fivetran and for submitting this feature improvement. I'm sorry to hear you've had these intermittent problems and that they have taken so long to troubleshoot and resolve. I going to bring your idea to the engineering team and see what we can get onto the roadmap.
Much appreciated
Alison
Please sign in to leave a comment.
Comments
2 comments