Hubspot Connector Should Sync Deleted Records at the same frequency as changes
AnsweredWhich connector?:
Hubspot
Additional details:
A sync at a frequency other than 1x every 24 hours is useless if deleted records are not synced at the same time. Currently, deleted records are synced only 1x every 24 hours for Hubspot records.
Additionally, a sync missing deleted records is not a sync at all and should not be described as a sync. The missing data corrupts the destination data warehouse and the incorrect 'sync' terminology sets expectations that do not match reality.
For example, we currently bring in Hubspot data hourly in order to show us near realtime pipeline. If a user deletes a million dollar line item to replace it with a different sku on a quote, our pipeline will show an increase of a million dollars because the deleted line item is not recognized as deleted in the hourly sync. That discrepancy is significant, and our only answer is that Fivetran doesn't work well. That is not a good look for us or for Fivetran.
Requested functionality:
Regardless of the sync interval configured by a fivetran admin, every time a sync is performed with Hubspot tables, all deleted records are recognized and synced along with changed records.
-
Official comment
Hi Chris,
Thanks for the detailed feedback, specifically highlighting the pipeline example. It was really helpful in understanding this problem much better.
A bit of context on how we handle deletes today for some of the objects in HubSpot, since HubSpot doesn't have a single delete API:
- Webhooks - HubSpot can take 30+ minutes to notify us of a delete
- Delete APIs - polled once every 24 hours
- Daily re-import - catches deletes for most objects once every 24 hours
- Weekly re-import - for high-volume tables (Contact List Member, Company, List Member) due to API performance constraints
More frequent delete detection does come with a performance tradeoff, especially when HubSpot doesn't have a dedicated way to fetch them, but accurate data at your destination matters - so we're happy to revisit our delete capture approach for specific objects if there's a clear case for it. To do that well, it would help to know which objects are most affected for you.
Best,
Vignesh
Product team -
Hi Chris,
Just a quick update. We saw your support ticket flagging the need for more frequent delete capture on Line Items, thanks for raising it.
Good news: HubSpot webhooks support delete events for Line Items, which means we'll be able to capture deletes on every sync rather than waiting 24 hours. We're currently working on capturing more events from webhook and Line Items is on the roadmap. We've started with deletes and association changes for Contacts, Companies, and Deals (which aren't captured today), with Line Items to follow. We're targeting end of June for the Line Items fix.
We'll keep you posted as this rolls out.
Best,
Vignesh
Product team -
Thank you Vignesh, Line items followed by Deals would be the biggest requests that would impact anyone trying to bring pipeline into a data warehouse. Please call me out if I can offer any contributions to this feature enhancement, I'm happy to help.
Please sign in to leave a comment.
Comments
3 comments