Connector Improvement: Capture deletes within more Lever tables and Enable resyncs at the Table level
AnsweredCurrently, Lever only captures deletes within a limited amount of tables. When data is misaligned in tables where deletes aren't captured, the only option is to resync at the connector level. This is an issue because resyncs at the connector incurs significant MAR and time.
It would be very useful to capture deletes within more tables. For our case, the application table is our main concern.
It would also be useful to allow resyncs at the table level where capturing deletes is not possible due to API restrictions.
-
Official comment
Hi Annelies -
When we can't capture deletes directly from the source, we do re-imports on those tables. Are you seeing issues where even the re-imports aren't reflecting the deleted records? I would be curious to understand a bit more what's going on here.
Table level resyncs can sometimes be tricky where a lot of tables stem from a single API endpoint.Best,
Erin
-
Hi Erin,
Thanks for your reply. What does a re-import mean? Where can I see the frequency of re-imports on specific tables? Is there a certain indicator of this within the logs?
It has been about a month since posting the issue and the data that has been deleted in the Lever source is still showing in our schema without an indication that it was deleted in the source.
Best,
- Annelies -
Hi Erin,
Were you able to see my previous reply? We are still experiencing the issue.
Best,
- Annelies -
Hi Annelies -
By reimport, we mean syncing the entire table rather than a specific time series. We do this when either A) there is no field do filter on in order to have a cursor for incremental capture and/or B) there is no way to capture deleted data incrementally (no webhooks, or event api for example). In order to guarantee data integrity we reimport the entire table either every sync or in a particular cadence in order to capture the deleted data.
It sounds like there is an issue on your connector though and I would ask you to open a support ticket as we should be reimporting that table and thus marking old records as deleted.
Please sign in to leave a comment.
Comments
4 comments