Options to Handle Unique Constraint Errors in HVR6 Burst Mode
AnsweredWhich destination?:
Options to Handle Unique Constraint Errors in HVR6 Burst Mode
Additional details:
if a constraint error occurs, we should have an option to temporarily switch back to continuous mode and then return to burst mode afterward.
for additional details please refer - https://support.fivetran.com/hc/en-us/requests/296508
-
Hi Rajasekhar,
We are investigating how HVR can best support this scenario and will provide an update as soon as we have more clarity.
Best regards,
Edwin -
HI Rajasekhar,
From the support ticket, it was not clear how you got duplicates with burst mode. Can you explain?
Edwin
-
Hi Edwin,
Thanks for checking.
To clarify, our concern is not that Burst Mode itself is generating duplicate records. The scenario we are looking at is when a unique constraint violation occurs during Burst Mode, regardless of the underlying reason for the duplicate.
In Continuous Mode, we have more flexibility to investigate and recover from such constraint errors. For Burst Mode, we would like to have a supported recovery mechanism where:
-
If a unique constraint error occurs, HVR can temporarily switch the channel/workflow from Burst Mode to Continuous Mode.
-
The replication can then continue in Continuous Mode so that we can resolve/recover from the constraint issue.
-
Once the issue is resolved and the channel is back to a stable state, we can switch it back to Burst Mode.
The key requirement is therefore the ability to temporarily fall back to Continuous Mode when a constraint error occurs and then safely return to Burst Mode, rather than having the channel remain stopped until manual intervention.
The original support ticket [296508] contains the additional context and the specific scenario we encountered.
Please let me know if you need the exact HVR logs/error details from our environment, and I can provide them.
Best regards,
Rajasekhar -
-
two additional questions:
--> The replication can then continue in Continuous Mode so that we can resolve/recover from the constraint issue.
edwin: How will you know it is correct? How will you solve the underlying issue?
--> Once the issue is resolved and the channel is back to a stable state, we can switch it back to Burst Mode.
edwin: we can switch? Is that a manual intervention in your idea?
-
Hi Edwin,
Thanks for the questions. Let me clarify the scenario based on what we have observed in our environment.
When we encounter the unique constraint error while running in Burst Mode, we have tested switching the channel to Continuous Mode, and the constraint error goes away and replication is able to proceed.
The reason we want this to be automated is because these channels have high data volumes and control latency. Staying in Continuous Mode for an extended period is not practical for us, as it can result in increased latency/backlog.
Our requirement is therefore an automated fallback and recovery mechanism, rather than manual intervention:
-
Channel is running in Burst Mode.
-
HVR detects a unique constraint violation.
-
HVR automatically switches the channel to Continuous Mode.
-
Replication continues in Continuous Mode, allowing the backlog/latency to be reduced and the constraint error to clear based on the behavior we have observed.
-
Once the channel reaches a defined healthy/stable condition, HVR automatically switches the channel back to Burst Mode.
-
The channel continues operating in Burst Mode as normal.
So, to answer your questions:
How will we know it is correct / how will we solve the underlying issue?
From our testing, switching from Burst Mode to Continuous Mode itself resolves the constraint error for this scenario. We are not proposing to manually investigate and correct every occurrence. We would like HVR to use the successful Continuous Mode behavior as the recovery mechanism.Would switching back to Burst Mode be manual?
No. The main requirement is that this should also be automated. We would like HVR to switch back to Burst Mode once the channel meets a configurable healthy condition, such as the constraint error no longer occurring and/or latency/backlog being below a defined threshold.This would allow us to handle these transient constraint errors without requiring manual intervention, while still getting the performance benefits of Burst Mode for our high-volume channels.
Please let us know if HVR can support this through an existing configuration, automation hook, or if this would need to be considered as a product enhancement.
Best regards,
Rajasekhar -
Please sign in to leave a comment.
Comments
5 comments