Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: Ability to rewind Capture for PostgreSQL on v6.2.x running on pgoutput APIs

Answered
LT Tan User
The reason why rewind is required for one of our customer is because it is quite impossible to do a repair by date for one of their largest table in the channel.
  • This table has 2 billion records.
  • The unique key for this table is made up by 5 columns (no datetime)
  • The date time column in this table has no index (an index of a datetime column will take up at least 24 GB) and may affect the OLTP performance of this source table
  • Everyday there will be about 50 million records being added to this table but across different account numbers
  • If there is any issue capturing, it will be almost impossible to know which account number to flag for repairs and it could be a million different account numbers
  • A full refresh of this table may take more than a day and will affect the target operational reporting (billing) and slowed down the source considerably.
 
We sincerely hope Engineering team to consider building a rewind for pgoutput in v6.2.x (e.g. using LSN or datetime)

Please sign in to leave a comment.

Comments

1 comment

Date Votes
  • Official comment
    Valentin Kulichenko Fivetranner
    Hi LT Tan,

    Thank you so much for submitting this feature request and sharing your thoughts with us. The ability to rewind is a valuable idea, and I appreciate you bringing it to our attention. I've added your request to our feature improvements backlog for review by our engineering team.

    We'll be sure to keep the community updated on this thread with any progress or changes related to this feature. Thanks again for your feedback and for helping us improve Fivetran!

    Best regards,
    Val Kulichenko