Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Other: Fivetran continuous replication as a stream

Safoora Jalali User

As per my understanding, in hvr, files are captured and copied to the hub machine, and then in the next step integrated to destination. I understand this might cause a lag in replication.

Is it possible to stream the data replication in a continuous fashion instead. So instead of copying files to hub machine first, we stream them directly to a destination like event hub.

Please sign in to leave a comment.

Comments

1 comment

Date Votes
  • Mark Van de Wiel User

    Safoora,

    HVR is fundamentally designed with a separate capture and integrate process. Queuing takes place on the hub for recovery reasons, and to avoid downstream slow downs from impacting change data capture speed.

    This concept - originally designed to support change data capture for databases - was extended to support files. File replication is a feature in HVR, but arguably not its sweet spot (database replication is).

    We have no plans to build a competitor to Kafka or one of the other stream replication solutions that are out there (e.g. Kinesis, Pub Sub, Eventhub).

    Hope this makes sense to you.

    Thank you,
    Mark.