Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: Functionality to assess time travel eligibility only for enabled tables

Completed
Dileep Reddy User

I would like to raise a concern regarding the behaviour observed during data replication from Databricks connector.

We have set up the connector with a dedicated warehouse and have explicitly scoped it to replicate only specific tables within a particular schema. However, during replication window, when inspecting the query activity on the associated warehouse, we consistently observe the Fivetran is attempting to Alter tables in the other schemas- Specifically to enable Change Data Feed (CDF).
This unexpected behaviour is resulting in a significant increase in the replication time and warehouse resource consumption, even though those other schemas/tables were not selected as part of replication.
Which has been taken to your support in #272843 ticket, As this not the ideal behaviour for our use case. They suggested to raise this feature request to get this sorted. please get back to us with an ETA.

Regards,
Dileep

Please sign in to leave a comment.

Comments

2 comments

Date Votes
  • Official comment
    Egidio Terra Fivetranner

    Hi everyone,


    The feature enabling time travel eligibility assessment exclusively for enabled tables in the Databricks connector is now live and generally available. This enhancement ensures that only specified tables within your selected schemas are included in Change Data Feed (CDF) operations, reducing unnecessary warehouse resource consumption and improving replication performance.


    Thank you to everyone who provided feedback and for your patience as we worked to deliver this improvement.


    Best regards

  • Egidio Terra Fivetranner

    Hi Dileep,

    Thank you for sharing this feature request with us! We appreciate you bringing up the need to disable Databricks' Change Data Feed (CDF) for unselected tables.

    I've added your suggestion to our feature improvements backlog for team review. We'll keep this thread updated with any progress or developments regarding this request to ensure the community stays informed.

    Thanks again for your input!

     

    Best regards,