Skip to main content


Connector Improvement: Databricks Historical table sync using single merge command

Please sign in to leave a comment.



  • Official comment

    Hi Omar, 

    Thanks for reaching out! We are investigating the use of a single staging table for the Databricks connector where we'd load into the staging table and then use a single MERGE operation for that staging table. These improvements are on our short term roadmap!

    Would you be able to describe your use case further? How are you querying the data? 

    +1 for this feature Omar. Thanks for the response Coral, glad to hear this is on the roadmap. The big issue for us is we have different tables in an upstream SQL source that are committed in single transaction, we dont see this down stream, so if other users are querying this data as "Bronze" data in Delta/DBX for reporting, they may see inconsistent results (for example account balances may not align with aggregate transactions table).