New Destination: Google Drive Destination
CompletedWe are currently having Fivetran connected to our Databricks platform (as our destination). We are exploring the option to use Fivetran to deliver data from Databricks to a Google Drive (in a form of Google Sheet or CSV file).
-
Official comment
I'm excited to share that Google Drive as a destination is now possible with Fivetran's acquisition of Census! You can learn how to do this in Census's official documentation.
To get started with Census, please visit www.getcensus.com, or reach out to your account manager. Thank you all for your feedback and patience. We truly appreciate your engagement and support. -
Thanks for writing in Jimmy,
We don't have this on our roadmap currently, but I believe you would be able to accomplish this workflow using a dbt transformation job that periodically extracts data and writes to Google Drive.
dbt supports integrated python code, and the script you would need to write might look something like this (I did not write this myself, the below is output from an LLM — you would need to review and fix-up the code):
import csv
import os
import google.auth
from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload
import pyodbc # or databricks-sql-connector
# 1) Connect to Databricks
connection = pyodbc.connect(
'DSN=Databricks', # or the appropriate connection string
autocommit=True
)
cursor = connection.cursor()
# 2) Query your newly refreshed table
cursor.execute("SELECT * FROM your_schema.export_data")
rows = cursor.fetchall()
columns = [desc[0] for desc in cursor.description]
# 3) Write rows to a CSV
csv_filename = "export_data.csv"
with open(csv_filename, "w", newline="") as f:
writer = csv.writer(f)
writer.writerow(columns)
writer.writerows(rows)
# 4) Upload to Google Drive
# Make sure you’ve set up Google API credentials (service account JSON, etc.)
creds, _ = google.auth.default(scopes=["https://www.googleapis.com/auth/drive.file"])
drive_service = build('drive', 'v3', credentials=creds)
file_metadata = {
'name': csv_filename,
'parents': ['<your_drive_folder_id>'] # folder you want in Drive
}
media = MediaFileUpload(csv_filename, mimetype='text/csv')
uploaded_file = drive_service.files().create(
body=file_metadata,
media_body=media,
fields='id'
).execute()
print("Uploaded CSV to Drive - File ID:", uploaded_file.get('id'))
Please sign in to leave a comment.
Comments
2 comments