Connector Improvement: Add optional field-level description retrieval to the Salesforce connector
AnsweredSalesforce supports the retrieval of column metadata using SOQL. This metadata includes the column's description. Once retrieved, these descriptions could then be synced to a Snowflake destination (and perhaps other types) in it's field metadata's comment attribute. Snowflake will then display these comments within its UI. Furthermore, the Snowflake comments can be pulled in using SQL, and used to populate dbt YAML files so the documentation is available for dbt-based documentation, as well.
Here's code to pull field-level descriptions from Salesforce:
SELECT QualifiedApiName, Description
FROM FieldDefinition
WHERE EntityDefinition.QualifiedApiName = '<SF_ObjectName>'
Here's the Snowflake SQL for inserting a comment of a column:
COMMENT ON COLUMN my_table.my_column IS 'now comment4';
I have played around w/ doing this in Python, but doing so requires reverse engineering Fivetran's Salesforce to Snowflake field naming algorithm and then setting up an environment where the Salesforce query and Snowflake update code could be synchronized and invoked on-demand and/or at regular intervals. But then it dawned on me that this basically describes what Fivetran's already doing!
So, how about it? I think that it could be pretty seamlessly integrated into and work just like what Fivetran does when syncing data from Salesforce. Only sync the description if it's changed, allow the first description sync to be MARless and include it in a Historical re-sync.
-
Official comment
Hi Albert,
Sadie from the Product team here. Thanks for sharing this idea, we can definitely see the value in syncing these descriptions over! I've put it in our backlog for now while we collect other metadata-related requests and get a sense of what other customers need. No timeline as of now, but we'll continue to monitor for demand and update you here if that changes.
Best,
Sadie
-
Would also love to see this feature!
-
I think this is an amazing idea, and would be INCREDIBLY useful here at Rocket Software.
-
Amazing idea !
-
A great idea!
-
Upvoted
-
Would love to see this feature as well!
Please sign in to leave a comment.
Comments
7 comments