Connector Improvement: Salesforce Event Monitoring / Files Management
Answered
I would like to pull Event Monitoring data from Salesforce into Snowflake. Each hour a record per log type is created in the EventLogFile object, with an attached CSV of the actual data.
Unfortunately this isn't supported due to the base64 limit: https://fivetran.com/docs/connectors/applications/salesforce#unsupporteddatatypesandobjecttypes
It would be great to have a way to accomplish this integration without involving another system.
-
Official comment
Hi John,
Appreciate you reaching out about this. We would love to find a way for you to accomplish this without involving another system.
Ultimately, what will you do with these files? Will they be accessed programmatically? Parsed to look for particular information?
We have a few options to explore: either syncing the files as is to an object store in your destination, or, since they are structured files, looking for a way to parse them out into tables for you.
More about exactly how you plan to access and use them would be helpful in making this decision.
Best,
Sadie
-
Hi Sadie,
Ideally we'd store the contents of each of the log types as a table in Snowflake. We could then build queries around this data or restructure more robustly via our data pipelines. Event Monitoring data gives visibility into what's happening on the Salesforce platform (apex errors, api usage, etc.).
We exploring dedicated observability tools (e.g. DataDog) as there are OOTB connectors that are more purpose built. There are likely advantages to having this data more broadly available within Snowflake as well (e.g. we can then bump error logs against actual data in one place, larger joins are easier, etc.), which is why we are exploring both paths.If you have any other Q's feel free to ask!
Please sign in to leave a comment.
Comments
2 comments