Connector Improvement: Enhanced Schema Change Alerting for Fivetran Connectors
We recently encountered an issue where tables became unselectable due to an outdated schema from Fivetran's side. This impacted our reporting and overall reporting for some of the most business critical metrics. Please refer to Case 229013.
To prevent similar incidents in the future, we strongly propose implementing the following enhancements to Fivetran's alerting system:
1. Schema Change Detection:
- Implement automatic detection of schema changes, including additions or removals of tables - both by users and Fivetran.
- Alert users when the number of tables in a schema changes unexpectedly.
2. Proactive Schema Refresh Notifications:
- Send notifications to users when a schema refresh is recommended, especially after detecting potential inconsistencies.
- Include a one-click option in the notification to perform the schema refresh.
3. Table-Level Sync Monitoring:
- Implement granular alerts for individual tables that haven't synced data within an expected timeframe.
- Allow users to set custom sync frequency thresholds at the table level, independent of the overall connector settings.
- Provide options to exclude specific tables from sync delay alerts to accommodate tables with naturally infrequent updates.
4. Historical Schema Change Logging:
- Maintain schema changes over time, accessible through the Fivetran dashboard
- Enable users to review and audit schema modifications easily.
5. Enhanced Schema Change Logging:
- Maintain a detailed log of schema changes over time, accessible through the Fivetran dashboard.
- Include logs for permission-related issues that prevent table syncs.
- Log events when a table sync is requested but denied due to insufficient permissions.
- Provide clear error messages in the logs, detailing which permissions are missing.
- Enable users to easily review and audit schema modifications and permission issues.
Please sign in to leave a comment.
Comments
0 comments