Connector Improvement: SAP ERP on HANA Connector - Granular Control Over Data Pipeline Scheduling and Object-Level Sync
Not plannedDear Fivetran Team,
I am writing to propose a new feature that would significantly enhance the flexibility and efficiency of data pipelines within Fivetran when using SAP ERP on HANA based connector.
Current Limitation:
At present, we are unable to create custom schedules for different objects within a single pipeline or pause ingestion for specific objects. This limitation has led us to consider duplicating the connector, which is not an ideal solution in terms of resource usage and potential cost implications.
Proposed Feature:
- Option to Group tables in a connector based on user selection
- Set custom sync schedules for individual objects or groups of objects within a single data pipeline.
- Pause and resume ingestion for specific objects or groups of objects without affecting the entire pipeline.
Use Case:
In our current scenario, we need to:
- Run our main SaaS data pipeline on a regular schedule.
- Extract data from Compatibility Views on an ad-hoc basis (weekly or fortnightly).
- Have the flexibility to pause ingestion for certain objects when not needed, without affecting the entire pipeline.
Benefits:
- Improved Resource Utilization: By allowing granular control, we can optimize our data syncs based on the actual needs of different data objects.
- Cost Efficiency: This feature could potentially reduce unnecessary data syncs, leading to cost savings under the new MAR pricing model.
- Enhanced Flexibility: It would allow us to adapt our data ingestion strategies more dynamically to changing business needs.
- Simplified Management: We could manage all related data objects within a single pipeline, reducing complexity and potential for errors.
We believe this feature would not only benefit our specific use case but would also be valuable to many other Fivetran customers who require more granular control over their data pipelines.
Thank you for your consideration.
Best regards,
Himanshu Jain
-
Official comment
For now, this functionality is not on the roadmap, but we’re actively tracking interest via votes and comments. Please continue to share your use cases. Strong demand enables us to prioritize features that deliver the greatest value to our customer community.
-
Hi Himanshu,
Thank you for the great suggestion. I agree that this would be a nice addition to the functionality that Fivetran offers. We will evaluate this internally and see how we can incorporate it to enhance our services. Please note that this scheduling is a capability of Fivetran and is not solely related to the SAP ERP connector; therefore, the impact of a change will be greater.
Best regards,
Edwin
Please sign in to leave a comment.
Comments
2 comments