Connector Improvement: CRON-based scheduling for connector runs
Requesting to add ability to schedule connectors on a frequency within a set period, or by explicit times.
Use cases:
We have some APIs that only provide data at certain times, such as business hours M-F. Pulling from those during other times results in 0-record log entries, and we'd prefer to alert on 0-records only during certain periods. It is also a waste of compute to invoke Lambda functions outside these time periods. I.e. "run every 30 minutes from 7am to 7pm M-F"
We have other APIs where we're asked to not query during certain periods (such as business hours) to avoid contributing to API usage limits on the source, but we'd like to do infrequent polling during those times and more frequent polling during the period's we're allowed to query. I.e. "run every 2 hours from 8am to 6pm and every 30 minutes 6pm to 8am".
We have source DBs that we're only allowed to pull from during certain periods. We can do a 24 hour schedule and define a time, but we'd like the flexibility to pull more often and the DBAs prefer explicit schedules to not conflict with other usage. I.e. "Run at exactly 11pm and 6am"
-
Such a basic feature missing in Fivetran
Please sign in to leave a comment.
Comments
1 comment