The data ingested by my Adobe Analytics Data Feed (AADF) connector lands at sporadic times in my warehouse.
- To ensure data is ingested as soon as it is available, configure the sync frequency of the connector to be as short as possible
- To ensure integrity of the data, ensure that the daily file is always configured as a minimum
Our AADF connector uses Adobe's Data Feed functionality to ingest hit data from the cloud storage bucket you have configured for files to land in. Data Feed data is written based on the schedule you apply:
- Hourly files are dropped as multiple files on an hourly basis
- Daily files are dropped as multiples files on a daily basis
However, if you configure both hourly and daily files then Fivetran will also utilise the daily file to ensure data integrity by the end of the day.
- The AADF connector uses a composite Primary Key, so any long-running sessions that might span over multiple files are always reconciled
- All files - whether they’re hourly or daily - have a specific naming convention with a date and timestamp that Fivetran can cursor on, ensuring there is never a conflict in the data being ingested