Skip to main content

Community

Destination Improvement: Add Support for Daylight Savings Time in Destination Time Zone and Schedules

Please sign in to leave a comment.

Comments

10 comments

  • Official comment

    Hi All - Thanks for the feedback here and sorry for the frustration this has caused. We do want to improve how we're handling daylight savings, so I appreciate you bringing this up. It sounds like there are two cases where our lack of robust support impacts you:

    • 24 hour sync frequencies no longer run at the same time as before (+/- 1 hour)
    • Sync frequencies longer than 1 hour either occur 1 hour earlier or later than the frequency

    Am I correct in thinking that the issue with 24 hour syncs is a bigger problem as it continues to run at a different time than before? For the second example (sync frequencies longer than an hour), it sounds like the issue is about the one sync following DST but then the rest are fine. Is that correct?

    It's actually surprising that this has to be requested as a "feature". Our 25 year old mainframe manages daylight saving, AWS, GCP, and Azure manage daylight saving, all of our ETL/ELT tools manage daylight savings, and then we get to Fivetran that is "modern" and it can't handle daylight savings. Not only does it not handle daylight savings but it completely throws the end user off because the interface says the job is schedule for X time but it actually runs an hour earlier. If the schedule tells you one thing and the actual sync time is at a completely different time that's a bug and not a feature. 

    Someone really missed the boat on this one.

    Being told by support that we could fix the daylight savings issue by scheduling outside of Fivetran and kicking off syncs using their API is a real slap in the face. We use Fivetran because of it's simplicity and the fact we don't have to write code around the solution. If we have to start writing code to fix simple flaws like this, the ROI of this product quickly falls off.

    Bryan Meier I completely agree and had the same conversation with Fivetran support prior to submitting this request.

    Andrew Morse I would consider both scenarios to be the same problem since the outcome of both is data not being available when users or other systems expect it to be. The impact of the 24hr syncs could end up being greater if the downstream processing is also occurring once per day, but both present similar challenges. This problem can then be compounded (like in our case for many key sources) if the source of the data is also based off a batch process. 

    I'm happy to discuss specific use cases if that is helpful at any point. 

    I would agree what Kyle Spotts mentioned. The 1 hour either direction could effect any situation depending on a company's requirements. I could see the least impact being syncs that are running every 15 mins or less but as that time length grows the criticality does as well.

    We are actually seeing a data quality issue related to how our Workday and Coupa connectors ingest datetime columns during a Daylight savings time period.  Once ingested into our target warehouse, the data is off by an hour. When comparing reports from the source system to what is replicated in our Snowflake environment, the datetime columns are off by 1 hour.  This occurred during our initial load and continues to occur with timestamp field that fall within that window(mid-March to early November).  This is a production data quality issue for us and should not be considered a feature request.   

    John Blankenship - I agree that what you're describing is in fact a data quality issue and should be treated as a bug. This request is centered around when a connector runs and how the schedule is impacted by DST. I've gone ahead and talked to our support team and they're reopening the ticket. You can anticipate someone reaching out tomorrow as they dig into the issue around data in the destination being incorrect.

    It also sounds like there is feedback on how the scheduling shifts during DST, I've asked to be included on your next sync with our team so that we can discuss your feedback directly. 

    Kyle Spotts & Bryan Meier - Can you confirm that your issue relates to how the schedule shifts during DST and that it is something you're looking for us to address before the next shift when we fall back? If so, I'll be reaching out to each of your shortly to get time scheduled so that we can discuss your specific needs/use-case.

    Andrew Morse Thanks for checking. I can confirm that getting this addressed prior to the next time change would be great for us and I'm happy to discuss.

    Hi All - Wanted to just share an update that we're still scoping out this project and should be able to share another update in a few weeks. 

    Thanks for the update Andrew. Looking forward to hearing about the solution Fivetran comes up with. Thanks again!

Didn’t find what you need?

Contact support