Question
I am experiencing issues with a cloud function but the logs provided by Fivetran do not indicate where the root cause lies. How can I see more detailed logs?
Fivetran Recommendation:
Please make sure that you are printing out errors throughout your function, specifically at points where you receive data in the response or transform that data in some way (such as parsing or deserializing JSON)
Context
When Fivetran interacts with a cloud function, the platform is simply making a request to invoke the function (in the form of a request object) and waiting for a response back (in the form of a response object).
With this context, you will experience one of three groups of errors:
- 4xx errors
- 5xx errors
- Non-4xx or 5xx errors
These errors indicate different root causes:
- 4xx errors
- A permissions error
- An issue with the configured JSON in the ‘secrets’ object
- Non-4xx or 5xx errors
- An issue with Fivetran
The steps to rectify these errors are:
- 4xx errors
- Work through and fix the permissions in the cloud platform
- Work through and fix the configuration of the values in the Fivetran setup form
- Non-4xx or 5xx errors
- Raise a ticket with the Fivetran Support team
However, in the case of a 5xx error, the rectification steps are not obvious; Fivetran has successfully invoked the function, but it hasn’t been completed successfully for an unknown reason because the error took place within the run of the function.
In this case, it would be up to you to ensure that you have added logging throughout your function. The recommendation is then:
- Run your function within the cloud environment
- Use your logging to identify and fix the line(s) of code
- Rerun the cloud function connector
Considerations
Should your cloud function be incurring a 5xx error, this will stop the sync of data until the function is fixed.
You can fix and redeploy the function within your cloud platform and do not need to pause the Fivetran connector. If you have built idempotency successfully into your function, Fivetran will pick up the sync of data from where it was left off before the connector began erroring.