We have been using the Fivetran Lambda connector to ingest non-Fivetran supported sources. At the end of each ingestion iteration, the lambda code will return a customised state variable back to Fivetran to store the states.
In some scenarios the state variable gets stuck with incorrect values which are returned to Fivetran so the next execution Fivetran will send these bad values back to AWS Lambda. In order to recover from the error and having the Fivetran store the correct state, we currently have to:
- Re-sync historical data (may take a long time) Or;
- Modify the lambda code temporarily to correct the state vairable manually
It would be great if we could have a feature to allow user to update/modify the state variable at the frontend. After saving the changes in the frontend, Fivetran will return the new values in the next request.