Question
How should I best utilize the state
object in a cloud function?
Environment
Cloud function connectors.
Answer
- Ensure your function is always expecting an empty ‘state’ object on first invocation of the function, so can handle scenarios where the ‘state’ is null
- Ensure your function iterates the value returned from the ‘state’ object on every subsequent invocation of the function where the ‘state’ object is not null
Example pseudo-code for this scenario:
// Request
if (req.body.state.value = null) {
hostname = endpoint.com
Path = /giveMeEverything
}
else {
hostname = endpoint.com
Path = /value
}
// Response
state: {
value: value == null ? req.state.value : value
}
Context:
The "state" object is what your function will use to maintain cursoring,
For example:
- On the initial run of a function, the ‘state’ will be passed as an empty string to the function:
{ }
- If your cursor is keyed on a date and the last date you received in the response from the source system was 01/01/21, you would pass the following back to Fivetran:
{"state":01/01/21}
- On the next run, this will be passed into the function as part of that initial invocation from Fivetran:
{"state":01/01/21}
- Given you want to run from 02/01/21, you will then have logic in your function that will allow for that:
{"state":(01/01/21 + 1)}
Considerations
Whilst you do not have to utilise the ‘state’ object, Fivetran does require a JSON string passed back including that object.