Connector Improvement: Google Cloud Function: Allow users to explicitly define schema
It would be good to have the ability to explicitly define the schema for the fields in the response object data.
As defined in the BigQuery documentation, it would be good to be able to provide a JSON schema object as part of the response: https://cloud.google.com/bigquery/docs/schemas#specifying_a_json_schema_file
Currently, the response format already has a schema object: https://fivetran.com/docs/functions#responseformat
However, this is only to define the primary keys. Ideally we could also pass a JSON object which contains the BigQuery schema, i.e. description, name, type, and mode. If this schema exists for a field in the response then BigQuery should use it instead of trying to auto-detect the correct schema.
In addition, this would also allow us to define nested JSON responses in fields - which currently is not possible. Instead, these just get loaded as string representations which have to be processed in the destination.
-
Official comment
Hi Fabio,
- thanks for submitting this request! We don't have this planned at this time. We will continue to collect customer demand to justify the commitment to support & maintain a high-quality Fivetran function behavior.
Can you share a little bit more detail around the use cases that would be unlocked by the nested JSON structure you're describing wanting?
Every upvote on this request increases the case to build it.
Thank you,Alison
Please sign in to leave a comment.
Comments
1 comment