Connector Improvement: Option to enable/disable all datasets/tables/columns from schema
AnsweredIn order to manage connectors schema in an automated way, it would be useful to have a way to enable or disable every datasets/tables/columns without explicitly listing them.
With current API "PATCH {{baseUrl}}/v1/connectors/:connectorId/schemas", the fields internally flagged as excludedByUser will not be updated if not explicitly listed.
Thus, there is currently no way to revert manual changes to apply a predefined schema configuration (that does not list every columns of every tables).
Example of usecase :
1. I have the following schema config applied to my connector
{
"schemas": {
"my_schema": {
"enabled": "true",
"tables": {
"my_table {
"enabled": "true"
}
}
}
},
"schema_change_handling": "ALLOW_COLUMNS"
}
2. Someone manually disable some columns (either using the UI or the API)
3. I want to revert changes by deploying my base schema config. Currently, using the PATH API will not enable the disabled columns.
This feature would also be very useful to manage properly schemas using the Terraform provider.
-
Official comment
Hi Victor,
I believe you might be able to solve this by using the schemas endpoint: https://fivetran.com/docs/rest-api/api-reference/connection-schema/pre-create-connection-schema-config
In Terraform, we have pretty extensive schema management available here: https://registry.terraform.io/providers/fivetran/fivetran/latest/docs/resources/connector_schema_configI believe this should solve for what you're looking to do, let me know!
Jimmy
Please sign in to leave a comment.
Comments
1 comment