Question
Why is the replication of PostgreSQL to Google BiqQuery is taking a long time to load to the destination regardless of the dataset size?
Environment
- Connector: Google Cloud PostgreSQL
- Destination: BigQuery
Answer
To avoid delays, please follow the two guides below:
- Split up the connector into multiple smaller connectors.
- Deselect tables from the sync that are not a priority
Cause
Google Cloud PostgreSQL does not support the logical replication of the WAL. Fivetran, therefore, uses the XMIN sync strategy. This results in Fivetran performing full table scans each sync to identify new or updated records.