Other: Increasing the limit for string/binary data
CompletedWe encountered multiple encoded_binary_string_too_long issues during data replication from PostgreSQL to BigQuery. Please refer to cases #98366 and #104205 for details.
According to Fivetran Engineering, BigQuery doesn't have any limit on the length of STRING/BINARY, the limit imposed is on the size of the entire row which should not exceed 100MB. Whereas Fivetran limits the length to 8MB as a safety check so the row size doesn't exceed 100 MB. Hence the root cause is the limitation in Fivetran code.
This is an urgent feature request as the issue impacts our production data accuracy at BigQuery. Please advise when this limitation in Fivetran code can be addressed.
Please sign in to leave a comment.
Comments
0 comments