MySQL Release Notes

May 2021link

To correct data integrity issues, we will now trigger a re-sync for a table when we detect a reduction in the size of one of its SET columns. Before the re-sync begins, you may notice a SET type change detected error in your logs.

When we detect a reduction in the size of a table’s ENUM columns, we now create an ENUM type change detected error in your logs before we trigger a re-sync for that table. Previously, we added FIVETRAN_ERROR_ENUM_DELETED values to your ENUM columns before we triggered a re-sync.

March 2021link

Data syncs no longer fail without explanation when we encounter prohibited DATETIME values (for example, 29 Feb 2021). How we respond to prohibited values varies by sync type:

• Initial syncs: The sync fails when it encounters a prohibited value. We create a Task in your Fivetran dashboard with instructions on how to resolve the issue.

• Incremental updates: The sync does not fail when it encounters a prohibited value. We process the prohibited value and create a Warning in your Fivetran dashboard with instructions on how to resolve the issue. The way we process the prohibited value depends on your sync mode:

  • Live mode: We ignore the prohibited value.
  • Legacy mode: We ignore the prohibited value.
  • History mode: If the prohibited value is a primary key, we ignore it. If the prohibited value is not a primary key, we change it to NULL.

Learn more about Tasks and Warnings in our Alerts documentation.

January 2021link

We have fixed a bug where TIME values were sometimes synced to the destination incorrectly. Previously, we added a .0 to values with no decimals and dropped the hundredths place for values that did. We now sync TIME values to the destination as they appear in the source:

If any values were incorrectly synced prior to this change, you will need to initiate a historical re-sync or table re-sync to correct them.

April 2020link

We have increased the fractional second precision of our binary log parser from millisecond to microsecond for DATETIME, TIMESTAMP, and TIME types. To correct existing values of these types in your destination, trigger a table re-sync or a historical re-sync in the connector dashboard.

We now add a fivetran_synced column while syncing empty tables and columns from the source.

We have added Magento MySQL and Magento MySQL RDS to the list of connectors that can be created via our REST API.

March 2020link

We have fixed a problem where unsigned BIGINT values larger than 9223372036854775807 were sometimes parsed incorrectly during incremental updates. If any values were incorrectly synced prior to this change, you will need to initiate a re-sync to correct them.

We now replicate empty tables in a MySQL source database as empty tables in the destination.

To correct data integrity issues, we will now trigger a re-sync for a table when we detect a reduction in the size of one of its ENUM columns. Before the re-sync begins, you may notice FIVETRAN_ERROR_ENUM_DELETED values in these columns.

October 2019link

• We are now rolling out support for BLOB, TINYBLOB, MEDIUMBLOB, LONGBLOB, and YEAR data types in version 4 of our MySQL connector.

• We are now rolling out the ability to sync MySQL DATETIME columns to the destination as TIMESTAMP_NTZ columns for version 4 of our MySQL connector.

• We have a fixed a bug that caused zeroed DATE and DATETIME values to sync incorrectly. MySQL allows 0000-00-00 and 0000-00-00 00:00:00 as values for DATE and DATETIME primary key columns respectively. Previously, we converted these values to 1970-01-01 and 1970-01-01 00:00:00. Therefore, zeroed times and the real values of 1970-01-01 and 1970-01-01 00:00:00 were mapped to the same value in the destination. Fivetran now converts these values as follows:

  • DATE: from 1970-01-01 to -999999999-01-01
  • DATETIME: from 1970-01-01 00:00:00 to -999999999-01-01 00:00:00

September 2019link

• If you have a logging service connected to your Fivetran account, the import progress of your tables in the event logs will be visible in your logs as an import_progress event. Table names will be in either the COMPLETE or IN_PROGRESS state, depending on their sync status. Tables that are not reported in the event have not started their import.

• We have begun rolling out support for the following spatial types in version 4 of our MySQL connector:

  • GEOMETRY
  • GEOMETRYCOLLECTION
  • LINESTRING
  • MULTILINESTRING
  • POINT
  • MULTIPOINT
  • POLYGON
  • MULTIPOLYGON

August 2019link

We have changed the MySQL version requirements for Fivetran to store data as JSON in a MySQL warehouse and create tables with primary keys.

We now convert JSON to text in versions below 5.7.8.

We now create primary keys in either of the following scenarios:

• MySQL version earlier than 5.7.7:

  • If the total number of primary keys provided by the source is less than 16, and the sum of the byte length of String primary keys is less than 191.

• MySQL version 5.7.7 and later:

  • If the total number of primary keys provided by the source is less than 16, and the sum of the byte length of String primary keys is less than 768.

Fivetran has released version 4 of our MySQL connector, which offers improved sync reliability. We will add a number of upgrades to this version in the coming months:

• We will sync DATETIME values as TIMEZONE_NTZ values in your destination.

• We will support parsing for TRUNCATE DDL events.

• We will support the following new data types:

  • BINARY
  • VARBINARY
  • YEAR
  • BLOB
  • SMALLBLOB
  • MEDIUMBLOB
  • LONGBLOB
  • GEOMETRY
  • GEOMETRYCOLLECTION
  • LINESTRING
  • MULTILINESTRING
  • POINT
  • MULTIPOINT
  • POLYGON
  • MULTIPOLYGON

This month, we have begun rolling out support for BINARY and VARBINARY types.

For more information about data types, see our MySQL documentation.

June 2019link

We have added a test to the MySQL setup forms which checks if you are using RDS. We added the test because if you select the wrong variant in the setup form, your connector may not work properly.

The test fails when there is a discrepancy between your selection and what Fivetran detected. If you are not confident in your selection, you may go back to the Add Connector view and select the connector type that matches your MySQL database variant.

Fivetran no longer supports invalid Date, Datetime and Timestamp values. For example, 2019-00-01 00:00:00 is invalid. This change immediately affects all new MySQL connectors, and we will roll it out gradually to existing connectors.

The supported ranges of Date, Datetime and Timestamp in MySQL are:

We will load these invalid values as NULL into the destination and display a warning on your Fivetran dashboard. The warning will indicate the schemas, tables, and columns which contain NULL values.

January 2019link

Fivetran will create primary keys for a MySQL warehouse in either of the following scenarios:

• MySQL version earlier than 5.7 or MariaDB version earlier than 10.2: If the source provides fewer than 16 primary keys and the sum of the byte length of string primary keys is less than 191.

• MySQL version 5.7 and later or MariaDB version 10.2 and later: If the source provides fewer than 16 primary keys and the sum of the byte length of string primary keys is less than 768.

November 2018link

We have identified and fixed an extremely rare bug that can cause data to be missed during binary log replication. The bug meant that, in some cases, our system believed that all data had been successfully synced, when the binary log replication process had actually stopped prematurely.

October 2018link

For the following warehouses, Fivetran now drops the primary_key field from existing tables in your warehouse if there is a primary_key coming in from your data source:

• MySQL
• PostgreSQL
• SQL Server
• Redshift
• Periscope
• Panoply

September 2018link

The MySQL connector is now able to handle unescaped control characters in textual data as if they had been properly escaped.

The MySQL warehouse now uses the load data infile strategy to write data. It requires the load_infile system variable to be set to on. For additional details, refer to our MySQL warehouse documentation.

August 2018link

String values will now be created as LONGTEXT columns in the MySQL warehouse instead of VARCHAR fields if they are not the primary key. We made this change to better handle MySQL’s maximum row size limit.

July 2018link

You can now customize the replica ID used to access your database’s binary logs. By default, the replica ID is a random integer greater than 1000.

May 2018link

We now fully support the SET data type and consistently sync it as a JSON array. Previously, we synced SET as a string during the initial import and as an integer when processed from the binary logs. If any values were incorrectly synced prior to this change, you will need to initiate a re-sync to correct them.

April 2018link

If your data warehouse uses a MySQL version earlier than 5.7, you can’t use the timestamp column as a primary key because MySQL incorrectly handles the decimal part of the timestamp. We now detect this scenario, skip tables with this column during our sync, and warn you to upgrade your data warehouse MySQL version. Note that this limitation only applies to MySQL destinations, not MySQL sources.