Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Other: support binary log transaction compression and row_image minimal

Answered
Jason Rosenbaum User

Hello folks, we recently had an issue with MySQL Aurora generated a single binlog transaction that was larger than 1GB, which was larger than the max packet size allowed by MySQL Aurora, thus preventing replication via HVR. To fix it, these options are recommended, but HVR does not support them:

https://dev.mysql.com/doc/refman/8.0/en/binary-log-transaction-compression.html

https://dev.mysql.com/doc/refman/8.4/en/replication-options-binary-log.html#sysvar_binlog_row_image

Please support them!

Please sign in to leave a comment.

Comments

4 comments

Date Votes
  • Official comment
    Mark Van de Wiel User

    Correct Jason if you'd enable this then you'd break replication.

    I was just wondering if you had a feel for how effective the compression is on your data. Some data - depending on the compression algorithm - compresses notoriously poorly. I was just wondering if just supporting the compressed log fragments would allow you to keep going for a while, or if this is only a temporary solution and soon you'll exceed 1GB compressed log fragments.

    Mark.

  • Mark Van de Wiel User

    Hi Jason,

    Do you know how large your binlog transactions get? Do you know what compression ratio you get when you enable compression (at the highest level), and what is the (performance) impact of enabling this?

    Is it a single LOB that you are processing when you exceed the 1GB limit?

    Thanks,
    Mark.

  • Jason Rosenbaum User

    @mark We've seen single transaction logs (ie a log file with just one transaction in it) exceed the max packet size which is 1GB I believe. 

    This is an array stored as text in a LONGTEXT field. 

    In terms of compression, I don't think we can enable it because HVR doesn't support binlog compression yet, so how could we enable it without breaking replication?

  • Jason Rosenbaum User

    good question, they are arrays of numeric IDs, so I'd suspect they'd compress at least pretty well, given the character space is only 0-9