Question

Why are all my rows being counted as MAR?

Environment

All file connectors (except for Google Sheets).

Answer

If a file is modified in any way, it will cause the complete file to be resynced. This causes all the rows to be counted towards MAR if the file is modified in the new billing cycle.

Cause

Modifying files in the S3 connector causes undesirable outcomes. As per our docs: Updating Data,

If you modify a file, we update the table in the destination to match the contents of the file. Adding rows to a CSV that has already been synced will change the last_modified date, which will then trigger a re-sync of the contents of that file.

On the new billing cycle, all rows that are modified on this day (deleted/inserted/updated) are charged. These rows can then be modified for the rest of the month without in-curing any additional charges at all. However, they will keep getting counted on the new billing cycle if they are modified on that day.

Information about file connectors and MAR can be found at Consumption-Based Pricing and File Connectors.