Hello,

[follow-up to support ticket 103253]

Context:

- We use snowflake as destination

- All our Kafka messages of all our topics use Avro format for the message value and the message key (company standardisation)

Issue:

Fivetran allows us to sync kafka topics from Confluent cloud to Snowflake but the message key is lost if it is in Avro. Our assumption is that Fivetran only supports string in message key, and tries to parse Avro as String, causing data loss.

Expected behaviour:

We would expect Fivetran to treat the message key the same that it treats the message value, if it is in Avro. Meaning, when ingesting kafka messages with Key and Value in Avro into snowflake, we expect that the generated snowflake table contains 2 columns (Key and Value) with their nested appropriated content.

At least we would expect Fivetran to dump the Avro binary as it is, without any tempering of data, so we can parse it on snowflake if needed.

Impact for us:

We are migrating our postgresDB (20Mil MAR monthly) to micro-services where kafka will be the new source of truth (expect 50mil message next year).

- Without this feature, we will look into another solution. We would remove 20Mil monthly MAR

- With this feature, we will use Fivetran for all our Kafka->DWH. Expected gain ~30Mil monthly MAR over next year.

Thanks for your consideration. Happy to provide more information / examples if needed.

Kind regards,

Christian.