Hello - we want to use the AWS MSK connector to send our Kafka topic data to Snowflake. We have many union schemas where it's very important to know the type. Fivetran support has let us know that the specific event type indicator from the union is not included in the output, but without this information, our data is incomplete.

For example, we may have the following field in our avro schema:

                {
                    "name": "email_events",
                    "type": [
                        {
                            "type": "record",
                            "fields": [{"name": "id", "type": "long"}],
                            "name": "EmailDelivered",
                        },
                        {
                            "type": "record",
                            "fields": [{"name": "id", "type": "long"}],
                            "name": "EmailDropped",
                        },
                    ],
                },

Currently we get this data in Snowflake

{ "id": 2}

But we want to see this

{

      "EmailDelivered": {

          "id": 2

      }

}

Knowing the union type is critical to our business processing, and aligns with what we see from other Kafka topic processing tools such as RedPanda and open source MSK connectors.