Connector Improvement: Support for Snowplow IGLU Custom Schemas
Currently on a snowplow custom event Fivetran expects any custom schema definitions to be a hardcoded URL. Most snow plow infrastructure support a protocol called IGLU (https://docs.snowplow.io/docs/pipeline-components-and-applications/iglu/common-architecture/schema-resolution/) for resolving custom schemas. This lack of option of using the IGLU schema resolution will cause issues with our migration to fivetran.
* We are currently splitting our traffic to the old and new as part of our migration strategy.
* Currently we have an in place mechanism that does the translation.
* After migrating to fivetran we will need to reverse this translation.
* Since the hardcoded URL for resolution is non-standard, we will have to update all of our clients.
* Particularly mobile clients are not always updated.
* Since we will have multiple clients, we will need to maintain, and pay for Fivetran and our current infrastructure for an extended amount of time (probably around 3 months). This double implementation will cost a significant amount of money, and likely consume ~1 years worth of return on the migration.
Ideally we can continue to use the IGLU protocol, and not update our clients. This could be an option on the connector setup. This would allow for a seamless transition to Fivetran with our snowplow volume, which is substantial.
Please sign in to leave a comment.
Comments
0 comments