Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: Enhancing Kafka Connector: Adding Custom Topic Specification Feature

Answered
James Huang User

The current Kafka connector is limited to a predetermined topic, which restricts its applicability. To enhance flexibility and accommodate diverse use cases, it's essential to introduce a feature that allows users to specify different topics. This addition would greatly expand the connector's functionality and utility.

Please add the option to let us specify topic

Please sign in to leave a comment.

Comments

3 comments

Date Votes
  • Official comment
    Alison Fivetranner

    Hi James,

    Thank you for trying out our Apache Kafka connector. I'm interested to hear if you completed the setup process and explored the schema tab for the completed connector?

    We have tried to optimize our setup flow to be simple so we just ask for general access and present all the topics we find for de/selection on the schema tab.

    Can you share if this approach didn't work for you because we didn't describe it clearly in our docs or because there is a reason you cannot setup the more general access? 

    Thank you

    Alison

  • James Huang User

    Hi, 

    It doesn't seem you are familiar with the service Fivetran is offering.
    When setting up the Kafka connector, there's no schema tab to explore...

    The issue with Kafka connector is that there's nowhere for user to configure the topic, which assumes user topic is on dedicated cluster (i.e. Fivetran is assuming that there's ONLY one topic in the cluster).

    Also, you're right the document is unhelpful.

    Therefore, could you look into this issue on how to sepecify the topic again please?

     

  • Parmeet Kohli Fivetranner

    Hi James, 

    The Apache Kafka connector does allow multiple topics. See https://fivetran.com/docs/events/apache-kafka#schemainformation for more detail. The schema tab can be accessed after you "Save & Test" the setup form and allows you to choose which topics to sync.

    Thank you for pointing out that the setup documentation can be more helpful in this case. We will review and update it shortly.

    Best,
    Parmeet