Connector Improvement: FT with Alation Integration for Metadata
AnsweredHi Team,
We have opened a case with Fivetran #308191 and they suggested to open a feature request for FT with Alation Integration for Metadata.
Please find details we discussed.
- The above API calls return all the connections/destinations in the account, including the
incompleteones. When Alation makes further API calls and tries to fetch details of theseincompleteconnections/destinations, it fails with 404 error. - There is no option to filter the connections/destinations based on the status and by default all the connections/destinations are returned in the response.
- We want to configure Alation to fetch metadata from Fivetran using REST API. Alation is initially making API call to fetch all connections and desertions in the account using below API calls and then tries to fetch details for individual connection/destination
-
We successfully made network connectivity and attempting to fetch Fivetran Lineage data from our data catalog tool (Alation). However, each run shows an error (shown below). After digging in the logs, I saw multiple 404 destination ids which my assumption could’ve caused the pre-flightcheck to fail during each run. I performed the GET calls for connectors and destinations from Postman as a test and attached the json file outputs to this communication.
Error Message:
Operation: Checking that the Fivetran account has destinations, sources, and the Fivetran Platform Connector configured
Message: The provided credentials may lack the required DESTINATION: VIEW or CONNECTOR: VIEW permissions, or there may be no destinations configured in the Fivetran account. Please verify the permissions and destination setup.
Action: Provide credentials that have DESTINATION: VIEW permission for destinations and CONNECTOR: VIEW permission for connectors in Fivetran. Create a destination in Fivetran and configure the sources. Install the Fivetran Platform Connector in any destination and enable it at the account level.
-
Official comment
Hi Rini,
It sounds like the core ask here is to be able to filter on status, is that right? Is there any reason you keep incomplete connectors in your account?
Jimmy
-
Hi Team,
Please share if we have any updates regarding this feature request.
Thanks and Regards,
Rini
-
Hi Jimmy,
Since we are on shared model, so these we will be need for other lines as well
- Filter to capture metadata based on LOB.
- Filter to capture metadata based on env, like dev, prod [ environment specific ]
- And of course, lineage for completed connections not for broken or incomplete flows.
Regards,
Rini
Please sign in to leave a comment.
Comments
3 comments