Dashboard Improvement: Schema Browser improvementsNot planned
I'd love to have more options in the schema browser. Even basic things like:
- Number of records in the table
- Field Data types (at least the category)
- Date added to replication (if applicable)
I'm sure there's a ton more that would be useful.
Hi Tim- Thanks for the suggestion. Can you help me understand how these pieces of data would help you better within our schema tab? Specifically, what actions are you taking or research are you doing that would be improved by having this information?
Sure Andrew, happy to help. There's a couple motivations. One is helping me find the right data to select. the other is cost control. Here's a couple examples:
I can see what tables my users query in Snowflake, I would like to remove tables from replication if they're not being used, but if a table was just added a few days ago I don't want to disable it for low activity. The date added to replication for example would help me determine if a table should remain in replication or if it's not being used (user would be nice too now that I'm thinking about it more). I'm 100% certain there are other ways to find this information, but it would be a time saver if that information was simply on screen in Fivetran.
Field data types and number of records on the table would help when trying to find the correct table to add to replication. As you know, Fivetran charges based on MAR, so we only replicate tables if there's a need for that data. As our analysis needs grow we often need add tables to replication. If we know the exact table we need it's easy, but that's often not the case. In my experience, most SaaS programs have a default database schema with tables that support all the functionality of the product, even the parts of the product not being used by our organization. Some, such as Salesforce and Netsuite, have a huge number of these tables. As you might imagine, there's also a host of custom objects and tables as well. Table names, while unique, aren't always helpful in identifying if they're being used and if so what. Lets take an example where I need information about "Sprockets" and there are three tables with "Sprocket" in the name. Today, I would have to select all three for replication, query each one in the database, manually drop the two unnecessary tables, and come back to Fivetran to remove them from replication. All those additional steps would be removed if I could have seen that two of the three were empty (or close to it) from the schema page.
Thanks Tim for the detailed explanation on this. I've noted down the value of this type of information onto the schema tab. As we look into future improvements to address these type of needs, I'll be sure to reach out on this thread.
Please sign in to leave a comment.