Skip to main content

Community

Connector Improvement: Being able to de-select SUB TABLES

Please sign in to leave a comment.

Comments

3 comments

    Another specific example of this that we encountered was the product_tag table. We use the top level product table and product_variant of the sub tables, but not product_image, product_tag, or product_option and these drive >90% of the MAR we see for some Shopify customers.

    Given the schema it doesn't even look like they should be required for Product.

    I echo this request. The absence of this feature causes your connectors to have exhaust a lot of unnecessary MAR.

    Relatedly, I suspect that a "free" resync causes downstream child tables to sync and count against paid MAR. Whatever Fivetran does to implement this separation of child-tables must also take the resync scenario into account -- i.e. if a resync occurs on a parent table, either propagate the "freeness" downstream or else allow the caller to specify that downstream tables do not get resynced.

    For example, if a Fivetran employee were to trigger a resync of the `ORDER` table, then of course that table's MAR will be free for the duration of the resync. But child tables like `ORDER_LINE`, `ORDER_URL_TAG`, etc.. will also be synchronized consuming Paid MAR.

    Here is another side-effect of the UI that fivetran presents when these "child tables" are selectable / de-selectable.

    A user can disable those checkboxes in the hope of reducing MAR, but that won't work. But it looks like it will do something. Turns out, that because the primary key column(s) effectively cannot be disabled on child tables, the rows will still write but with empty data.

    This means that Fivetran's UI enables a situation where a user can choose to spend MAR to sync empty rows. Either implement this feature request or make this configuration impossible. I can think of zero scenarios where a user would on purpose want to make this configuration.

Didn’t find what you need?

Contact support