Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: S3 Connector REST API enhancement - Support for upsert file using custom primary key

Completed
Nikola Jovanovic User

Hi team,

It would be awesome if you could enhance S3 connector exposed REST API to allow us to configure Field: Modified File Merge -> Upsert file using custom primary key via Terraform provider. So that would mean that new release of Fivetran provider should be created with added support for above mentioned feature. We can do that via Fivetran console ( GUI ), but there is no available API for that option ( doc link ) so that we can do it via terraform. We can only configure Modified File Merge -> Upsert file using file name and line number. Or to use append file option.

Many thanks in advance.

Please sign in to leave a comment.

Comments

5 comments

Date Votes
  • Official comment
    Parmeet Kohli Fivetranner

    Hi Nikola,

    Thank you for reaching out! We are looking into this and will let you know when it's done.

    Thank you,
    Parmeet

  • Sivakanthavel Shanmugam User

    Hi,

    Do we have any update on this request ?

    We are planning to use the REST API feature to create an S3 connector. However, the feature to select primary keys is not available through the REST API. As a result, we are forced to use the UI to select primary keys after creating the connector via the REST API. This defeats the purpose of using the REST API.

     

      

  • Justin Ainsworth User

    Would be great to see this same functionality extended to the SFTP connector setup.  

  • Parmeet Kohli Fivetranner

    Hi everyone, Upsert file using custom primary key is now available via the API for S3 - https://fivetran.com/docs/rest-api/interactive-api-reference/api-configuration/connector-configuration#upsertwithcustomprimarykey Other File connectors to follow soon.

    Thank you,
    Parmeet

  • Jure Černjak User
    • Edited

    Hello, 

    the original author requested Terraform support for defining primary keys to allow "upsert_file_with_primary_keys" to be used as the value of "append_file_option" property. I can see the latest update was that support for this was added on the API but there is no mention of Fivetran's Terraform provider. Judging by the error I get when trying to set the "primary_key" option to a column in my connector schema I would say that this is still not supported via Terraform provider.

    Error message for reference:

    Cannot set value for this attribute as the provider has marked it as read-only. Remove the configuration line setting the value.

    Refer to the provider documentation or contact the provider developers for additional information about configurable and read-only attributes that are supported.