Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: Add current timestamp to filename and/or prefix of S3 uploads

Answered
Sami Pekkala User

It would be nice if we could comply with our AWS data platform's naming conventions when using Fivetran S3 Data Lake as a destination.

So either the S3 Prefix Path or the filename created should be editable so that they can include the current timestamp as variables. Something like constant-prefix/%YYYY/%MM%DD/%hh/%mm/%ss.

Please sign in to leave a comment.

Comments

1 comment

Date Votes
  • Official comment
    Casey Karst Fivetranner
    Hi Sami,

    Thank you for taking the time to submit your feature request for adding a current timestamp to the filename and/or prefix of S3 uploads.

    Because the files in MDLS are associated with Iceberg and Delta tables, we strongly suggest that you do not alter the names of the files as this will lead to Table corruption. For the data files themselves have datetime embedded into the file name to make tracking easier. While this may not confirm exactly to your policy, we hope that this is sufficient for your needs.

    Thanks,
    Casey

    Best regards,
    Casey Karst