Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: File Preview for SFTP (in particular) expansion

Answered
Ian Dadswell User

We use multiple sftp connections for different sources. Some have user/password connections which we can easily access and investigate issues with files sent this way. 

This is hared (impossible?) with SSH Key protected SFTP Connections as the only SSH used is Fivetran's, so we can't directly access the SFTP site.

The File Preview option is a great help in identifying what files are available and what they're called (often the filename reality and what was promised by the user don't always match up!).

But it would be even more enhanced with a couple of extra pieces of info for the files displayed in File Preview. 

Would it be possible to add the follow for each file listed: 

  1. File size
  2. File date (created date)

Which should be available from the SFTP sites.

Are highlighted in Red : 

Would it also be possible to add the total number of files to the "X files matching your parameters" message. 

So it would say something like "X files listed out of Y files matching your parameters".

This message here: 

 

Please sign in to leave a comment.

Comments

2 comments

Date Votes
  • Official comment
    Parmeet Kohli Fivetranner
    Hi Ian,

    Thank you for sharing your suggestion with us! We appreciate you taking the time to contribute ideas to help improve Fivetran.
    To ensure I fully understand the use case, could you please provide more detail on why the size and created date is helpful? I agree that listing the total file count would be ideal but it's a trade off we made against running into a long lag when the source folder has too many files.

    Best,
    Parmeet Kohli
  • Ian Dadswell User

    Hi Parmeet, 

    It will help us diagnose any issues where a Connection is re-run (or runs as per schedule) but our DB doesn't show any new data. We can check the files in place on the FTP site (or bucket) to see if the client has actually uploaded the file they said at the time they said. 

    It would also help to define when the Connection should be run and when our downstream processes should be run as we can see when the client is actually delivering the files (ie is it 1am or 5am etc).

    Appreciate not wanting to do a full list and we can work round that by adjusting the "File Pattern" to limit the files detected to be more focused.

    Cheers,


    Ian.