Hello. I’m using the SFTP plugin in Connect Workflow (Windows 10 Pro; newer fairly beefy system) to download thousands of files each day. Wondering if anyone is aware of a way to increase download speed? It takes about twice as long to download the PDF files as it does with FileZilla on my 2013 iMac on the same network. One thing I already tried was disabling the Windows Nagle algorithm. Not sure if changing settings in the winSCP GUI preferences would affect anything or not? Just reaching for any tricks or hacks someone may have up their sleeve.
FileZilla is downloading multiple files on multiple threads, which means many files are being downloaded simultaneously. With the SFTP plugin, you’re downloading one file at a time. You could try creating several different processes that each download a specific type of file (i.e. one process for *.pdf, another for *.jpg, etc.), which would allow several threads to run simultaneously.
Alternatively to Phil’s suggestion, WinSCP has very detailed documentation on scripting/automating their software. You could look into that for creating your own script to perform a multithreaded download.
I’ve been messing around with this for the last few days and I can say it works quite nicely. By interfacing with their COM library (WinSCP .NET Assembly and COM Library :: WinSCP) you have full control over the download, logs, errors returned, etc. This would allow you to run a fairly robust Script object in your workflow to do these downloads for you.
The major downside to this is that it can’t be run as an Input task as our SFTP plugin allows. It would only be executed with some other trigger. Still, you could simply set the process to run on a Create File trigger that then launches a multithreaded download through WinSCP.
Hey Guys, thank you for your replies and your thoughts. I have actually been pursuing AlbertsN recommendation of going directly to WinSCP via it’s scripting functionality. I have seen very promising results so far. But like AlbertsN mentioned having the ability to run from task to task is very nice within Workflow. I had a Database Query plugin following the SFTP plugin inserting data directly to a SQL DB for reporting that worked awesome, albeit slowly. Testing proved the DB inserting was not slowing down the process. So, I can still accomplish the same results faster but now with some added fancy scripting. Thanks again, I think these forums are great!
Hi every one,
I would like to reopen this topic. Recently, after the upgrade, we have noticed a slowdown in downloading speed from sftp server. It takes about 30 secs for a 100kb pdf. It is too long when you consider we have 44k files to download.
Is there any other suggestion how to improve this beside winScp scripts?