Review of methods to download files using PowerShell

557483263_190baee82f_m
The goal of this post is to review and compare different methods to download files using PowerShell. As part of the review I would like to share (in addition to the inline source code you can also download a module (Get-FileMethods) that contains all functions via GitHub) some wrapper functions that follow the same pattern:

  • function name = Get-FileMETHODNAME
  • Parameters:
  • Name Description
    url URL to download from
    destinationFolder Defaults to “$env:USERPROFILE\Downloads”
    includeStats Switch parameter if specified the function will output stats for comparison purpose

  • If possible the function implements a progress bar including:
    • Remaining time (hh:mm:ss)
    • Elapsed time (hh:mm:ss)
    • Average download speed (Mb/s)/li>
    • Total download size (formatted according to size KB/MB/GB)
    • Currently downloaded size (formatted according to size KB/MB/GB)
  • After the download has finished the downloaded file is automatically unblocked
  • Provided that the includeStats is used the function outputs its name, the size of the file downloaded, and the time it took to download the file (the time is not representative and comparable as I’m currently using a quite bad 3G connection)

The following methods are compared and reviewed:

Method Function Name(s)
1. Invoke-WebRequest Get-FileInvokeWebRequest
2. Microsoft.VisualBasic.Devices.Network Get-FileVB
3. System.Net.WebClient Get-FileWCSynchronous, Get-FileWCAsynchronous
4. Background Intelligent Transfer Service Get-FileBitsTransferSynchronous, Get-FileBitsTransferAsynchronous


Get-FileSize filter
This is a little helper to convert file size units based on the number of actual bytes in a file, which I make use of within all the download functions:

 

↑Invoke-WebRequest

Invoke-WebRequest is a built-in cmdlet (since version 3) that can be used (amongst many other things) to download files.
Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileInvokeWebRequest $url -includeStats

Result:

We get a built-in progress bar showing only the currently downloaded bytes and the download of 10MB took around 1 minute:

ProgressInvokeWebRequest


resultInvokeWebRequest

↑Microsoft.VisualBasic.Devices.Network

 

The DownloadFile method of the Network class within the Microsoft.VisualBasic.Devices Namespace is an oldie but goldie from VisualBasic which can be also used (once the respective assembly is loaded) from within PowerShell:

Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileVB $url -includeStats

Result:

This method also contains a built-in progress bar which pops up in a separate window but doesn’t contain any additional information. The download of 10MB took 17 seconds:

progressBarVB

resultVB

↑System.Net.WebClient

 

The WebClient class provides two different means to download files. One works in synchronous mode (DownloadFile) and one in asynchronous mode (DownloadFileAsync). The difference between the two is, that in synchronous mode the execution of further commands halts until the download has finished while in asynchronous mode the execution continues since the download happens in the background (on another thread). Let’s first have a look at the synchronous one.

Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileWCSynchronous $url -includeStats

Result:

No progress bar (and no way to add one) and the download of 10MB took 35 seconds:
 

resultWCSync
Now the asynchronous version. The helper function is this time a bit more involved to get a proper progress bar by plugging into the respective events:

Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileWCAsynchronous $url -includeStats
 

Result:

This time we can add a custom progress bar including current and overall download speed and time. The download of 10MB took 33 seconds this time:

DownloadWCAsyncProgressbar

resultVB

GetFileWCAsync_Result

 

↑Background Intelligent Transfer Service

 

Background Intelligent Transfer Service (short BITS) is built into Windows since Windows 7. It is a file-transfer service designed to transfer files using only idle network bandwidth therefore BITS does not use all available bandwidth, so you can use it to download large files without affecting other network applications. BITS transfers are also very reliable and can continue when users change network connections or restart their computers. Therefore MS makes use of this technology to download Windows Updates (try Get-BitsTransfer -AllUsers | Select -ExpandProperty FileList to see a list of currently running BITS jobs). While BITS also provides a command-line interface…

bitsadmin /transfer ajob http://speedtest.reliableservers.com/10MBtest.bin %USERPROFILE%\Desktop\10MBtest.bin

… its use is deprecated since Windows 8. Instead the use of the BITS PowerShell module is encouraged. Similar to the WebClient class BITS also provides the user with a synchronous and asynchronous download method. Again we first take a look at the synchronous version:

Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileBitsTransferSynchronous $url -includeStats
 

Result:

This method provides a built-in progress bar but without much information. The download of 10MB took 1 minute 26 seconds which can be improved by specifying ‘Foreground’ to the ‘Priority’ paramater:

BitsSync_progressbar
BitsSync_Result

 

Again similar to WebClient the asynchronous mode provides means to add a custom progress bar. This time by using the properties of the job object:

Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileBitsTransferAsynchronous $url -includeStats
 

Result:

Also with the async mode for BITS we can add a custom progress bar including current and overall download speed and time. The download of 10MB took exactly the same as using BITS in synchronous mode (00:01:25):

BitsAsync_progressbar
BitsAsync_result

 

Conclusion

All methods have their advantages and disadvantages and the choice of course also depends on the actual task at hand (e.g. are we downloading a small file from the command prompt or are we downloading multiple files as part of a bigger scripts) and personal preferences.
The most versatile is the BITS asynchronous mode since it provides everything the other methods also provide plus the ability to run the download in the background using only idle network bandwidth and also options to pause and resume the download.

shareThoughts


Photo Credit: Nicolas Valentin via Compfight cc

Advertisements