Review of methods to download files using PowerShell

557483263_190baee82f_m
The goal of this post is to review and compare different methods to download files using PowerShell. As part of the review I would like to share (in addition to the inline source code you can also download a module (Get-FileMethods) that contains all functions via GitHub) some wrapper functions that follow the same pattern:

  • function name = Get-FileMETHODNAME
  • Parameters:
  • Name Description
    url URL to download from
    destinationFolder Defaults to “$env:USERPROFILE\Downloads”
    includeStats Switch parameter if specified the function will output stats for comparison purpose

  • If possible the function implements a progress bar including:
    • Remaining time (hh:mm:ss)
    • Elapsed time (hh:mm:ss)
    • Average download speed (Mb/s)/li>
    • Total download size (formatted according to size KB/MB/GB)
    • Currently downloaded size (formatted according to size KB/MB/GB)
  • After the download has finished the downloaded file is automatically unblocked
  • Provided that the includeStats is used the function outputs its name, the size of the file downloaded, and the time it took to download the file (the time is not representative and comparable as I’m currently using a quite bad 3G connection)

The following methods are compared and reviewed:

Method Function Name(s)
1. Invoke-WebRequest Get-FileInvokeWebRequest
2. Microsoft.VisualBasic.Devices.Network Get-FileVB
3. System.Net.WebClient Get-FileWCSynchronous, Get-FileWCAsynchronous
4. Background Intelligent Transfer Service Get-FileBitsTransferSynchronous, Get-FileBitsTransferAsynchronous


Get-FileSize filter
This is a little helper to convert file size units based on the number of actual bytes in a file, which I make use of within all the download functions:

 

↑Invoke-WebRequest

Invoke-WebRequest is a built-in cmdlet (since version 3) that can be used (amongst many other things) to download files.
Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileInvokeWebRequest $url -includeStats

Result:

We get a built-in progress bar showing only the currently downloaded bytes and the download of 10MB took around 1 minute:

ProgressInvokeWebRequest


resultInvokeWebRequest

↑Microsoft.VisualBasic.Devices.Network

 

The DownloadFile method of the Network class within the Microsoft.VisualBasic.Devices Namespace is an oldie but goldie from VisualBasic which can be also used (once the respective assembly is loaded) from within PowerShell:

Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileVB $url -includeStats

Result:

This method also contains a built-in progress bar which pops up in a separate window but doesn’t contain any additional information. The download of 10MB took 17 seconds:

progressBarVB

resultVB

↑System.Net.WebClient

 

The WebClient class provides two different means to download files. One works in synchronous mode (DownloadFile) and one in asynchronous mode (DownloadFileAsync). The difference between the two is, that in synchronous mode the execution of further commands halts until the download has finished while in asynchronous mode the execution continues since the download happens in the background (on another thread). Let’s first have a look at the synchronous one.

Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileWCSynchronous $url -includeStats

Result:

No progress bar (and no way to add one) and the download of 10MB took 35 seconds:
 

resultWCSync
Now the asynchronous version. The helper function is this time a bit more involved to get a proper progress bar by plugging into the respective events:

Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileWCAsynchronous $url -includeStats
 

Result:

This time we can add a custom progress bar including current and overall download speed and time. The download of 10MB took 33 seconds this time:

DownloadWCAsyncProgressbar

resultVB

GetFileWCAsync_Result

 

↑Background Intelligent Transfer Service

 

Background Intelligent Transfer Service (short BITS) is built into Windows since Windows 7. It is a file-transfer service designed to transfer files using only idle network bandwidth therefore BITS does not use all available bandwidth, so you can use it to download large files without affecting other network applications. BITS transfers are also very reliable and can continue when users change network connections or restart their computers. Therefore MS makes use of this technology to download Windows Updates (try Get-BitsTransfer -AllUsers | Select -ExpandProperty FileList to see a list of currently running BITS jobs). While BITS also provides a command-line interface…

bitsadmin /transfer ajob http://speedtest.reliableservers.com/10MBtest.bin %USERPROFILE%\Desktop\10MBtest.bin

… its use is deprecated since Windows 8. Instead the use of the BITS PowerShell module is encouraged. Similar to the WebClient class BITS also provides the user with a synchronous and asynchronous download method. Again we first take a look at the synchronous version:

Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileBitsTransferSynchronous $url -includeStats
 

Result:

This method provides a built-in progress bar but without much information. The download of 10MB took 1 minute 26 seconds which can be improved by specifying ‘Foreground’ to the ‘Priority’ paramater:

BitsSync_progressbar
BitsSync_Result

 

Again similar to WebClient the asynchronous mode provides means to add a custom progress bar. This time by using the properties of the job object:

Function:

Test:

$url = 'http://speedtest.reliableservers.com/10MBtest.bin'
Get-FileBitsTransferAsynchronous $url -includeStats
 

Result:

Also with the async mode for BITS we can add a custom progress bar including current and overall download speed and time. The download of 10MB took exactly the same as using BITS in synchronous mode (00:01:25):

BitsAsync_progressbar
BitsAsync_result

 

Conclusion

All methods have their advantages and disadvantages and the choice of course also depends on the actual task at hand (e.g. are we downloading a small file from the command prompt or are we downloading multiple files as part of a bigger scripts) and personal preferences.
The most versatile is the BITS asynchronous mode since it provides everything the other methods also provide plus the ability to run the download in the background using only idle network bandwidth and also options to pause and resume the download.

shareThoughts


Photo Credit: Nicolas Valentin via Compfight cc

Show-CommandGUI an enhanced Show-Command for PowerShell

2036545266_e1f19d8cdd_m
After my last post (PowerShell tricks – Use Show-Command to add a simple GUI to your functions). I was thinking how one could write a function that would not have the deficiencies that Show-Command has when it comes to providing a GUI for functions. In addition to what Show-Command does I wanted a function that:

  • Considers advanced function parameters (i.e. ValidateScript, ValidatePattern, ValidateRange, ValideLength)
  • Populates the fields with the respective default values (if applicable)
  • Provides a browse for file or browse for folder option (if the parameter name contains ‘file’ or ‘folder’)
  • Provides meaningful error messages via message boxes
  • Doesn’t require the use of Invoke-Expression in order to run the parameters against the command
  • Can be customized to my own preferences

Say hello to Show-CommandGUI.

Show-CommandGUI
The screenshot above is produced with the following test function:

function test{
    [CmdletBinding(DefaultParameterSetName='Basic')]
    Param
    (
        [Parameter(Mandatory=$true, 
                   Position=0,
                   ParameterSetName='Basic')]
        [ValidateNotNullOrEmpty()]
        [ValidateSet("sun", "moon", "earth")] 
        [string]$choice,
        [ValidateRange(2,5)]
        $number,
        [ValidatePattern("^[a-z]")]
        [ValidateLength(2,5)]
        [String]$pattern,
        [ValidateScript({$_ -like '*test'})]
        $string,
        [Parameter(ParameterSetName='Advanced')]
        [switch]$switch,
        [Parameter(Mandatory=$true, ParameterSetName='Advanced')]
        $filePath = 'c:\test.txt',
        [Parameter(Mandatory=$true, ParameterSetName='Basic')]
        $folderPath = 'c:\'
    )
    "pattern: $pattern"
    "number: $number"
    "string: $string"
    if ($PSCmdlet.ParameterSetName -eq 'Advanced'){
        "switch: $switch"
        "filePath: $filePath"
    }
    else{
        "choice: $choice"
        "folderPath: $folderPath"
    }
}

Producing the GUI is as simple as this:

Import-Module "$PATHTOMODULE\Show-CommandGUI.psm1"
Show-CommandGUI test

The module and the test function can be downloaded via GitHub. Please let me know if you have any questions or ideas on how to improve the function.

shareThoughts


Photo Credit: Fernando X. Sanchez via Compfight cc

PowerShell tricks – Use Show-Command to add a simple GUI to your functions

211707_b06dae339d_m
The Show-Command cmdlet has been introduced in PowerShell Version 3 and is very useful to help discovering and learning more about PowerShell cmdlets and their respective parameters (also built into the ISE as the Show-Command Add-on).:

#Discover commands by running Show-Command without parameters
Show-Command
#Run Show-Command for a specific cmdlet
Show-Command Get-ChildItem

Show-Command can be also utilized for your own functions in order to provide your users with a simple GUI as it builds a graphical user interface for the provided function on the fly. Show-Command displays:

  • A drop-down for parameters that use the ValidateSet option
  • A check-box for switch parameters
  • A text box for any other type of parameter
  • An asterisk behind the parameter name in case the parameter is mandatory (the mandatory parameters are also enforced by disabling the run/copy buttons until the mandatory parameter is provided)
  • Each parameter set is displayed on a separate tab

Below is an example showing the features mentioned above using the NoCommonParameter switch to hide those parameters and PassThru in combination with Invoke-Expression in order to run the function with the chosen parameters (I couldn’t get this working otherwise):

Show-MyCommand
Limitations of this approach are:

  • No enforcement for any other advanced function parameter option (e.g. ValidatePattern). The error message is displayed on the command prompt after clicking Run.
  • No option to disable/change the built-in functionality (e.g. disable buttons at the bottom, change minimum height)
  • No sophisticated options for specific parameter types (e.g. browse for files)

What are the GUI options you like to you use for your functions?
Update: I’ve added a follow-up post with a new function that removes the disadvantages mentioned above
shareThoughts


Photo Credit: Sun Spiral via Compfight cc

PowerShell tricks – Useful default parameters to add to your profile

15389627623_7ef1f7595f_m
Since version 3 PowerShell introduced $PSDefaultParameterValues which is a built-in preference variable which lets you specify default values for any cmdlet or advanced function. You can read much more about it inside the respective help file. In a nutshell $PSDefaultParameterValues is a hash-table where (in its most common version) the key consists of the cmdlet name and parameter name separated by a colon (:) and the value is the custom default value:

<CmdletName>:<ParameterName>"="<DefaultValue>

I’ve added the following default parameter values to my profile (You can read here and here on how to work with profiles):

$PSDefaultParameterValues.Add("Get-ChildItem:Force",$true)
$PSDefaultParameterValues.Add("Receive-Job:Keep",$true)
$PSDefaultParameterValues.Add("Format-Table:AutoSize",$true)
$PSDefaultParameterValues.Add("Import-Module:Force",$true)
$PSDefaultParameterValues.Add('Export-Csv:NoTypeInformation', $true)
$PSDefaultParameterValues.Add('Get-Member:Force', $true)
$PSDefaultParameterValues.Add('Format-List:Property', '*')
$PSDefaultParameterValues.Add('Set-Location:Path', '..')
$PSDefaultParameterValues.Add('Get-Help:Detailed', $true )

What are other default parameter values that you use?

shareThoughts


Photo Credit: coloneljohnbritt via Compfight cc

Expanding aliases in PowerShell ISE or any PowerShell file

393790664_da5b0ddb12_m
Further extending my PowerShell ISE module (ISEUtils) I’ve added a function to convert aliases either in the currently active ISE file or (in case a a path is provided) within any PowerShell file (that way the function can be also used from the PowerShell Console) to their respective definitions.
Aliases are very useful when working interactively, since they help saving extra keystrokes when you just want to get things done fast. At the same time if we are speaking about production code where readability, and easy comprehension of the code are much more important the usage of aliases should be avoided ( read here for a good article on best practices for PowerShell alias usage).
With the Expand-Alias function you can get the best of both worlds. Writing clearer code while avoiding extraneous keystrokes. For the code samples in my blog posts I’m also using aliases quite a lot, but would like to start using the new function from now on.
Below is the source code for Expand-Alias:

Usage:

  1. If the function is called without any parameter it will expand all aliases within the current ISE file.
  2. Providing a full-path to the path parameter will expand all aliases within the respective PowerShell file instead.

Adding the following lines to your PowerShell profile (you can read here and here on how to work with profiles) will automatically load the function and add an item to the AddOn menu to call it on every ISE startup:

$FULLPATHTOEXPANDALIAS = 'c:\expand-alias.ps1'
. $FULLPATHTOEXPANDALIAS
$null  = $psISE.CurrentPowerShellTab.AddOnsMenu.Submenus.Add('Expand-Alias', { Expand-Alias }, 'CTRL+SHIFT+X')

shareThoughts


Photo Credit: Martin Gommel via Compfight cc

PowerShell tricks – Using dot(s) to refer to the current location

9663950111_c97678228e_m
Most people are aware that PowerShell supports commandline navigation in the same way as the good old command prompt (see my previous post Improve PowerShell commandline navigation for ways to enhance this):

cd $env:USERPROFILE\Desktop
Resolve-Path '.'
#change to the current direction (doing nothing)
cd .
#move up one level
Resolve-Path '..'
cd ..

The above is using cd as the alias for the Set-Location Cmdlet providing:

  • One dot as an argument for the Path parameter representing the current location (changing the location to the current location does effectively nothing)
  • Two dots for the Path as an argument for the Path parameter representing one level higher than the current location

Since this is implemented by the Provider (at least that’s what I believe) the same concept can be used in many different places, basically every built-in command that has a Path parameter.:

#get all the built-in commands that have a Path parameter
Get-Command | where { $_.Parameters -and $_.Parameters.ContainsKey('Path') -and $_.HelpURI -and $_.HelpURI.StartsWith('http://go.microsoft.com') }

#navigation works also with the registry provider
cd HKLM:\Software\Microsoft
cd ..
cd c:
#open windows explorer using the current location (ii is the alias for Invoke-Item)
ii .
#same one level higher
ii ..
#Within ISE open all files in the current folder or one level higher
psedit .
psedit ..
#copy the current folder and its content to a folder one level higher
copy * ..\test

Please share if you know of more tricks using dot(s) as an argument to the Path parameter.

shareThoughts


Photo Credit: Rein -e- Art via Compfight cc