Using Everything search command line (es.exe) via PowerShell

tree8

Everything by voidtools is a great search utility for Windows. It returns almost instantaneous results for file and folder searches by utilizing the Master File Table(s). There is also a command-line version of everything (es.exe) and this post is about a wrapper I wrote in PowerShell around es.exe.
The full version including full help (which I’m skipping here to keep it shorter) can be downloaded from my GitHub repository

function Get-ESSearchResult {
    [CmdletBinding()]
    [Alias("search")]
    Param
    (
        #searchterm
        [Parameter(Mandatory=$true, Position=0)]
        $SearchTerm,
        #openitem
        [switch]$OpenItem,
        [switch]$CopyFullPath,
        [switch]$OpenFolder,
        [switch]$AsObject
    )
    $esPath = 'C:\Program Files*\es\es.exe'
    if (!(Test-Path (Resolve-Path $esPath).Path)){
        Write-Warning "Everything commandline es.exe could not be found on the system please download and install via http://www.voidtools.com/es.zip"
        exit
    }
	$result = & (Resolve-Path $esPath).Path $SearchTerm
    if($result.Count -gt 1){
        $result = $result | Out-GridView -PassThru
    }
    foreach($record in $result){
        switch ($PSBoundParameters){
	        { $_.ContainsKey("CopyFullPath") } { $record | clip }
	        { $_.ContainsKey("OpenItem") }     { if (Test-Path $record -PathType Leaf) {  & "$record" } }
	        { $_.ContainsKey("OpenFolder") }   {  & "explorer.exe" /select,"$(Split-Path $record)" }
	        { $_.ContainsKey("AsObject") }     { $record | Get-ItemProperty }
	        default                            { $record | Get-ItemProperty | 
                                                    select Name,DirectoryName,@{Name="Size";Expression={$_.Length | Get-FileSize }},LastWriteTime
                                               }
        }
    }
}

The function contains a call to “Get-FileSize” a helper filter in order to return the file size of the selected items in proper format:

filter Get-FileSize {
	"{0:N2} {1}" -f $(
	if ($_ -lt 1kb) { $_, 'Bytes' }
	elseif ($_ -lt 1mb) { ($_/1kb), 'KB' }
	elseif ($_ -lt 1gb) { ($_/1mb), 'MB' }
	elseif ($_ -lt 1tb) { ($_/1gb), 'GB' }
	elseif ($_ -lt 1pb) { ($_/1tb), 'TB' }
	else { ($_/1pb), 'PB' }
	)
}

How does it work? The Get-ESSearchResult function (alias search) searches for all items containing the search term (SearchTerm parameter is the only mandatory parameter). The search results (if multiple) are piped to Out-GridView with the -PassThru option enabled so that the result can be seen in GUI and one or multiple items from within the search results can be selected. By default (no switches turned on) the selected item(s) are converted to FileSystemInfo objects and their Name, DirectoryName, FileSize and LastModifiedDate are output. The resulting objects can be used for further processing (copying, deleting….).

The switch Parameters add the following features and can be used in any combination:

  • -OpenItem : Invoke the selected item(s) (only applies to files not folders)
  • -CopyFullPath : Copy the full Path of the selected item to the clipboard
  • -OpenFolder : Opens the folder(s) that contain(s) the selected item(s) in windows explorer
  • -AsObject : Similar to default output but the full FileSystemInfo objects related to the selected item(s) are output

I hope that the function can also help some of you to find your files and folders faster from the commandline.
I’ve written another blog post in relation to Everything and PowerShell:
Search fiel content by keyword using Everyting + PowerShell + GUI

shareThoughts


photo credit: 983 Foggy Day via photopin (license)

Advertisements

Convert between US and European date formats in Excel

tree7

The easiest way I know of (please let me know if you know a better way) to convert between US (“mm/dd/yy”) and European (“dd/mm/yy”) dates without using VBA in Excel is via “Text to Columns”. Let’s look at an example:
Convert date1
My system’s regional settings are setup for US dates, therefore I need to convert the dates to US format in order to make the Weekday function return a proper result. Here are the steps:

  1. Highlight the range of dates to convert (A2:A6)
  2. Click on “Text to Columns” in the Data ribbon
  3. Go with the defaults in the first two steps of the wizard
  4. Select “Date” as Column data format and pick the appropriate Format (DMY) from the dropdown
  5. Modify the Destination to paste the results somewhere else if necessary (needs to be on the same sheet)
  6. Click on “Finish”

ConvertDates2
ConvertDates3

shareThoughts


photo credit: Bouleau d’hiver, Megève, Haute-Savoie, Rhône-Alpes, France. via photopin (license)

Use PowerShell to measure the time to download a file

tree 7

It is actually surprisingly easy to measure how long it takes to download a file with PowerShell. Let’s look at an example: My internet speed is according to speedtest.net around 4.2 Mbps (megabits/second). If I would like to know how long it takes me to download a file of 1GB I can use a simple one liner:

[TimeSpan]::FromSeconds(1GB/(4.2/8*1MB))

How does that actually work?

  1. Divide the size of the files in bytes (1GB/)
  2. by the download speed/second in bytes (*1MB). First converting Mb into MB (4.2/8)
  3. The results is the time it takes to download the specified size in decimal seconds
  4. The result is used to create a new TimeSpan object (around 32 minutes to download 1GB on my PC)

It doesn’t take much more effort to turn this into a re-usable function:

function Get-DownloadTime($SizeInBytes,$SpeedInMbps,[switch]$AsTimeSpan){
    $ts = [TimeSpan]::FromSeconds($SizeInBytes/($SpeedInMbps/8*1MB))
    if ($AsTimeSpan){
        $ts
    }
    else{
        $ts.ToString('hh\:mm\:ss')
    }
}
#usage
Get-DownloadTime 1GB 4.2
Get-DownloadTime 1GB 4.2 -AsTimeSpan

The function takes the size in bytes and the speed in Mbps and returns by default the time in the format “hh:mm:ss” or the TimeSpan object when used with the -AsTimeSpan switch. More details on the TimeSpan format specifiers can be found here.

shareThoughts


photo credit: Pine Forest Colorful Tree via photopin (license)

“Please wait while windows configures Microsoft Visual Studio…” when starting Excel

tree6

I got this dialog (“Please wait while windows configures Microsoft Visual Studio Professional 2013”) on every Excel (2010) start up shortly after I had installed Visual Studio 2013 Community Edition. In my case it delayed the Excel start-up for several minutes. In order to get rid of the quite annoying dialog I just created a new directory under C:\WINDOWS\Microsoft.NET\Framework\URTInstallPath_GAC. To do so just run the command below from an elevated command- or PowerShell prompt:

mkdir C:\WINDOWS\Microsoft.NET\Framework\URTInstallPath_GAC

photo credit: Mañana… – Tomorrow… via photopin (license)

PowerShell scope write-up

tree 5

I have seen and experienced myself quite some confusion and frustration around the application and understanding of scope using PowerShell. This is an attempt to write up a summary of the different aspects of how PowerShell handles scope which hopefully helps some people to understand it better.

TOC:
Definition and purpose of scope
Lexical vs. dynamic scoping
Types of scope
Scoping rules
Scope modifiers
Dot sourcing
Remote sessions and scope
Modules and scope
Recommendations

↑ Definition and purpose of scope

The scope of an object is the region of the program in which it is accessible. All objects have a scope that defines (from) where the name of the object can be used to refer to it.

Scopes help to protect objects from being unintentionally modified and also to avoid the risk of a name collision between two identically named objects.

Within PowerShell the following objects support scope:

  • Variables
  • Aliases
  • Functions
  • PowerShell Drives

↑ Lexical vs. dynamic scoping

PowerShell uses dynamic scoping as compared to most of the other languages that use lexical scoping. What is the difference? Lexical scope means that the scope is defined by the environment where an object was created. Dynamic scope means to use the environment where the object is called. An example should help to illustrate this:

$foo = 3
function Out-Foo { $foo }
function Set-Foo{
    $foo = 5
    Out-Foo
}
Out-Foo
Set-Foo
Out-Foo 

Running the example in PowerShell will output 3 5 3. Running a similar example within a programming language that is using lexical scoping will output 3 3 3. Lexical scoping treats the foo variable as having only one instance within the global scope. The dynamic scoping in PowerShell makes the modified value of foo that is set within the Set-Foo function available to the Out-Foo function.

↑ Types of scope

A new scope is created by:

  • running a script or function
  • creating a remote session
  • or by starting a new instance of PowerShell

When a new scope is created it will act as a child scope of the scope from where it was created. A child scope is created with a set of items. It includes all the items that have the AllScope option plus some variables that can be used to customize the scope, such as MaximumFunctionCount.

Global

All scopes are child scopes of the global scope (i.e. the mother of all scopes). Global is the scope that is in effect when PowerShell starts including Automatic variables, preference variables and the variables, aliases, and functions that are in the active profile. Items in global scope are visible everywhere.

Local

Only available in scripts or functions. Default scope for variables and aliases within scripts or functions.

Script

This scope is created while a script file runs. Items in script scope are visible everywhere inside the script and inside any scripts called within the parent script. Default scope for functions within scripts. Using the script scope modifier permits access to objects in the parent script scope.

Private

Items in private scope are not visible outside of the current scope. Private scope can be used to create a private version of an item with the same name in another scope. The Option parameter of the New-Variable, New-Alias can be used to set the value of the Option property to Private.

AllScope

Variables and aliases have an Option property that can take a value of AllScope. Items that have the AllScope property become part of any child scopes. Changes to the item in any scope affect all the scopes in which the variable is defined.

Numbered Scopes

A scope can be referred to by a number that describes the relative position of one scope to another. Scope 0 represents the current, or local, scope. Scope 1 indicates the immediate parent scope. Scope 2 indicates the parent of the parent scope, and so on.

↑ Scoping rules

  1. An item included in a scope is visible in the scope in which it was created and in any child scope, unless it is within private scope.
  2. The parent scope cannot access variables, which are defined in a child scope.
  3. An item created within a scope can be changed only in the scope in which it was created, unless one explicitly specifies a different scope by using a scope modifier. The only exception to this rule are aliases and variables that are created with the AllScope (link) option
  4. If you create an item in a scope, and the item shares its name with an item in a different scope, the original item might be hidden under the new item. But, it is not overridden or changed. In other words, if a child scope attempts to modify a parent scope element without using the special syntax, a new element of the same name is created within the child scope, and the child scope effectively “loses” access to the parent scope element of that name.

Some examples:

#foo has global scope if pasted to the console but script scope if run from within a script
$foo = 'I am global'
function bar { 
  #will read the global foo variable
  $foo 
  #will create a new variable $foo within the local scope since the child scope cannot modify the global foo variable
   $foo = 'I am local to the function'
   #access the local variable
   $foo
}
$foo
bar
$foo
<#output
I am global
I am global
I am local to the function
I am global
#>
$foo = 'I am global'
function bar { 
  $foo 
  #access the global/script scope by using a scope modifier
  $script:foo = 'Me too'
  #same as above using Get-Variable
  Get-Variable foo –Scope Script -ValueOnly
  $foo
}
$foo
bar
$foo
<#output
I am global
I am global
I am global
I am global
I am global
#>

↑ Scope modifiers

Scope modifiers (global, local, private, and script.) can be used for variables and functions:

function private:myFunction {}
$script:myVar

To find the items in a particular scope, use the Scope parameter of Get-Variable or Get-Alias. Use Get-Item to get the functions in a particular. Get-Item does not have a scope parameter but it returns all functions within the scope it is called from.

↑ Dot sourcing

A script or function can be added to the current scope by using dot sourcing. This will in effect make any functions, aliases, and variables that the script creates available in the current scope.

For example, to run the Sample.ps1 script from the C:\Scripts directory in the local scope, use the following command:
. C:\Scripts\sample.ps1

I would recommend using modules instead of dot sourcing in most of the cases but actually use it myself quite often in order to have a main function at the top of the script:

Function main {
	Function one
	Function two
}
Function one {
	“helper function”
}
Function two {
	“Another helper function”
}
#dot source the main function in order to make it “see” the other functions
. main

↑ Remote sessions and scope

A remote session has its own global scope, but a session is not a child scope of the session in which it was created. Child scopes within a remote session can be created by running a script.
To pass on an “external” variable into a remote session scriptBlock the $using qualifier can be utilized:

$foo = ‘test’
Invoke-Command –ComputerName bar	{
  $dir = 'c:\' + $using:foo
  Get-ChildItem	 $dir
}

↑ Modules and scope

Variables and Aliases in a module are by default not accessible outside the module (This can be controlled using Export-ModuleMember). The privacy of a module behaves like a scope, but adding a module to a session does not change the scope. And, the module does not have its own scope, although the scripts in the module do have their own scope.

Visibility

The Visibility property of a variable or alias determines whether it can be seen outside the container, such as a module, in which it was created. Visibility works for containers in the same way that the Private value of the Option property works for scopes.
The Visibility property takes can be set to either Private or Public. Items that have private visibility can be viewed and changed only in the container in which they were created. Because Visibility is designed for containers, it works differently in a scope. If you create an item that has private visibility in the global scope, you cannot view or change the item in any scope. If you try to view or change the value of a variable that has private visibility, Windows PowerShell returns an error message.
One can use the New-Variable and Set-Variable cmdlets to create a variable that has private visibility.

↑ Recommendations

1. Try to avoid using scope modifiers in order to modify items in a parent scope.
a. Better pass the item into a function through an argument by reference

$foo = 'some value'
function bar ([ref]$myFoo){
    #instead of $script:foo = 'new value'
    $myFoo.Value = 'new value'
}
$foo
bar ([ref]$foo)
$foo

<#output
some value
new value
#>

2. Use Set-Strict in order to prohibit references to uninitialized variables (see Get-Help Set-StrictMode for details)
3. Use modules instead of dot sourcing (see .dot sourcing for an exception)

shareThoughts


photo credit: Standing out from the Crowd. via photopin (license)

PowerShell and the “missing” ternary operator

tree4

People switching to PowerShell from other languages like JavaScript or C# might be looking for a ternary operator at one point. While there is no built-in ternary operator in PowerShell it is not too hard to come up with something that comes pretty close to it. Let’s look at an example:

filter Get-FourthLetter{
    if ($_.Length -gt 3){
        $_[3]
    }
    else{
        -1
    }
}
'Pete', 'Joe', 'Dirk' | Get-FourthLetter

This defines a (admittedly pretty useless) filter (like a function but with a process block only) to get the fourth letter of any word that is “piped” into the filter, which has more than 4 characters or -1 (in case the word has less than 4 chars). Let’s try to re-write this with “something like” a ternary operator in PowerShell:

filter Get-FourthLetter{
    (-1, $_[3])[$_.Length -gt 3]
}
'Pete', 'Joe', 'Dirk'  | Get-FourthLetter

Here we replace the if/else construct with a simple array construct of the form (FALSEEXPRESSION, TRUEEXPRESSION). The CONDITION part is then used to index into the array by making use of the fact that $true evaluates to 1 and $false to 0 in PowerShell. To increase the “uselessness” factor even further, let’s wrap the “ternary” operator functionality into a more generic filter:

filter ?:($trueExpression, $falseExpression){
    ($falseExpression, $trueExpression)[$_]
}
'Pete', 'Joe', 'Dirk'  | % { ($_.Length -gt 3)  | ?: $_[3] -1 }

I hope this will help anyone that has been desperately searching for a PowerShell ternary operator ;-).

shareThoughts


photo credit: Purple storm. Old Jacarandas avenue in Goodna, near Brisbane via photopin (license)

Waiting for elements presence with Selenium Web driver

tree3

Selenium is a great way to automate browsers for testing or scraping purposes. One thing that is missing when using the driver via PowerShell is an easy way to specify an explicit wait for methods on the driver. This is useful if you would like to call a method on an element but don’t know whether the element has been loaded yet:

#load the web driver dll
Add-Type -path c:\WebDriver.dll
#initiate a driver
$script:driver = New-Object OpenQA.Selenium.Chrome.ChromeDriver
#specify an implicit wait interval
$null = $driver.Manage().Timeouts().ImplicitlyWait((New-TimeSpan -Seconds 5))
#browse to a website
$driver.Url = 'https://www.google.com'
$driver.FindElementByName('q')
#potential error message if element hasn't been loaded yet
$driver.Close()
$driver.Dispose()
$driver.Quit()

One workaround would be to just adding a ‘sleep 10’, but this is not a very elegant solution. Since we also don’t know for how long to wait. Another approach (probably also not the most elegant solution, but the best that I came up with so far. It turns out that there actually is another probably more elegant approach available (see below for more details)) is to use a little helper function like this:

function isElementPresent($locator,[switch]$byClass,[switch]$byName){
    try{
        if($byClass){
            $null=$script:driver.FindElementByClassName($locator)
        }
        elseif($byName){
            $null=$script:driver.FindElementByName($locator)
        }
        else{
            $null=$script:driver.FindElementById($locator)
        }
        return $true
    }
    catch{
        return $false
    }
}

There is another approach available using the webdriver’s built-in capabilities. With this, one can specify a time in seconds the webdriver will wait for the element’s presence before throwing an exception (openqa.selenium.TimeoutException):


function waitForElement($locator, $timeInSeconds,[switch]$byClass,[switch]$byName){
    #this requires the WebDriver.Support.dll in addition to the WebDriver.dll
    Add-Type -Path C:\WebDriver.Support.dll
    $webDriverWait = New-Object OpenQA.Selenium.Support.UI.WebDriverWait($script:driver, $timeInSeconds)
    try{
        if($byClass){
            $null = $webDriverWait.Until([OpenQA.Selenium.Support.UI.ExpectedConditions]::ElementIsVisible( [OpenQA.Selenium.by]::ClassName($locator)))
        }
        elseif($byName){
            $null = $webDriverWait.Until([OpenQA.Selenium.Support.UI.ExpectedConditions]::ElementIsVisible( [OpenQA.Selenium.by]::Name($locator)))
        }
        else{
            $null = $webDriverWait.Until([OpenQA.Selenium.Support.UI.ExpectedConditions]::ElementIsVisible( [OpenQA.Selenium.by]::Id($locator)))
        }
        return $true
    }
    catch{
        return "Wait for $locator timed out"
    }
}

With this in place we can more or less avoid unnecessary wait time and also the potential for error when trying to access an element that is not loaded yet:

while(!(isElementPresent 'q' -byName)){
    sleep 1
}
$driver.FindElementByName('q')

or

waitForElement 'q' 10 -byName

shareThoughts


photo credit: I paint my own reality via photopin (license)