PowerShell: Get unique values from array

This post is about finding unique values from PowerShell array by removing redundant/duplicate items. We will discuss various options available to list the unique values in array and their benefits/downsides.

There are multiple ways to do this.

1. Get-Unique cmdlet
2. Select-Object cmdlet
3. Sort-Object cmdlet (the winner)

Get-Unique

This is the most well known and famous option. It is because of intuitiveness in the name. But many feel it is cumbersome to use and some couldn’t figure out how to use this. This is because Get-Unique cmdlet works by comparing each item in sorted list to the next item to eliminate the duplicates. That means if you plan to use this cmdlet you need to make sure the input list of sorted. Also this cmdlet is case-sensitive. See what happens when I pass array of duplicate items to this to get the unique ones.

As you noticed in the output, it returned everything in the array. It because they differ by case. So you need to convert everything to lowercase/upper case, sort it, and then pass to this cmdlet to get the actual result.

Is there no other easy way? Yes, read on…

Select-Object

Let us see what happens when we pass the same array to select-object cmdlet by specifying -Unique cmdlet.

$letters = @("A","a","b","c","F","c","a","E","d")

$letters | select -unique

As you can see it worked better than Get-Unique but it still failed to match items which are different in case. The -Unique switch of Select-Object looks for duplicate items through out the list to remove them. But it does case sensitive search to do that. So, you need to convert the list to lower or upper case before passing it to select-object to get unique items.

Sort-Object

Looking at the name of the cmdlet, you might wonder whether it will really remove the duplicates. But it does and does better job than other two above. Now let us pass the same array to Sort-Object by specifying the -Unique switch.

$letters = @("A","a","b","c","F","c","a","E","d")

$letters | sort -unique

Hurray.. it worked. It searched for duplicate items throughout the list and removed the duplicates irrespective of case. This is what most of the systems need when they want to remove duplicates from array using PowerShell.