Sort text file content and get unique values in Powershell
How to sort the content of a text file (a very big one) and get only unique values (the content is a huge list) in a new text file in Powershell?
Let's say that the contents of the file are as follows:
doe, john
twist, oliver
doe, jane
simmons, richard
twist, oliver
doe, john
lincoln, abraham
You need to sort this list alphabetically. In Powershell, we can create a new file named Sort.ps1 in the same directory as the text file we want to sort, for example unsorted.txt, or even just using the following command. For example, let's put the following simple command in a .ps1 file (gc stands for Get-Content cmdlet):
gc unsorted.txt | Sort | Get-unique > sorted.txt
Fire up PowerShell and change to the directory the files are located in. Enter the command ./sort.ps1 and hit Enter. The command will process and give you a new file named sorted.txt. Take note that the resulting file could be larger of the original one because it could use the UCS-2 LE BOM encoding and not UTF-8. The contents of that file will be:
doe, jane
doe, john
lincoln, abraham
simmons, richard
twist, oliver
Everything is sorted alphabetically and purged from duplicates with one simple command.