Powershell – Search and Replace multiple text strings in file/variable


To totally unlock this section you need to Log-in

A common, but not so often, requirement in some scenarios is to search and replace one or multiple text strings in variables or files.

This can be quite easily achievied by using a bit of Powershell code.

String functions are an integral part of Powershell and there are various functions present to handle the string manipulation related tasks. In Powershell, everything is an object and string is also an object of type System.String. To remove certain part of a string or replace a part or an entire string with some other text can be achieved using the replace cmdlet in Powershell.

The first, easiest, way to replace a text into a file or variable is the following. First we consider modifying a text file (Demo.txt) after searching for a single word (assuming to replace the word one to two into the Demo.txt file):

((Get-Content -path C:\Demo.txt -Raw) -Replace 'one','two') | Set-Content -Path C:\Demo.txt

The replace operator returns the new string. Notice that the Get-Content command execution is in parentheses. This is required to ensure Get-Content is complete before attempting any replace operation on its output.

After Get-Content reads the file and passes the string to the replace operator, PowerShell then hands off that new string to Set-Content, which takes that string as pipeline input and overwrites the existing file.

Instead, if we want to replace text into a variable, in the following example the $test variable, we can use the following approach:

### Replacing a text in string
$test="Old text to be replaced"
### Going to replace old
$test.Replace("Old","New")
### Text is replaced

There are countless other ways to replace strings in a text file with PowerShell. It's possible to use regular expressions with the replace operator to find complex strings, use regular expression groups and a whole lot more.

The following example is another way to replace a sub-string and, at the same time, save the modification into the same original variable, $input:

$input="A sample text string"
$input= $input -Replace "string","sentence"

The other effective way of replacing multiple text simultaneously would be to use hash table. Hash table is a type of an array, which stores values as key value pair. The key values must be unique, and values can be non-unique. The built-in properties of a hash table are key, value and count. In the case of replacing text using hash table, the key would represent the text to be replaced and the value would represent the text value to be used for replacement.

$hashtable=@
{
Text1=value1;
Text2=value2;
Text3=value3;
Text4=value4
}

The values of a hash table can be altered as follows, adding a new pair to hash table or editing a value:

$hashtable.Add(“text5”,”value5”)
$hashtable.Set_Item(“text5”,”value6”)

Or:

$hastable.”text5”=”value6”

To remove a value and finally display the hash table:

$hashtable.Remove(“text5”)
$hashtable

Now, let's consider an example of replacing text sub strings (and numbers and special characters) into a variable using an hash table:

$input=" This is an example. Let us see how text string are replaced with hash table."

#### Building the hash table ####
$hashtable = @{}
$hashtable.'table'='error'
$hashtable.'This'='That'
$hashtable.'is'='was'
$hashtable.'text'='random'
$hashtable.'.'='!'

#### Replacing text into $input using the hash table ####
Foreach ($key in $hashtable.Keys) {
   $input = $input.Replace($key, $hashtable.$key)
}

##### Replacing numbers and special characters ######
$input1="this 71 35 62 20. Random example text string. % * # $. 62 23 11 89 8 29 85"

$hashtable1 = @{}
$hashtable1.'23'='350'
$hashtable1.'35'='780'
$hashtable1.'8'='900'
$hashtable1.'%'='replaced'
$hashtable1.'*'='sfdsfdsfds'
$hashtable1.'#'='####'
Foreach ($key in $hashtable1.Keys) {
$input1 = $input1.Replace($key, $hashtable1.$key)
}

An additional, and very useful in many scenarios, way to search and replace multiple text strings (especially when we do not know, since the beginning, exactly what text we will need to replace, but we have a general pattern that we need to follow) is by using the powerful Regex approach:

#### Replacing all numbers with two digits in .##. form #####
$input="692.10.445.30.358"
$input=$input -Replace "\.\d{2}\.","**"

#### Swapping name and surname using regex #####
$input= "Tizio Sempronio"
$input= $input -Replace "([a-z]+)\s([a-z]+)",'$2, $1'
Write-Host "After swap: " $input -ForegroundColor Green

A special case of replace would be the following, in which we will use the regex negation symbol to remove, using the -Replace parameter, blank spaces in a text sentence (the result will be Glasshalfemptyglasshalffull):

$string = 'Glass half empty, glass half full.'
$string -replace '[^a-z]'

An additional example of replacing text (this approach can be also used in other examples in this article) using Regex is the following, in which we will append, to the original string variable $string, the -NOT_VALID suffix:

$string = "SerialNumber=YPLagykabFkh7GyriY3P2nDtXKV48Mjb"
$serialNumber = $string -replace "SerialNumber=(.*)","SerialNumber=`$1-NOT_VALID"
Write-output $serialNumber

Note that in the above example, The dollar sign in $1 is escaped using a backtick. Otherwise, PowerShell would treat that $1 as a variable instead of a special regex character.

What we can see here is the usage of a Capture Group (by surrounding a search term with parentheses, PowerShell is creating a capture group. Capture groups capture the content of a regex search into a variable): in this example this special regex character $1, that will contain the value defined with the (.*) pattern, in this case the YPLagykabFkh7GyriY3P2nDtXKV48Mjb text.

Another scenario that we could consider, in Powershell, is when we need, for example, to replace few different text string inside a very large file (like a SQL dump from mySQL DBs, with a resulting file of hundreds of MBs or GBs). The following approach use multiple -Replace parameters against the same text file we need to alter:

$original_file = 'C:\dump_script.sql'
$destination_file =  'C:\import_script.sql'

(Get-Content $original_file) | Foreach-Object {
    $_ -replace 'http://example.net', 'http://localhost/example' `
       -replace 'https://example.net', 'http://localhost/example' `
       -replace 'http://www.example.net', 'http://localhost/example' `
       -replace 'https://private.example.net', 'http://localhost/example'
    } | Set-Content $destination_file

A similar approach to the above one is the following, in which we will concatenate multiple replace blocks with a different notation:

(Get-Content $sourceFile) | ForEach-Object {
    $_.replace('something1', 'something1').replace('somethingElse1', 'somethingElse2')
 } | Set-Content $destinationFile

The Lookup Method

A special position for the available methods to search and replace for text strings is the "lookup method", both using an external file or an array defined in the same Powershell code block.

Let's begin by viewing this method by using the array approach: the $lookupTable array will hold all the pairs of values (old and replacing strings) that the code will use to search and replace in the specified file to analyze.

$lookupTable = @{
    'something1' = 'something1aa'
    'something2' = 'something2bb'
    'something3' = 'something3cc'
    'something4' = 'something4dd'
    'something5' = 'something5dsf'
    'something6' = 'something6dfsfds'
}

$original_file = 'path\filename.txt'
$destination_file =  'path\filename.new.txt'

Get-Content -Path $original_file | ForEach-Object {
    $line = $_

    $lookupTable.GetEnumerator() | ForEach-Object {
        if ($line -match $_.Key)
        {
            $line = $line -replace $_.Key, $_.Value
        }
    }
   $line
} | Set-Content -Path $destination_file

The lookup method can be used also used with the support of an external file (a .csv file), in which are defined the pairs of values, on the same row, to search and replace values in group of files (the $Files parameter using wildcards). The following the code:

Param (
    [String]$List = "NewValues.csv",
    [String]$Files = ".Files*.*"
)
$ReplacementList = Import-Csv $List;
Get-ChildItem $Files |
ForEach-Object {
    $Content = Get-Content -Path $_.FullName;
    foreach ($ReplacementItem in $ReplacementList)
    {
        $Content = $Content.Replace($ReplacementItem.OldValue, $ReplacementItem.NewValue)
    }
    Set-Content -Path $_.FullName -Value $Content
}

The structure of the CSV file (values separated by comma), needed to populate the variable on which the search and replace process will rely, is the following:

OldValue,NewValue
OldFirstValue,NewFirstValue
OldSecondValue,NewSecondValue
OldThirdValue,NewThirdValue

Once the CSV file is prepared and the script is ready, we can call it like in the following example (specifying multiple files with wildcards is not mandatory, so we can use it also by specifying a specific filename):

.Replace-InFilesUsingList.ps1 -List ".NewValues.csv" -Files ".FilesToTarget*.txt"

In this specifically telling the script to only lookup at the files we want it to look, using a file name mask.


Summary
Article Name
Powershell - Search and Replace multiple text strings in file/variable
Description
A common, but not so often, requirement in some scenarios is to search and replace one or multiple text strings in variables or files. This can be quite easily achievied by using a bit of Powershell code.
Author
Publisher Name
Heelpbook.net