How to use sed to find and replace text in files in Linux / Unix shell
Find and replace text within a file using sed command
The procedure to change the text in files under Linux/Unix using sed:
- Use Stream EDitor (sed) as follows:
- sed -i ‘s/old-text/new-text/g’ input.txt
- The s is the substitute command of sed for find and replace
- It tells sed to find all occurrences of ‘old-text’ and replace with ‘new-text’ in a file named input.txt
- Verify that file has been updated:
- more input.txt
Let us see syntax and usage in details.
Syntax: sed find and replace text
The syntax is:
sed ‘s/word1/word2/g’ input.file
## *bsd/macos sed syntax#
sed ‘s/word1/word2/g’ input.file > output.file
sed -i ‘s/word1/word2/g’ input.file
sed -i -e ‘s/word1/word2/g’ -e ‘s/xx/yy/g’ input.file
## use + separator instead of / ##
sed -i ‘s+regex+new-text+g’ file.txt
The above replace all occurrences of characters in word1 in the pattern space with the corresponding characters from word2.
Examples that use sed to find and replace
Let us create a text file called hello.txt as follows:
$ cat hello.txt
The is a test file created by nixCrft for demo purpose.
foo is good.
Foo is nice.
I love FOO.
I am going to use s/ for substitute the found expression foo with bar as follows:
sed ‘s/foo/bar/g’ hello.txt
Sample outputs:
To update file pass the -i option:
sed -i ‘s/foo/bar/g’ hello.txt
cat hello.txt
The g/ means global replace i.e. find all occurrences of foo and replace with bar using sed. If you removed the /g only first occurrence is changed:
sed -i ‘s/foo/bar/’ hello.txt
The / act as delimiter characters. To match all cases of foo (foo, FOO, Foo, FoO) add I (capitalized I) option as follows:
sed -i ‘s/foo/bar/g I ‘ hello.txt
cat hello.txt
Sample outputs:
Please note that the BSD implementation of sed (FreeBSD/MacOS and co) does NOT support case-insensitive matching. You need to install gnu sed. Run the following command on Apple Mac OS:
$ brew install gnu-sed
######################################
### now use gsed command as follows ##
######################################
$ gsed -i ‘s/foo/bar/g I ‘ hello.txt
$ cat hello.txt
sed command problems
Consider the following text file:
$ cat input.txt
http:// is outdate.
Consider using https:// for all your needs.
Find word ‘http://’ and replace with ‘https://www.cyberciti.biz’:
sed ‘s/ http:// / https://www.cyberciti.biz /g’ input.txt
You will get an error that read as follows:
Our syntax is correct but the / delimiter character is also part of word1 and word2 in above example. Sed command allows you to change the delimiter / to something else. So I am going to use +:
sed ‘s+ http:// + https://www.cyberciti.biz +g’ input.txt
Sample outputs:
How to use sed to match word and perform find and replace
In this example only find word ‘love’ and replace it with ‘sick’ if line content a specific string such as FOO:
sed -i -e ‘/FOO/s/love/sick/’ input.txt
Use cat command to verify new changes:
cat input.txt
Recap: Using sed to find and replace
The general syntax is:
## find word1 and replace with word2 using sed ##
sed -i ‘s/word1/word2/g’ input
## you can change the delimiter to keep syntax simple ##
sed -i ‘s+word1+word2+g’ input
sed -i ‘s_word1_word2_g’ input
## you can add I option to GNU sed to case insensitive search ##
sed -i ‘s/word1/word2/gI’ input
sed -i ‘s_word1_word2_gI’ input
How to Find and Replace Text in a File
Last modified: May 8, 2020
1. Introduction
In this tutorial, we’re going to take a look at how we can harness the power of built-in Linux commands to search for and replace a string of characters quickly and easily. This is a very handy technique for when we need to update all occurrences of a particular string in a large number of files.
For example, one typical scenario could be when we want to update the copyright notice in a collection of static HTML files.
2. Search and Replace With sed
The first command that we’re going to look at is sed, a powerful stream editor that is useful for performing basic transformations on an input stream. Take a look at our Guide to Stream Redirections in Linux for a refresher on what streams are.
Using sed, we’re able to quickly and easily find and replace a set of characters within a file.
Let’s begin by creating a test file to use in our examples. We’ll use a here document to create this test file quickly:
Let’s confirm that we created the test file successfully:
Now everything is as it should be, so let’s begin using sed.
2.1. Find and Replace the First Occurrence
We’re going to use sed to manipulate our test file and update the year to the current year.
An easy way to remember the syntax of the sed command for replacing the first occurrence of a word or term is:
Now let’s apply this command to our example:
And now, we print the contents of our file to confirm that we only replaced the first occurrence of “2019”:
Let’s break down our statement to get a deeper understanding of how this works:
- First, we pass the -i option to instruct sed to make the changes inside our test.txt file. By default, sed prints changes to the terminal
- Next is the regular expression that specifies what we want to change
- We start the expression with the letter “s” so that sed knows we’re performing a substitution
- Next up, we have the string “2019” which is the value for the OLD_TERM placeholder. This is the section of text that we want to replace
- Following that is the value of the NEW_TERM placeholder and that’s “2020”
- We separated these parameters using forward slashes (/)
As we can see, sed is a quick and easy way to perform search and replace operations.
2.2. Find and Replace All Occurrences
By default, sed only replaces the first occurrence that it finds. We can easily override this behavior to replace all instances within a file.
Let’s see the expression that we can use to achieve this:
We’ve added “g” to the end of our search expression. This instructs sed to replace all occurrences globally.
So let’s apply this to our text file:
Now let’s print out the contents of the test.txt again. This time we’ve replaced all occurrences of “2019” with “2020”:
3. Search and Replace With awk
In this section, we’ll take a look at awk. A powerful scripting language that is designed for text processing and is often used for data extraction and reporting purposes. As is common with other Linux utilities, awk can perform operations on both streams and files.
Let’s look at how awk performs find-and-replace operations on our test.txt sample file:
In this statement, we’re invoking awk and sending in a list of tasks for awk to perform.
Let’s break this down:
- The first task in our list is a substitute, which we write as sub
- Sub directs awk to find all occurrences of OLD_TERM and replace them with NEW_TERM
- Our next instruction directs awk to print the output to the standard output stream; the console
- Finally, we have the name of the file that awk will be working on
A notable difference with sed is that awk will not perform an in-place substitution; meaning that the updates will not be made inside the file.
We’ll tackle that shortly but for now, let’s see this in action:
Our resulting output confirms that the replacement has been carried out correctly:
As expected, awk has replaced all instances of “2019” with “2020”.
However, our output is directed to the console instead of updating our original file. We have a trick for taking care of that.
We’ll use our knowledge of streams and stream redirection to update our awk find-and-replace command and use it to update our original file in a single step:
This time nothing is printed to the console but let’s dump the contents of test.txt to see what’s happened:
Here we’re seeing that the output of the awk command was correctly redirected to our input file thereby updating it for us.
4. Conclusion
In this tutorial, we looked at how we could use two of the most common Linux utilities to find and replace a string of characters in a single file or a set of files without manually having to edit each individual file.
These utilities are extremely powerful and come in handy for a variety of day-to-day tasks whilst working on the Linux command line. Be sure to have a look at the documentation of these commands to learn more about them.
Find and replace text within a file using commands
How can I find and replace specific words in a text file using command line?
7 Answers 7
- sed = Stream EDitor
- -i = in-place (i.e. save back to the original file)
The command string:
- s = the substitute command
- original = a regular expression describing the word to replace (or just the word itself)
- new = the text to replace it with
- g = global (i.e. replace all and not just the first occurrence)
file.txt = the file name
There’s multitude of ways to achieve it. Depending on the complexity of what one tries to achieve with string replacement, and depending on tools with which user is familiar, some methods may be preferred more than others.
In this answer I am using simple input.txt file, which you can use to test all examples provided here. The file contents:
Bash isn’t really meant for text processing, but simple substitutions can be done via parameter expansion , in particular here we can use simple structure $
This small script doesn’t do in-place replacement, meaning that you would have to save new text to new file, and get rid of the old file, or mv new.txt old.txt
Side note: if you’re curious about why while IFS= read -r ; do . done is used, it’s basically shell’s way of reading file line by line. See this for reference.
AWK, being a text processing utility, is quite appropriate for such task. It can do simple replacements and much more advanced ones based on regular expressions. It provides two functions: sub() and gsub() . The first one only replaces only the first occurrence, while the second — replaces occurrences in whole string. For instance, if we have string one potato two potato , this would be the result:
AWK can take an input file as argument, so doing same things with input.txt , would be easy:
Depending on the version of AWK you have, it may or may not have in-place editing, hence the usual practice is save and replace new text. For instance something like this:
Sed is a line editor. It also uses regular expressions, but for simple substitutions it’s sufficient to do:
What’s good about this tool is that it has in-place editing, which you can enable with -i flag.
Perl is another tool which is often used for text processing, but it’s a general purpose language, and is used in networking, system administration, desktop apps, and many other places. It borrowed a lot of concepts/features from other languages such as C,sed,awk, and others. Simple substitution can be done as so:
Like sed, perl also has the -i flag.
Python
This language is very versatile and is also used in a wide variety of applications. It has a lot of functions for working with strings, among which is replace() , so if you have variable like var=»Hello World» , you could do var.replace(«Hello»,»Good Morning»)
Simple way to read file and replace string in it would be as so:
With Python, however, you also need to output to new file , which you can also do from within the script itself. For instance, here’s a simple one: