How to use sed command in linux?

SED is most commonly used command in UNIX it is known as stream editor. This command can be used to perform following different types of operations:

  • searching
  • find and replace
  • insertion
  • deletion

SED command in linux supports regular expression which allows you to perform complex pattern matching.

sed <options> <script> <input-file>

In order to learn this command better let's take following text as an example. Create a file called test.txt and add following contents:

Then he waddled away, waddle waddle
Then he waddled away, waddle waddle
Then he waddled away, waddle waddle

Using sed command to search and replace a string in a file

Let's replace word We in above example to Our using sed command as shown below:

# search waddle in test.txt and replace with toddle
sed 's/waddle/toddle/' test.txt

Output:

Then he toddled away, waddle waddle
Then he toddled away, waddle waddle
Then he toddled away, waddle waddle

Replacing the nth occurrence of a pattern in a line

Let say that in above example we only wanted to replace nth occurance in a line. Use following syntax:

sed 's/[search]/[replace]/[nth]' <file-name>

You can use following code to replace nth occurance in a line.

# search waddle in test.txt and replace with toddle
sed 's/waddle/toddle/2' test.txt

Output:

Then he waddled away, toddle waddle
Then he waddled away, toddle waddle
Then he waddled away, toddle waddle

Replacing from the nth occurrence till end of a pattern in a line

You can use following code to replace from nth occurance until the end of a line.

# search waddle in test.txt and replace with toddle
# this will start searching from 2nd occurance until end of the line
sed 's/waddle/toddle/2g' test.txt

Output:

Then he waddled away, toddle toddle
Then he waddled away, toddle toddle
Then he waddled away, toddle toddle

Replacing from the nth occurrence till end of a pattern on a specific line

You can use following code to replace from nth occurance until the end of a line for line number 3.

# search waddle in test.txt and replace with toddle
# this will start searching from 2nd occurance until end of the line
sed '3 s/waddle/toddle/2g' test.txt

Output:

Then he waddled away, waddle waddle
Then he waddled away, waddle waddle
Then he waddled away, toddle toddle

Followings are some more advanced examples:

Command Summary
sed 's/waddle/toddle/p' test.txt Duplicate the replaced lines
sed -n 's/waddle/toddle/p' test.txt Printing only the replaced lines
sed '1,2 s/waddle/toddle/' test.txt Replacing string for specific lines
sed '2,$ s/waddle/toddle/' test.txt Replace string from line 2 till end of the file
sed '2d' test.txt Delete the 2nd line
sed '$d' test.txt Delete the last line
sed '2,$d' test.txt Delete from 2nd till end of the file
sed '/^$/d' test.txt Delete all blank lines in a file
sed 's/^/.    /' test.txt Insert 5 spaces to the left of every lines
sed = test.txt | sed 'N;s/\n/ /' add line number in front of each line
sed '/./=' test.txt | sed '/./N; s/\n/ /' add line number in front of each line and avoid blank line number
sed '/waddle/d' test.txt delete all the line that contains word waddle
sed '3~2d' test.txt start from line 3 and delete every 2nd line
sed ‘/pattern/,+2d’ test.txt Delete the lines which matches the pattern and 2 lines after to that
sed -n 'x,yp' test.txt Viewing a file from x to y range
sed 'x,yd' test.txt View entire file except range x to y
sed -n '4'p test.txt Print nth line of the file
sed -n 'n,$'p test.txt Print from nth line until the end of the file
sed -n /pattern/p test.txt Print only lines that matches the pattern
sed -n ‘/pattern/,+xp’ test.txt Print the lines which matches the pattern up-to the next xth lines
sed G test.txt Double space line in a file
sed '/^$/d;G' test.txt Double space lines in a file there should not be more then one blank line
sed 'G;G' test.txt Triple space a file
sed 'n;d' test.txt Undo double-spacing (assumes even-numbered lines are always blank)
sed -n '$=' test.txt Count total lines in a file
sed 's/^[ \t]*//' test.txt Delete leading whitespace (spaces, tabs) from front of each line
sed 's/[ \t]*$//' test.txt Delete trailing whitespace (spaces, tabs) from end of each line
sed 's/^[ \t]*//;s/[ \t]*$//' test.txt Delete BOTH leading and trailing whitespace from each line
sed '/baz/s/foo/bar/g' test.txt Substitute "foo" with "bar" ONLY for lines which contain "baz"
sed '/baz/!s/foo/bar/g' test.txt Substitute "foo" with "bar" EXCEPT for lines which contain "baz"
sed '$!N;s/\n/ /' test.txt Join pair of lines together by removing newline
sed 2q test.txt Print first two lines of the file just like head command
sed -e :a -e '$q;N;11,$D;ba' test.txt Print last 10 lines of the file just like tail command
sed '/AAA/!d; /BBB/!d; /CCC/!d' test.txt Grep for AAA and BBB and CCC (in any order)
sed '/AAA.*BBB.*CCC/!d' test.txt Grep for AAA and BBB and CCC (in that order)
sed -n '/^.\{65\}/p' Print only lines of 65 characters or longer
sed -n '/Iowa/,/Montana/p' test.txt Print section of file between two regular expressions (inclusive),  case sensitive
sed '/Iowa/,/Montana/d' test.txt Print all of file EXCEPT section between 2 regular expressions
sed '$!N; /^\(.*\)\n\1$/!P; D' test.txt Delete duplicate, consecutive lines from a file, keep the first line as is

Reference: https://gist.github.com/ssstonebraker/6140154