Command-line Knowledge Base

sed

Use a sed script/file
sed -i -f commands.sed outputfile.txt
Ranges by patterns

You can specify two regular expressions as the range. Assuming a "#" starts a comment, you can search for a keyword, remove all comments until you see the second keyword. In this case the two keywords are "start" and "stop:"

sed '/start/,/stop/ s/#.*//'
sed '/<\/head>/,/.*<\/h1>/ d'
To add text to the end of a file [1][2]
sed -i -e "\$a some text" $file
Using variables in sed commands [3]

Use double quotes to get the shell to expand variables.

sed -i "s/$var1/ZZ/g" "$file"
Replace one or more digits [1]
sed 's/slide[0-9]*/slide/' links.html
Replace one or more digits [2] [5]
sed 's/ id="ingredient_[0-9]*"//' test.html
Replace newlines with a space [4]
sed ':a;N;$!ba;s/\n/ /g' file
Delete HTML Tags [6]
sed 's/<[^>]*>//g'
Simple Multiline Tag Matching

To replace:

<p>Some important paragraph text</p>
<ul>
<ul>
<ul>
<li>Some important list item</li>

With:

<p>Some important paragraph text</p>

<ul>
<li>Some important list item</li>

Use:

sed 'N;N;s/<\/p>\n<ul>\n<ul>/<\/p>\n/'
Delete two blank lines in a row [7]*

* This article does a good job of explaining how sed works. Also gives an example for deleting three blank lines in a row.

sed 'N;/^\n$/d;P:D' $file
Remove the first word of a line with sed (1/2) [8] [9]
root@linux:~# echo "first second third fourth fifth" | sed "s/^[^ ]* //"
second third fourth fifth
sed 's/\(^[^ ]\)* /<p>\1/' twelve_steps__multi_purpose.html
Replace an entire line by line number
sed '5s/.*/boo/' 00__template__recipe.php
Remove a Range of Lines 10
sed '4,7d' file
Escaping Characters 11

There are two levels of interpretation here: the shell, and sed.

In the shell, everything between single quotes is interpreted literally, except for single quotes themselves. You can effectively have a single quote between single quotes by writing '\'' (close single quote, one literal single quote, open single quote).

Sed uses basic regular expressions. In a BRE, in order to have them treated literally, the characters $.*[\^ need to be quoted by preceding them by a backslash, except inside character sets ([…]). Letters, digits and (){}+?| must not be quoted (you can get away with quoting some of these in some implementations). The sequences \(, \), \n, and in some implementations \{, \}, \+, \?, \| and other backslash+alphanumerics have special meanings. You can get away with not quoting $^ in some positions in some implementations.

Furthermore, you need a backslash before / if it is to appear in the regex outside of bracket expressions. You can choose an alternative character as the delimiter by writing, e.g., s~/dir~/replacement~ or \~/dir~p; you'll need a backslash before the delimiter if you want to include it in the BRE. If you choose a character that has a special meaning in a BRE and you want to include it literally, you'll need three backslashes; I do not recommend this, as it may behave differently in some implementations.

Match character repeated 'n' times 12
sed -E 's/y{4,}/SUCCESS!!!!/' file.txt
NULL