Stream Editor.doc

7/26/2019 Stream Editor.doc

1/60

Stream Editor - Quick Guide

Stream Editor - Overview

The acronym SED stands for Stream EDitor.It is a simple yet powerful utility thatparses the text and transforms it seamlessly. SED was developed during 197!7" #y $eeE. %c%ahon of &ell $a#s. Today' it runs on all ma(or operating systems.

%c%ahon wrote a general)purpose line)oriented editor' which eventually #ecame SED.SED #orrowed syntax and many useful features from ed editor. Since its #eginning' it hassupport for regular expressions. SED accepts inputs from files as well as pipes.*dditionally' it can also accept inputs from standard input streams.

SED is written and maintained #y the +ree Software +oundation ,+S+- and it is

distri#uted #y /0$inux. 2ence it is often referred to as GNU SED.To a novice user'the syntax of SED may loo3 cryptic. 2owever' once you get familiar with its syntax' youcan solve many complex tas3s with a few lines of SED script. This is the #eauty of SED.

Typical Uses of SED

SED can #e used in many different ways' such as4

Text su#stitution'

Selective printing of text files'

In)a)place editing of text files'

/on)interactive editing of text files' and many more.

Stream Editor - Environment

This chapter descri#es how to set up the SED environment on your /0$inux system.

Installation Using Package Manager

enerally' SED is availa#le #y default on most /0$inux distri#utions. 0se whichcommand to identify whether it is present on your system or not. If not' then install SED

on De#ian #ased /0$inux using aptpac3age manager as follows4

[jerry]$ sudo apt-get install sed

*fter installation' ensure that SED is accessi#le via command line.

[jerry]$ sed --versio


2/60

5n executing the a#ove code' you get the following result4

sed (GNU sed) 4.2.2Copyright (C) 2!2 "ree #ot%are "oundation& 'n.iense G*v+, GNU G* version + or later .his is ree sot%are you are ree to hange and redistri/ute it.

here is N0 133N& to the e5tent per6itted /y la%.1ritten /y 7ay "enlason& o6 ord& 8en *i99ini&and *aolo :on9ini.GNU sed ho6e page .General help using GNU sot%are .;-6ail /ug reports to .:e sure to inlude the %ord "ree #ot%are "oundation& 'n.his is ree sot%are? see the soure or opying onditions. here isN0%arranty? not even or @;3CAN:'' or "'N;## "03 *3'CU3*U3*0#;&

to the e5tent per6itted /y la%.GNU sed ho6e page .General help using GNU sot%are .;-6ail /ug reports to .:e sure to inlude the %ord


3/60

Decompress and extract the downloaded source code.

[jerry]$ tar 5v sed-4.2.2.tar./92

hange into the directory and run configure.

[jerry]$ .Bonigure

0pon successful completion' the configuregenerates %a3efile. To compile the

source code' issue a makecommand.

[jerry]$ 6ae

:ou can run the test suite to ensure the #uild is clean. This is an optional step.

[jerry]$ 6ae he

+inally' install the SED utility. %a3e sure you have superuser privileges.

[jerry]$ sudo 6ae install

That is it; :ou have successfully compiled and installed SED.


4/60

!ead4 SED reads a line from the input stream ,file' pipe' or stdin- and stores it in

its internal #uffer called pattern "uffer. E#ecute4 *ll SED commands are applied se=uentially on the pattern #uffer. &y

default' SED commands are applied on all lines ,glo#ally- unless line addressingis specified.

Dispa$4 Send the ,modified- contents to the output stream. *fter sending the

data' the pattern #uffer will #e empty.

The a#ove process repeats until the file is exhausted.

Points to Note

attern #uffer is a private' in)memory' volatile storage area used #y the SED.

&y default' all SED commands are applied on the pattern #uffer' hence the input

file remains unchanged. /0 SED provides a way to modify the input file in)a)place. 8e will explore a#out it in later sections.

There is another memory area called hod "ufferwhich is also private' in)

memory' volatile storage area. Data can #e stored in a hold #uffer for laterretrieval. *t the end of each cycle' SED removes the contents of the pattern #uffer#ut the contents of the hold #uffer remains persistent #etween SED cycles.2owever SED commands cannot #e directly executed on hold #uffer' hence SEDallows data movement #etween the hold #uffer and the pattern #uffer.

Initially #oth pattern and hold #uffers are empty.


5/60

If no input files are provided' then SED accepts input from the standard input

stream ,stdin-.

If address range is not provided #y default' then SED operates on each line.

Examples

$et us create a text file %uote.t#tto contain a =uote of the famous author aulo oelho.

[jerry]$ vi Duote.t5there is only one thing that 6aes a drea6 i6possi/le to ahieve theear o ailure.- *aulo Coelho& he lhe6ist

To understand the wor3flow of SED' let us display the contents of the file =uote.txt usingSED. This example simulates the catcommand.

[jerry]$ sed EE Duote.t5t

8hen the a#ove code is executed' it will produce the following result.

here is only one thing that 6aes a drea6 i6possi/le to ahieve theear o ailure.

In the a#ove example' =uote.txt is the input file name and #efore that there is a pair ofsingle =uote that implies the SED command. $et us demystify this operation.

+irst SED reads a line from the input file =uote.txt and stores it in its pattern #uffer. Then

it applies SED commands on the pattern #uffer. In our case' no SED commands are there'hence no operation is performed on the pattern #uffer. +inally it deletes and prints thecontents of the pattern #uffer on the standard output. Isn>t it simple?

In the following example' SED accepts input from the standard input stream.

[jerry]$ sed EE




2ere' the first line is entered through 3ey#oard and the second is the output generated #ySED. To exit from the SED session' press ctrl)D ,@D-.

Stream Editor - &asic S$nta#


6/60

This chapter introduces the #asic commands that SED supports and their command)linesyntax. SED can #e invo3ed in the following two forms4

sed [-n] [-e] Eo66and(s)E ilessed [-n] - sriptile iles

The first form allows to specify the commands in)line and they are enclosed within single=uotes. The later allows to specify a script file that contains SED commands. 2owever'we can use #oth forms together multiple times. SED provides various command)lineoptions to control its #ehavior.

$et us see how we can specify multiple SED commands. SED provides the deetecommand to delete certain lines. $et us delete the 1st' And' and Bth lines. +or the time#eing' ignore all the details of the delete command. 8e will discuss more a#out the deletecommand later.

+irst' display the file contents using the catcommand.

[jerry]$ at /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

/ow instruct SED to remove only certain lines. 2ere' to delete three lines' we havespecified three separate commands with )e option.

[jerry]$ sed -e E!dE -e E2dE -e EdE /oos.t5t


+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2F) Ga6e o hrones& George 3. 3. @artin& IF4

*dditionally' we can write multiple SED commands in a text file and provide the text file

as an argument to SED. SED can apply each command on the pattern #uffer. Thefollowing example illustrates the second form of SED.

+irst' create a text file containing SED commands. +or easy understanding' let us use thesame SED commands.

[jerry]$ eho -e


7/60


!d2dd

/ow instruct the SED to read commands from the text file. 2ere' we achieve the sameresult as shown in the a#ove example.

[jerry]$ sed - o66ands.t5t /oos.t5t


+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2F) Ga6e o hrones&George 3. 3. @artin& IF4

Standard Options

SED supports the following standard options4

)n4 Default printing of pattern #uffer. +or example' the following SED command

does not show any output4

[jerry]$ sed -n EE Duote.t5t

)e 4 /ext argument is an editing command. 2ere' angular #rac3ets imply

mandatory parameter. &y using this option' we can specify multiple commands.$et us print each line twice4

[jerry]$ sed -e EE -e EpE Duote.t5t


here is only one thing that 6aes a drea6 i6possi/le to ahieve theear o ailure.here is only one thing that 6aes a drea6 i6possi/le to ahieve theear o ailure.- *aulo Coelho& he lhe6ist- *aulo Coelho& he lhe6ist

)f 4 /ext argument is a file containing editing commands. The angular #rac3etsimply mandatory parameter. In the following example' we specify print commandthrough file4

[jerry]$ eho


8/60

here is only one thing that 6aes a drea6 i6possi/le to ahieve theear o ailure.- *aulo Coelho& he lhe6ist

GNU Specific Options

$et us =uic3ly go through the /0 specific SED options. /ote that these options are/0 specificC and may not #e supported #y other variants of the SED. In later sections'we will discuss these options in more details.

)n' ))=uiet' ))silent4 Same as standard )n option.

)e script' ))expressionscript4 Same as standard )e option.

)f script)file' ))filescript)file4 Same as standard )f option.

))follow)symlin3s4 If this option is provided' the SED follows sym#olic lin3s

while editing files in place.

)iS0++IFG' ))in)placeS0++IFG4 This option is used to edit file in place. If

suffix is provided' it ta3es a #ac3up of the original file' otherwise it overwrites theoriginal file.

)l /' ))line)lenght/4 This option sets the line length for l command to /

characters.

))posix4 This option disa#les all /0 extensions.

)r' ))regexp)extended4 This option allows to use extended regular expressions

rather than #asic regular expressions.

)u' ))un#uffered4 8hen this option is provided' the SED loads minimal amount of

data from the input files and flushes the output #uffers more often. It is useful forediting the output of Htail )fH when you do not want to wait for the output.

)' ))null)data4 &y default' the SED separates each line #y a new)line character. If

/0$$)data option is provided' it separates the lines #y /0$$ characters.

Stream Editor - 'oops

$i3e other programming languages' SED too provides a looping and #ranching facility tocontrol the flow of execution. In this chapter' we are going to explore more a#out how touse loops and #ranches in SED.

* loop in SED wor3s similar to a gotostatement. SED can (ump to the line mar3ed #ythe la#el and continue executing the remaining commands. In SED' we can define a a"eas follows4


9/60

la/elstartendup

In the a#ove example' a name after colon,4- implies the la#el name.

To (ump to a specific la#el' we can use the "command followed #y the la#el name. If thela#el name is omitted' then the SED (umps to the end of the SED file.

$et us write a simple SED script to understand the loops and #ranches. In our #oo3s.txtfile' there are several entries of #oo3 titles and their authors. The following examplecom#ines a #oo3 title and its author name in one line separated #y a comma. Then itsearches for the pattern HauloH. If the pattern matches' it prints a hyphen,)- in front ofthe line' otherwise it (umps to the (rintla#el which prints the line.

[jerry]$ sed -n E

h?n?A?5sBJnB& BB*auloBL/ *rintsBMB- B*rintpE /oos.t5t


#tor6 o #%ords& George 3. 3. @artinhe %o o%ers& 7. 3. 3. olienhe lhe6ist& *aulo Coelhohe "ello%ship o the 3ing& 7. 3. 3. olien

- he *ilgri6age& *aulo Coelho Ga6e o hrones& George 3. 3. @artin

*t first glance' the a#ove script may loo3 cryptic. $et us demystify this.

The first two commands are self)explanatory h)n)*)#and s+,n+ +com#ine the

#oo3 title and its author separated #y a comma,'-. The third command (umps to the la#el (rintonly when the pattern does not

match' otherwise su#stitution is performed #y the fourth command.

(rintis (ust a la#el name and as you already 3now' pis the print command.

To improve reada#ility' each SED command is placed on a separate line. 2owever' onecan choose to place all the commands in one line as follows4

[jerry]$ sed -n Eh?n?A?5?sBJnB& B?B*auloBL/ *rint? sBMB- B? *rint?pE/oos.t5t



10/60

#tor6 o #%ords& George 3. 3. @artinhe %o o%ers& 7. 3. 3. olienhe lhe6ist& *aulo Coelhohe "ello%ship o the 3ing& 7. 3. 3. olienhe *ilgri6age& *aulo Coelho Ga6e o hrones& George 3. 3. @artin

Stream Editor - &ranches

&ranches can #e created using the t command. The tcommand (umps to the la#el only ifthe previous su#stitute command was successful. $et us ta3e the same example as in theprevious chapter' #ut instead of printing a single hyphen,)-' now we print four hyphens.The following example illustrates the usage of the tcommand.

[jerry]$ sed -n Eh?n?A?5sBJnB& B

oopB*auloBsBMB-BB----BLt ooppE /oos.t5t


#tor6 o #%ords& George 3. 3. @artinhe %o o%ers& 7. 3. 3. olien----he lhe6ist& *aulo Coelhohe "ello%ship o the 3ing& 7. 3. 3. olien----he *ilgri6age& *aulo Coelho Ga6e o hrones& George 3. 3. @artin

In the a#ove example' the first two commands are self)explanatory. The third commanddefines a la#el 'oop. The fourth command prepends hyphen,)- if the line contains thestring HauloH and the tcommand repeats the procedure until there are four hyphens atthe #eginning of the line.

To improve reada#ility' each SED command is written on a separate line. 5therwise' wecan write a one)liner SED as follows4

[jerry]$ sed -n Eh?n?A?5? sBJnB& B? oop?B*auloBsBMB-B? B----BLt oop?pE /oos.t5t


#tor6 o #%ords& George 3. 3. @artinhe %o o%ers& 7. 3. 3. olien----he lhe6ist& *aulo Coelhohe "ello%ship o the 3ing& 7. 3. 3. olien----he *ilgri6age& *aulo Coelho Ga6e o hrones& George 3. 3. @artin


11/60

Stream Editor - (attern &uffer

5ne of the #asic operations we perform on any file is display its contents. +or thispurpose' we can use the printcommand which prints the contents of the pattern #uffer.

So let us learn more a#out the pattern #uffer

+irst create a file containing the line num#er' the name of the #oo3' its author' and thenum#er of pages. In this tutorial' we will #e using this file. :ou can use any text fileaccording to your convenience. 5ur text file will loo3 li3e this4

[jerry]$ vi /oos.t5t!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho&2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

/ow' let us print the file contents.

[jerry]$ sed EpE /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +22) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H

+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+24) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2II) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4F) Ga6e o hrones& George 3. 3. @artin& IF4

:ou might wonder why each line is #eing displayed twice. $et us find out.

Do you remem#er the wor3flow of SED? &y default' SED prints the contents of thepattern #uffer. In addition' we have included a print command explicitly in our command

section. 2ence each line is printed twice. &ut don>t worry. SED has the -noption tosuppress the default printing of the pattern #uffer. The following command illustratesthat.

[jerry]$ sed -n EpE /oos.t5t



12/60


ongratulations; we got the expected result. &y default' SED operates on all lines. &utwe can force SED to operate only on certain lines. +or instance' in the example #elow'SED only operates on the rd line. In this example' we have specified an address range#efore the SED command.

[jerry]$ sed -n E+pE /oos.t5t


+) he lhe6ist& *aulo Coelho& !>H

*dditionally' we can also instruct SED to print only certain lines. +or instance' thefollowing code prints all the lines from A to B. 2ere we have used the comma,'- operatorto specify the address range.

[jerry]$ sed -n E2& pE /oos.t5t


2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2II

There is also a special character Dollar,J- which represents the last line of the file. So letus print the last line of the file.

[jerry]$ sed -n E$ pE /oos.t5t


F) Ga6e o hrones& George 3. 3. @artin& IF4

2owever we can also use Dollar,J- character to specify address range. &elow example

prints through line to last line.

[jerry]$ sed -n E+&$ pE /oos.t5t


+) he lhe6ist& *aulo Coelho& !>H 4) he "ello%ship o the 3ing& 7. 3.3. olien& 4+2 ) he *ilgri6age& *aulo Coelho& 2II F) Ga6e ohrones& George 3. 3. @artin& IF4


13/60

8e learnt how to specify an address range using the comma,'- operator. SED supportstwo more operators that can #e used to specify address range. +irst is the plus,K- operatorand it can #e used with the comma,'- operator. +or instance / 0nwill print the next nlines starting from line num#er /. Sounds confusing? $et us chec3 it with a simpleexample. The following example prints the next " lines starting from line num#er A.

[jerry]$ sed -n E2&,4 pE /oos.t5t


2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

5ptionally' we can also specify address range using the tilde,L- operator. It uses /1n

form. It indicates that SED should start at line num#er % and process every n,th- line.+or instance' 2312matches line num#er BM' BB' NM' NB' and so on. $et us print only oddlines from the file.

[jerry]$ sed -n E!2 pE /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F+) he lhe6ist& *aulo Coelho& !>H) he *ilgri6age& *aulo Coelho& 2II

The following code prints only even lines from the file.

[jerry]$ sed -n E22 pE /oos.t5t


2) he %o o%ers& 7. 3. 3. olien& +24) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2F) Ga6e o hrones& George 3. 3. @artin& IF4

Stream Editor - (attern !ange

In the previous chapter' we learnt how SED handles an address range. This chapter covershow SED ta3es care of a pattern range. * pattern range can #e a simple text or a complexregular expression. $et us ta3e an example. The following example prints all the #oo3s ofthe author aulo oelho.

[jerry]$ sed -n EB*auloB pE /oos.t5t


14/60


+) he lhe6ist& *aulo Coelho& !>H) he *ilgri6age& *aulo Coelho& 2II

In the a#ove example' the SED operates on each line and prints only those lines thatmatch the string aulo.

8e can also com#ine a pattern range with an address range. The following example printslines starting with the first match of *lchemist until the fifth line.

[jerry]$ sed -n EBlhe6istB& pE /oos.t5t


+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2

) he *ilgri6age& *aulo Coelho& 2II

8e can use the Dollar,J- character to print all the lines after finding the first occurrenceof the pattern. The following example finds the first occurrence of the pattern The andimmediately prints the remaining lines from the file

[jerry]$ sed -n EBheB&$ pE /oos.t5t


2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H

4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

8e can also specify more than one pattern ranges using the comma,'- operator. Thefollowing example prints all the lines that exist #etween the patterns Two and ilgrimage.

[jerry]$ sed -n EB%oB& B*ilgri6ageB pE /oos.t5t


2) he %o o%ers& 7. 3. 3. olien& +2

+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2II

*dditionally' we can use the plus,K- operator within a pattern range. The followingexample finds the first occurrence of the pattern Two and prints the next " lines after that.

[jerry]$ sed -n EB%oB& ,4 pE /oos.t5t


15/60


2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2II


8e have supplied here only a few examples to get you ac=uainted with SED. :ou canalways get to 3now more #y trying a few examples on your own.

Stream Editor - &asic 4ommands

This chapter descri#es several useful SED commands.

Delete Command

SED provides various commands to manipulate text. $et us first explore a#out the deetecommand. 2ere is how you execute a delete command4

[address![&address2]]d

address5and address6are the starting and the ending addresses respectively' which can#e either line num#ers or pattern strings. &oth of these addresses are optional parameters.

*s the name suggests' the delete command is used to perform delete operation and sincethe SED operates on line' we can say that this command is used to delete lines. /ote that

the delete command removes lines only from the pattern #ufferC the line is not sent to theoutput stream and the original file remains unchanged. The following example illustratesthe point.

[jerry]$ sed EdE /oos.t5t

&ut where is the output? If no line address is provided' then the SED operates on everyline #y default. 2ence' it deletes all the lines from the pattern #uffer. That is why thecommand does not print anything on the standard output.

$et us instruct the SED to operate only on certain lines. The following example removes

the "th line only.

[jerry]$ sed E4dE /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H


16/60

) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

*dditionally' SED also accepts address range using comma,'-. 8e can instruct the SED toremove /1 to /A lines. +or instance' the following example deletes all the lines from Athrough ".

[jerry]$ sed E2& 4 dE /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

SED>s address range is not only limited to num#ers. 8e can also specify patterns as anaddress. The following example removes all the #oo3s of the author aulo oelho.

[jerry]$ sed EB*aulo CoelhoBdE /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +24) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2F) Ga6e o hrones& George 3. 3. @artin& IF4

8e can also specify an address range using textual pattern. The following exampleremoves all lines #etween the patterns Stormand 7eowship.

[jerry]$ sed EB#tor6B&B"ello%shipBdE /oos.t5t) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

In addition to this' we can also use dollar,J-' plus,K-' and tilde,L- operators with SED.

rite Command

5ne of the important operations we perform on any file is #ac3up' i.e.' we ma3e anothercopy of the file. SED provides the writecommand to store the contents of the pattern#uffer in a file. iven #elow is the syntax of the writecommand which is similar to thedeetecommand.

[address![&address2]]% ile

2ere' address5and address6are the starting and the ending address respectively' whichcan #e either line num#ers or pattern strings. &oth of these addresses are optionalparameters.


17/60

In the a#ove syntax' wrefers to the write command and fieis the file name in which youstore contents. &e careful with the fieparameter. 8hen a file name is provided' the SEDcreates a file on the fly if it is not present' and overwrites it if it is already present.

$et us ma3e an exact copy of the file using SED. /ote that there must #e exactly one

space #etween wand fie.

[jerry]$ sed -n E% /oos./aE /oos.t5t

8e created another file called "ooks."ak./ow verify that #oth the files have identicalcontent.

[jerry]$ di /oos.t5t /oos./a[jerry]$ eho $O


:ou may assume that the cpcommand does exactly the same thing. :es; The cpcommand does the same thing' #ut SED is a matured utility. It allows creating a filecontaining only certain lines from the source file. $et us store only even lines to anotherfile.

[jerry]$ sed -n E22 % jun.t5tE /oos.t5t[jerry]$ at jun.t5t


2) he %o o%ers& 7. 3. 3. olien& +24) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2F) Ga6e o hrones& George 3. 3. @artin& IF4

:ou can also use comma,'-' dollar,J-' and plus,K- operators with the write command.

In addition to this' SED also supports pattern matching with the write command. Supposeyou want to store all the #oo3s of individual authors into a separate file. 5ne #oring andlengthy way is do it manually' and the smarter way is to use SED.

[jerry]$ sed -n -e EB@artinB % @artin.t5tE -e EB*auloB % *aulo.t5tE -e

EBolienB %olien.t5tE /oos.t5t

In the a#ove example' we are matching each line against a pattern and storing thematched line in a particular file. It is very simple. To specify multiple commands' weused -eswitch of the SED command. /ow let use see what each file contains4

[jerry]$ at @artin.t5t


18/60


!) #tor6 o #%ords& George 3. 3. @artin& !2!FF) Ga6e o hrones& George 3. 3. @artin& IF4

$et us display the file contents.

[jerry]$ at *aulo.t5t


+) he lhe6ist& *aulo Coelho& !>H) he *ilgri6age& *aulo Coelho& 2II

$et us display the file contents.

[jerry]$ at olien.t5t


2) he %o o%ers& 7. 3. 3. olien& +24) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2

Excellent; 8e got the expected result. SED is really an amaing utility.

!ppend Command

5ne of the most useful operations of any text editor is to provide append functionality.SED supports this operation through its append command. iven #elow is the syntax ofappend4

[address]aJppend te5t

$et us append a new #oo3 entry after line num#er ". The following example shows howto do it

[jerry]$ sed E4 a H) dultry& *aulo Coelho& 2+4E /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2H) dultry& *aulo Coelho& 2+4) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4


19/60

In the command section' 8implies the line num#er' ais the append command' and theremaining part is the text to #e appended.

$et us insert a text line at the end of the file. To do this' use 9as the address. Thefollowing example illustrates this4

[jerry]$ sed E$ a H) dultry& *aulo Coelho& 2+4E /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4H) dultry& *aulo Coelho& 2+4

*part from line num#er' we can also specify an address using textual pattern. +orinstance' the following example appends text after matching the string :he ;chemist.

[jerry]$ sed EBhe lhe6istB a H) dultry& *aulo Coelho& 2+4E /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>HH) dultry& *aulo Coelho& 2+44) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2

) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

/ote that if there are multiple patterns matching' then the text is appended after eachmatch. The following example illustrates this scenario.

[jerry]$ sed EBheB a H) dultry& *aulo Coelho& 2+4E /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2

H) dultry& *aulo Coelho& 2+4+) he lhe6ist& *aulo Coelho& !>HH) dultry& *aulo Coelho& 2+44) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2H) dultry& *aulo Coelho& 2+4) he *ilgri6age& *aulo Coelho& 2IIH) dultry& *aulo Coelho& 2+4F) Ga6e o hrones& George 3. 3. @artin& IF4


20/60

C"ange Command

SED provides changeor repacecommand which is represented #y c. This commandhelps replace an existing line with new text. 8hen line range is provided' all the lines arereplaced as a group #y a single text line. iven #elow is the syntax of the change

command4

[address![&address2]]J3eplae te5t

$et us replace the third line with some other text.

[jerry]$ sed E+ +) dultry& *aulo Coelho& +24E /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F

2) he %o o%ers& 7. 3. 3. olien& +2+) dultry& *aulo Coelho& +244) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

SED also accepts patterns as an address. In the following example' a line is replacedwhen the pattern match succeeds.

[jerry]$ sed EBhe lhe6istB +) dultry& *aulo Coelho& +24E /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) dultry& *aulo Coelho& +244) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

SED also allows replacement of multiple lines with a single line. The following exampleremoves lines from fourth through sixth and replaces them with new text.

[jerry]$ sed E4& F 4) dultry& *aulo Coelho& +24E /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) dultry& *aulo Coelho& +24


21/60

Insert Command

The insert command wor3s much in the same way as append does. The only difference isthat it inserts a line #efore a specific position. iven #elow is the syntax of the insertcommand4

[address]iJ'nsert te5t

$et us understand the insert command with some examples. The following commandinserts a new entry #efore the fourth line.

[jerry]$ sed E4 i H) dultry& *aulo Coelho& +24E /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F

2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>HH) dultry& *aulo Coelho& +244) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

In the a#ove example' 8is the location num#er' iimplies the insert command' and theremaining part is the text to #e inserted.

To insert text at the start of a file' provide the line address as 5. The following commandillustrates this4

[jerry]$ sed E! i H) dultry& *aulo Coelho& +24E /oos.t5t


H) dultry& *aulo Coelho& +24!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

*dditionally' we can insert multiple lines. The following command inserts two lines#efore the last line.

[jerry]$ sed E$ i H) dultry& *aulo Coelho& +24J


I) ;leven @inutes& *aulo Coelho& +4E /oos.t5t


22/60

!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age&*aulo Coelho& 2IIH) dultry& *aulo Coelho& +24I) ;leven @inutes& *aulo Coelho& +4F) Ga6e o hrones& George 3. 3. @artin& IF4

/ote that the entries to #e inserted are entered on separate lines and delimited #y the#ac3slash,O- character.

Translate Command

SED provides a command to translate characters and it is represented as $. It transformsthe characters #y position. iven #elow is the syntax of the translate command4

[address![&address2]]yBlist-!Blist-2B

/ote that translation is #ased on the position of the character from ist 5to the characterin the same position in ist 6and #oth lists must #e explicit character lists. 6egularexpressions and character classes are unsupported. *dditionally' the sie of ist 5and ist6must #e same.

The following example converts *ra#ic num#ers to 6oman num#ers.

[jerry]$ eho

' Q 'Q RR

l command

an you differentiate #etween words separated #y spaces and words separated #y ta#characters only #y loo3ing at them? ertainly not. &ut SED can do this for you. SED usesthe command to display hidden characters in the text. +or example' ta# character with ,tand End)5f)$ine with 9character. iven #elow is the syntax of the command.

[address![&address2]]l[address![&address2]]l [len]

$et us create a file with ta# characters for demonstration. +or simplicity' we are going touse the same file' (ust #y replacing spaces with ta#s. 8ait; &ut how to do that P #yopening the file in a text editor and replacing each space with ta#? ertainly not; 8e canma3e use of SED commands for that.

[jerry]$ sed EsB BJtBgE /oos.t5t K jun.t5t


23/60

/ow let us display the hidden characters #y using the command4

[jerry]$ sed -n ElE jun.t5t


!)JtJt#tor6JtoJt#%ords&GeorgeJt3.Jt3.Jt@artin&!2!F$2)JtheJt%oJto%ers&7.Jt3.Jt3.Jtolien&+2$+)JtheJtlhe6ist&*auloJtCoelho&!>H$4)JtheJt"ello%shipJtoJttheJt3ing&7.Jt3.Jt3.Jtolien&4+2$)JtheJt*ilgri6age&*auloJtCoelho&2II$F)JtJtGa6eJtoJthrones&GeorgeJt3.Jt3.Jt@artinJt&IF4$

$i3e other SED commands' it also accepts line num#ers and patterns as an address. :oucan try it yourselves.

$et us ta3e a close loo3 at another interesting feature of SED. 8e can instruct the SED toperform line wrapping after a certain num#er of characters. The following example wrapslines after AB characters.

[jerry]$ sed -n El 2E /oos.t5t


!) #tor6 o #%ords&GeoJrge 3. 3. @artin&!2!F$2) he %o o%ers&7. 3. J3. olien&+2$+) he lhe6ist&*aulo CJoelho&!>H$

4) he "ello%ship o theJ3ing&7. 3. 3. olien&4J

+2$) he *ilgri6age&*aulo JCoelho&2II$F) Ga6e o hrones&GeoJrge 3. 3. @artin &IF4$

/ote that in the a#ove example' wrap limit is provided after l command. In this case' it isAB characters. This option is /0 specific and may not wor3 with other variants of theSED.

* wrap limit of M means never #rea3 the line unless there is a new line character. Thefollowing simple command illustrates this.

[jerry]$ sed -n El E /oos.t5t


!) #tor6 o #%ords&George 3. 3. @artin&!2!F$2) he %o o%ers&7. 3. 3. olien&+2$


24/60

+) he lhe6ist&*aulo Coelho&!>H$4) he "ello%ship o the 3ing&7. 3. 3. olien&4+2$) he *ilgri6age&*aulo Coelho&2II$F) Ga6e o hrones&George 3. 3. @artin&IF4$

#uit Command

Quit command instructs the SED to =uit the current execution flow. It is represented #ythe %command. iven #elow is the syntax of the =uit command4

[address]D[address]D [value]

/ote that the =uit command does not accept range of addresses' it only supports a singleaddress. &y default' SED follows read' execute' and repeat wor3flowC #ut when the =uitcommand is encountered' it simply stops the current execution.

$et us print the first lines from the file.

[jerry]$ sed E+ DE /oos.t5t



In addition to line num#er' we can also use textual patterns. The following command=uits when pattern match succeeds.

[jerry]$ sed EBhe lhe6istB DE /oos.t5t



In addition to this' SED can also accept a vauewhich can #e used as the exit status. Thefollowing command shows its exit status as 1MM.

[jerry]$ sed EBhe lhe6istB D !E /oos.t5t




25/60

/ow let us verify the exit status.

[jerry]$ eho $O


!

$ead Command

8e can instruct the SED to read the contents of a file and display them when a specificcondition matches. The command is represented #y the alpha#et r. iven #elow is thesyntax of the read command.

[address]r ile

/ote that there must #e exactly one space #etween the rcommand and the file name.

$et us understand it with a simple example. reate a sample file called


26/60

his is jun te5t.4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2his is jun te5t.) he *ilgri6age& *aulo Coelho& 2IIhis is jun te5t.F) Ga6e o hrones& George 3. 3. @artin& IF4

$i3e other SED commands' the read command also accepts pattern as an address. +orinstance' the following command inserts the contents of '# 2!4+) he lhe6ist& *aulo Coelho& !>H

4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

$i3e other commands' it also accepts patterns as an address. +or example' the followingexample executes datecommand when a pattern match succeeds. /ote that after eachpattern match' first the command is executed and then the contents of the pattern #ufferare displayed.


27/60

[jerry]$ sed EB*auloB e dateE /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2

#un #ep H !IF4 '# 2!4+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2#un #ep H !IF4 '# 2!4) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

If you o#serve the syntax of the ecommand carefully' you will notice that commandisoptional. 8hen no command is provided after eit treats the contents of the pattern #ufferas an external command. To illustrate this' let us create a commands.txt file with a fewsimple commands.

[jerry]$ eho -e


28/60

Miscellaneous Commands

&y default' SED operates on single line' however it can operate on multiple lines as well.%ulti)line commands are denoted #y uppercase letters. +or example' unli3e the ncommand' the Ncommand does not clear and print the pattern space. Instead' it adds a

newline ,On- at the end of the current pattern space and appends the next line from theinput)file to the current pattern space and continues with the SED>s standard flow #yexecuting the rest of the SED commands. iven #elow is the syntax of the Ncommand.

[address![&address2]]N

$et us print a comma)separated list of #oo3 titles and their respective authors. Thefollowing example illustrates this.

[jerry]$ sed EN? sBJnB& BgE /oos.t5t


#tor6 o #%ords& George 3. 3. @artinhe %o o%ers& 7. 3. 3. olienhe lhe6ist& *aulo Coelhohe "ello%ship o the 3ing& 7. 3. 3. olienhe *ilgri6age& *aulo Coelho Ga6e o hrones& George 3. 3. @artin

$et us understand how the a#ove example wor3s. The Ncommand reads the first line'i.e.'* Storm of Swords into the pattern #uffer and appends On followed #y the next line.The pattern space now contains * Storm of Swords,neorge 6. 6. %artin. In the nextstep' we are replacing the newline with a comma.

$i3e pcommand' we have a (command to print the first part ,up to em#edded newline-of the multi)line pattern space created #y the Ncommand. iven #elow is the syntax ofthe (command which is similar to the pcommand.

[address![&address2]]*

In the previous example' we saw that the Ncommand creates a newline) separated list of#oo3 titles and their authors. $et us print only the first part of it' i.e.' only the titles of the#oo3. The following command illustrates this.

[jerry]$ sed -n EN?*E /oos.t5t


#tor6 o #%ordshe %o o%ershe lhe6isthe "ello%ship o the 3inghe *ilgri6age


29/60

Ga6e o hrones

/ote that in the a#sence of N' it #ehaves same as the pcommand. The following simplecommand illustrates this scenario.

[jerry]$ sed -n E*E /oos.t5t


#tor6 o #%ordsGeorge 3. 3. @artinhe %o o%ers7. 3. 3. olienhe lhe6ist*aulo Coelhohe "ello%ship o the 3ing7. 3. 3. olienhe *ilgri6age*aulo Coelho Ga6e o hronesGeorge 3. 3. @artin

In addition to this' SED also provides a vcommand which chec3s for version. If theprovided version is greater than the installed SED version' then the command executionfails. /ote that this option is /0 specific and may not wor3 with other variants of SED.iven #elow is the syntax of the vcommand.

[address![&address2]]v [version]

+irst' find out the current version of SED.

[jerry]$ sed --version


sed (GNU sed) 4.2.2

In the following example' the SED version is greater than version ".A.A' hence the SEDcommand a#orts its execution.

[jerry]$ sed Ev 4.2.+E /oos.t5t


sed -e e5pression =!& har H e5peted ne%er version o sed

&ut if the provided version is lesser than or e=ual to version ".A.A' then the commandwor3s as expected.

[jerry]$ sed Ev 4.2.2E /oos.t5t


30/60


#tor6 o #%ordsGeorge 3. 3. @artinhe %o o%ers7. 3. 3. olien

he lhe6ist*aulo Coelhohe "ello%ship o the 3ing7. 3. 3. olienhe *ilgri6age*aulo Coelho Ga6e o hrones George 3. 3. @artin

Stream Editor - Specia 4haracters

SED provides two special characters which are treated as commands. This chapter

illustrates the usage of these two special characters.

& Command

The HH command deals with line num#ers. iven #elow is the syntax of the HHcommand4

[BpatternB]S[address![&address2]]S

The command writes the line num#er followed #y its contents on the standard outputstream. The following example illustrates this.

[jerry]$ sed ESE /oos.t5t


!!) #tor6 o #%ords& George 3. 3. @artin& !2!F22) he %o o%ers& 7. 3. 3. olien& +2++) he lhe6ist& *aulo Coelho& !>H4

4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIFF) Ga6e o hrones& George 3. 3. @artin& IF4

$et us print the line num#ers and the contents of the first four lines. The followingcommand prints the first four lines with line num#ers and the remaining without linenum#ers.


31/60

[jerry]$ sed E!& 4SE /oos.t5t


!!) #tor6 o #%ords& George 3. 3. @artin& !2!F

22) he %o o%ers& 7. 3. 3. olien& +2++) he lhe6ist& *aulo Coelho& !>H44) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

*dditionally' we can instruct the SED to print line num#ers when a pattern matchsucceeds. The following example prints the line num#er that contains the pattern HauloH.

[jerry]$ sed EB*auloB SE /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2++) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age& *aulo Coelho& 2IIF) Ga6e o hrones& George 3. 3. @artin& IF4

an you guess what the following SED command does?

[jerry]$ sed -n E$ SE /oos.t5t


F

:es' you are right. It counts the total num#er of lines present in the file. $et us demystifythe code. In the command section' we used HJ H which prints the line num#er of the lastline followed #y its contents. &ut we also provided the -n flag which suppresses the

default printing of the pattern #uffer. 2ence' only the last line num#er is displayed.

' Command

SED supports the special character R. 8henever a pattern match succeeds' this specialcharacter stores the matched pattern. It is often used with the su#stitution command. $etus see how we can leverage this efficient feature.


32/60

Each line in the #oo3.txt file is num#ered. $et us add the words &ook num"erat the#eginning of each line. The following example illustrates this.

[jerry]$ sed EsB[[digit]]B:oo nu6/er TBE /oos.t5t


:oo nu6/er !) #tor6 o #%ords& George 3. 3. @artin& !2!F:oo nu6/er 2) he %o o%ers& 7. 3. 3. olien& +2:oo nu6/er +) he lhe6ist& *aulo Coelho& !>H:oo nu6/er 4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2:oo nu6/er ) he *ilgri6age& *aulo Coelho& 2II:oo nu6/er F) Ga6e o hrones& George 3. 3. @artin& IF4

This example is very simple. +irst' we search for the first occurrence of a digit' which isthe line num#er ,that is why we used 4digit4GG- and the SED automatically stores thematched pattern in the special character R. In the second step' we insert the words &ooknum"er#efore each matched pattern' i.e.' #efore every line.

$et us ta3e another example. In the #oo3.txt file' the last digit implies the num#er ofpages of the #oo3. $et us add Hages H #efore that. To do this' find the last occurrence ofthe digit and replace it with Hages RH. 2ere' R stores the matched pattern' i.e.' thenum#er of pages

[jerry]$ sed EsB[[digit]]$B*ages S TBE /oos.t5t

5n executing the a#ove syntax' you get the following result4

!) #tor6 o #%ords& George 3. 3. @artin& *ages S !2!F

2) he %o o%ers& 7. 3. 3. olien& *ages S +2+) he lhe6ist& *aulo Coelho& *ages S !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& *ages S 4+2) he *ilgri6age& *aulo Coelho&*ages S 2IIF) Ga6e o hrones& George 3. 3. @artin& *ages S IF4

+or the time #eing' (ust remem#er that ==digit>>?9finds the last occurrence of the digit.In the chapter H6egular Expressions' we will explore more a#out regular expressions.

Stream Editor - Strings

Su(stitute Command

Text su#stitution operations li3e Hfind and replaceH are common in any text editor. In thissection' we illustrate how SED performs text su#stitution. iven #elow is the syntax ofthe su#stitution command.

[address![&address2]]sBpatternBreplae6entB[lags]


33/60

2ere' address5and address6are the starting and ending addresses respectively' whichcan #e either line num#ers or pattern strings. &oth these addresses are optionalparameters. The pattern is the text which we want to replace with the replacement string.*dditionally' we can specify optional flags with the SED.

In the #oo3s.txt file' we have used comma,'- to separate each column. $et us use vertical#ar,- to separate each column. To do this' replace comma,'- with vertical #ar,-.

[jerry]$ sed EsB&B P BE /oos.t5t


!) #tor6 o #%ords P George 3. 3. @artin& !2!F2) he %o o%ers P 7. 3. 3. olien& +2+) he lhe6ist P *aulo Coelho& !>H4) he "ello%ship o the 3ing P 7. 3. 3. olien& 4+2) he *ilgri6age P *aulo Coelho& 2IIF) Ga6e o hrones P George 3. 3. @artin& IF4

If you o#serve carefully' only the first comma is replaced and the second remains as it is.8hy? *s soon as the pattern matches' SED replaces it with the replacement string andmoves to the next line. &y default' it replaces only the first occurrence. To replace alloccurrences' use the glo#al flag ,g- with SED as follows4

[jerry]$ sed EsB&B P BgE /oos.t5t


!) #tor6 o #%ords P George 3. 3. @artin P !2!F

2) he %o o%ers P 7. 3. 3. olien P +2+) he lhe6ist P *aulo Coelho P !>H4) he "ello%ship o the 3ing P 7. 3. 3. olien P 4+2) he *ilgri6age P *aulo Coelho P 2IIF) Ga6e o hrones P George 3. 3. @artin P IF4

/ow all occurrences of commas,'- are replaced with vertical #ar,-.

8e can instruct the SED to perform text su#stitution only when a pattern match succeeds.The following example replaces comma,'- with vertical #ar,- only when a line containsthe pattern The ilgrimage.

[jerry]$ sed EBhe *ilgri6ageB sB&B P BgE /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin& !2!F2) he %o o%ers& 7. 3. 3. olien& +2+) he lhe6ist& *aulo Coelho& !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien& 4+2) he *ilgri6age P *aulo Coelho P 2II


34/60


In addition to this' SED can replace a specific occurrence of the pattern. $et us replaceonly the second instance of comma,'- with vertical #ar,-.

[jerry]$ sed EsB&B P B2E /oos.t5t


!) #tor6 o #%ords& George 3. 3. @artin P !2!F2) he %o o%ers& 7. 3. 3. olien P +2+) he lhe6ist& *aulo Coelho P !>H4) he "ello%ship o the 3ing& 7. 3. 3. olien P 4+2) he *ilgri6age&*aulo Coelho P 2IIF) Ga6e o hrones& George 3. 3. @artin P IF4

In the a#ove example' the num#er at the end of the SED command ,or at the place offlag- implies the And occurrence.

SED provides an interesting feature. *fter performing su#stitution' SED provides anoption to show only the changed lines. +or this purpose' SED uses the pflag which refersto print. The following example lists only changed lines.

[jerry]$ sed -n EsB*aulo CoelhoB*U0 C0;A0BpE /oos.t5t


+) he lhe6ist& *U0 C0;A0& !>H) he *ilgri6age& *U0 C0;A0& 2II

8e can store changed lines in another file as well. To achieve this result' use the wflag.The following example shows how to do it.

[jerry]$ sed -n EsB*aulo CoelhoB*U0 C0;A0B% jun.t5tE /oos.t5t

8e used the same SED command. $et us verify the contents of the


35/60


+) he lhe6ist& *U0 C0;A0& !>H) he *ilgri6age& *U0 C0;A0& 2II

So far' we have used only the foreslash,- character as a delimiter' #ut we can also usevertical #ar,-' at sign,-' caret,@-' exclamation mar3,;- as a delimiter. The followingexample shows how to use other characters as a delimiter.

$et us assume you need to replace the path +"in+sedwith +home+


36/60

[jerry]$ eho


37/60

0ne&%o&hree

/ote that now there is comma,'- instead of space in the regular expression.

String $eplacement )lags *GNU SED only+

In the previous section' we saw some examples of the su#stitution command. The /0SED provides some special escape se=uences which can #e used in the replacementstring. /ote that these string replacement flags are /0 specific and may not wor3 withother variants of SED. 2ere we will discuss string replacement flags.

O$4 8hen O$ is specified in the replacement string' it treats all the remaining

characters of the the word after O$ as lowercase characters. +or example' thecharacters H0$5H are treated as lowercase characters.

[jerry]$ sed -n EsB*auloB*JU0BpE /oos.t5t


+) he lhe6ist& *ulo Coelho& !>H) he *ilgri6age& *ulo Coelho& 2II

Ou4 8hen Ou is specified in the replacement string' it treats the immediate character

after Ou as an uppercase character. In the following example' Ou is used #efore thecharacters >a> and >o>. 2ence SED treats these characters as uppercase letters.

[jerry]$ sed -n EsB*auloBpJuaulJuoBpE /oos.t5t


+) he lhe6ist& pul0 Coelho& !>H) he *ilgri6age& pul0 Coelho& 2II

O04 8hen O0 is specified in the replacement string' it treats all the remaining

characters of the the word after O0 as uppercase characters.

[jerry]$ sed -n EsB*auloBJUpauloBpE /oos.t5t


+) he lhe6ist& *U0 Coelho& !>H) he *ilgri6age& *U0 Coelho& 2II

OE4 This flag should #e used with O$ or O0. It stops the conversion initiated #y the

flag O$ or O0. In the following example' only the first word is replaced withuppercase letters.

[jerry]$ sed -n EsB*aulo CoelhoBJUpaulo J;oelhoBpE /oos.t5t


38/60


+) he lhe6ist& *U0 oelho& !>H) he *ilgri6age& *U0 oelho& 2II

Stream Editor - /anaging (atterns8e have already discussed the use of pattern and hold #uffer. In this chapter' we aregoing to explore more a#out their usage. $et us discuss the ncommand which prints thepattern space. It will #e used in con(unction with other commands. iven #elow is thesyntax of thencommand.

[address![&address2]]n

$et us ta3e an example.

[jerry]$ sed EnE /oos.t5t

8hen the a#ove code is executed' it will produce the following result4


The ncommand prints the contents of the pattern #uffer' clears the pattern #uffer' fetches

the next line into the pattern #uffer' and applies commands on it.

$et us consider there are three SED commands #efore nand two SED commands after nas follows4

#ed o66and =!#ed o66and =2#ed o66and =+n o66and#ed o66and =4#ed o66and =

In this case' SED applies the first three commands on the pattern #uffer' clears the pattern#uffer' fetches the next line into the pattern #uffer' and thereafter applies the fourth andfifth commands on it. This is a very important concept. Do not go ahead without having aclear understanding of this.

The hold #uffer holds data' #ut SED commands cannot #e applied directly on the hold#uffer. 2ence' we need to #ring the hold #uffer data into the pattern #uffer. SED provides


39/60

the #command to exchange the contents of pattern and hold #uffers. The followingcommands illustrate the #command.

$et us slightly modify the #oo3s.txt file. Say' the file contains #oo3 titles followed #ytheir author names. *fter modification' the file should loo3 li3e this4

[jerry]$ at /oos.t5t


#tor6 o #%ordsGeorge 3. 3. @artinhe %o o%ers7. 3. 3. olienhe lhe6ist*aulo Coelhohe "ello%ship o the 3ing7. 3. 3. olien

he *ilgri6age*aulo Coelho Ga6e o hronesGeorge 3. 3. @artin

$et us exchange the contents of the two #uffers. +or instance' the following exampleprints only the names of authors.

[jerry]$ sed -n E5?n?pE /oos.t5t


George 3. 3. @artin7. 3. 3. olien*aulo Coelho7. 3. 3. olien*aulo CoelhoGeorge 3. 3. @artin

$et us understand how this command wor3s.

Initially' SED reads the first line' i.e.' * Storm of Swords into the pattern #uffer.

#command moves this line to the hold #uffer.

nfetches the next line' i.e.' eorge 6. 6. %artin into the pattern #uffer.

The control passes to the command followed #y n which prints the contents of the

pattern #uffer.

The process repeats until the file is exhausted.


40/60

/ow let us exchange the contents of the #uffers #efore printing. uess' what happens?:es' it prints the titles of #oo3s.

[jerry]$ sed -n E5?n?5?pE /oos.t5t


#tor6 o #%ordshe %o o%ershe lhe6isthe "ello%ship o the 3inghe *ilgri6age Ga6e o hrones

The hcommand deals with the hold #uffer. It copies data from the pattern #uffer to thehold #uffer. Existing data from the hold #uffer gets overwritten. /ote that the hcommanddoes not move data' it only copies data. 2ence' the copied data remains as it is in thepattern #uffer. iven #elow is the syntax of the hcommand.

[address![&address2]]h

The following command prints only the titles of the author aulo oelho.

[jerry]$ sed -n EB*auloBLh? B*auloBW5?pXE /oos.t5t


he lhe6isthe *ilgri6age

$et us understand how the a#ove command wor3s. The contents of #oo3s.txt follow aspecific format. The first line is the #oo3 title followed #y the author of the #oo3. In thea#ove command' H;H is used to reverse the condition' i.e.' line is copied to the hold #ufferonly when a pattern match does not succeed. *nd curly #races UV are used to groupmultiple SED commands

In the first pass of the command' SED reads the first line' i.e.' * Storm of Swords into thepattern #uffer and chec3s whether it contains the pattern aulo or not. *s the patternmatch does not succeed' it copies this line to the hold #uffer. /ow #oth the pattern #ufferand the hold #uffer contain the same line i.e.' * Storm of Swords. In the second step' it

chec3s whether the line contains the pattern aulo or not. *s the pattern does not match'it does not do anything.

In second pass' it reads the next line eorge 6. 6. %artin into the pattern #uffer andapplies the same steps. +or the next three lines' it does the same thing. *t the end of thefifth pass' #oth the #uffers contain The *lchemist. *t the start of the sixth pass' it readsthe line aulo oelho and as the pattern matches' it does not copy this line into the hold


41/60

#uffer. 2ence' the pattern #uffer contains aulo oelho' and the hold #uffer contains The*lchemist.

Thereafter' it chec3s whether the pattern #uffer contains the pattern aulo. *s the patternmatch succeeds' it exchanges the contents of the pattern #uffer with the hold #uffer. /ow

the pattern #uffer contains The *lchemist and the hold #uffer contains aulo oelho.+inally' it prints the contents of the pattern #uffer. The same steps are applied to thepattern The ilgrimage.

The hcommand destroys the previous contents of the hold #uffer. This is not alwaysaccepta#le' as sometimes we need to preserve the contents. +or this purpose' SEDprovides the *command which appends the contents to the hold #uffer #y adding a newline at the end. The only difference #etween hand *command is' the former overwritesdata from the hold #uffer' while the later appends data to the hold #uffer. Its syntax issimilar to the hcommand.

[address![&address2]]A

$et us ta3e another example. This time' instead of printing only #oo3 titles' print thenames of their authors too. The following example prints the #oo3 titles followed #y theirauthor names.

[jerry]$ sed -n EB*auloBLh? B*auloBWA?5?pXE /oos.t5t


he lhe6ist*aulo Coelho

he *ilgri6age*aulo Coelho

8e learnt how to copyappend the contents of pattern #uffer to hold #uffer. an weperform the reverse function as well? :es certainly; +or this purpose' SED provides the gcommand which copies data from the hold #uffer to the pattern #uffer. 8hile copying'existing data from the pattern space gets overwritten. iven #elow is the syntax of the gcommand.

[address![&address2]]g

$et us consider the same example ) printing #oo3 titles and their authors. This time' wewill first print the name of the author and on the next line' the corresponding #oo3 title.The following command prints the name of the author aulo oelho' followed #y its#oo3 title.

[jerry]$ sed -n EB*auloBLh? B*auloBWp?g?pXE /oos.t5t



42/60

*aulo Coelhohe lhe6ist*aulo Coelhohe *ilgri6age

The first command is 3ept as it is. *t the end of fifth pass' #oth the #uffers contain The

*lchemist. *t the start of the sixth pass' it reads the line aulo oelho and as the patternmatches' it does not copy this line into the hold #uffer. 2ence' the pattern spacecontainsaulo oelho and the hold space contains The *lchemist.

Thereafter' it chec3s whether the pattern space contains the pattern aulo. *s the patternmatch succeeds' it first prints the contents of the pattern space' i.e.' aulo oelho' then itcopies the hold #uffer to the pattern #uffer. 2ence' #oth the pattern and hold #ufferscontain The *lchemist. +inally' it prints the contents of the pattern #uffer. The same stepsare applied to the pattern The ilgrimage.

Similarly' we can append the contents of the hold #uffer to the pattern #uffer. SED

provides the Gcommand which appends the contents to the pattern #uffer #y adding anew line at the end.

[address![&address2]]G

/ow let us ta3e the previous example which prints the name of author aulooelhofollowed #y its #oo3 title. To achieve the same result' execute the following SEDcommand.

[jerry]$ sed -n EB*auloBLh? B*auloBWG?pXE /oos.t5t


*aulo Coelhohe lhe6ist*aulo Coelhohe *ilgri6age

an you modify the a#ove example to display the #oo3 titles followed #y their authors?Simple' (ust exchange the #uffer contents #efore the Gcommand.

[jerry]$ sed -n EB*auloBLh? B*auloBW5?G?pXE /oos.t5t


he lhe6ist*aulo Coelhohe *ilgri6age*aulo Coelho

Stream Editor - !eguar E#pressions


43/60

It is the regular expressions that ma3e SED powerful and efficient. * num#er of complextas3s can #e solved with regular expressions. *ny command)line expert 3nows the powerof regular expressions.

$i3e many other /0$inux utilities' SED too supports regular expressions' which are

often referred to as as rege#. This chapter descri#es regular expressions in detail. Thechapter is divided into three sections4 Standard regular expressions' 5SIF classes ofregular expressions' and %eta characters.

Standard $egular E%pressions

Start of line (^)

In regular expressions terminology' the caret,@- sym#ol matches the start of a line. Thefollowing example prints all the lines that start with the pattern HTheH.

[jerry]$ sed -n EBMheB pE /oos.t5t


he %o o%ers& 7. 3. 3. olienhe lhe6ist& *aulo Coelhohe "ello%ship o the 3ing& 7. 3. 3. olienhe *ilgri6age& *aulo Coelho

End of Line ($)

End of line is represented #y the dollar,J- sym#ol. The following example prints the lines

that end with HoelhoH.

[jerry]$ sed -n EBCoelho$B pE /oos.t5t


he lhe6ist& *aulo Coelhohe *ilgri6age& *aulo Coelho

Single Character (.)

The Dot,.- matches any single character except the end of line character. The following

example prints all three letter words that end with the character HtH.

[jerry]$ eho -e


44/60

/atrat6at

Match Character Set ([])

In regular expression terminology' a character set is represented #y s=uare #rac3ets ,G-. Itis used to match only one out of several characters. The following example matches thepatterns HallH and HTallH #ut not H&allH.

[jerry]$ eho -e


45/60

#ero on ne ccurrence (%&)

In SED' the =uestion mar3 ,O?- matches ero or one occurrence of the precedingcharacter. The following example matches H&ehaviourH as well as H&ehaviorH. 2ere' wemade HuH as an optional character #y using HO?H.

[jerry]$ eho -e

22!2+2+4222

#ero or More ccurrence ()

*steris3s ,W- matches the ero or more occurrence of the preceding character. Thefollowing example matches HcaH' HcatH' HcattH' and so on.

[jerry]$ eho -e


46/60

!!!!!!!!!!

$et us write the SED expression.

[jerry]$ sed -n EBM[->]JW+JX$B pE nu6/ers.t5t


!

/ote that the pair of curly #races is escaped #y the HOH character.

-t least n ccurrences +n,

Un'V matches at least HnH occurrences of the preceding character. The following exampleprints all the num#ers greater than or e=ual to five digits.

[jerry]$ sed -n EBM[->]JW&JX$B pE nu6/ers.t5t


!!!!!!

M to * ccurrence +m n,

Um' nV matches at least HmH and at most HnH occurrences of the preceding character. Thefollowing example prints all the num#ers having at least five digits #ut not more thaneight digits.

[jerry]$ sed -n EBM[->]JW&IJX$B pE nu6/ers.t5t


!!!


47/60

!

/ipe (0)

In SED' the pipe character #ehaves li3e logical 56 operation. It matches items fromeither side of the pipe. The following example either matches Hstr1H or HstrH.

[jerry]$ eho -e


48/60


str!Jrstr2

Escaping 1%dnnn1

This matches a character whose decimal *SII value is HnnnH. The following examplematches only the character HaH.

[jerry]$ eho -e HB pE


a

Escaping 1%onnn1

This matches a character whose octal *SII value is HnnnH. The following examplematches only the character H#H.

[jerry]$ eho -e


49/60


0ne!2+

[2alpha2]

It implies alpha#etical characters only. The following example matches only the wordH5neH.

[jerry]$ eho -e


50/60

[2upper2]

It implies uppercase letters only. The following example matches only HT85H.

[jerry]$ eho -e


51/60

Non/ord -oundary *.-+

In regular expression terminology' HO&H matches non)word #oundary. +or example'HtheO&H matches HtheseH and HtheyH #ut not HtheH. The following example illustrates this.

[jerry]$ eho -e


52/60

Single Non/ord C"aracter *.+

In SED' HO8H implies single non)word character which is exactly opposite to HOwH. Thefollowing example illustrates this.

[jerry]$ eho -e


53/60

[jerry]$ sed -n EpE /oos.t5t


#tor6 o #%ords& George 3. 3. @artinhe %o o%ers& 7. 3. 3. olien

he lhe6ist& *aulo Coelhohe "ello%ship o the 3ing& 7. 3. 3. olienhe *ilgri6age& *aulo Coelho Ga6e o hrones& George 3. 3. @artin

$emo2ing Empty 3ines

In the following example' H@JH implies empty line' and empty lines are deleted when apattern match succeeds.

[jerry]$ eho -e


54/60


=inlude \iostrea6Kusing na6espae std?

int 6ain(void)

Wout KK


55/60

7. 3. 3. olienhe lhe6ist*aulo Coelhohe "ello%ship o the 3ing7. 3. 3. olienhe *ilgri6age*aulo Coelho

Tail /6 Command

The Htail )1H prints the last line of the file. The following syntax shows its simulation.

[jerry]$ eho -e


56/60

ne%.t5t #C'' te5t

/ow let us display the file contents.

[jerry]$ at -vte ne%.t5t


ine =!$ine =2$

Uni%7dos command

Similar to HdosAunixH' there is HunixAdosH command which converts 0/IF newlinecharacter to D5S newline character. The following example shows simulation of thesame.

[jerry]$ eho -e


57/60


ine =!$ine =2$

$et us simulate the command using SED.

[jerry]$ sed EsP$PT$PE test.t5t


ine =!$ine =2$

Cat /ET Command

The Hcat )ETH command shows the Dollar,J- sym#ol at the end of each line and displays

the T*& characters as H@IH. The following example shows the simulation of Hcat )ETHcommand using SED.

[jerry]$ eho -e


58/60

The first SED expression prints line num#ers followed #y their contents' and the secondSED expression merges these two lines and converts newline characters to T*&characters.

cp Command

The HcpH command crates another copy of the file. The following SED script simulatesthis #ehavior.

[jerry]$ sed -n E% dup.t5tE data.t5t[jerry]$ di data.t5t dup.t5t[jerry]$ eho $O


E%pand Command

The HexpandH command converts T*& characters to whitespaces. The following codeshows its simulation.

[jerry]$ eho -e


59/60

cat /s Command

0/IF Hcat )sH command suppresses repeated empty output lines. The following codeshows the simulation of Hcat )sH command.

[jerry]$ eho -e


60/60

[jerry]$ eho -e

Stream Editor.doc

Documents

Transcript of Stream Editor.doc