Linux System and Network Administration 8

7/30/2019 Linux System and Network Administration 8

1/68

1

Regular Expressions

You can use and even administer Linux systemswithout understanding regular expressions butyou will be doing things the hard way

Regular expressions are used by: vi, ed, sed, and emacs

Awk, Tcl, Perl and Python

grep, egrep, fgrep


2/68

2

So What Is a Regular Expression?

Aregular expression is simply a description of apattern that describes a set of possible charactersin an input string

Weve already seen some simple examples ofregular expressions (known as regex from here on)

In vi when searching :/c[aou]t searches for cat, cot, or

cut In the shell

ls *.txt

cat chapter?

cp Week[1234].pdf /home/clyde/krf


3/68

3

Downside of Regular Expressions

There is considerable variation from utility toutility

The shell is limited to fairly simple metacharacter

substitution (*,?, []) and doesnt really support regex Regex in vi are also fairly limited

Regex in sedare not exactly the same as regex in Perl, orAwk, orgrep, or egrep

This puts the onus on the user to examine the manpage or other documentation for these utilities todetermine which flavor of regex are supported


4/68

4

So How Do We Build a Regex?

The simplest regex is a normal character

c , for example, will match a c anywhere while an a willdo the same for an a

The next thing is a . (period) This will match any single occurrence of any character

except a newline

For example . will match a z or an e or a ? or even

another . w.n will match win, wan, won, wen, wmn, went, and

wanton as well as w*n and w9n


5/68

5

Protecting Regex Metacharacters

Since many of the special characters used in regexsalso have special meaning to the shell, its a goodidea to get in the habit of single quoting your

regexs This will protect any special characters from being

operated on by the shell

If you habitually do it, you wont have to worry aboutwhen it is necessary


6/68

6

Multiple Occurrences in a Pattern

The * (asterisk or star) is used to define zero ormore occurrences of the single character precedingit

abc*d will match abd, abcd, abccd, abcccd, or evenabcccccccccccccccccccccccccccccccccccd

Note the difference between the * in a regex and theshells usage

In a regex, a * only stands for zero or more occurrences of asingle preceding character,

In the shell, the * stands for any number of characters thatmay or may not be different


7/687

Specifying Begin or End of Line The ^ specifies the beginning of a line

^The then will match any The that are the firstcharacters on a line

The $ matches the end of line

well$ will match well only if they are the lastcharacters on a line prior to the NEWLINE character

Note that well (notice the space at the end) wouldNOT match well$

^Ken$ would only match a line that started withKen and then had no other characters on the line


8/688

Character Classes [ ]

The square brackets [ ] are used to define characterclasses

[aA]wk will match awk or Awk

Ranges can also be specified in character classes [1-9] is the same as [123456789]

[abcde] is equivalent to [a-e]

You can also combine multiple ranges

[abcde123456789] is equivalent to [a-e1-9]

Note that the - character has a special meaning in acharacter class BUT ONLY if it is used within a range, [-123] would match the characters -, 1, 2, or 3


9/689

Negating a Character Class

The ^, when used as the first character in acharacter class definition, serves to negate thedefinition

For example [^aeiou] matches any character except a,e, I, o, or u

Used anywhere else within a character class, the ^simply stands for a ^

[ab^&] would match a a, b, ^, or &

Note also that within a character class, the ^ does notstand for beginning of line


10/6810

Escaping Special Characters

Even though we are single quoting our regexs so theshell wont interpret the special characters,sometimes we still want to use a special characteras itself

To do this, we escape the character with a \(backslash)

Suppose we want to search for the character

sequence 8*9* Unless we do something special, this will match zero or

more 8s followed by zero or more 9s, not what we want

8\*9\* will fix this - now the asterisks are treated as

regular characters


11/6811

Alternation

Regex also provides an alternation character ( | )for matching one or another subexpression

(K|T)en will match Ken or Ten

^(From|Subject): will match the From and Subject linesof a typical email message

It matches a beginning of line followed by either thecharacters From or Subject followed by a :

The parenthesis ( ) are used to limit the scope ofthe alternation

At(ten|nine)tion then matches Attention orAtninetion, not Atten or ninetion as would happen

without the parenthesis - Atten|ninetion


12/68

Repetition

The * (asterisk or star) has already been seen tospecify zero or more occurrences of theimmediately preceding character

+ (plus) means one or more

abc+d will match abcd, abccd, or abccccccd but willnot match abd while abc?d will match abd and abcd

but not abccd

12


13/68

13

Regex Summary

Character Name Meaning

.[][^]

dotcharacter classnegated character class

any one characterany character listedany character not listed

^

$\

caret

dollarbackslash less-thanbackslash greater-than

position at start of line

position at end of lineposition at beginning of wordposition at end of word

?*+

{n,m}

question markasterisk or starplus sign

n to m

matches optional preceding charactermatches zero or more occurrencesmatches one or more occurrences

matches m to n occurrences|

()

\1, \2,

bar, orparenthesis

backreference

matches either expression it separateslimits scope of | or enclosessubexpressions for backreferencingMatches text previously matched withinfirst, second, etc set of parenthesis


14/68

14

grep

grep comes from global regular expression print

This was such a useful command that it waswritten as a standalone utility

There are two other variants, egrep andfgrep thatcomprise thegrep family

grep is the answer to the moments where youknow you want a the file that contains a specificphrase but you cant remember its name


15/68

15

grep Family

Syntaxgrep [-hilnw] [-e expression] [filename]

egrep [-hiln] [-e expression] [-f filename] [expression][filename]

fgrep [-hilnx] [-e string] [-f filename] [string] [filename]

-h - Do not display filenames

-i - Ignore case

-l - List only filenames containing matching lines

-n - Precede each matching line with its line number

-w - Search for the expression as a word (grep only)

-x - Match whole line only (fgrep only)


16/68

16

Family Differences

grep - uses regular expressions for pattern matching

fgrep - file grep, does not use regular expressions,only matches fixed strings

egrep - exponential grep, uses a more powerful setof regular expressions call extended regularexpression:

Regular expressions and special characters

+ adds items to search pattern ? have zero or one occurrence of a string

| acts as or operator


17/68

17

Common Regular Expressions


18/68

18

Extended Regular Expressions


19/68

19

grep Example

grep cat myfile

grep root /etc/passwd

grep my files file1

grep i medical insurance .

grep iw dog pets


20/68

Using grep/egrep/fgrep

KEY CONCEPTS:

grep locates regular expressions

grep -E behaves same way as egrep

egrep allows use of extended regularexpressions

fgrep does not allow use of wildcards or

regular expressions grep family of command writes to standard

output

20


21/68

sed command

sed stream editorA stream editor is used to perform basic text

transformations on an input stream (a file or

input from a pipeline). Replace some substring with another

$ cat abird barks

mouse runs

$ sed 's/barks/flies/' < abird flies

mouse runs

21


22/68

sed command

Replace some characters with others

Replacing b with Q, i with X

$ cat abird barks

mouse runs

$ cat a | sed 'y/bi/QX/'

QXrd Qarks

mouse runs

22


23/68

Shell scripts

Everything that can be called from command line,can also be called from shell script

Shell Script is series of commands written in plain

text file.

Shell script is just like batch file is MS-DOS buthave more power than the MS-DOS batch file.

Interpreted line by line The same effect when entering the lines one by one

in interactive shell


24/68

shell script

First line of shell Cannot be a comment Is a shebang line

Used to identify program that is used toexecute lines in script In Bash written as #!/bin/bash In tcsh written as #!/bin/tcsh

#! Called a magic number Used by kernel to identify program that interprets

lines in script file


25/68

Running a shell script

As a parameter to shell interpreter

bash some_script.sh

Specifying interpreter on first line

First line:

#!/bin/bash

Make it executable (chmod +x some_script.sh)

./some_script.sh


26/68

Running a shell script

As a parameter to shell interpreter

bash some_script.sh

Specifying interpreter on first line

First line:

#!/bin/bash

Make it executable (chmod +x some_script.sh)

./some_script.sh


27/68

Bourne Shell 1-27

Debugging Shell Scripts

Display each command before it runs the command Set the x option for the current shell

$set x

Use the x to invoke the script

$bash x script.sh

Add the set command at the top of the script

set x or #!/bin/bash -x

Then each command that the script executes is preceded

by a plus sign (+) Distinguish the output of trace from any output that the script

produces

Turn off the debug with set +x


28/68

Sample shell script

#!/bin/bash

# comment line

echo "what a fine day: "

date

Output, when called by./test.sh:

what a fine day:

Thu Oct 28 23:37:39 EEST 2004

Interpreter to be used

Regular commands to execute


29/68

Variables

Sample hello world with variables#!/bin/bash

STR="Hello World!"

echo $STR

When assigning, no $ is used

When getting the contents use $

No data types string, number, character, all thesame

No declaring, just assign


30/68

Another sample shell script

EXAMPLE: backissa.sh

#!/bin/bash

DATE=`date +%Y%m%d`WHAT='/home/issa'

DEST="$DATE.tgz"

tar cvzf $DEST $WHAT

Results in calling:tar cvzf 20071028.tgz /home/issa


31/68

Command line arguments

Automatically defined variables

$0 contains shell script name

$1 contains first argument

$2 2nd

$# gives count of command line arguments

$* lists all command line arguments

$@ lists command line arguments individually quoted


32/68

The shift Command

The shift command Used to shift parameters to the left, one by

one

Makes number of parameters on a linelimitless.

Whenever script file read shift, it movedeach parameter over one position with

exception of $0 $0 is always name of shell script

32


33/68

Using the shift Command

33


34/68

Using the shift Command

Ch 10 34


35/68

The set Command

setCan reset positional parameters when usedwith arguments

set --Used to unset positional parameters

set with command substitutions

Output from command that can bemanipulated in a shell script

Example: set `date`

35


36/68

Exit Status

Exit statusProcess stops executingreturns an exit

status to its parent process

0command successful (true condition)

not zerocommand failed (false condition)

Can specify exit status that you want your

shell script to return

Ch 10 36


37/68

Arithmetic expansion

Arithmetic expansion allows the evaluation of anarithmetic expression and the substitution of theresult. The format for arithmetic expansion is:

$((expression))A=5

B=4

C=$(($A*$B))

echo $C


38/68

Bourne Shell 1-38

test

Command test is a built-in command Syntax

test expression

[ expression ]

The test command evaluate an expression

Returns a condition code indicating that the expression is eithertrue (0) or false (not 0)

Argument Expression contains one or more criteria

Logical AND operator to separate two criteria: -a

Logical OR operator to separate two criteria: -o

Negate any criterion: !

Group criteria with parentheses

Separate each element with a SPACE


39/68

Test Command Operations

Ch 10 39


40/68


Ch 10 40


41/68


Ch 10 41


42/68

Bourne Shell 1-42

Control structures

if then

for in

while

until

case

break and continue


43/68

Bourne Shell 1-43

if then

Structureiftest-command

then

commandsfi

Example:if test $word1 = $word2

then

echo Match

fi


44/68

Bourne Shell 1-44

Example

Create a shell script to check there is at least oneparameter

Something like this:

if test $# -eq 0

then

echo you must supply at least one arguments

exit 1

fi


45/68

Bourne Shell 1-45

Example

Check weather or not the parameter is a non-zeroreadable file name

Continue with the previous script and add something

likeif [ -r $filename a s $filename ]

then

fi


46/68

Bourne Shell 1-46

Example

Check users confirmation

First, read user inputecho -n Please confirm: [Yes | No]

read user_input Then, compare it with standard answer yes

if [ $user_input = Yes ]

then

echo Thanks for your confirmation!

fi

What will happen if no around $user_input and userjust typed return?


47/68

Bourne Shell 1-47

ifthenelse


then

commands

else

commandsfi

You can use semicolon (;) ends a command thesame way a NEWLINE does.

if [ ]; then

fi

Example: if [ 5 = 5 ]; then echo "equal"; fi


48/68

Bourne Shell 1-48

ifthenelif


then

commands

eliftest-commandthen

commands

.

.

.else

commands

fi


49/68

Bourne Shell 1-49

for

Structurefor loop-index

do

commandsdone

Automatically takes on the value of each of command

line arguments, one at a time. Which impliesfor arg in $@


50/68

Bourne Shell 1-50

Example

Structurefor loop-index in argument_listdo

commands

done

Example:for file in *

do

if [ -d $file ]; thenecho $file

fi

done


51/68

Example-2

for i in 1 2 3 4 5

do

echo "Welcome $i times"

done


52/68

Example-3

for (( expr1; expr2; expr3 ))do

repeat while expr2 is true

done

for (( i = 0 ; i


53/68

Example-4

You can use for together with file name expansionto do some action for several files

#!/bin/bashfor x in *txt;

do

cat $x

done;


54/68

Bourne Shell 1-54

while

Structure

while test_command

do

commands

done

Example:while [ $number lt 10 ]

do

number=`expr $number + 1`

done


55/68

Bourne Shell 1-55

until

Structureuntil test_command

do

commands

done

Example:secretname=jenny

name=noname

until [ $name = $secretname ]do

echo Your guess: \c

read name

done


56/68

Bourne Shell 1-56

break and continue Interrupt for, while or until loop

The break statement transfer control to the statement AFTER the done

statement terminate execution of the loop

The continue statement Transfer control to the statement TO the done

statement Skip the test statements for the current iteration

Continues execution of the loop


57/68

Bourne Shell 1-57

Example

for index in 1 2 3 4 5 6 7 8 9 10

do

if [ $index le 3 ]; then

echo continuecontinue

fi

echo $index

if [ $index ge 8 ]; thenecho break

break

fi

done


58/68

Bourne Shell 1-58

case

Structurecase test_string in

pattern-1 )

commands_1

;;

pattern-2 )

commands_2

;;

esac

default case: catch all pattern

* )


59/68

Bourne Shell 1-59

case

Pattern Matches

* Matches any string of characters.

? Matches any single character.

[] Defines a character class. A hyphen specifiesa range of characters

| Separates alternative choices that satisfy aparticular branch of the case structure

Special characters used in patterns


60/68

Bourne Shell 1-60

Example

#!/bin/bash

echo \n Command MENU\n

echo a. Current data and time

echo b. Users currently logged inecho c. Name of the working directory\n

echo Enter a,b, or c: \c

read answer

echo


61/68

Bourne Shell 1-61

Example-continuecase $answer in

a)date

;;

b)

who

;;c)

pwd

;;

*)

echo There is no selection: $answer

;;

esac


62/68

Bourne Shell 1-62

echo and read

The backslash quoted characters in echo \c suppress the new line

\n new line

\r return

\t tab

Read read variable1 [variable2 ]

Read one line of standard input

Assign each word to the corresponding variable, with theleftover words assigned to last variables

If only one variable is specified, the entire line will be assignedto that variable.


63/68

Bourne Shell 1-63

Built-in: exec

Execute a command:

Syntax: exec command argument

Run a command without creating a new process

Run a command in the environment of the original process Exec does not return control to the original program

Exec can be the used only with the last command that youwant to run in a script

Example, run the following command in your current shell,what will happen?

$exec who


64/68

Bourne Shell 1-64

Catch a signal: builtin trap Built-in trap

Syntax: trap commands signal-numbers

Shell executes the commands when it catches one of the signals

Then resumes executing the script where it left off.

Just capture the signal, not doing anything with ittrap signal_number

Often used to clean up temp files

Signals SIGHUP 1 disconnect line

SIGINT 2 control-c SIGKILL 9 kill with -9

SIGTERM 15 default kill

SIGSTP 24 control-z


65/68

Bourne Shell 1-65

Example

$ cat inter

#!/bin/bash

trap 'echo PROGRAM INTERRUPTED' 2

while true

do

echo "programming running."

sleep 1done


66/68

Bourne Shell 1-66

functions

A shell function is similar to a shell script It stores a series of commands for execution at a later time.

The shell stores functions in the memory

Shell executes a shell function in the same shell that called it.

Where to define In .profile

In your script

Or in command line

Remove a function Use unset built-in


67/68

Bourne Shell 1-67

functions

Syntaxfunction_name()

{

commands

}


68/68

Example$whoson()

> {

> date

> echo "users currently logged on"

> who

}

$whoson

Tue Feb 1 23:28:44 EST 2005

users currently logged on

issa :0 Jan 31 08:46

issa pts/1 Jan 31 08:54 (:0.0)

issa pts/2 Jan 31 09:02 (:0.0)

Linux System and Network Administration 8

Documents

Transcript of Linux System and Network Administration 8