OS54 Slides UNIX Shell Scripting
-
Upload
tpsarvanan -
Category
Documents
-
view
323 -
download
15
Transcript of OS54 Slides UNIX Shell Scripting
-
7/24/2019 OS54 Slides UNIX Shell Scripting
1/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
2/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 22
Confidential Information
This Document is confidential to Infosys Technologies Limited. This document containsinformation and data that Infosys considers confidential and proprietary (Confidential
Information).
Confidential Information includes, but is not limited to, the following:
Corporate and Infrastructure information about Infosys;
Infosys project management and quality processes;
Project experiences provided included as illustrative case studies.
Any disclosure of Confidential Information to, or use of it by a third party, will be damaging toInfosys.
Ownership of all Infosys Confidential Information, no matter in what media it resides, remains withInfosys.
Confidential information in this document shall not be disclosed, duplicated or used in whole orin part for any purpose without specific written permission of an authorized representative of
Infosys.
This document also contains third party confidential and proprietary information. Such third partyinformation has been included by Infosys after receiving due written permissions and
authorizations from the party/ies. Such third party confidential and proprietary information shall notbe disclosed, duplicated or used in whole or in part for any purpose without specific written
permission of an authorized representative of Infosys.
-
7/24/2019 OS54 Slides UNIX Shell Scripting
3/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
4/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
5/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
6/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
7/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
8/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 88
Introduction to Unix
What is Unix?
Need for Unix in todays world Evolution of Unix
Flavors of Unix
http://www.bell-labs.com/history/unix/
Unix is a very popular multi-user, multitasking, time-sharing operating system.
Unix has become the operating system of choice for various engineering and scientificapplications. The need for Unix could be determined by the different categories ofapplication it suffices. viz. networking, programming, multimedia, high-performancecomputing to name a few.
Evolution of UNIX:1965 : AT&T, GE, IBM and Project MAC joined together to MULTICS (MultiplexedInformation and Computing Service), MULTICS is a time-sharing operating system.
1969 : AT&T Bell Labs was out of MULTICS project in 1969. The initial version of UNICSwas written by Ken Thompson. UNICS was only compatible with PDP-7 machine. UNICSstands for Uniplexed Information and Computing Services. UNICS is later changed to UNIX
1973 : Re-written in high level language C, thus making it machine-independent.
1977-1981 : Unix was widely available at minimal cost and became popular for scientificapplications.
1982 : Unix System III is released.
1984-85 : Unix System V is released.
1992-93 : 4.4 BSD is released.1994 : Linux 1.0 is released.
2001 : Linux 2.4 is released.
Unix is available in many flavors like :
AIX (Advanced IBM Unix), HP-UX ( Hewlett Packard Unix), MINIX (Minimal Unix)
SCO UNIX, SOLARIS, XENIX, SUN OS, LINUX
-
7/24/2019 OS54 Slides UNIX Shell Scripting
9/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
10/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
11/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
12/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
13/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
14/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 1414
Operating System Services
Exception: An exception is an unexpected event like reference toan illegitimate address. Exceptions usually occur in the middle of a
process where as interrupts occur between them. Differentalgorithms are used to handle interrupts/exceptions efficiently
Processor execution levels
Based on the priority level of an interrupt the kernel masks certaininterrupts in order that a critical process may proceed without error.
The priority levels in descending order are: Machine error
Clock
Disk
Network Devices
Terminals
Software Interrupts
Memory Management
The concept of a Virtual Machine is central to memory management under Unix. Thecompiler generates addresses for a virtual machine as if no other program willexecute simultaneously on the physical machine. Virtual addresses generated by thecompiler need not be identical to the actual phys ical addresses in the machine. Thememory management coordinates with the machine hardware to set-up a virtual to physicaladdress translation that maps the compiler-generated address to the physical machine
address.
-
7/24/2019 OS54 Slides UNIX Shell Scripting
15/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
16/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
17/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
18/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
19/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 1919
How Unix Works
USER TYPES COMMAND
SHELL EXECUTESUTILITY TO CARRYOUT COMMAND
SHELL ASKS FOR A COMMAND
USER INTERACTS WITHUTILITY
SHELL PROMPTS FORNEXT COMMAND
USER TYPES CONTROL-D
LOGOUT
LOGIN
The user connects to the Unix Server using an utility called telnet, and by mentioning the IPaddress of the Unix Server.
The Server requests for the username and password. The user needs to give the validusername and password, for log-in.
Once the username and the password is right, the shell-prompt is displayed, where-in the
user could execute the commands.
-
7/24/2019 OS54 Slides UNIX Shell Scripting
20/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
21/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
22/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
23/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
24/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 2424
File Descriptors
Standard Input (0)
This represents the file to which the process is tied to for accepting theinput
Standard Output (1) This represents the file to which the process is tied to for directing the
output
Standard Error (2) This represents the file to which the process is tied to for writing error
messages
Standard Input (0)
Takes the keyboard or f iles as input.
Example
$> mail user1 < s1 #s1 is the name of the file.
The contents of s1 is mailed to user1.
Here the input for the command mail is coming from file user1
Standard Output (1)
This is the default fi le.
Example
$> ls 1> infy #Redirects the output to a file called infy.
Is same as
$> ls > infy
Standard Error (2)
Is used for redirecting the errors.
Example
$> ls abcd11 2> def #Redirects the errors into a file called def
$> cat def
ls: abcd11: No such file or directory
-
7/24/2019 OS54 Slides UNIX Shell Scripting
25/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
26/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
27/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
28/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
29/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 2929
Summary
Introduction to the Unix operating system
Unix System Architecture General Commands and Utilities
Wild Cards, I/O redirection and Filters
Regular Expressions
-
7/24/2019 OS54 Slides UNIX Shell Scripting
30/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
31/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
32/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
33/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 3333
Types of shells
Bourne Shell Written by Steve Bourne
Bourne Again Shell Brian Fox wrote the first version released in 1988.
Later Chet Ramey joined and is responsible for numerous bug fixand addition of useful features
C Shell Written by Bill Joy
Korn Shell Written by David Korn
TC Shell (tcsh)
Public Domain Korn Shell (pdksh)
Bourne shell (/bin/sh) (written by Steve Bourne of Bell Labs)
This is the original Unix shell and it is available on all UNIX systems.
This shell does not have support of latest user interactive facilities provided by modernshells such as the Korn shell and C shell.
bourne shell does provide an easy to use language to write scripts.
Bourne again shell (/bin/bash) ( Free Software Foundation ).bash provides all the interactive features of the modern shells like Korn shell (ksh) and theC shell (csh) . Bashs programming language is also compatible with the Bourne shell (sh).
C shell (/bin/csh) (written at University of California, Berkeley)
C shell provides a C-like language to write shell scripts.
Korn shell, /bin/ksh, (written by David Korn (Bell Labs) )
Ksh is now the standard shell on all Unix-like systems. Provides all the interactive featuresof the C and TC shells. Kshs shell programming language is also similar/compatible to shellprogramming language of the original Bourne shell.
TC Shell (tcsh )
tcsh provides all the features of the C shell with emacs style editing of the command line.
References: http://unixhelp.ed.ac.uk/
-
7/24/2019 OS54 Slides UNIX Shell Scripting
34/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 3434
Welcome to Shell Script ing
Create a file called hello.sh using editor or from command line
Assign execute permissions for yourself
$ chmod u+x hello.sh
Execute it from command prompt
$ ./hello.sh
Creates a sub-shell and execute hello.sh
$ . ./hello.sh
Execute hello.sh in current shell
$ cat hello.sh
#!/bin/bash
date; cal #Di spl ays date f ol l owed by cal endar
echo Welcome to Shell Scripting
hello.sh when run, displays date and calendar for current month, followed by messageWelcome to Shell Scripting
First line of the script #!/bin/bash determines which shell to be used to execute the shellscript. This can be changed to /bin/sh (for Bourne shell), /bin/bash (for bash shell) , /bin/ksh
(for Korn Shell), etc.
Shell treats all characters after # till end of line as comments (ignores them). Comments arevery useful for documenting and elaborating the functionality achieved by command or groupof commands.
Note: While executing shell script using ., it is not required to have execute permissions set
Note: On some systems /bin/sh may refer to POSIX Shell. bash conforms to most of POSIX1003.2 standard. If one is too cautious about portability issues, one can use - -posix optionwhile invoking bash. This would make bash act like a POSIX shell.
-
7/24/2019 OS54 Slides UNIX Shell Scripting
35/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
36/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
37/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
38/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 3838
Quotes
Quotes help us in reinforcing the correct interpretation of
things which shell might interpret differently Single quotes
Shell treats everything between pair of quotes literally
We cannot have a single quote inside a pair of single quotes
Double quotes
Enclosed string considered as single string. Single quote has nospecial effect inside.
Characters $, , and \ must be escaped using \
Allow parameter and command substitution, evaluation of arithmeticexpressions
Back quotes (ticks or grave accent) ` `
Enclosed string is treated as a command and shell executes it
Single Quotes:
$ echo Let me study # Let me study
$ echo $var # $var
$ echo *[|>/$\? # *[|>/$\?
Double Quotes:
$ star=*$ echo $star # Displays list of all files in current directory
$ echo $star # Displays *
$ age=29; echo \\$age\=$age # Displays $age=29
$ age=30; echo $age=$age # Displays $age=30
$ varo=Hello vart=World
$ var=$varo $vart
$ echo $var # Displays Hello World
Back Quotes:
$ echo Working Directory is `pwd`
Working Directory is /root/cppdemos
Note: POSIX supports newer construct for command substitution and use of back-quote isdepreciated
$ x=10
$ echo 'echo $x# Displays echo$x
$ echo "echo $x" # Displays echo 10
$ echo `echo $x` # Displays 10
-
7/24/2019 OS54 Slides UNIX Shell Scripting
39/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
40/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 4040
test command
Used extensively for evaluating script conditions
It evaluates the condition on its right and if the result istrue, returns an exit status ($?) of zero
[ ] can be used instead of writing test explicitly
n1 eq n2 -Integers n1 and n2 are equal
n1 ne n2 -Integers n1 and n2 are not equal
n1 lt n2 -Integer n1 is < n2
n1 gt n2 -Integer n1 is > n2
n1 le n2 -Integer n1 is = n2
s1 = s2 Strings s1 is same as as s2
s1 != s2 Strings s1 and s2 are not same
-n s1 String s1 is not null (non-zero length)
-z s1 String s1 is null (zero length)
$ x=10 y=20
$ test $x -eq $y; echo $?
1
$ test $x -ne $y; echo $?
0
$ [ $x -ne $y ];echo $?
0
$ [ $x -eq $y ]; echo $?
1
Above list illustrates how test can be used for comparing numbers and strings. Test returntrue when comparison is successful, otherwise false. 0 represents truth while all othervalues (positive and negative) represent false
Note :A space is mandatory after [ and before ]
$ str1="Infosys str2="Infy"
$ test $str1 =$str2 ; echo $?
1 # indicates both the strings are different.
Logical operations: test supports logical and (-a), logical or (-o) and logical not (!).Parenthesis can be used for grouping
$ x=10;
$ test $x -lt 15 -o $x -gt 5; echo $?
0
$ test $x -lt 15 -a $x -gt 5; echo $?
0
$ test ! $x gt 5; echo $?
1
-
7/24/2019 OS54 Slides UNIX Shell Scripting
41/75
File test expressions: test expressions can be used to perform file tests as well
-e filename filename exists
-f filename filename exists and it is a regular file # $ test -f $f ; echo $?
-r filename filename exists and is readable
-w filename filename exists and is writable # $ [ ! x $file ]; echo $?
-s filename filename exists and is not empty (size greater than 0)
-d filename filename exists and is a directory
Note: to perform multiple tests on a file, we should use following syntax:
$ test option1 filename option2 filename
For ex: $test r file1 w file1 is used to check whether a file is both readable and writeable.
-
7/24/2019 OS54 Slides UNIX Shell Scripting
42/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
43/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 4343
Iteration control
while, until and for loops
# Iterates, if condition is true
while condition; do
statements
done
---------------------------------
# Iterates, if condition is false
until condition; do
statements
done
---------------------------------
# Iterates, if list is not empty
for var in list_of_values; do
statements using $var
done
#!/bin/bash
var=2;
while [ "$var" -le 10 ]; do
echo "Var is $var"
var=$((var+3)) # var= expr $var + 3`
done
------------------------------------
var=2;
until [ "$var" -gt 10 ]; do
echo "Var is $var"
var=$((var+3)) # var= expr $var + 3`
done
------------------------------------
for var in * ; do
echo "$var # Displays all filenames
done
Example: Changes the extension of all .txt file to .doc
for filename in *.txt; do
tmpname=$(basename $filename txt)
mv $filename ${tmpname}doc
done
Example: Finds factorial of number passed from command line
fact=1 num="$1"
while [ "$num" -gt 0 ]; do
fact=$((fact*num))
num=$((num-1))
done
echo "Factorial of $1 is $fact
bash supports non POSIX iteration construct called select. Select generates a menu(formatted by numbers 1, 2, 3 etc) of each item in the list. Asks user to enter a number. Storesthe selected choice number in a built in variable REPLY. Loop terminates when we use break.
#!/bin/bash
select choice in "cal" "who" "exit"; do
case "$REPLY" in
1) echo "$choice"; cal;;
2) echo "$choice"; who;;
3) echo "$choice"; break;;
esac
done
-
7/24/2019 OS54 Slides UNIX Shell Scripting
44/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
45/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 4545
I/O redirection and piping data with loops
#!/bin/bash
lineno=1
while read line; do
echo " $lineno $line"
lineno=$((lineno+1))
done < history.txt
----------------------------
#!/bin/bash
for filename in *.txt; do
echo "$filename"
cat $filename
echo "--- End ---"
done > textfiles.log
for filename in *.dat; do
sort r $filename
done&
--------------------------------------
#!/bin/bash
lineno=1
cat history.txt | while read line; do
echo " $lineno $line"
lineno=$((lineno+1))
done
--------------------------------------
#!/bin/bash
for filename in *.txt; do
echo "$filename"
done | wc -l
A loop can be executed in the background by adding & to done statement.
First example of I/O redirection reads input from history.txt line by line and displays each lineprefixed by line-number. Second example stores all .txt files in current directory intotextfiles.log after re-formatting. During re-formatting name of file is prefixed to its contentsand message --- End --- is suffixed.
Note:All commands that write to standard output inside loop take part in redirection from
loop. If there is an statement like echo hello >/dev/null inside loop; this would not take partin output redirection from loop.
Note: Some shells take care of piping with loops by creating a sub-shell. Any modificationsmade to variables inside loop, would not be reflected outside iteration control construct thatuses piping. Most of the shells (except few old shells) perform I/O redirection from loopswithout invoking a new sub-shell; hence any modifications to variables inside the loop wouldbe reflected outside the loop as well. Refer to sh, bash and ksh manual for details.
#!/bin/bash
count=5
while [ "$count" -lt 10 ]; do
newcount=$((count*2))echo "$count"
count=$((count+1))
done > count.txt
echo ":$count:$newcount: # Displays :10:18:
If we replace done > count.txt with done | wc l, echo :$count:$newcount: would display5:: (original value of count is not modified)
-
7/24/2019 OS54 Slides UNIX Shell Scripting
46/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 4646
Functions
Advantages of using functions Helps us in avoiding repetition of script code if it is required a
number of times
When we run a program that requires a number of functions, wecan store functions in another file and load them into our scriptwhen we wish to use them
Features Arguments passed to function can be accessed using $1, $2 and
so on
Value can be returned from functions return $var
Value returned from a function can be accessed using $?
We can write recursive functions
Functions loaded into memory can be unset $ unset -f function_name
When making standard operations, to avoid having to repeatedly create each line of thescript, we create functions that we call into out scripts.
All functions must be defined before use. A function runs in the current environment and canbe recursive. functions cannot be exported to subshells.
Below example illustrates that $1, $2, etc inside a function refers to arguments passed to
function and not the shell script$ cat >test.sh
#!/bin/bash
argsinfo () {
echo '--argsinfo()--'
echo " $# : $* "
}
echo '--test.sh--'
echo " $# : $*"
shift 2
argsinfo these arguments are different
$ ./test.sh 1 2 3 4 5
--test.sh--
5 : 1 2 3 4 5
--argsinfo()--
4 : these arguments are different
-
7/24/2019 OS54 Slides UNIX Shell Scripting
47/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
48/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
49/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 4949
Customizing your environment
Environment variables are set during startup of shell HOME, IFS, PATH, PS1, PS2, PWD, TERM, DISPLAY, etc
are automatically exported to sub-shells Value can be displayed using echo $HOME, echo $PS1, etc
User can also assign new values PS1=Enter a command>
HOME : Users home directory
PS1, PS2 : Primary/secondary prompt
PWD : Present working directory
IFS: Input Field Separator
PATH : List of directories to be searched for command
SHELL : Default shell
EDITOR: Default editor
Export
Makes a variable a part of environment and also be accessible to thechild shell
$ echo $PATH #To display the PATH
/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/ajit:.
$ PATH=$PATH:/home #To append to the existing path.
$ echo $PATH
/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/ajit:.:/home
A sub-shell has no knowledge of any user defined variables in parent shell.
If exported variables are modified in subshell, it has no effect on original variables in parent shell.Exported variables are not only available in sub-shell but for all processes spawned by thosesubshell.
Example:
$> a=10
$> echo $$ #PID of the shell
6034
$> echo $a
10
$> sh #Creates a child shell
$> echo $$ #PID of child shell
9072$>echo $a
Af ter using export
$> export a
$> sh #Creates a child shell
$>echo $a
10
-
7/24/2019 OS54 Slides UNIX Shell Scripting
50/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 5050
Customizing your environment (Contd)
set Used to display all variables (user defined as well as environmental)
Used to set positional parameters Can be used to debug shell scripts as well
alias Shorthand notation for referencing commands
unalias Remove aliases from alias list
unset Used to unset variable or function
Syntax unset [-vf] name
readonly Used to mark variables readonly, after this value of variable cannot be
changed
$ set #Displays all variable and their values
$ set $(date) # runs date command sets it output as positional parameters
$ echo $# # Displays 6
$ echo $1 $2 $6# Displays Mon Jun 2008
$ set -- # Unsets all positional parameters
$ set one two three four
$ echo $# # Displays 3
$ unset v varname # Unsets variable varname
$ unset f func_name # Unsets function func_name
$ readonly # Displays all readonly variables
$ var=100; readonly var # Makes var readonly
$ var=200 # Fails, var is readonly
$ alias # Displays all aliases
$ alias ll=ls l # ll aliased to ls -l
$ ll # executes ls l
$ unalias ll # alias removed
$ unalias -a # Removes all aliases
-
7/24/2019 OS54 Slides UNIX Shell Scripting
51/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 5151
Customizing your environment (Contd)
Shell start up process
Bourne shell
Locates file .profile in home directory and reads requisite settings from it
.profile is executed only once when you login and not for every invocation ofa subshell
Korn shell
Locates system startup file in /etc/profile, reads it; locates .profile file inhome directory, reads it; finally reads environment file
If a subshell is invoked, only environment file is read
Bash
Locates system startup file in /etc/profile, reads it. Locates .bash_profile inhome directory, reads it
If .bash_profile is not found, bash tries to read .bash_login or .login files inhome directory
.bash_logout is executed every time one logs out
If a subshell is invoked, only .bashrc is read
What is so special about files /dev/tty and /dev/null?
Note: In bourne shell, .profile file contains the settings for the user to customize his shell
When shell starts up, it reads various values from different files; that determine setting for aparticular system. Some of these files are located in our home directory while some may be insystem directories.
/dev/tty and /dev/null are very useful. Any data redirected to /dev/null (bit bucket or trash file) isdiscarded. This is useful if our requirement is to find out the exit status rather than the output.
Example:
if who | grep $username >/dev/null ; thenecho $username is logged in
else
echo $username is not logged in presently
fi
/dev/tty represents a terminal file and cannot store data. Useful for reading passwords.
Example:
#!/bin/bash
echo 'Enter password?'
stty -echo # echo turned off
read varo
-
7/24/2019 OS54 Slides UNIX Shell Scripting
52/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 5252
Customizing your environment (Contd)
What should shell script do when a signal is raised? Let the default-activity associated with raised signal take over
Ignore the raised signal Specify an user defined action to be performed
trap Used to trap process signals Shell can register a signal handler to catch specified signal Syntax
trap command1; command2; signal_name
Common signals SIGEXIT 0 Normal termination of script SIGHUP 1 Hangup SIGINT 2 Interrupt (ctrl+c) SIGFPE 8 Floating point exception SIGKILL 9 Kill signal
SIGPIPE 13 Broken pipe: write to pipe with no readers SIGTERM 15 Termination signal
When we press Ctrl+c during the execution of a shell script, a signal is sent; normally theprogram is terminated. However, sometimes this is undesirable. One should ensure removalof temporary files and other book-keeping (clean up) must be done before a shell scriptshould terminate. UNIX provides the trap command which allows the user to alter the effectsof certain events that generate signals, which generally tends to halt a process.
#!/bin/bash
trap 'echo SIGINT disabled' INTtrap 'echo $0 terminated' EXIT
trap ' ' HUP # SIGHUP Ignored
echo 'Enter your name?'
read name
echo "Name: $name"
$ ./test.sh
Enter your name?
SIGINT disabled # User pressed ctrl+c
Amit Tiwary # User enters Amit Tiwary
Name: Amit Tiwary
./test.sh terminated # Displayed due to trapping of SIGEXIT
Note: If one ignores a signal, all subshells ignore that signal as well. If one has set trap totake action on receiving the desired signal, the subshell would continue to thake the defaultaction and not the one specified by the user. SIGKILL cannot be trapped or ignored
trap - INT TERM EXIT # Resets the handlers for SIGINT, SIGTERM and SIGEXIT
-
7/24/2019 OS54 Slides UNIX Shell Scripting
53/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential
REGULAR EXPRESSIONS
5353
-
7/24/2019 OS54 Slides UNIX Shell Scripting
54/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
55/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 5555
Regular Expressions (Contd)
Atoms
Character This may contain any printable character (alpha-numeric or special)
. (dot) represents any single character except newline
Range/Class matches any one character from the set [ ]
Anchor
Back Reference
Character: $ grep U song.txt matches all lines containing U
Regular expression meta-characters like * . ? $ ^ etc must be preceded by \ to suppress theirmeaning.
$ grep \$ song.txt matches all lines containing $
Dot: $ grep W . . song.txt matches all lines containing W followed by any two characters
Range/Class: $ grep [0-9] song.txt matches all lines containing a digit
A ^ placed as first item of range/class negates the meaning.
[^A-Z] match anything except uppercase letters, [^aeiou] matches anything except vowels, [\^]matches a , [^\^] matches anything except a ^, [A-Za-z0-9] matches an alpha-numeric character,[][-] matches ] or [ or , [0-5\-] matches 0 or 1 or 2 or 3 or 4 or 5 or
Anchor: There are four types of anchors. ^(beginning of line), $ (end of line), \< (beginning of word)and \> (end of word)
$ grep ^F song.txt matches all lines beginning with F, $ grep \^ song.txt matches all linesbeginning with ^ , $ grep e$ song.txt matches all lines ending with e
$ grep \.$ song.txt matches all lines ending with . , $ grep \ song.txt matches all lines having words ending with a comma, $ grep ^$ song.txtmatches all blank lines
$ grep ^.$ song.txt matches all single character lines
Back Reference: Used to restore or replay a pattern saved in buffer. There are nine buffersnumbered 1 to 9. \n restores the pattern from nth buffer. We would be covering more of this whenwell talk about tagged regular expressions under operators.
-
7/24/2019 OS54 Slides UNIX Shell Scripting
56/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 5656
Regular Expressions (Contd)
Operators
Alternation | Repetition \{s,e\}
Grouping ( )
Shorthand * + ?
Operators: Operators are used to combine atoms.
Al ternat ion: This is denoted by | and behaves like logical or
$ egrep you|You song.txt matches lines containing either you or You
$ egrep '\|\' song.txt matches lines containing either word this or that
Repetition: is denoted by \{s,e\} and matches previous character s to e times, where s and e are
positive integers.
ab\{2,4\} matches abb abbb and abbb
ab\{2,\} matches abb abbb abbb i.e. a followed by atleast 2 bs
ab\{,2\} matches a ab abb i.e. a follwed by atmost 2 bs
ab\{2\} matches abb i.e. a followed by exactly 2 bs
\{2,4\} matches {2,4} (Why? Repetition is not prefixed by any character)
Grouping: is denoted by () and is used to group two or more regular expresions
Hel(l|lo) matches Hell or Hello
a(bc)\{2\} matches abcbc
Nested Grouping is also possible
(a(bc)\{2\}d)\{2\} matches abcbcdabcbcd
Short Hand Operators: There are three shorthand operators * + and ?
* matches zero or more repeats of previous atom, equivalent to \{0,\}
+ matches one or more repeats of previous atom, equivalent to \{1,\}
? matches zero or one repeat of previous atom, equivalent to \{0,1\}
Note that AA* is equivalent to A+
-
7/24/2019 OS54 Slides UNIX Shell Scripting
57/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 5757
Summary of Regular Expressions
. Matches any single character except newline \ Suppress special meaning of meta-characters
[ ] Matches any one character from the set (- indicates range and ^inside [ ] negates the meaning i.e. character not marching thecharacter set is matched)
^ Matches at the beginning of line $ Matches at the end of line \n Replay the nth pattern saved by tagged regular expression \( \) | Matches alternate regular expressions ( ) Groups regular expressions to be matched \{s,e\} Matches occurrences of preceding character s through e times * Matches zero or more occurrences of preceding regular expression + Matches one or more occurrences of preceding regular expression ? Matches zero or one occurrence of preceding regular expression
\( \) Saves the matched pattern enclosed between \( and \) into abuffer
Tagged Expressions (Saving for later replay): Regular expressions may be tagged andsaved in one of the 9 buffers to be replayed later. Symbol used is \( \)
\([A-Z]\)\1 matches repeated (twice) upper case letters
\(.\) \(.\) . \2\1 matches 5 letter palindromes
\(UNIX\).*\1 finds two occurrences of word UNIX in a line
\(UNIX\)\1 finds repeated occurrences of word UNIX in a line
Note: There is no regular expression to mach a newline
-
7/24/2019 OS54 Slides UNIX Shell Scripting
58/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential
ADVANCED SHELL SCRIPTING
5858
-
7/24/2019 OS54 Slides UNIX Shell Scripting
59/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
60/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 6060
Advanced I/O redirection (Contd)
Miscellaneous I/O redirectors
file : Use file as both standard input/output
nfile : Use file as both standard input/output for file descriptor n
n>file, n >> file : redirect file descriptor n to file
n& , n&m , nfile : redirect standard output and standard error to file
&- : close the standard input/output
n&- : close input/output from file descriptor n
>|file : force standard output to file even if noclobber option is set
n>|file : force output to file from file descriptor n even if noclobber is set
errorlog # redirects all standard output to file result and all errors to fileerrorlog
$ command >result 2>&1 # redirects all standard output and error to file result since fd2 is
made a copy of fd1 Example: cat song.txt >newsong.txt 2>&1$ command 2>&1 >result # Shell processed redirection left to right. This is not equivalent toprevious command Example: cat song.txt 2>&1 >newsong.txt
$ exec > test.out # test.out is now standard output, all standard output goes to test.out
$ exec > /dev/tty # Redirection standard output to terminal
$ exec 2> /tmp/error.log # Standard error now redirected
$ exec 4< /cppdemos/file # Open new file descriptor 4
$ exec 4
-
7/24/2019 OS54 Slides UNIX Shell Scripting
61/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
62/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 6262
Advanced Parameter Expansion
Substitution operators
Pattern matching operators
${var:-word} If var is set (exists and not nul l) , return i ts value; else return word
${var:=word} If var is set (exists and not null), return its value; else set var to word and return word
${var:?word}If var is set (exists and not null), return its value; else display var:word and abortscurrent command or shell script. Omitting word displays parameter null or not set
${var:+word} If var is set (exis ts and not null), return word; else return null
${var%pattern}If the pattern matches the end of the variable's value, delete the shortest part thatmatches and return the rest.
${var%%pattern}If the pattern matches the end of the variable's value, delete the longest part thatmatches and return the rest.
${var#pattern}If the pattern matches at the beginning of the variable's value, then deletes theshortest part that matched and return the rest.
${var##pattern}If the pattern matches the beginning of the variable's value, delete the longest partthat matches and return the rest.
Substitution operators are used test existence and substitution of default value of variables.
$ count= ; echo ${count:-5} # Displays 5
$ echo $count # Displays blank line
$ count=2; echo ${count:-5} # Displays 2
$ echo $count # Displays 2
$ count= ; echo ${count:=5} # Displays 5$ echo $count # Displays 5
$ count=2; echo ${count:+5} # Displays 5
$ echo $count # Displays 2
$ count= ; echo ${count:?undefined} # Displays count:undefined
Note: pattern refers to shell wildcard patterns
Note: ${#var} can be used to count the no of characters stored inside var.
$var=hello world; echo ${#var} # displays 11
$ var="thinktwice"
$ echo ${var#th} # Displays inktwice
$ echo ${var#*i} # Displays nktwice
$ echo ${var##*t} # Displays wice
$ echo ${var##*t*i} # Displays ce
$ echo ${var%t*i} # Displays thinktwice
$ echo ${var%i*e} # Displays thinktw
$ echo ${var%%i*e} # Displays th
$ echo ${var#tt} # There is no match so displays thinktwice
-
7/24/2019 OS54 Slides UNIX Shell Scripting
63/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 6363
eval
The eval command can be used to execute a given string as it is ashell command
Syntax $ eval command_string Command_string is first processed by the shell and then passed to eval
for execution
Examples $ count=| wc l ; $ ls l $count # Error, use eval ls l $count $ pro=Perl; pre=pro; echo $pre # Displays pro $ eval echo \$$pre # Displays Perl $ great=>; $ echo Perl $great out # Displays Perl > out $ great=>; $ eval echo Perl $great out # Sends Perl to out $ cat < *.c # Error $ eval cat \< *.c # Displays all .c files Displaying the value of last command line argument
$ echo $$# $ echo \$$#
$ echo $\$# $ eval echo \$$# # Ok
-
7/24/2019 OS54 Slides UNIX Shell Scripting
64/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 6464
Evaluation order
1. split into tokens; if single quote go to step 11; if double quote go to step 5
2. 1st token keyword, read next command and go to step 1
3. 1st token alias, expand alias and go to step 1
4. tilde(~) expansion
5. variable/parameter substitution
6. command substitution
7. arithmetic expression substitution; has double quotes go to step 11
8. word splitting
9. pathname/wildcard expansion
10. command lookup: special built-in, function, built-in, executable file in $PATH
11. run command (if eval is used, output of step 11 is passed to step 1. bash performsbrace expansion between steps 3 and 4)
Attempt writing a function cd that should not only change directory but alsodisplay current working directory as part of command prompt
Every line read from standard input is called pipeline (containing one or more commandsseparated by zero or more | characters). Shell breaks each pipeline into commands, shellthen sets up I/O for the pipeline and then follows above mentioned 11 step process for eachcommand.
Example: Assume nerds home directory is /home/nerd, present working directory is/home/nerd/utils and there are files named data5, test.txt, test.sh, test.c in the current
directory$ alias li=ls i
$ var=5000
$ li $(pwd)/main.c ~nerd/cppdemos/*$(($var/1000))
ls i $(pwd)/main.c ~nerd/cppdemos/*$(($var/1000))
ls i $(pwd)/main.c /home/nerd/cppdemos/*$(($var/1000))
ls i $(pwd)/main.c /home/nerd/cppdemos/*$((5000/1000))
ls i /home/nerd/util/main.c /home/nerd/cppdemos/*$((5000/1000))
ls i /home/nerd/util/main.c /home/nerd/cppdemos/*5
ls i /home/nerd/util/main.c /home/nerd/cppdemos/data5
/usr/bin/ls i /home/nerd/util/main.c /home/nerd/cppdemos/data5
Examples of special built-in: break, continue, eval, exit, readonly, return, export, et, trap, etcExamples of shell built-in: cd, wait, test, etc
Note: command builtin bypasses shells inclusion of functions in the search for commandsto run
Example: $ command cd /home/nerd
-
7/24/2019 OS54 Slides UNIX Shell Scripting
65/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
66/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 6666
Non portable features from bash (Contd)
Extended substitution and pattern matching operators
${var:offset:length}
Substring operation. Returns substring starting at offset and uptolength characters. Offset starts at 0 (first character). If length isommited, all characters form offset and onwards are returned.
echo ${var:5} # Displays twice
echo ${var:2:3} # Displays ink
${var/pattern/string}
First longest match of pattern in $var is replaced with string. Ifpattern begins with #, pattern should match at the beginning of$var. If pattern begins with %, pattern should match with the endof $var
echo ${var/*k/act} # Displays acttwice
${var//pattern/string}
Every longest match of pattern in $var is replaced with string. Ifpattern begins with #, pattern should match at the begining of$var. If pattern begins with %, pattern should match with the end
of $varecho ${var//t?i/X} # Displays XnkXce
echo ${var//i?/X} # Displays thXktwXe
echo ${var/%i*e/X} # Displays thX
var=thinktwice
-
7/24/2019 OS54 Slides UNIX Shell Scripting
67/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 6767
Non portable features from bash (Contd)
Arrays
Index always starts from 0 Array name refers to first element of array at index 0
# Declaring an array
$ declare -a array
$ array[0]="Amit"
$ array[2]="Tiwary"
$ array[1]="Kumar"
$ array[5]="Linux
# Other ways of declaring and initializing
$ friends=( Bill Arnold Randal Eric John )
$ friends=( $(cat namelist) )
Note: array is spare whereas friends is
continuous
$ echo ${array[0]} ${array[2]}
Amit Tiwary
$ echo $array[0] $array
Amit[0] Amit
# echo ${array[*]}
Amit Kumar Tiwary Linux
# echo ${#array[*]}
4 # Total no of elements
# echo ${#array[2]}
6 # Length of element at 2
Note: behavior of *, @, * and @ in arrays is in line with behavior of $*, $@, $* and $@respectively
Values can be deleted using unset
$ unset array # Deletes array
$ unset array[3] # Deletes element and index 3
-
7/24/2019 OS54 Slides UNIX Shell Scripting
68/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
69/75
We enable you to leverage knowledge
anytime, anywhere! 2008 Infosys Technologies
Ltd.
2008 Infosys TechnologiesLtd .
ConfidentialConfidential 6969
Debugging shell scripts
Steps involved
Try to understand the error message Figure out location of error in the script
Fix the error
Using set options -n : read commands but not execute
-v : display shell input lines
-x : execution trace
-t : exit after reading one command and executing it
-e : exit if any command terminates with non zero status
-u : treat unset variables as error condition
$ nl test.sh
1 #!/bin/bash
2 rem=$($1%2) # rem=$(($1%2))
3 if [ "$rem" -eq 0]; then # space missing before ]
4 echo "Even"
5 else
6 echo "Odd"
7 fi
$ sh -v ./test.sh 5
#!/bin/bash
rem=$($1%2)
$1%2
./test.sh: line 2: 5%2: command not found
if [ "$rem" -eq 0]; then
echo "Even"
else
echo "Odd; fi
./test.sh: l ine 3: [: miss ing `]'
Odd
--- After correcting the errors ---
$ sh -x ./test.sh 5
+ rem=1
+ '[' 1 -eq 0 ']'
+ echo Odd
Odd
-
7/24/2019 OS54 Slides UNIX Shell Scripting
70/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
71/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
72/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
73/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
74/75
-
7/24/2019 OS54 Slides UNIX Shell Scripting
75/75