Ryan Hulguin...1. Call your script with /bin/bash –x myscript.sh 2. Insert the line set –x near...
27
Ryan Hulguin
Transcript of Ryan Hulguin...1. Call your script with /bin/bash –x myscript.sh 2. Insert the line set –x near...
Ryan Hulguin
Why Shell Programming is Part of the Seminar Series on High Performance Computing?
• The majority of the Portable Batch System (PBS) submission scripts are written in Bash – This allows you to queue up jobs on clusters/
supercomputers
• Many pre/post processing steps are easier when automated – gathering and analyzing input/output data
Active participation
• It is highly encouraged to try all the examples as we go along
• You may use your own shell from a Linux/Mac terminal
• You may also use the free linux shell provided by http://simpleshell.com
Archive data output example
• Suppose you had a program which generated output for density, velocity and temperature in the files: den.dat, vel.dat, and temp.dat
• This example will copy the files into a folder based on the date, and then archive them
• Before we write the shell script, let’s create these .dat files using IO redirection
Generating .dat files
echo –e “1.0\n2.5\n3.0” > den.dat
echo –e “0.0 3.0\n0.2 2.5\n 0.4 2.0” > vel.dat
echo –e “273.5\n284.2\n286.7” > temp.dat
• The \n inserts a newline or hard return • the –e flag is needed to enable the interpretation the
backslash escapes • the > redirects the output from the terminal to the file
specified • Verify the contents of the .dat files using cat
cat den.dat cat vel.dat cat temp.dat
The Archive data script # Get today’s date TODAYS_DATE=$(date “+%m_%d_%y) # create directory of today’s date mkdir $TODAYS_DATE # For every .dat file copy it to the new # folder for DATFILE in $(ls *.dat) do
cp –v $DATFILE ./$TODAYS_DATE done cd $TODAYS_DATE # Archive all the .dat files tar –zcvf archived_data.tar.gz *.dat rm *.dat cd ..
The test utility
• test evaluates conditional expressions test expression [ expression ]
• If the expression to be evaluated is true, 0 is returned
• If the expression to be evaluated is false, 1 is returned
• If an error occurs, a number greater than 1 is returned
• The return value can be displayed using $?
Comparing integers with test
• Two integers can be compared using the following comparisons
• [ INTEGER1 –eq INTEGER2 ] [ INTEGER1 –ne INTEGER2 ] [ INTEGER1 –gt INTEGER2 ] [ INTEGER1 –ge INTEGER2 ] [ INTEGER1 –lt INTEGER2 ] [ INTEGER1 –le INTEGER2 ]
• The equivalent symbols can also be used The < and > signs need to be escaped \< and \> Make sure there are spaces around = [ INTEGER1 = INTEGER2 ]
Comparing strings with test • Test if the length of STRING1 is nonzero [ -n STRING1 ]
• Test if the length of STRING1 is zero [ -z STRING1 ]
• Test if STRING1 and STRING2 are identical [ STRING1 = STRING2 ]
• Test if STRING1 and STRING2 are not identical [ STRING1 != STRING2 ]
Common file tests
[ -operator FILENAME ]
operator characteristic -d directory -e or -a exists -f file -h or -L symbolic link -r readable by you -s not empty -w writable by you -N has been modified since last
being read
Comparing pairs of files with test
• Test if FILE1 is newer than FILE2 [ FILE1 –nt FILE2 ]
• Test if FILE1 is older than FILE2 [ FILE1 –ot FILE2 ]
• Test if FILE1 and FILE2 refer to the same thing (either hard or symbolic link)
[ FILE1 –ef FILE2 ]
Integer arithmetic
• Integer arithmetic expressions can be evaluated using $(( EXPRESSION ))
• Example COUNT=1 COUNT=$((2+COUNT)) echo $COUNT
• Note how the shell variable did not a $ in front of it
• Input parameters still need the $
The let statement
• The previous arithmetic expression COUNT=$((2+COUNT)) can also be expressed using the let statement
• let COUNT=COUNT+2 • let COUNT+=2
The compound command
• Integer arithmetic expressions can serve as command by dropping the $
• (( EXPRESSION )) • If the EXPRESSION evaluates to zero, then a
return value of 1 is given • If the EXPRESSION evaluates to a nonzero
number, then a return value of 0 is given • Example: (( 3 + 3 )); echo $? (( 3 – 3 )); echo $?
The if…then control structure
• if test-command then commands
fi
• The test-command is a conditional expression i.e. [ $AGE –gt 18 ]
• For every if there must be a matching fi
The if…then…else control structure
• if test-command then commands else commands
fi
• Here we can specify what should be done if the test-command evaluates to false
• You may also use a semicolon to make it look cleaner • if test-command; then
commands else commands
fi
The if…then…elif control structure
• if test-command then commands elif test-command
then commands
fi
• elif is a combination of else and if • Multiple elif blocks can be nested
The while control structure
• while test-command do
commands done
• Example: # Echo the contents of input1.in while read LINE do
echo $LINE done < "input1.in"
The until control structure
• until test-command do
commands done
• This is equivalent to while !test-command • The loop will continue to run until the test-
command evaluates to true
break and continue
• The for, while, or until control structures can be interrupted using a break or continue statement
• break terminates the execution of the loop and then begins executing statements immediately following the done statement
• continue jumps immediately to the done statement and skips all the statements in between
The case structure
• case test-string in pattern-1) commands-1 ;; pattern-2) commands-2 ;; pattern-3) commands-3 ;;
… esac
case structure example
read –p “Choose either A, B, or C: ” letter case “$letter” in
A) echo “You chose A” ;; B) echo “You chose B” ;; C) echo “You chose C” ;; *) echo “You did not choose properly” ;;
esac
The select control structure
• select varname [ in arg … ] do
commands done
• A menu will be formatted with numbers before each item
• When a user selects a number, varname will be set to the appropriate arg
The select example
# Create menu from input files using select
# This particular example only accepts input1.in
select OPTION in $(ls *.in)
do
if [ $OPTION = "input1.in" ]
then
echo "Using $OPTION"
break
else
echo "$OPTION is not a valid choice at this time"
fi
done
Debugging shell scripts
• There may be times where a shell script does something unexpected (due to user error of course)
• It may be helpful to see exactly what commands the shell is currently executing
• This can be done in several ways 1. Call your script with /bin/bash –x myscript.sh 2. Insert the line set –x near the top of the script 3. Modify the SHEBANG to be #!/bin/bash –x
• If you want to monitor the input as well, use -xv instead of -x
References
• A Practical Guide to Linux® Commands, Editors, and Shell Programming by Mark G. Sobell
• Bash Cookbook: Solutions and Examples for Bash Users by Carl Albing, JP Vossen, and Cameron Newham
• http://en.wikibooks.org/wiki/Bash_Shell_Scripting
• These slides will be posted on http://www.nics.tennessee.edu/hpc-seminar-series
