Lesson 6-Using Utilities to Accomplish Complex Tasks

Overview

Creating and executing a script to list user information.

Listing directories and files separately.

Identifying changes made to files in a directory.

Creating a complex script.

Creating and Executing a Script to List User Information

Creating a script.

Executing a script.

Creating a Script

A series of commands can be executed

repeatedly/automatically by placing them in a script file.

Placing commands in scripts helps avoid errors and save

time.

The vi editor can be used to create a file.

The ":wq" commands are used to write to a file and return

to the shell.

Executing a Script

The commands in a file can be executed in two ways:

Instructions can be given to the current shell to read the file

(source) and execute all the commands.

The script file can be made executable, and a child shell can

be started to read the script file and execute the commands.

Executing a Script

The command used to make the shell executable is “chmod

+x filename”.

By default, the output of the script is redirected to the

screen.

The output of the script can be redirected to a file.

Listing Directories and Files Separately

Selecting only directories / files:

The “ls –F | grep /” command is used to select only lines that

contain directory names.

The “ls –F | grep / | column” command can be used to put the

output into columns.

The “ls –F | grep –v / | column” command is used to select only

files.

Listing Directories and Files Separately

Selecting Directories Using Multiple Utilities

Listing Directories and Files Separately

Interpretation by grep Using –v option

Listing Directories and Files Separately

Adding comments to scripts:

The # (pound) sign is used to add comments to scripts.

Ensure that the # sign is placed at the beginning of each line.

The pound (#) sign cannot be used inside a long command

line.

Identifying Changes Made to Files in a Directory

Collecting data about files in a directory:

The “ls –l directory name” command can be used to provide

information about the files in a particular directory.

The output of the code can also be redirected to another file by

the command "ls -l directoryname > filename”.

The information includes the current status of each file, such

as permissions, date of modification, owner, etc.

Identifying Changes Made to Files in a Directory

Comparing file information:

The “comm” command is used to identify the common aspects

between two files.

It requires two arguments – the names of the two the files that

are to be compared.

The command gives three columns of output – lines unique in

the first file, lines unique in the second file, and lines in

common.

Identifying Changes Made to Files in a Directory

Comparing file information:

The “diff” command is used to identify the differences between

two files.

It requires two arguments – the names of the two files to be

compared.

The command gives three columns of output – lines unique in

the first file, lines unique in the second file, and lines common

to the two files.

Creating a Complex Script

Determining the number of unique words in a file.

Removing punctuation.

Converting characters to lowercase.

Putting each word on a line.

Removing blank lines.

Sorting the lines.

Determining the Number of Unique Words in a File

The “uniq” utility is used to:

Provide a list of unique words in a file.

The number of times that each word is used.

The number of unique words.

Determining the Number of Unique Words in a File

The uniq utility outputs both, unique lines as well as single

copies of any lines that are duplicate and adjacent.

The utility deletes identical lines.

The uniq utility deletes only those lines that are identical

and adjacent.

Determining the Number of Unique Words in a File

The following things must be ensured for the uniq utility to

work properly:

Punctuation and blank lines must be removed.

Differences in case for the same word must be reconciled.

The words must be one word to a line in a sorted order.

Removing Punctuation

Shell and Utility Interpretation

Converting Characters to Lowercase

Shell and tr Interpretation

Putting Each Word on a Line

To remove duplicate words using the uniq utility, the data

must be modified to ensure that each word is on a line by

itself.

The new line character “\n” is used to separate lines in a

file.

The ASCII character code 012 can also be used to separate

lines in a file.

Putting Each Word on a Line

The “man ascii” command is used to examine the ASCII

characters and their associated codes.

TAB characters can also be replaced with new line

characters.

Putting Each Word on a Line

Using tr Utility to Replace Characters

Removing Blank Lines

A View from the Shell

Sorting the Lines

The “sort” command can be used to sort the output such

that all lines containing the same word are on adjacent

lines.

The “uniq –c” command can be used to remove all

duplicate lines.

The “sort –rn” command can be used to list the most

frequently used words first.

Summary

A series of commands placed in a script file can be

repeatedly executed, avoiding errors and saving time.

The ls utility provides a list of all the files and subdirectories

in the current directory.

The # sign is used to add comments to script files.

The pipeline feature of UNIX is very useful and is central to

manipulating data effectively with UNIX utilities.