Software Tools Design
description
Transcript of Software Tools Design
Software Tools Design
Architecture – the design and and specification of overall system structure
• Classical architecture– Design Representations
• plan, elevation, perspective
• drawings/models
– Architectural Styles• Victorian
• Colonial
• Spanish
– Requirements,Constraints• circulation patterns
• acoustics
• lighting
• air flow
• Software architecture– Design Representations
• multiple views
• models for different roles (customer, developer)
– Architectural Styles• Client/server
• Layered
• Object-Oriented
– Requirements,Constraints• reliability, fault tolerance
• performance
• maintainabililty
Software Architecture• As size/complexity of software increases,
design/spec of overall system structure becomes more significant than the choice of algorithms/data structures
• Currently not a well-defined terminology available for software arch.– but good software engineers do use architectural
principles
• Software architecture: defines a system in terms of computational components and interactions among them.
Architectural Elements– components: define computation,data storage
• clients/servers, databases, filters,...
– connectors: define the interactions between components
• procedure calls, pipes, event multicast, …
– configurations: collections of interacting components and connectors
• diagrams of boxes and arrows, formal mathematical models
– constraints: provide semantic understanding of the style requirements
• what makes the style correct, legal, efficient, secure, reliable
Architectural Styles• Dataflow systems
– batch sequential– pipes and filters
• Call-and-return systems– main program and subroutine– OO systems– Hierarchical layers
• Independent components– communicating processes– event systems
Architectural Styles, cont.• Virtual machines
– interpreters– rule-based systems
• Data-centered systems (repositories)– databases– hypertext systems– blackboards
This class covers:• Pipes and Filters: Unix shell
• Object-Oriented: Java
• Client/Server: – Web Applications and XML, Javascript– CGI programming with Perl– Applets, Servlets
Architectural Style Questions:• What is the design vocabulary? (components,
connectors?)• What are the allowable structural patterns?• What is the underlying computational model?• What are the essential invariants of the style?• What are some common examples of its use?• What are the advantages/disadvantages of using
the style?• What are some common specializations?
Common Framework• Use a common framework in order to
compare different arch. styles• framework: collection of computational components
(components), description of interactions (connectors), set of (constraints) on how they can be combined
• For each style:• structural pattern
• underlying computation model
• essential invariants
• common examples
• advantages/disadvantages
• common specializations
Pipes & Filters• Components:
– filters (processing components): incrementally transform data (stream to stream) from a set of inputs to a set of outputs.
• Connectors:– pipes: conduits for the data stream. transmit output of
one filter to input of next filter
• Invariants: • filters (components) must be independent entities; • little context info is utilized, no state info preserved between
instantiations• filters do not know the identity of their upstream and
downstream filters.
• Examples: unix shell, compilers
Pipes & Filters• Advantages:
– neat design; support reuse; easy to maintain
– hierarchical decomposition
– black box approach
– easy to reuse; support concurrency
– support specialized analysis (deadlock, throughput)
• Disadvantages:– batch organization of processing
• not interactive
– difficult to synchronize pipes, order filters
– performance penalty : forces lowest common denominator of data (often ASCII)
– parsing/unparsing overhead
– infinite queues may be required
– possible deadlock with finite buffers
Data abstraction, OO• Components: objects/instances of ADT
• Connectors: procedure invocation
• Variations:– allow multiple interfaces
• (member function access)
– allow concurrency
• Advantages – encapsulation/info hiding
• can change implementation without affecting clients
– inheritance: shared definitions of functionality
– polymorphism: actual operation determined at runtime
– reuse/maintenance: promotes separation of concerns
• Disadvantages– objects must know id for interaction, change of id requires
notification
Event-based, Implicit Invocation• Components: objects/processes with interfaces that:
– define allowable incoming, outgoing events• Connectors: event procedure bindings
– procedures are registered with events
– components interact by announcing events
– announcing component does not know who registered for the event
– upon event occurrence, associated procedures are implicitly invoked
– order of invocation is non-deterministic
• Main invariant: event announcer does not know who will respond
• Examples: GUIs, databases
• Advantages:– strong support of reuse (event registration)
– eases system evolution (replace components without affecting interfaces)
• Disadvantages:– less control of computation, data exchange difficult
– difficult to reason about correctness
– harder to understand flow, test
Client/Server• Components: server contains processing and data
resources required by clients
• Connectors: packets of client requests and server replies
• Examples: distributed systems, web applications
• Advantages:– strong support of reuse
– support parallel computation
• Disadvantages:– complexity
– performance (data transmission)
– security
The Structure Of a Unix OS
• Major Unix OS features:
– Kernel
– Shell
– File System
• Kernel
• Manages memory and allocates it to each process
• Schedules work done by the CPU
• Organizes transfer of data from one part of machine to another
• Accepts instructions from shell and carries them out
• Enforces access permission on the file system
UNIX OS• Shell• Command interpreter• Create customized environments• Write shell scripts• Define command aliases• Manipulate command history• File and command completion• Edit the command line
Unix OS• File System
• Logical method for organizing and storing large amounts of information.
• Easy to manage.
• File: basic storage unit.
• Types:– ordinary file (stores info)– directory (holds other files and directories)– special file (represents physical devices like
printers, terminals, etc)– pipe (temporary file for command linkage)
Unix Operating System Structure
• OS mediates between the user and the computer
User
Application Programs
Kernel
Hardware
Shell
UNIX: Multi-user Multi-tasking• More than one user can run at the same time and
more than one task can run at the same time• In Unix, each program is started as a process.• A process is a program in execution.• Usually only one copy of a program, but there
may be many processes running the same program.
• To each interactive user (window):– only one process in foreground– may have several processes in background
Unix Process• The fundamental unit of a multi-user UNIX
operation.
• A program in execution, doing some work.
• Multiple users and multiple processes.
• CPU is switched rapidly from one process to the other.
• The shell uses the UNIX process structure to carry out user commands.
• Command Execution From Shell:• Typically a shell process forks a child process to
execute the command.• The child process execs the utility program named
in the command.• Now two processes are running for the user - the
shell (parent) and the utility (child) process.• Running the command in the foreground: shell
waits until the child process is complete.• Running the command in the background:
– the shell prints the process id of the child process and waits for the next command.
httpdlpd
Processes
/etc/init
kernel Process 0: Kernel bootstrap. Start process 1.
Process 1: create processes to allow login.
inetd/etc/getty
forkexec
/bin/login
exec
shell
exec
/etc/getty
forkexec
/bin/login
exec
shell
exec
condition terminal for login
check password
command interpreter
kernal mode
user mode
Unix Process• Init process
– last step in booting procedure
– create other processes to allow the users to login
• Getty process
– conditions for terminal connection
– wait for user-id
• display login on the screen
• Login process
– check password with the uid
– execute .profile or .login (depends on default shell)
– display shell prompt
• Shell process (command line interpreter)
• Shell prompt ($, %)
UNIX Process• Process environment
– Process id, parent-process-id, process-group-id– Opened files– Working directory– File creation mask– User ID, Group ID– Resource limits– Environment variables– Code
• A child process inherits parent’s environment.
Processesuse ps to see the processes that you are running.
$ ps
PID TTY TIME CMD
221 pts/4 4:15 netscape
201 pts/4 0:05 bash
215 pts/4 1:15 emacs-19– use “&” to execute a task in background
• Example: $ sort infile > outfile &
– use ctrl-Z to suspend the foreground task, and then use bg.
– use fg to move a background task to foreground.
Shell: Command Interpreter• Bourne Shell: the default shell (sh)
– original unix shell
– does not have interactive features of newer shells
– widely used for writing shell scripts
– standard on Unix System V
• C Shell (csh): available on BSD Unix and most other systems
– with syntax similar to the C language
– with many enhancement over the Bourne shell.
• Korn Shell (ksh): AT&T’s answer to C Shell
– combine features of Bourne and C shells
– very efficient
• Other shells: tcsh, bash
Day-to-Day Use
Command Function Meaningcat Display a file
conCATenatecp Copies a file CoPymv Renames a file or moves it MoVerm Delete files ReMovelpr Sends a file to a printer Line Printerlp (Sys V)ls Lists the contents of a directory LiStchmod Changes the “mode” of permissions Change MODepwd Shows the current directory Print WorkingDircd Change current directory Change Dirmkdir Create a directory MaKe DIRrmdir Delete a directory ReMove DIRps Shows the processes on the system Process Statusman Shows info. about commands Manualdf Shows file system status Disk Filedu Shows the space allocation of files Disk Utilizationgrep Search for patterns in files
Standard Command Format• command [options] [arguments]
wc [-c | -m | -C] [-lw] [file …]
• stuff in brackets is optional
• boldface words are literals (must be typed as is)
• italicized (or <> enclosed) words are args (replace appropriately)
• Options modify how the command works
• command [options] [--] [<file> …]
options ::= -option white-space options*
option ::= noargoption* argoption | noargoption+
noargoption ::= letter
argoption ::= letter whitespace string
$cc -po zap zap.c
Some Examples• ls [-alRF…] file-list
– a for listing all files including the dot files– l for long format (file type, permissions, #links,
owner, group, etc)– R for recursive, list subdirectories.– F for listing directories with a trailing /
• ps [<options>]– List the information about running processes– Example:
%ps -el # print the info about all processes (e) in the long format (l)
I/O Redirection• Redirection and Pipe
“>“ redirects standard output (screen) to a file
E.g. ls > dirlist
“<“ redirects standard input (keyboard) from a file
E.g. sort < infile > outfile
“|” pipe output of program 1 to input of program 2
E.g. who | wc
Or getit < in | check_it | process_it | format_id > out
“>>“ appends output to a file
E.g. ls -l >> oldfile
Exercise: find out the definition of “tee”.
Sequential vs. Concurrent Process• Sequential:
%date
%ps -ef OR %date; ps -ef; who
%who
• Concurrent:%pgm1 & prgm2 > file1 & pgm3
%make > log &
%sort +1 pdir; ((pr dir | lpr) & sort +1 local))
File Name Expansion• Each shell program recognizes a set of special characters
called meta characters.
• The metacharacters are used to create patterns to match filenames and command names.
• Bourne and Korn shell meta/wildcard characters
– * matches any string (including null)
– ? matches any one character
– [a-dA-D] matches any one character in the range
– [!AB123] matches a char not in this range
– \ escape
• ~<username> : (not bourne shell) the home dir of the user.
File Name Expansion• Assume we have the following files under
the current directory: 120, 235, 2#5, a.out, c22, c*2, Doc.1, Doc.2, Doc.3, one.c,two.c, three.c
ls *.c ls c*2
ls [a-z]*[!0-9] a.* *.*
ls ??? cd ~foo
ls *
Filters• Most UNIX utilities are filters• A filter is a program which
– reads input (text) from standard input only– writes output to standard output only– writes error to standard error only– may use temporary files for intermediate results
• Filters can be combined to work together using pipes
• Pipe: takes stdout of one command and uses it as stdin of another command
ls | wc
Command Alias• Assign your own name for a command • Syntax is shell dependent
alias ll ‘ls -l’ C shell
alias ll=‘ls -l’ Korn, Bourne shell• Displaying the value of an alias
alias ll (displays value)
Unix File Systems• File: a collection of data
– Just a sequence of bytes– no record or block structure required
• Directory– A file that contains information for files– distinction between a directory and a file
• system can alter the contents of a directory
– rooted tree file structure (inverted tree)• directories can contain both files and other
directories info.
5binbinetc
Unix File System Road Map
• Special files: /dev/* represents I/O devices.
/
/etc /var /bin /tmp /usr /mnt /home
sue john
passwdhosts…...
spool admprintermail(in)mail(out)uucp
messageswtmp…
libinclude
/dev
File Systems and the I-nodes• Each disk drive contains one or more file systems
– Each file system occupies a number of cylinder groups.
– Each file system has a superblock, an i-node table and files
• The superblock is at a fixed location relative to the beginning of the file system. It contains a description of the file system.
• One can find the location of the I-node table thru superblock.
• Each entry of the I-node table is an I-node, which uniquely represents a file in the file system.
• An I-node contains uid, gid, time of creation, modification, and access, permissions, location of the file on the disk.
superblock I-node table file1 file2 free file3 free
• An I-node includes the following information:– File access permissions– Owner and group ids– File link count– Time of last modification– Time of last file access– Location of blocks for each regular file and
directory file– Device id for special files
I-node 2 contains the location of the block with the root directory
chmod• Change the access permissions of a file
• chmod <permissions> <filename>– permissions can be specified as 3 octal digits,
<user,group,others>, the three bits of an octal means r,w,x
• Example: chmod 755 test
– permissions can be specified as +x, or u+x, or g+r, …
– “chmod +s test” sets the set-uid bit for file test.
• If a directory has “x” in its permission, the dir is searchable, ie., one can do “ls” on the directory.
Pathnames• Every file and directory in the file system can
be identified by a “full path name” (route from root to file)/home/sue/email/f1
• Relative path name– location relative to current directory
. Current directory
.. Parent directory
– if cwd is /home/sue:ls email
ls ./email
cd ..
ls ../fred
fred
/
home
sue
docs email
f1 f2
Some Details• cp [-ir…] file1 file2
cp [-ir…] file-list directory– i for interactive. prompt use whenever a file will be
overwritten– r for recursive. copy a directory tree
• ls [-alRF…] file-list– a for listing all files including the dot files– l for long format– R for recrusive. list the all subdirectories.– F for listing directories with a trailing /
• date [+format]– % date ‘+%h %d, 19%y’
Some Details (cont.)• wc file-list
– display the number of lines, words and characters• more file-list
– Browse through text files on page at a time.• head [-n …] file-list
– Display the first n lines of the files (default=10)• tail [+n|-n| -f| …]
– Display the last few lines in the files (default = 10)– Example:
%tail +5 foo # display the last parf of foo starting from line 5
%tail -5 foo # display the last five lines of foo
%tail +30 foo | head -15 | more #display line 30-45 of foo
%tail -f foo # wait and display the new lines appended to foo
Some Details (cont.)• cut -c list file• cut -f list [-dChar] file
– Cut out selected characters or fields from each line of a file– Examples:
%cut -c 1-5,15-20 foo # extract chars 1-5 and 5-20 from each line of foo.
%cut -f 1,3 -d” “ moo # extract field 1 and 3 from each line of moo.
• paste file1 file2– Concatenate corresponding lines of the given input files– Example (reverse two fields of the file abc)
%cut -f1 abc > abc1%cut -f2 abc > abc2%paste abc2 abc1 > xyz
Some Details (cont.)• grep, egrep, fgrep
grep [-nv...] pattern [file-list]– Search the input text files for lines
matcing the pattern
%grep Unix doc.1 # Display the lines in doc.1 that contains “Unix”
%grep -n Unix doc.* # Display the lines with line numbers
%grep -v Unix doc.1 # Display the lines which do not contain “Unix”
Some Details (cont.)• sort [-tC…] [-o outfile] [field-list] [file-
list]– sort the files
%sort +1 list1 # sort list 1 starting from field 2 to the end of the line
%sort +2-3 list2 # sort list2 on the third field
%sort -n -o list4 list3 sort list3 numerically and place the output in list4
diff • diff file1 file 2
– Display different lines that are found when comparing two files
– It prints a message that users ed-lide notation (a - append, c - change, d -delete) to describe how a group of lines has changed.
– It also describes what changes (additions, deletions and changes) need to be made to the first file to make it the same as the second file.
diff Example
file1 file2 file3
apples apples oranges
oranges oranges bananas
bananas kumquat s kiwis
peaches
diff (cont.)%diff file1 file2
3c3
<bananas
-----------------
>kumquats
%diff file1 file3
1d0
<apples
3a3,4
>kiwis
>peaches
comm file1 file2– Takes two sorted text fields and print common lines and lines
which appear exclusively in one file on separate columns.– column1: lines only in file1, column 2: lines only in file2; col 3:
comm– Example
file1 file2 %comm file1 file2
apple apple apple
banana cantaloup banana
grape grade cantaloup
orange kiwi grape
lemon kiwi
%comm -12 file1 file2
apple
grape
tr [-csd…] pattern1 pattern2– Translate input character to output character based on the
input and output patterns– Example%tr ‘[A-Z]’ ‘[a-z]’ <in >out
# xlate all letters to lower case.%tr -s ‘\012\011\040’ ‘\012\012\012’ < in > out
# xlate newline, tab and blank chars to newline chars and squeeze (-s) consecutive newline char into one
%tr -cs ‘[a-z][A-Z]’ ‘[\012*]’ < in > out# change all non-aplphbetic (-c) chars to newline
chars and squeeze consecutive newlne char into one.
The * (asterisk) tells tr to repeat the # (number sign) enough times
to make the second string as long as the first one. %tr -d ‘\040’ < in > out
# delete all blanks.
find <dir-name> <exp>– Recursively search the directory tree rooted at
<pathname> and find all files whose names satisfy <exp>
– find pathname search options action options– There are many details in the expression.– Examples:%find . -name \*.doc -print # list all files names ending
with .doc%find /etc/source -atime 2 -print
# print the names of the files under /etc/source whose lst access time was 2 days ago.
find <dir-name> <exp>%find . -name “[a-z]*” -exec rm {} \;
# remove the files under the curent directory whose names begin with a lower case letter.
%find / \(-name a.out -o -name “*.o” \) -atime +7 -exec {} \;
# remove the object and binary executable files under the root directory which have not be accessed more than 7 days.