Chapter11

CS2028 - Unix Internals

Chapter 1

General Overview of the Unix

History

System structure

User Perspective

Operating System Services

Assumptions about hardware

AgendaAgenda

3

What is Unix?

What is an operating system?

The lowlevel (first layer loaded) software which handles the interface to system hardware (input/output devices, memory, file system, etc), schedules tasks, and provides common core services such as a basic user interface.all of the software that comes with a system before applications are installed.

4

What Is Unix?

Examples of Operating Systems:Unix and Unixlike: A/UX, AIX, *BSD (Free, Micro, Net, Open, etc), Darwin (Mac OS X), HPUX, Hurd, IRIX, Linux, LynxOS, Minix, QNX, SELinux, Solaris, System V, Triance, TRUSIX, Tru64, UnixWare, VSTa, z/OS, etc.Embedded: BeOS, Chorus OS, eCos, FreeRTOS, IOS, JUNOS, LynxOS, QNX, VRTX, VxWorks, Windows CE, RTLinux, RTAI, Symbian, etc.Others: AOS, JavaOS, MorphOS, Primos, Windows 3.1/95/98/NT/XP/2000/2003, etc.

5

What is Unix? A portable, multi-tasking and multi-user operating system

Portable: runs on many different hardware architectures (Intel x86 and IA-64, Alpha, MIPS, HP PA-RISC, PowerPC, IBM S/390, SPARC, Motorola 680x0, etc.).

Preemptive multi-tasking: several programs can run at the same time (time slices, interrupts, and task switching).

Multi-user: many users can share the computer system at the same time.

6

What is Unix? Other Features

Uses a simple, uniform file model which includes devices and access to other services in a flexible, hierarchical file system.

Written in a high-level language (“ C” ) making it easy to read, understand, change and port.

The command prompt is a simple user process, the Unix shell, which is also a convenient job programming language.

Includes support for regular expressions which are convenient for complex searching.

7

Why Do I Need to Know This?

Ubiquitous: Most big computers and much of the Internet infrastructure runs on some variant of Unix (SUN, SGI, HP, etc.).

Linux is the fastest growing operating system in the market. Source code availability provides the ability to “ get under the

hood” of operating system design and function (and TCP/IP). By understanding the Unix/Linux community and its culture, it is

easy to collect critical information.

HISTORY Multics: 1965 Bell with General Electronic Company and

Massachusetts Institute of Technology.GE645. Multiuser, Large data storage & power, data sharing.

UNIX: 1969 Thompson and Ritchie at AT&T Bell Labs – Unix File system – Thompson (Space travel program) – PDP 7

UNIX moved to PDP – 11 in 1971 UNIX in C (Ritchie) 1977 – UNIX ported into non PDP machine , Interdata 8/32 BSD: 1978 Berkeley Software Distribution- UNIX system III –

V(1983) Commercial Vendors: Sun, HP, IBM, SGI, DEC GNU: 1984 Richard Stallman, FSF POSIX: 1986 IEEE Portable Operating System unIX Minix: 1987 Andy Tannenbaum SVR4: 1989 AT&T and Sun Linux: 1991 Linus Torvalds Intel 386 (i386) Open Source: GPL, LGPL, Cathedral and the Bazaar

9

Unix History

1965 joint project between AT&T Bell Labs, GE(General Electric company), and MIT(Massachusetts Institute of Technology) to develop a new OS.

Goal : develop an OS that could provide computational power, data storage and the ability to share data among multiple users.

Result: Multiplexed Information & Computer Service - MULTICS.

10

Unix History

1969 Bell Labs withdraws from group.

Two Bell Lab scientists, Ken Thompson and Dennis Ritchie, continue research. They were still left without a “ Convenient interactive computing service” *.

* Ritchie, D.M. “ The Evolution of the Unix Time-sharing System” , AT&T Bell Laboratories Technical Journal, Oct. 1984, Vol 63, No.8, Part 2, pp. 1577-1594.

11

Unix History At the same time Ken Thompson

wrote a game “ space travel” in Fortran to run on GECOS OS (Honeywell 635).

The spaceship was hard to control and it was expensive to run. He was told to get the game off his work computer.

12

Unix History Thompson ported the game to a little used PDP-7

computer.

Unics (later Unix) was born as a pun on Multics.

13

Unix HistoryPDP-7

14

Unix History

Dennis Ritchie developed “ B” . Then wrote “ C” a compiled language.

In 1973 entire OS ported to “ C” .

15

Unix History➲ Because of a 1956 Consent Decree AT&T

could not market Unix so it provided it to academia.

➲ Late 70s : Thompson took a sabbatical to teach Unix at UC Berkley –

Birth of BSD Unix. Introduced many new features.

➲ AT&T Bell Labs realized the commercial potential and began distributing System V.

➲ Commercialization of Unix (70s / 80s) AT&T, Sun, SGI, HP, DEC, NCR, IBM.

16

Linus Torvalds

1991 Linux 0.02 is first released to the public. 1994 Linux 1.0 is released.

Three Definitions of Linux

Linux Kernel: The very low-level software that manages your computer hardware and provides a library (POSIX) interface for user-level software. The Linux kernel runs on many platforms (Intel x86 and IA-64, Alpha, MIPS, HP PA-RISC, PowerPC, IBM S/390, SPARC, Motorola 680x0, etc.).

GNU/Linux OS: The Linux kernel plus utility software to provide a useful working environment.

Linux Distributions: The packaging of the Linux Kernel, the GNU/Linux OS and lots of other software to make Linux easy to install, configure, and use (at least for the target audience).

Tux, the Linux Mascot

What is UNIX?

Unix is a Operating System that popular since 1969. System divided into two parts First parts-programs and services such as shell, mail,

text processing packages and source code control systems

Second parts-system calls and hardware for supports to first parts

Example is UNIX system V produced by AT & T. Now Unix is maintained by Open Group.

Unix Features

High level language ‘ C’ Simple user interface Complex programs built from simpler programs Hierarchical file system Consistent format for files-byte streams Consistent interface to peripheral devices. Multi user, multi process system Hides machine architecture – portable Even though Unix is in ‘ C’ we can run other languages like cobol,

fortran, ada, lisp, prolog, basic, pascal, java with their Compiler or interpreter.

Linux Features

“ UNIX-like” operating system “ aims at” standards compliance “ all the features you would expect in a modern UNIX”

preemptive multitasking virtual memory (protected memory, paging) shared libraries demand loading, dynamic kernel modules shared copy-on-write executables TCP/IP networking

other features: SMP support, large memory, large files advanced networking, advanced filesystems efficient, stable, highly portable, supports most device hardware active development community, support, documentation, open source GUIs, applications

Architecture of Unix System

hardware

kernelsh who

date

ed

wc

grep

as

nroff

ldcc

cpp

emacs

Other apps

• OS interacts directly with the hardware • Such OS is called system kernel

Unix: Structure

• Operating System: a system that manages the resources of a computer.• Resources: CPUs, Memory, I/O devices, Network• Kernel: the memory resident portion of Unix system• File system and process control system are two major components of Unix Kernel.

Unix System Kernel

• Three major tasks of kernel: Process Management Device Management File Management

• Three additional Services for Kernel:✯ Virtual Memory✯ Networking✯ Network File Systems

• Experimental Kernel Features:✱ Multiprocessor support✱ Lightweight process (thread) support

Block Diagram of System Kernel

System Call Interface

File SubsystemInterprocess

communication

Scheduler

Memorymanagement

Process

control

subsystemDevice drivers

hardware control

hardware

LibrariesUser ProgramsUser LevelKernel Level

Hardware Level

Process Control Subsystem

• Process Synchronization• Interprocess communication• Memory management: • Scheduler: process scheduling

(allocate CPU to Processes)

File subsystem

• A file system is a collection of files and directories on a disk or tape in standard UNIX file system format. • Kernel’s file sybsystem regulates data flow between the kernel and secondary storage devices.

Hardware Control

• Hardware control is responsible for handling interrupts and for communicating with the machine.• Devices such as disks or terminals may interrupt the CPU while a process is executing.• The kernel may resume execution of the interrupted process after servicing the interrupt.

System Call

• A process accesses system resources through system call.• System call for

❂ Process Control: fork: create a new process wait: allow a parent process to synchronize its

execution with the exit of a child process. exec: invoke a new program. exit: terminate process execution❂ File system: File: open, read, write, lseek, close inode: chdir, chown chmod, stat fstat others: pipe dup, mount, unmount, link, unlink

User Perspective-File system

31

Architecture of Unix Unix file system

root (/)

bin

dev etc

tmpvar

home

spool

mail

(users’ mail)

log

usr

X11R6

bin

(users’ accounts)

32

Typical Directories

➲ / : Root of the tree. Where it starts.

➲ bin, sbin, usr/bin: software for the shells and most common Unix commands.

➲ dev: short for devices, holds the files necessary to operate peripherals such as printers and terminals.

➲ home: contains the home directories of users (/export/home on sun computers).

33

Typical Directories

tmp: holds temporary files.

var: contains files that vary in size; (Mail directories, printer spool files, logs, etc.)

etc: administrative files such as lists of user names and passwords.

34

Typical Directories

usr: Contains application programs

lib: Contains libraries for programs

proc: a pseudo-filesystem used as an interface to kernel data structures.

Unix File System

File system is organised as a heirarchy of directories It starts from a single directory called root(represented

by a /).Types

Ordinary files Directories Special files Pipes

Directory

Directory is file containing list of files and subdirectories It has fixed size records of 16 bytes each which contains

- a filename(14 bytes)- an inode number (2 bytes) which acts as a pointer

to where the system can find info about the file.

Special Files

Special files are contained in the directory /dev. They are used to represent a real physical device such

as a printer, tape device etc. Ex: Special device - /dev/null(unwanted output can be

redirected).Pipes UNIX allows us to link commands together using a pipe. The pipe acts as a temporary file which only exists to

hold data from one command until its read by another.

ex: command1 | command2 | command3....

Processes

• A program is an executable file.• A process is an instance of the program in execution.• For example: create two active processes

$ emacs &$ emacs & $ psPID TTY TIME CMD12893 pts/4 0:00 tcsh12581 pts/4 0:01 emacs12582 pts/4 0:01 emacs$

Processes

• A process has text: machine instructions

(may be shared by other processes) data stack

• Process may execute either in user mode and in kernel mode.• Process information are stored in two places:

❋ Process table ❋ User table

System call: fork()

• fork: the only way for a user to create a process in Unix operating system.• The process that invokes fork is called parent process and the newly created process is called child process.• The syntax of fork system call:

newpid = fork();• On return from fork system call, the two processes have identical copies of their userlevel context except for the return value pid. • In parent process, newpid = child process id• In child process, newpid = 0;

/* forkEx1.c */#include <stdio.h>

main(){ int fpid; printf("Before forking ...\n"); fpid = fork(); if (fpid == 0) { printf("Child Process fpid=%d\n", fpid); } else { printf("Parent Process fpid=%d\n", fpid); } printf("After forking fpid=%d\n", fpid); }

$ cc forkEx1.c o forkEx1$ forkEx1Before forking ...Child Process fpid=0After forking fpid=0Parent Process fpid=14707After forking fpid=14707$

/* forkEx2.c */#include <stdio.h>

main(){ int fpid; printf("Before forking ...\n"); system("ps"); fpid = fork(); system("ps"); printf("After forking

fpid=%d\n", fpid);}

$ forkEx2Before forking ... PID TTY TIME CMD 14759 pts/9 0:00 tcsh 14778 pts/9 0:00 sh 14777 pts/9 0:00 forkEx2 PID TTY TIME CMD 14781 pts/9 0:00 sh 14759 pts/9 0:00 tcsh 14782 pts/9 0:00 sh 14780 pts/9 0:00 forkEx2 14777 pts/9 0:00 forkEx2After forking fpid=14780$ PID TTY TIME CMD 14781 pts/9 0:00 sh 14759 pts/9 0:00 tcsh 14780 pts/9 0:00 forkEx2After forking fpid=0

$ ps PID TTY TIME CMD 14759 pts/9 0:00 tcsh$

System Call: getpid() getppid()

• Each process has a unique process id (PID). • PID is an integer, typically in the range 0 through 30000.• Kernel assigns the PID when a new process is created.• Processes can obtain their PID by calling getpid().• Each process has a parent process and a corresponding parent process ID.• Processes can obtain their parent’s PID by calling getppid().

/* pid.c */#include <stdio.h>#include <sys/types.h>#include <unistd.h>

main(){ printf("pid=%d ppid=%d\n",getpid(), getppid());}

$ cc pid.c o pid$ pidpid=14935 ppid=14759$

/* forkEx3.c */#include <stdio.h>#include <sys/types.h>#include <unistd.h>main(){ int fpid; printf("Before forking ...\n"); fpid = fork(); if (fpid == 0) { printf("Child Process fpid=%d pid=%d ppid=%d\n", fpid, getpid(), getppid()); } else { printf("Parent Process fpid=%d pid=%d ppid=%d\n", fpid, getpid(), getppid()); } printf("After forking fpid=%d pid=%d ppid=%d\n", fpid, getpid(), getppid());}

$ cc forkEx3.c o forkEx3$ forkEx3Before forking ...Parent Process fpid=14942 pid=14941 ppid=14759After forking fpid=14942 pid=14941 ppid=14759$ Child Process fpid=0 pid=14942 ppid=1After forking fpid=0 pid=14942 ppid=1

$ ps PID TTY TIME CMD 14759 pts/9 0:00 tcsh

System Call: wait()

• wait system call allows a parent process to wait for the demise of a child process.

#include <stdio.h>#include <sys/types.h>#include <unistd.h>main(){ int fpid, status; printf("Before forking ...\n"); fpid = fork(); if (fpid == 0) { printf("Child Process fpid=%d pid=%d ppid=%d\n",

fpid, getpid(), getppid()); } else { printf("Parent Process fpid=%d pid=%d ppid=%d\n",

fpid, getpid(), getppid()); } wait(&status); printf("After forking fpid=%d pid=%d ppid=%d\n",

fpid, getpid(), getppid());}

$ cc forkEx4.c o forkEx4$ forkEx4Before forking ...Parent Process fpid=14980 pid=14979 ppid=14759Child Process fpid=0 pid=14980 ppid=14979After forking fpid=0 pid=14980 ppid=14979After forking fpid=14980 pid=14979 ppid=14759$

System Call: exec()

• exec() system call invokes another program by replacing the current process• No new process table entry is created for exec() program. Thus, the total number of processes in the system isn’t changed.• Six different exec functions: execlp, execvp, execl, execv, execle, execve, (see man page for more detail.)• exec system call allows a process to choose its successor.

/* execEx1.c */#include <stdio.h>#include <unistd.h>

main(){ printf("Before execing ...\n"); execl("/bin/date", "date", 0); printf("After exec\n"); }

$ execEx1Before execing ...Sun May 9 16:39:17 CST 1999$

/* execEx2.c */#include <sys/types.h>#include <unistd.h> #include <stdio.h>

main(){ int fpid; printf("Before execing ...\n"); fpid = fork(); if (fpid == 0) { execl("/bin/date", "date", 0); } printf("After exec and fpid=%d\n",fpid); }

$ execEx2Before execing ...After exec and fpid=14903$ Sun May 9 16:47:08 CST 1999$

Building Block Primitives

User to create complex program effectively. Users capability to (1)redirect I/O.

User much access three files Std input file Std output file Std error file

Example: ls, ls > ssn.txt, mail x < letter, 2>(err) (2)Pipe: mechanisms allows a stream of data to be

passed between reader and writer processes. Example: grep ssn file1.c file2.c file3.c file4.c | wc -l

< file - means open a file for reading and associate with STDIN. << token - Means use the current input stream as STDIN for

the program until token is seen. > file - means open a file for writing and truncate it and

associate it with STDOUT. >> file - means open a file for writing and seek to the end and

associate it with STDOUT. This is how we append to a file using a redirect.

n>&m means redirect FD n to the same places as FD m. Eg, 2>&1 means send STDERR to the same place that STDOUT is going to.

FD 0 – STDIN , FD 1 – STDOUT, FD 2 -STDERR

Operating System Services

Controlling Scheduling Memory Management

Allocating to main memory Allocating to secondary memory

Process management – controlled access to devices Transparent Service – Kernel hides the type of file from

user.

ASSUMPTIONS ABOUT H/W:User mode and Kernel mode

• At any given instant a computer running the Unix system is either executing a process or the kernel itself is running• The computer is in user mode when it is executing instructions in a user process and it is in kernel mode when it is executing instructions in the kernel.• Executing System call ==> User mode to Kernel mode perform I/O operations system clock interrupt

Difference between two modes

Process in user mode can access their own instruction not kernal instructions and data of other processes but in kernal mode can access both. Example: virtual address space access only in kernal mode.

Some machine instructions are privileged. Example: the process status register is not capable to access

from the user mode.

Kernel runs on behalf of the user processes. Part of each user process.

Interrupts and Exceptions

Interrupts: for multitasking CPU is shared by many processes

Blocks low priority interrupts but services high priority interrupts.

Exception refers to unexpected events caused by process.

Example: addressing illegal memory, executing privileged instructions and dividing by zero.

Restart the instruction after handling exception. Exceptions in middle of an instruction whereas interrupts

in between 2 instructions.

Processor Execution Levels

Kernal prevent the occurrence of interrupts during critical activity.

Manipulating linked list pointers High priority interrupts

Machine Errors Clock Disk

Lower Priority interrupts Network Devices Terminals Software Interrupts

Masking a level will block below and allow above.

Memory Management

Kernal resides in main memory permanently Compiler generates virtual address space Kernal Mapping virtual address to physical address

Chapter11

Documents

Transcript of Chapter11