npp_ch1_2

Filehandles:

• connections to outside world: disk file, hardware device, local process, remote process, “bit bucket”

• STDIN, STDOUT, STDERR are opened automatically

$ ./myprog.pl > /dev/null

the bit-bucket

redirects the STDOUT filehandle inside

myprog.pl from the terminal to the bit bucket

finding out about a process and its filehandles:

[root@joyous fd]# cd /proc[root@joyous proc]# ls1 11230 1791 2346 27177 33 4611 execdomains mounts1052 11236 1792 2395 27180 3353 5 fb mtrr107 1 1240 1820 2410 …

[root@joyous proc]# cd 2410[root@joyous 2410]# ls -Fattr/ cmdline environ fd/ maps mounts stat status wchanauxv cwd@ exe@ loginuid mem root statm task/

[root@joyous 2410]# cat cmdlinexinetd–stayalive–pidfile/vst/run/xinetd.pid

[root@joyous 2410]# cd fd[root@joyous fd]# ls -latotal 6dr-x------ 2 root root 0 Jan 8 13:25 .dr-xr-xr-x 3 root root 0 Dec 27 15:22 ..lr-x------ 1 root root 64 Jan 8 13:26 0 -> /dev/nulllr-x------ 1 root root 64 Jan 8 13:26 1 -> /dev/nulllr-x------ 1 root root 64 Jan 8 13:26 2 -> /dev/nulllr-x------ 1 root root 64 Jan 8 13:26 3 -> pipe:[5522]l-wx------ 1 root root 64 Jan 8 13:26 4 -> pipe:[5522]lrwx------ 1 root root 64 Jan 8 13:26 7 -> socket:[5524]

file handles

char** argv

Examples of STDIN and STDOUT:

$input = <STDIN>;chomp($input);printf STDOUT “You said, \”$input\”\n”;

chomp($input = <>);printf “Again you said, \”$input\”\n”;

strings in perl don’t drop the\newline character so chomp()does this (if it exists)

STDIN is the default

= returns an lvalue and not, as in C, the value of theexpression on the right-hand side of the operator

redirection:

• The internal filehandles, STDIN and STDOUT, are disconnected from the the keyboard and terminal and attached to data_in.txt and data_out.txt respectively. STDERR is untouched.

• In this case STDERR is also redirected to the same file as STDOUT (filehandle 2 follows filehandle 1).

$ myprog.pl < data_in.txt > data_out.txt

$ myprog.pl < data_in.txt > data_out.txt 2>&1

IO operations 1:

• <> is context sensitive. This means the function “knows” what kind of variable is waiting for its return value and returns the correct type of value.

$line = <FILEHANDLE>; # scalar context@lines = <FILEHANDLE>; # list context

while ( <> ) { printf “ found a gnu\n” if /GNU/i;}

void context; input assigned to $_ comparison made against $_

ignore case

IO operations 2:

• Both read a maximum of $length bytes into $buffer from FILEHANDLE after skipping $offset bytes.

• The difference is that read() blocks until exactly $length bytes are read or end-of-file is encountered; sysread() returns immediately after reading at least one byte. So it blocks only if no input at all is available.

$bytes = read(FILEHANDLE, $buffer, $length [, $offset]);$bytes = sysread(FILEHANDLE, $buffer, $length [, $offset]);

= actual number of bytes read or$bytes = 0 if end-of-file and no data = undef if error occurred

IO operations 3:

$result = print FILEHANDLE $data1, $data2 …;$result = print $data1, $data2 …;$result = printf $format, $data1, $data2 …;$bytes = syswrite( FILEHANDLE, $data [, $length [, $offset]]);

filehandle name optional; default STDOUT

no comma

read man –S 3 printf man pagesto understand formatting

writes $length bytes from $data toFILEHANDLE starting at location $offset;returns the number of bytes actually written

print blocks until all data is written;writes to buffer

writes what it can and returns

$bytes = 0;while ( $bytes < $length ) { $len = syswrite(FILEHANDLE, $data, $length - $bytes, $bytes); $bytes += $len; }

in case syswrite() fails towrite everything in a single operation; no buffering

IO operations 4:

• select() changes the default output filehandle for print()

$previous = select FILEHANDLE;

open(FILEOUT, “>myfile”);print “Hello, world!\n”$previous = select FILEOUT;print “Hello, world!\n”select $previous;

prints to STDOUT

prints to FILEOUT

makes STDOUT default again

End of File:

• What does EOF mean when reading from a filehandle?

• How we see the EOF condition depends on whether we are reading one line at a time or in a byte stream.

file: end-of-filekeyboard: ^D (Unix), ^Z (Windows)socket: other end closes the socket

EOF Byte Stream:

• read() and sysread() return 0 on EOF, undef if error. However you should always test for failure first

while (1) { my $bytes = read(STDIN, $buffer, 100); if ( ! defined($bytes) ) { printf “Error: $!\n”; # $! contains error message exit 1; } # die “Error: $!” unless defined($bytes); # same thing last unless $bytes > 0;}

same as break in C++ and Java

EOF line-at-a-time:

• In line-mode, using <>, EOF and error are the same; both return undef

undef $!;while (defined(my $line = <STDIN>) ) { $data.= $line;}die “Error: $!” if defined($!);

clear $! before series of reads

check if any reads really failedundef $!;while ( my $line = <STDIN> ) { $data .= $line;}if ( defined($!) ) { printf “Error: $!\n”; exit 1;}

you can drop use of the defined() function since while-loop fails onfalse and undefined.

still need to check $! since the loopmay have stopped because of EOF

eof() function:

• eof() will return true if the next read to FILEHANDLE will return end-of-file.

$eof = eof(FILEHANDLE);

End-of-Line (\newline, \n) Anarchy:

• In perl, $/ is the current value of \newline (\n) • <> reads until it finds $/• changing the value of $/ changes behaviour of <>• chomp() tries to drop the value of $/• \n is the “logical” \newline character.

Windows: EOL == CRLF \015\012Unix: EOL == LF \012Mac: EOL == CR \015network: EOL == CRLF \015\012

for example, moving lines of text betweentwo Linux servers you still should write each \newline as two characters.

more \n

• You can’t define

since on Windows this would define a 3-character \newline symbol. You need to define

• The Socket and IO::Socket modules define $CRLF and CRLF() as exported globals that return the right thing.

$/ = \r\n;

$/ = “\015\012”;

more on CRLF

• Text mode: Automatic conversion

• Binary mode: Automatic conversion could corrupt data

unix network unix

LF CRLF CRLF LF

binmode(FILEHANDLE);

disables character translation

Opening files:

open(FH, “< myFile”); # opens myFile to readopen(FH, “> myFile”); # truncates myFile and opens it to writeopen(FH, “>> myFile”); # opens myFile to write without truncationopen(FH, “+> myFile”); # truncates myFile and opens it for read/writeopen(FH, “<+ myFile”); # opens myFile for read/write; no truncation

filehandle of opened file

$result = open(…);

open(…) or die “file failed to open”;

# check $! for specific error

true/false

Closing files:

close(FILEHANDLE);

Do you believe in magic?

#!/usr/bin/perl# redirect.pl

printf "Redirecting STDOUT\n";open(SAVEOUT, ">&STDOUT");open(STDOUT,">myfile.dat");printf STDOUT "STDOUT is redirected\n";system("date");open(STDOUT, ">&SAVEOUT");printf "STDOUT restored\n";

$ ./redirect.plRedirecting STDOUTSTDOUT restored

$ cat myfile.datSTDOUT is redirectedMon Jan 9 21:23:45 EST 2006

prints to STDOUT duplicates STDOUTfilehandle to write

close STDOUT connection toterminal; “redirects”it to myFile prints to STDOUT;

ie myFile

close new STDOUTconnection tomyFile; “redirects”it to old STDOUT

writes to STDOUT;ie terminal

Alternatives to open():

$result = sysopen(FILEHANDLE, $filename, $mode );# $result == (true/false); $! gives the reason for failure

O_RDONLY Open read only

O_WRONLY Open write only

O_RDWR Open read/write

O_CREAT Create file if it doesn’t exist

O_EXCL (O_EXCL | O_CREAT) creates file if it doesn’t exist but fails if file already exists

O_TRUNC If file exists; truncate to zero length

O_APPEND Open in “append” mode

O_NOCTTY If file is terminal, it won’t the process’s controlling terminal

O_NONBLOCK Open file in nonblocking mode

O_SYNC Open file in synchronous mode; all writes block until physical write takes place

Modes available to sysopen()

Buffering and Blocking:

• print() to a filehandle usually involves intermediate buffers while syswrite() does not.

• processing speed and IO speed are mismatched; hence buffered IO

• buffering decouples IO calls from IO activity

program

buffer disk

print() fast slowos write

memory

How buffering works:

X X X X X X X X X . . .

H e l l o \n X X X . . .

X X X l o \n X X X . . .

write ptrfromprogram

read ptr to device

printf “Hello\n”; write ptr

read ptr

read ptr

opsys slowly writes to devicewrite ptr

Things to think about:

• What is the buffer discipline?• How much “free space” is to be found in the buffer?• Under what circumstances does printf “block”?

• Prepare a similar diagram for input and answer the same questions.

Standard IO Buffering

• multiple buffers at different layers (write to disk):– disk hardware buffer– IDE controller driver– file system driver (OS)– C library (stdio)

• only stdio buffers until it has enough; the other layers try to get rid of any data they receive asap

stdio buffering problem (write):

program

stdio

file tcp

IDE IP

hwether

buffer

buffered write

unbuffered write

wait until bufferfull before write

write asap

printprintf syswrite()

$| = 1; turnsoff buffering

Q: Why use a buffer if the write is unbuffered?A: makes layer functionality asynchronous

code snipits for turning off write buffering:

my $prev_handle = select(FH); # makes FH new default handle and # saves the old default handle$| = 1; # sets special variable $| to true so writes for default handle # ( currently FH) are not bufferedselect($prev_handle); # makes old handle the default again.

use IO::Handle;FHautoflush(1); ## OO syntax for turning off buffering

or

stdio buffering problem (read):

program

stdio

file tcp

IDE IP

hwether

buffer

buffered read

unbuffered read

waits for exactly the rightamount of data

reads whateveravailable

read sysread()

unfortunately, readsat least 1 byte so it can “block”

blocks until exact number ofbytes available orEOF

How does <STDIN> work in this picture?

sysread() blocking problem:

• sysread() blocks if no data available. If non-blocking behaviour is required you must either– use separate read thread– use select() or poll() to determine if data is available

• TCP is not record-oriented so data structures need to be built; sysread() is ideal.

Filehandles:

• internal names for external entities• names for filehandles are “unadorned”; not scalars• filehandles are stuck in a single package; to move

references to filehandles from package to package we need to turn them into a typeglob

$fh = *MY_FH; # $fh is a typeglob$fh = \*MY_FH; # $fh is a typeglob reference

# passing a typeglob (reference) to a routine&hello_world($fh); # ok, even if hello_world() from another package

&hello_world(\*MY_FH); # the author’s favourite style

Filehandles and typeglobs:

• filehandles and filehandle typeglobs are interchangeable.

my $fh = &get_fh();

sub get_fh() { open(FOO,”<foo.txt”) or die “foo: $!”; return \*FOO;}

…

printf $fh “hello, world!\n”;

Could use a filehandle here too.

fileno():

• fileno() returns a file descriptor (0, 1, 2, …) if FH is a valid filehandle; undef otherwise

$X = &fileno(FH);

die “not a filehandle” unless defined fileno($fh);

filehandle File desciptor

SDTIN 0

STDOUT 1

STDERR 2

Detecting errors:

• all IO functions return undef on failure• $! contains specific error info; string or number

depending on context

use Errno qw(EACCES ENOENT);my $result = open(FH, “>/etc/passwd”);if (!$result) { #something went wrong if ($! == EACCES) { warn “no permissions”; elsif ($! == ENOENT) { warn “file or directory no found”; } else { warn “some other error: $!”; }}

numeric context

stringcontext

constant tags imported explicitly from Errno package

splits string intolist of words

OO Syntax:

• two OO extensions: IO::Handle and IO::File

$a = “hi there”;$a_ref = \$a;@b = (‘this’, ‘is’, ‘an’, ‘array’);$b_ref = \@b;%c = ( first_name => ‘Fred’, last_name => ‘Freud’);$c_ref = \%c;

$a = $$a_ref;@b = @$b_ref;%c = %$c_ref;

creatingreferences

dereferencing

$b_ref->[2] eq “an”;

$c_ref->{last_name};

referencing components

bless me!

• An object is a reference that is blessed – it knows what class it belongs to.

• A class is a package with methods that deal with object references.

• A method is a subroutine that expects an object reference as its first argument.

• same as

$obj_ref->method_name($p1, $p2)

ClassName::method_name($obj_ref, $p1, $p2);

package name

OO sugarcoating

constructors:

• constructors can be called anything: usually called new()• constructors are class methods

• same as

$obj_ref = ClassName->new();

$obj_ref = ClassName::new(‘ClassName’);

class hierarchy:

Handle

File Socket Pipe

user friendly towardsfile handling

super class

holds generic methods common to all filehandles

all three kinds of filehandlesare accessible via the same set of Handle methods

OO example:

#! /usr/bin/perl# file: count_lines.pl

use strict;use IO::File;

my $file = shift;my $counter = 0;my $fh = IO::File->new($file) or die “Can’t open file: $file”);while ( defined (my $line = $fh->getline)) { $counter++;}STDOUT->print(“Counter $counter lines\n”);

lazy evaluation of or; die only if new() returnsundef.

simple wrapper around standard perl print() subroutine

built-in Handleobject so itcan use Handlemethods

IO::File Methods 1:

• main constructor; replaces open(), same rules for $mode and $perms

• Still need to look at $! if it fails

• invisible in file system; file goes away when object destroyed

• called automatically if you forget; an IO::Handle method

$fh = IO::File->new($filename [, $mode [,$perms]])

$fh = IO::File->new_tmpfile;

$result = $fh->close;

IO::File Methods 2:

• used to reopen a file in a redirection situation; comes from IO::Handle

$result = $fh->open($filename [,$mode [,$perm]]);

STDOUT->open(“>log.txt”) or die “Can’t reopen STDOUT: $!”;

IO::File Methods 3:

• main constructor for IO::File; w/ 1 argument it acts as the 2 argument version of open(). Returns undef and $! if an error occurs

• just in case you need a temporary file

• happens automatically if you forget

$fh = IO::File->new($filename [,$mode [,$perm]]);

$fh->IO::File->new_tmpfile;

$result = $fh->close();

IO::File Methods 4:

• These work just like their standard counterparts

• replace <>

• $| = [0|1]; but for any filehandle

$result = $fh->print(@args);$result = $fh->print($fmt. @args);$bytes = $fh->write($data [,$length [,$offset]]);$bytes = $fh->syswrite($data [,$length [,$offset]]);$bytes = $fh->read($buffer,$length[,$offset]);$bytes = $fh->sysread($buffer,$length[,$offset]);

$line = $fh->getline@lines = $fh->getlines

$previous = $fh->autoflush([$boolean]);

IO::File Methods 5:

• Returns true if a file handle is valid.

• Returns true if next read of the filehandle will return EOF

• Performs a one-time flush of the filehandle buffer. If write buffer then write occurs; if read buffer then data discarded.

• Turns blocking on and off (Chapter 13).

$boolean = $fh->opened; # same as defined fileno($fh);

$boolean = $fh->eof

$fh->flush

$boolean = $fh->blocking([$boolean]);

IO::File Methods 6:

• Used together these report on any error in the intervening code (. . .).

$fh->clearerr;. . .$boolean = $fh->error;

Copying Filehandles:

• Creates a duplicate handle for an existing filehandle object, previously opened with the same $mode.

• $fd can be an IO::Handle object, an IO::File object, a regular filehandle or a numeric file descriptor.

• similar to

$fh = IO::File->new_from_fd($fd,$mode);

$saveout = IO::File->new_from_fd(STDOUT,”>”);

open(SAVEOUT, “>&STDOUT”);

Copying Filehandles 2:

• Reopens an existing handle ($fh) as a copy of another existing filehandle object ($fd), previously opened with the same $mode.

• $fd can be an IO::Handle object, an IO::File object, a regular filehandle or a numeric file descriptor.

• Used with new_from_fd() to restore a saved filehandle

$result = $fh->fdopen($fd,$mode);

$saveout = IO::File->new_from_fd(STDOUT,”>”);STDOUT->open(‘>log.txt’);…STDOUT->print “Yippie yie yay!\n”;…STDOUT->fdopen($saveout,”>”);

STDOUT redirected to file

STDOUT filehandle saved

STDOUT redirected back to where it came from