Post on 20-Sep-2020
Slide 1
Input and Output Streams – part II
Slide 2
Lesson OverviewTopics covered
(1) Short recap on Input/Output Streams
(2) Using streams to read structured data – the problem
(3) The Decorator Design Pattern – the solution
(4) Stream Concatenation – plumbing revisited
(5) Reading and Writing objects
Slide 3
The General Scenario
Application
Problem : How can we communicate with various devices using a single common interface? Notice that we do not want to know the details of the hardware devices. Instead we would like to have a common interface for accessing all of them them!
Slide 4
I/O AbstractionWe can achieve the separation by designing a common interface for reading any kind of data, and common interface for writing any kind of data.
This interface is implemented by the notion of input and output streams.
Any input/output can be represented as a sequence of
bits. For convenience we divide the sequence into a sequence of bytes.
Slide 5
Input/Output StreamsAn input/output stream is a sequence of bytes that is attached to some input/output source. You can read/write data from/to the stream in a sequential order. One byte at a time or several bytes at a time.
12 72 32 17 83 11 7 91 108
43 55 31 37 34 13 17 1 15
Input streamreading direction
writing direction Output stream
Slide 6
Basic reading/writing algorithmsNo matter where the data is coming from or going to and no matter what its type, the algorithms for sequentially reading and writing data are basically the same:
Reading : (1) open a stream
(2) while (more information)
(2.1) read information
(3) close the stream
Slide 7
Basic reading/writing algorithmsWriting : (1) open a stream
(2) while (more information)
(2.1) write information
(3) close the stream
Slide 8
Character StreamsReader and Writer are the abstract superclasses for character streams in java.io.
Reader provides the API and partial implementation for readers — streams that read 16-bit characters.
Writer provides the API and partial implementation for writers — streams that write 16-bit characters.
Slide 9
Character Streams Subclasses of Reader and Writer implement specialized streams and are divided into two categories: those that read from or write to data sinks and those that perform some sort of processing .
data sinks processing
Slide 10
Byte StreamsTo read and write 8-bit bytes, programs should use the byte streams, descendents of InputStream and OutputStream.
InputStream and OutputStream provide the API and partial implementation for input streams (streams that read 8-bit bytes) and output streams (streams that write 8-bit bytes).
These streams are typically used to read and write binary data such as images and sounds.
Slide 11
Byte StreamsSubclasses of InputStream and OutputStream provide specialized I/O that falls into two categories
Slide 12
Input SuperclassesReader and InputStream define similar APIs but for different data types. Both are abstract classes!
Reader contains these methods for reading characters and arrays of characters.
int read() //Read a single character,-1 is ends.
int read(char cbuf[]) //Read chars into an array
//read chars into a portion of the array.
int read(char cbuf[], int offset, int length)
Slide 13
Input SuperclassesInputStream defines the same methods but for reading bytes and arrays of bytes:
int read() //Read a single byte,-1 if end.
int read(byte cbuf[]) //Read several bytes...
int read(byte cbuf[], int offset, int length)
Both Reader and InputStream provide methods for marking a location in the stream, skipping input, and resetting the current position.
Slide 14
Output SuperclassesWriter and OutputStream are similarly parallel.
Writer defines these methods for writing characters and arrays of characters:
int write(int c) //write a single character.
int write(char cbuf[]) //write an array of chars.
//write a portion of an array of chars.
int write(char cbuf[], int offset, int length)
Slide 15
Output SuperclassesOutputStream defines the same methods but for bytes:
int write(int c) //writes a single byte
int write(byte cbuf[]) //writes an array of bytes
int write(byte cbuf[], int offset, int length)
All of the streams — readers, writers, input streams, and output streams — are automatically opened when created.
Close a stream by calling its close method. A program should close a stream as soon as it is done with it, in order to free up system resources.
Slide 16
Example 1: Typing a text fileimport java.io.*;public class MyTextType {public static void main(String[] args){
try{
FileReader fileIn= new FileReader("temp.txt");
int c;
while((c=fileIn.read()) != -1)
System.out.print((char)c);
fileIn.close();
}catch(FileNotFoundException fnfe){
System.err.println("file temp.text not found!!!");
}catch (IOException ioe){
System.err.println("error reading from file!");}
}
}
Slide 17
Streams – current summaryInputStream and OutputStream gives us a low level for reading and writing binary data. We can only read/write a single byte or an array of bytes.
Likewise, Reader and Writer gives us a low level for reading and writing text files. We can only read/write a single char or an array of chars.
Slide 18
Using Streams to Read Structured Data (The Problem)
In many cases, the data we want to read/write usually has a more complex structure.
Examples:
Textual data ordered in a table
A list of short values where every 2 bytes represent a single short value
We would like to be able to read/write the data in a structured way.
Slide 19
Case study I: Reading a file of integers
Suppose we are given a file which contains a list of integers.
We would like to read all number from the file and store them somewhere.
Let us assume that the number of integers n is known in advance (e.g. N=100).
Here is what the code would look like if we only make use of the low-level input streams we know so far:
Slide 20
Reading a file of integers (The hard way)public static int[] readNumbersFromFile
( String fileName, int n) throws IOException{
int[] arr = new int[n];
FileInputStream fileIn= new FileInputStream(fileName);
int currByte;
int currInt; int base = 8;
for(int i = 1; i < n; i++){
for (int j = 3; j >= 0; j--)
currInt = s |
( unsignedByteToInt(fileIn.read()) << base*j );
}
fileIn.close();
}
Slide 21
Reading a file of integers (The hard way)/* a small hack function required to deal with the sign bit of
the integer, which is used in the 2's complement notation.
*/public static int unsignedByteToInt(byte b) {
return (int) b & 0xFF;
}
Slide 22
The problem of reading structured dataThe example above demonstrates that even reading a primitive data type such as an integer using low-level streams is a rather hard task, which requires intricate bitwise operations.
However, this is merely an example, there are many types of structured data which we may encounter – numbers, lines in a text file etc.
However, we would like to be able to read/write structured data from various input and output devices.
A straightforward (and bad) solution would be ... (?)
Slide 23
Case study II: Efficiently reading bytes from a large file
Suppose we are given a very large file which contains a list of n bytes.
Each byte represents some code word in a secret language.
Our task is to read the bytes one after the other, and then use a decoder that we have been provided with to decode their meaning.
Here is what our (bad) code would look like:
Slide 24
Decoding a file of bytes (The bad way)public static int[] decodeFile
( String fileName, int n) throws IOException{
FileReader fileIn= new FileReader(fileName);
int currByte;
for(int i = 1; i < n; i++){
currByte = fileIn.read();
System.out.println(Decoder.decode((byte)currByte));
}
fileIn.close();
}
What is bad here?
Slide 25
Case study II: Efficiently reading bytes from a large fileAlthough the read command of the FileInputStream class can actually directly read bytes from a file this is very very inefficient.
Any read/write operation from the disk usually involves the use of the Operating System (OS) and is therefore very time consuming.
The basic reading mechanism of the OS is built on reading much bigger chunks of data from the disk at once.
These chunks are called “Pages”. A typical page is of size ~4k (IS THIS TRUE TODAY???)
In our previous code snippet, we read “from the disk” byte by byte!
Slide 26
Efficiently reading bytes from a large file (The Problem)
Efficiency is a feature that we would like to have in other similar scenarios:
Reading a large file of characters
Reading a large file of booleans
Reading a large file of words (Strings)
Reading a large text file line after line.
Once more: a bad solution would be ?
Slide 27
Are the 2 problems related?
We began by introducing a problem of reading structured data.
We then introduced the problem of efficiently reading from a file (or any other input source).
Are these two problems related to one another?
The answer lies in discussing both in terms of OO design:
Slide 28
The Design ProblemObjective: Have methods for reading/writing data on a higher level. (structured, efficient etc.)
Problems:
There are many enhancements for reading/writing data
There are many types of input/output streams
If we would include all enhancements in all types of streams we will end up with a lot of duplicated code and it would be hard to add new enhancements or new types of streams
Slide 29
The Design Solution – The Decorator Pattern
The Decorator pattern is another type of structural pattern (like the Strategy pattern some of you used in ex2)
Design Pattern?????!!!! hmmmmm
Sounds familiar, now where have I heard that phrase before?
“Here is a first hypothesis: the drawing is blind, if not the draftsman or draftswoman. As such, and in the moment proper to it, the operation of drawing would have something to do with blindness, would in some way regard blindness” (Jacques Derrida, Memoires of the Blind)
Slide 30
A Design Pattern (GoF)“A design pattern names, abstracts, and identifies key aspects of a common design structure that makes it useful for creating a reusable object-oriented design.”
The GoF design patterns are “descriptions of communicating objects and classes that are customized to solve a general design problem in a particular context.”
Slide 31
Classification of Design PatternsPurpose - what a pattern does
Creational Patterns - Concern the process of object creation. (Example: Factory Method)
Structural Patterns - Deal with the composition of classes and objects. (Examples: Decorator,Strategy)
Behavioral Patterns - Deal with the interaction of classes and objects. (Example: Iterator)
Slide 32
Classification of Design PatternsScope - what the pattern applies to
Class Patterns - Focus on the relationships between classes and their
subclasses. Involve inheritance reuse
Object Patterns Focus on the relationships between objects Involve composition reuse
Slide 33
The Decorator Design PatternIntent
The Decorator pattern is used for adding additional functionality (or repsonsibilities) to an object dynamically.
Decorators provide a flexible alternative to sublcassing for extending functionality.
Slide 34
Decorator Design Pattern (cont.)Motivation
It is easy to add functionality to an entire class of objects by subclassing an object, but it is impossible to extend a single object this way. With the Decorator Pattern, you can add functionality to a single object and leave others like it unmodified.
The component which we want to extend is wrapped (or enclosed) within another object that adds the required functionality. This enclosing object is called a decorator.
The decorator conforms to the interface of the component it encloses, so that its presence is transparent to the component's clients.
Slide 35
Decorator Design Pattern (cont.)Motivation (cont.)
The decorator forwards all requests to the component which it encloses and may perform additional actions (such as shifting bits!) before or after forwarding.
The transparency allows to nest decorators recursively, thereby allowing an unlimited number of added responsibilities.
Slide 36
Decorator Design Pattern (cont.)
Slide 37
The Decorator Design Pattern (cont.)Structure
Slide 38
IOstreams and the Decorator Pattern (The solution)
In order to allow various extensions or “decorations” that could be applied to various types of Input/Output streams, the IO package contains a set of classes which implement the decorator design pattern.
Example: Reader Decorators - Use a “decorator”: a class that is both derived from Reader, and encloses another Reader object as a data member (received by constructor of new class).
All Reader methods are “forwarded” (or delegated) to the inner Reader object.
New attributes (methods) use the inner Reader object as well.
Slide 39
IOstreams and the Decorator Pattern (The solution)
We gain two things: (1) The “old” interface is preserved – Every Reader decorator can
“pass” as a Reader object (i.e. is transparent).
(2) We can “chain” several functionalities – We can now decorate a specific Reader with several types of decorations. Example: adding both line reading capabilities and efficient reading capability (see example below).
The Same solution exists for Writer, InputStream and OutputStream classes.
In Java, “decorators” are called “Filters”, and the base class for adding attributes is FilterXXX.
Slide 40
Java Filter classes
In Java, “decorators” are called “Filters”, and the base class for adding attributes is FilterXXX.
For each enhancement for reading from an input stream there is a suitable filter input stream.
For each enhancement for writing to an output stream there is a suitable filter output stream.
Slide 41
Example: BufferedReader
Reader
read()
...
...
BufferedReader
readLine()
read()
...
...
Slide 42
Example: DataInputStream
InputStream
read()
...
...
DataInputStream
readShort()
read()
...
...
Slide 43
Plumbing Pitstop – Revisited
Water
Reservoir
Main Pipe Sec. Pipe Filter
Slide 44
Plumbing Pitstop – Getting Water ≡ Reading
Water
Reservoir
Main Pipe Sec. Pipe Filter
File (Source)
FileInputStream
DataInputStream
Slide 45
Plumbing Pitstop – where does my used water go to?
Sink (Drain)Source Pipe FilterDest. Pipe
Destination
Nachal Soreq?
Slide 46
Plumbing Pitstop – Spilling Water ≡ Writing
Sink (Drain)Source Pipe FilterDest. Pipe
Destination
ByteArray
byte[] arr;
ByteArrayOutputStream
BufferedOutputStream
FileOutputStream
File
Slide 47
Source vs. Filter streams
In the plumbing example we saw that there are 2 different kinds of Input/Output Streams:
(1) Source Streams: Streams that are designed to connect to a specific source such as: FileInputStream, ByteArrayInputStream, etc.
(2) Filter Streams: Streams that are designed to connect to other streams – that may offer some more functionality. For example: BufferedInputStream, DataInputStream, etc.
Slide 48
Source vs. Filter streams
All Filter streams implement the Decorator design pattern and can therefore be connected to any source stream.
This is a very neat and useful example of how a design pattern actually allows extended functionallity and resuability.
Note however, that a decorator can never be used on its own. It will always be dependant on its enclosed component (or stream in our case) for receiving/sending the actual bytes/chars.
Decorators can also be chained on to another – or recursively applied. This is called chaining
Slide 49
Example I: Chaining Streams
try { DataInputStream input = new DataInputStream( new BufferedInputStream( new FileInputStream(args[0])));} catch (FileNotFoundException fnfe) { // ...}
readShort()
read() read() read()
BufferedDataFile
Slide 50
Example II: Chaining Streams
try { BufferedReader input = new BufferedReader( new PushbackReader( new FileReader(args[0])));} catch (FileNotFoundException fnfe) { // ...}
readLine()
read() read() read()
PushbackBufferedFile
Slide 51
Reading a file of integers revisited (The right way)
public static int[] readNumbersFromFile( String fileName, int n) throws IOException{
int[] arr = new int[n];
DataInputStream dataIn = new DataInputStream(
new BufferedInputStream(
new FileInputStream(fileName)));
int currByte;
int currInt;
for(int i = 1; i < n; i++){
arr[i] = dataIn.readInt();
}
fileIn.close();
}
Slide 52
Reading a text file line by line (The good way)//reads an input text file and creates a new file which does not contain any empty lines in it.
public static void removeEmptyLines( String fileName, String destFileName) throws IOException{
BufferedReader bufIn = new BufferedReader(
new FileReader(fileName));
BufferedWriter bufOut = new BufferedWriter(
new FileWriter(destFileName));
//continued on next slide...
Slide 53
Reading a text file line by line (The good way)//read the file line by line – and remove empty lines
String line;
while( (line = bufIn.readLine()) != null )
if(!line.equals(“”))
bufOut.write(line,0,line.length());
bufOut.newLine();
}
//don't forget to flush and close streams!
bufOut.flush();
bufOut.close();
fileIn.close();
}
Slide 54
Stream OverviewStreams Description
Memory
Files
Type of I/O
CharArrayReaderCharArrayWriterByteArrayInputStreamByteArrayOutputStream
Use these streams to read from and write to memory. You create these streams on an existing array and then use the read and write methods to read from or write to the array.
StringReaderStringWriterStringBufferInputStream
Use StringReader to read characters from a String in memory. Use StringWriter to write to a String. StringWriter collects the characters written to it in a StringBuffer, which can then be converted to a String.StringBufferInputStream is similar to StringReader, except that it reads bytes from a StringBuffer.
FileReaderFileWriterFileInputStreamFileOutputStream
Collectively called file streams, these streams are used to read from or write to a file on the native file system.
Slide 55
Stream Overview (cont.)Type of I/O Streams Description
Buffering
Used to serialize objects.
BufferedReaderBufferedWriterBufferedInputStreamBufferedOutputStream
Buffer data while reading or writing, thereby reducing the number of accesses required on the original data source. Buffered streams are typically more efficient than similar nonbuffered streams and are often used with other streams.
ObjectSerialization
N/AObjectInputStreamObjectOutputStream
DataConversion
N/A DataInputStreamDataOutputStream
Read or write primitive data types in a machine-independent format.
Slide 56
Stream Overview (cont.)Type of I/O Streams Description
Printing
Filtering
PrintWriterPrintStream
Contain convenient printing methods. These are the easiest streams to write to, so you will often see other writable streams wrapped in one of these.
FilterReaderFilterWriterFilterInputStreamFilterOutputStream
These abstract classes define the interface for filter streams, which filter data as it's being read or written.
Converting between
Bytes and Characters
InputStreamReaderOutputStreamWriter
A reader and writer pair that forms the bridge between byte streams and character streams.An InputStreamReader reads bytes from an InputStream and converts them to characters, using the default character encoding or a character encoding specified by name.An OutputStreamWriter converts characters to bytes, using the default character encoding or a character encoding specified by name and then writes those bytes to an OutputStream.
Slide 57
Object Input & Output Streams
Java allows us to read and write whole objects into a binary file. The process of saving an objects state requires saving all non-static data members it holds.These data members may be primitive or non-primitive (ref to an object).This process is termed Serialization: serializing the object’s data members into a stream.
Slide 58
ObjectOutputStream
The class ObjectOutputStream allows writing an object into a stream. It contains the following method:
public void writeObject(Object obj)
throws IOException
Slide 59
ObjectInputStream
The class allows reading objects from an underlying InputStream. It contains the following method:
public Object readObject() throws IOException, ClassNotFoundException
Slide 60
Object Serialization
In order for an object to be written into an Object Stream it must implement the interface Serializable.The interface is part of the java.io package.The interface is an empty interface – i.e. no methods are included in it!
Slide 61
Serialization - Exampleimport java.io.*;
public class WordEntry implements Serializable{
private String word;private int counter;
public WordEntry(String word){this.word= word;counter=0;
}
//continued on next slide…
Slide 62
Example (cont.)public void setCounter(int counter){
if(counter >0 )this.counter= counter;
}
public int getCounter(){return(counter);
}
public String getWord(){return(word);
}//continued on next slide…
Slide 63
Example (cont.)public String toString(){
return("word= " + word + " counter= " + counter);}
public boolean equals(Object obj){if(!(obj instanceof WordEntry))
return(false);
WordEntry other= (WordEntry)obj;return(this.word.equalsIgnoreCase
(other.word));}
}//end of class.
Slide 64
Saving a WordCounter Objectpublic static void main(String[] args){WordCounter wCount= new WordCounter(“yes”,10”);
try{ObjectOutputStream objOut= new ObjectOutputStream( new BufferedOutputStream(
new FileOutputStream(args[0]))); objOut.writeObject(wCount); objOut.flush(); objOut.close();}catch(IOException ioe){ }
}
Slide 65
Loading a WordCounter Objectpublic static void main(String[] args){WordCounter wCount;
try{ObjectInputStream objIn= new ObjectInputStream( new FileInputStream(args[0]));
wCount= (WordCounter)objIn.readObject(); objIn.close();}catch(IOException ioe){ //do something}catch(ClassNotFoundException cnfe){
} }
Slide 66
Final Example: An easy way to detect foolish copying - Diffimport java.io.*;
public class Diff {public static void main(String[] args){
//check for arguments if(args.length !=2){ System.out.println("Usage: java Diff <file1> <file2>"); System.exit(0); }
//used for comparing the files textually line by line LineNumberReader file1In,file2In;
//set to false if the files are different boolean noDiff= true; //used to keep all differences found between the 2 files. StringBuffer output= new StringBuffer(); try{
file1In= new LineNumberReader(new FileReader(args[0]));file2In= new LineNumberReader(new FileReader(args[1]));
Slide 67
Diff (cont.)String line1,line2;line1= file1In.readLine();line2= file2In.readLine();
while( (line1!= null) && (line2!= null) ){
//compare the 2 lines. if(! line1.equals(line2) ){
output.append("line no. " + file1In.getLineNumber() + " :\n");
output.append(args[0] +" "+ line1 + "\n"); output.append(args[1] +" "+ line2 + "\n"); noDiff= false;
}
line1= file1In.readLine(); line2= file2In.readLine();}
Slide 68
Diff (cont.)//check if file1 has more lines!!!if(line1 != null){ StringBuffer output1= new StringBuffer(); output1.append(args[0] +" " +
"contains more lines: \n"); output1.append("line no.: " + file1In.getLineNumber()
+": " + line1 + "\n");
//now let's read all the lines left in file1! while( (line1=file1In.readLine()) != null) output1.append("line no.: " + file1In.getLineNumber()
+": " + line1 + "\n" );
output.append(output1.toString());}
if(line2 != null){//same code as in upper case only now we work on
line2!
Slide 69
Diff (cont.)}catch(FileNotFoundException fnfe){ System.out.println
("one of the files doe's could not be opened!");}catch(IOException ioe){ System.out.println
("an error occured when reading from one of the files.” + “aborting...");}
//check if differencs . if so print a message and the //stringBuffer with all differencesif(!noDiff){
System.out.println("Diff found some differences between the files: ");
System.out.println(output.toString());}else //no differences! System.out.println
("Diff found no differences between the files: ");}
}//end of main