AWT Lastest Version

7/31/2019 AWT Lastest Version

1/66

Images In Java

Paul Cockshott

ALMA TADEMA RIVALES INCONSCIENTES 1893


2/66

Introduction

The AWT and how to put a simple image

onto the screen

Layout managers

The Jimage Class,


3/66

Summary of this section

At the end of this lecture you should have

an idea of how to display a JPEG image

on the screen, and how to load it into the

Jimage class to carry out further image

processing.


4/66

Agenda

AWT Images

Image Producers and Consumers

Jimage class

Pixel Representations

JPEG files


5/66

Overview

AWT abstract windows toolkit, supported

by JavaSoft

Operating system independent layer for

windowing in Java Fiendishly obscure

Designed around requirements of images

being streamed off the web


6/66

Connections

Simple image display program to show

how to display a JPEG file

Pipeline model of image production

Jimages act as image consumers Jimages allow arithmetic on image

Jimages provide output to AWT images

and JPEG


7/66

How to display a picture 1

import java.awt.*;

import java.awt.image.*;

import java.util.*;

class JPEGshow extends Frame {

...

static public void main(String[] args) {

if (args.length == 1) new JPEGshow(args[0]);

else System.err.println("usage: java JPEGshow ");

}

}

This is a standard Java Program class with a public static voidmain method


8/66

Constructor for JPEGshow

JPEGshow(String filename) {

super("JPEG show Example");

add(

new ImageCanvas(getToolkit().getImage(filename)

),BorderLayout.CENTER);

setSize(700, 540);

show();

}

See slide on these

Read in a JPEG file

Size of a frame


9/66

The toolkit

Each frame has associated with it a toolkit

object the provides an interface to OS

specific operations. CreateImage

CreateMenu

CreateLabel

CreateMenuBar . etc


10/66

Roll your own ImageCanvas

class ImageCanvas extends Component {

Image image;

ImageCanvas(Image image)

{this.image = image;}

public void paint(Graphics g)

{ g.drawImage(image, 0, 0, this);}

}

Paint is called whenever a component must be shown,

the Graphics object does the actual drawing, it has to be

passed in because it is what knows about physically

drawing on the screen

Constructor just stores

the image


11/66

Image Class

Pipeline flow model of image processing

Images are just tokens linking producers and

consumers

ImageImageProducerImageConsumer


12/66

ImageProducer Methods

addConsumer(ImageConsumer ic) This method is used

to register an ImageConsumer with the ImageProducer

for access to the image data during a later

reconstruction of the Image.

removeConsumer(ImageConsumer ic) This method

removes the given ImageConsumer object from the list

of consumers currently registered to receive image data.

startProduction(ImageConsumer ic) This method starts

an immediate reconstruction of the image data


13/66

ImageConsumer methods

void setDimensions(int width, int height)

The dimensions of the source image are reported

using the setDimensions method call.

Void setPixels(int x, int y, int w, int h,

ColorModel model, byte[] pixels, int off, int scansize)

The pixels of the image are delivered using one or

more calls to the setPixels method.


14/66

Image Class continued

ImageImageProducer ImageConsumer

Images contain a pointer to their producer which holds the actual

data for the image. This can be recovered using the getSource

method. This allows a consumer to get at the pixel data ofan image by adding itself to the producer and starting production

Image.getSource


15/66

Summary

AWT is operating system independent

Streaming image model

Images as tokens

Producer - consumer pipeline See chapters 6 of textbook


16/66

Buffered Image Class

Standard AWT images are just tokens for

data streams.

A BufferedImage actually contains the

data.

Colour model Raster

BufferedImage


17/66

JPEGCodec class

This class has factory methods to create

JPEG encoders and decoders: createJPEGDecoder(InputStream s)

createJPEGEncoder(OutputStream d)


18/66

Read a BufferedImage

FileInputStream in = new

FileInputStream(myfile.jpg);

JPEGImageDecoder dec=

JPEGCodec.createJPEGDecoder(in);

BufferedImage im =

decoder.decodeAsBufferedImage();


19/66

getRGB

You can access the pixels of a buffered

image using

int getRGB(int x, int y)

The default colour representation is:

alpha red green blue

Bit 0Bit 31


20/66

Writing pixels

This can be done with the setRGB

method.

This takes x, and y co-ordinates and a

pixel encoded as a 32 bit integer im . setRGB(2, 5, 255);

Would set pixel 2,5 to 255 = bright blue.


21/66

Creating sub images

You can create a sub area within a

buffered image using the

public BufferedImage getSubimage( int x,

int y,

int w,

int h);

Method of BufferedImage


22/66

Jimage implements ImageConsumer

Library of image processing classes

developed in the department

Available for student practicals

Algebraic rather than stream oriented Interfaces to MMX hardware under

windows


23/66

Algebraic orientation

By this we mean the it is structured around

algebraic expressions whose values are

images

Thus ifA

andB

are images and is someoperator thenA B

is also an image


24/66

Jimage operators

Arithmetic I+J Universal plus(Universal) I-J Universal minus(Universal) IJ Universal times(Universal) IJ Universal divide(Universal) IUniversal abs() Filtering Jimage convolve(double[] k)convolve with symmetrical separable

kernel.

public abstract Jimage convolve(double[][] kernel)with non

separable kernel


25/66

Scaling

Jimage getScaledInstance(int nwidth, int

nheight)

This scales with bicubic interpolation.

Jimage getScaledInstance(int nwidth, int

nheight, int ndepth)

This method allows the depth as well as area of animage to be altered if it is reduced the planes are

aggregated if increased they are interpolated.


26/66

More operations

Data accessint rgbpixel(int x,int y)

Converts the plane information into a pixel in the direct

color model of java.

public abstract int upixel(int x,

int y,int plane)

- returns unsigned integer pixelpublic abstract float fpixel(int x,

int y,

int plane)

Returns the pixel in the range -1 to +1.


27/66

Data Access

public abstract void setPixel(int x,

int y,

int plane,

double pix)

Pixel information in range -1 to +1 public void setSubImage(int x,

int y,

int z,

Jimage im)

Update an area of an image with another one. The other

one must not run off the edge of the one being written to.

The source of the copying is the 0th plane of the source

jimage.


28/66

Jimage input output

public voidputJPEGImage(

java.lang.String fileName,

int quality)

throws java.io.IOException

Outputs the image to a jpeg file public boolean getImage(java.lang.String

fileName)

Initialise the Jimage from the specified file. The file

must be jpeg or gif.


29/66

Jimage toAWT Image conversion

public java.awt.Image getAWTImage()

public java.awt.image.ImageProducer

getProducer()


30/66

Jimage implementations

JIMAGE CLASS HIERARCHY

IntelBImage

Runs best on MMX

Windows only

COM.C3D.IMAGE

ByteImage

Generic Java

IntelImage

Runs best on MMX

Windows Only

COM.C3D.IMAGE

ShortImage

Generic Java

IntelFImage

Runs best on PIII

Windows Only

COM.C3D.IMAGE

FloatImage

Generic Java

COM.C3D.IMAGE

Jimage

abstract


31/66

An example program

class Jimageshow extends Frame {

Jimageshow(String filename) {

super("Jimage show Example");

Jimage raw=new ByteImage(100,200,3);

if (raw.getImage(filename)){

Jimage cooked = (Jimage)raw.times(0.3);

add(new ImageCanvas(cooked.getAWTImage()),

BorderLayout.CENTER);

setSize(700, 540);

show();

}

}

Create Jimage

with byte pixels

Multiply by 0.3

Convert to AWT

for display


32/66

Pixel Representations

When dealing with displays it is conventional to assume that

pixels are bytes holding numbers in the range 0 to 255.

0 Is assumed to be black

1 Is assumed to be white or maximum brightness of any

given colour.

For multicolour displays with 3 colour components, the

convention is to have 3 fields of range 0..255 to hold the

colour information.


33/66

Pixel Representations AWT

For multicolour displays with 3 colour components, the

convention is to have 3 fields of range 0..255 to hold the

colour information. The AWT does this with the class

Color. public Color(int rgb)

Creates a color with the specified RGB value, where thered component is in bits 16-23 of the argument, the green

component is in bits 8-15 of the argument, and the blue

component is in bits 0-7. The value zero indicates no

contribution from the primary color component.

A Jimage returns this format with int rgbpixel().


34/66

Pixel Representations: Bytes

The byte data type in Java does not take on the values

0..255. Instead it takes on the values -128 to 127.

There are no unsigned bytes in Java.

This creates a problem for the representation of pixels in

Jimages.

The solution adopted is to adopt the following

representation

-128 = black

0 = mid grey

127 = white


35/66

Pixel Representations: Floats

If byte pixels are signed then so must other representations

be.

The solution adopted is to adopt the following

representation for floats

-1 = black

0 = mid grey

1 = white


36/66

Conversionsbetween representations

unsigned bytes shorts float

min value 0 -128 -2048

-1

maxval 255 127 2047 1

medianval m 127.5 -0.5 -0.50

range r 255 255 4095 2

As shown in table a pixel prin representation r is converted

to a pixel ps in representation s by the operation:

ps = ms+(rs(pr-mrrr


37/66

Signed Pixels : advantages

Signed pixels seem at first to be counter-intuitive but they

have numerous advantages.

A value of 0 or mid grey can be viewed as the most likely

value that a pixel takes on in the absence of other information.

If you do arithmetic on images, in particular subtract one image

from another, then negative values of pixels naturally arise. Signed pixels allow straightforward implementation of contrast

adjustments. For instance multiplying an image by 0.5 halves

the contrast in the image.


38/66

Signed Pixels : contrast adjustment

Signed pixels allow straightforward implementation of contrast

adjustments. For instance multiplying an image by 0.5 halves

the contrast in the image.

0.5

1

-1

0.5

Initial contrast rangeFinalcontrast

range

0.5

-0.5

0.25

-0.25


39/66

Image Multiplication


40/66

Image Addition

+


41/66

Image subtraction

-


42/66

What Is Convolution

Convolution takes a kernel of coefficients

and multiplies each pixel in a

neighbourhood by the corresponding

coefficient, and then sums the result

x y p[I+x, j+y]*k[x,y] Will give the convolved pixel at position i, j


43/66

1 D convolution

A 1 D convolution takes a one

dimensional array as a kernel and applies

it first in the X and then in Y dimension.

This can often be performed faster than a

2d convolution


44/66

Image Convolution: smoothing

double[] k= {0.1,0.1,0.2,0.2,0.2,0.1,0.1};

marble.convolve(k)=

marble =

Note sum of

coefficients =1


45/66

Image Convolution: sharpening

double[] k= {-0.3,1.6,-0.3}

marble.convolve(k)=

marble =

Note sum of

coefficients =1

number terms is odd


46/66

Convolution in Java2D

Java 2D provides a standard library for

convolution of buffered images

This uses the class Kernel and

ConvolveOp


47/66

Kernels in JAVA 2D

float[] blur={ 0.0f, 0.1f, 0.0f,

0.1f, 0.6f, 0.1f,

0.0f, 0.1f, 0.1f};

Kernel k= new Kernel(3,3, blur);

im = new ConvolveOp(K).filter(im,null);

This will blur the image im by applying the 3 by 3 kernel

blur to it.


48/66

Importance of speed

Image may contain a million pixels,

Arithmetic may be required on each one

Important to optimise operations or they

are very time consuming May need to use assembler kernels

May need to use special purpose

instructions


49/66

Multimedia Extensions MMX

Intel and other CPU manufacturers have

been adding to the instruction sets of their

computers new extensions that handle

multi-media data.

The aim is to allow operations to proceed

on multiple pixels each clock cycle


50/66

MMX 2

Standard Intel register set

eax

ebx

ecx

edx

espebp

esi

edi

8 General Registers 8 floating point registers

32 bit 64 bit

fp0

fp1

fp2

fp3

fp4fp5

fp6

fp7


51/66

MMX 3

Standard Intel register set operating in

MMX mode

eax

ebx

ecx

edx

espebp

esi

edi

8 General Registers 8 multimedia registers

32 bit 64 bit

mm0

mm1

mm2

mm3

mm4mm5

mm6

mm7


52/66

MMX 4 motivation

Existing operating systems must still work

unchanged

Applications not using MMX run

unchanged

No new state added to the CPU

Hence, shared use of the FP registers, since

these are already supported by exising

OSs


53/66

MMX data formats

One 64bit integer QUADWORD

Two 32 bit integer DOUBLEWORDS

Four 16 bit WORDS

Eight 8 bit BYTES

f f


54/66

Problem of overflows

A problem with limited precision arithmetic is that overflows frequently

occur. This can give rise to meaningless results: consider

200+175 = 375 but in 8 bit binary

11001000

+10101111

=101110111Leading 1 is discarded

This leaves an answer of 119

decimalclearly wrong

U i t ti


55/66

Using saturation

You can fix this by using conditionalsunsigned char p1,p2,p3;

int I3= (int)p1 + (int)p2;

p3=(I3>255?255:(unsigned char)I3);

E i f th d 1


56/66

Expansion of the code 1

12: j=(int)(*p1++)+(int)(*p2++);00401043 mov ecx,dword ptr [ebp-4]00401046 xor edx,edx00401048 mov dl,byte ptr [ecx]0040104A mov eax,dword ptr [ebp-8]0040104D xor ecx,ecx0040104F mov cl,byte ptr [eax]00401051 add edx,ecx00401053 mov dword ptr [ebp-14h],edx00401056 mov edx,dword ptr [ebp-8]00401059 add edx,10040105C mov dword ptr [ebp-8],edx0040105F mov eax,dword ptr [ebp-4]00401062 add eax,100401065 mov dword ptr [ebp-4],eax

E i 2


57/66

Expansion 2

13:14: *p3 = (unsignedchar)(j>255?255:j);00401068 cmp dword ptr [ebp-14h],0FFh0040106F jle main+6Ah (0040107a)00401071 mov dword ptr [ebp-18h],0FFh00401078 jmp main+70h (00401080)0040107A mov ecx,dword ptr [ebp-14h]0040107D mov dword ptr [ebp-18h],ecx00401080 mov edx,dword ptr [ebp-0Ch]00401083 mov al,byte ptr [ebp-18h]00401086 mov byte ptr [edx],al15: p3++;00401088 mov ecx,dword ptr [ebp-0Ch]0040108B add ecx,10040108E mov dword ptr [ebp-0Ch],ecx

Total of 26 instructions in the kernel

Alt ti i


58/66

Alternative using mmx

Iu8vec8 *v1,*v2,*v3; int i,j,k;

for(i=0;i


59/66

Optimised Assembler Loop

mov ecx ,32 ; load counter with 32

l1: movq mm0,[esi] ; load 8 bytes

add esi,8 ; inc src pntr

paddusb mm0,[edx] ; packed unsigned add bytes

add edx,8 ; inc src pntr

movq [edi],mm0 ; store 8 byte result

add edi,8 ; inc dest pntr

loop nz,l1 ; dec counter,

; repeat non zero

Go round only 32 times not 256

Total of 6 instructions in kernel

S d G i


60/66

Speed Gain

On image of 256x256 pixels

Old C code executes 26*256*256instructions = 1,703,936 instructions

Optimised mmx code executes 6*256*32

instructions = 49,152 Note that no compiler currently will give

the optimised code. It has to be handassembled.

Image Processing Librar


61/66

Image Processing Library

Intel Provide an image porcessing library

that can be downloaded from their web

site.

It provides efficient access to the MMX

hardware.

It provides frequently used Image

Processing Operations.

It requires a set of DLLs in your path to

run

Image Processing Library 2


62/66


At the core of IPL is the ability to write to a

single API and get the best possible

results for any Intel processor. The

libraries have as many as six processor-

specific branches for each function andsix sets of carefully written assembly

code, but only one entry point to each

function.



63/66




64/66


Use of Intel IPL complex and requires C

I have provided 2 java classes that call the IPL.

IntelBImage and IntelFImage. These aredocumented in the Jimage web pages. Theyinherit from ByteImage and FloatImage

To use them the Intel IPL must have beeninstalled on your machine and be on the path.

If you are forced to use Unix machines thelibraries will not be available to you.


65/66

Feedback


66/66

Feedback

What did you not understand here?

What would you like more information on?

AWT Lastest Version

Documents

Transcript of AWT Lastest Version