W3A.4

An analysis of object-based intelligent imageprocessing and retrieval system

Jia Bin Li, Chun Che Fung and Kok Wai WongSchool of Information Technology

Murdoch UniversityWestern Australia

j.li, H.fung, k.wonggmurdoch.edu.au

Abstract-In order to improve the process of analysis andretrieval of images, it is necessary to examine the execution ofsuch program at the lowest level. This paper reports the resultsobtained from profiling the execution of an object-oriented imageprocessing and analysis program termed ImageJ. Althoughprofiling has been used in software engineering to identifyexecution bottlenecks, to our knowledge, it has not beenconsidered as a means of analysis object based distributed imageprocessing systems. The paper summaries the characteristics ofthe program in four aspects: classes and method codes profiling,method calls, parameter types and return types and CPUpercentage. The profiling results show that the total amount ofclasses that was loaded at runtime is invariant to the change ofthe image size. Similar behaviours are observed for parametertypes and return types. Based on the information, an intelligentmonitor can be used to carry out automatic load scheduling andbalancing.

Index Terms-software profiling; image processing; objectoriented programming models; object based distributed systems

I. INTRODUCTION

PARALLELISM in image synthesis, analysis, processing andretrieval has been widely explored in a spectrum ofdisciplines. In applications of compression and

decompression, every image will be performed by the same

set of operations. Thus, specialised digital signal processors

(DSP) are pipelined to process continuous data strings toimprove the throughput of image processing. Such structure isso-called MISD (multiple instructions single data stream)architecture. On the other hand, an N-array processorsmachine, which is an example of SIMD (single instructionmultiple data streams), can manipulate N number of imagessimultaneously. There are a number of image processingsystems utilising both multiple instructions and multiple datastreams (MIMD) mechanisms. However, those systemsdramatically differ to each other in design andimplementation.

Shared memory processors (SMP) systems are generallywell understood and widely used. In such architecture, a

number of processors can concurrently write to (read from)the global memory. Access policy, however, has to beimplemented to ensure data consistency from concurrent writeoperations to the same memory address. Programming on a

SMP machine usually requires the use of synchronisedconcurrent processes or multiple threads.

Distributed systems are another form of parallel systems. Inthose systems, processors or processing units that consist of a

0-7803-9282-5/05/$20.00 ©2005 IEEE 315

processor and a memory unit are networked using some formof network topologies such as mesh. In order to programtheme efficiently, communicating entities have to beconsidered. These entities may be a set of communicatingsequential processes (CSP), or objects, or tokens. Mostdistributed systems reply on the CSP model such as MPI-C.To our knowledge, only few distributed systems consideredobject models or data-flow models.

Study of object based distributed systems is conductedthrough benchmarking and simulation. Benchmarking is usedfor analysing characteristics of object oriented programs andgenerating traces from these programs. Then, trace files areinputted to a software simulator for further examination of thedesign of such structure. This paper will focus on a discussionof the benchmarking analysis on an object oriented imageprocessing program. The program used in this study is termedImageJ.

This paper serves two purposes. First, it explores thepossibilities and advantages of using objects as a means ofconcurrency to improve performance of image processing andretrieval systems. Second, the profiling results are consideredto be used for optimising efficiency of object orienteddistributed programs. The rest of the paper is organised asfollows. Section II discusses the object oriented models.Section III presents the profiling methodology and theprofiling results are given in Section IV. Section V proposesan alternative implementation of the colour histogram usingfine grained objects.

II. OBJECT ORIENTED MODELSIt has been a misleading concept that execution of object-

oriented (00) programs is slower. Because someincompatibilities between the OOP model and currenthardware structures, a virtual machine has to be applied toovercome such incompatibilities. However, the virtualmachine causes overheads involved in execution and thereforeexecution of 00 programs is generally considered to beslower comparing to native codes such as C, when they arerunning on a conventional machine.

Efficient virtual machines have been developed such as theSun's Java Virtual Machine (JVM) or IBM's Jikes RVM.These virtual machines have been designed to improve theexecution efficiency and garbage collection. Due to theexistence of the intermediate layer between the high-level

ICICS 2005n A r-

Authorized licensed use limited to: Murdoch University. Downloaded on June 15, 2009 at 05:18 from IEEE Xplore. Restrictions apply.

program and the low level machine, the performance of 00software running on those virtual machines however are notable to match the performance of native codes such as C. Onesolution to overcome the limitation is to develop directexecution machines for 00 software such as Java. A numberof hardware designs have already been proposed for efficientexecution of Java programs in references [1-2]. While thesemachines can improve execution performance, they are stilllargely based on a single processor structure.

In the OOP model, communication between objects is viamessage passing. An object is inherently parallel and self-synchronised in nature. Therefore, the OOP model is wellsuited for a distributed hardware structure. Most existingdistributed systems rely on the Communicating SequentialProcesses (CSP) model. Although both the CSP model and theOOP model have an intrinsic nature of parallelism, theyapproach the issue in very different ways. The CSP model isbased on the concept of independent communicating processesacting in concert. It is necessary to make provision for someforms of synchronization mechanism and the model impliesthe concept of a global address space. In contrast, the OOPmodel relies on the simple concept of objects communicatingwith one another. An object includes a concept of state andmethods or code. In such case, the synchronisation is intrinsicand the contexts of memory are distributed. Furthermore, theexecution environment for the OOP model is highly dynamicas objects are constantly created and destroyed at runtime.

These variations suggest very different hardwarearchitecture and implementation. The CSP model suggestsfocus should be on a tightly coupled shared memory systemwith an efficient synchronisation mechanism. The OOP modelin contrast suggests a highly distributed system with acommunication structure that provides an efficient objectreferencing mechanism.

III. METHODOLOGYThe studied software package is an open source, image

processing and analysis software, which is implemented inJava [3]. Figure 1 shows the profiling procedure. Profilingfiles were generated by using a default profiler provided bySun JDK 1.4. These profiling files were inputted to theProfileAnalyser, a Java program that analyses the profilingfiles. Statistical analysis reports are produced as an outputfrom the ProfAnalyser. Based on these reports, four statisticaltables are set up and can be used to illustrate different aspectsof the characteristics of the benchmarks:

* Classes and methods profiling: measures the totalnumber of classes and method codes loaded duringexecution.

* Method calls: measures the number of messagesexchanged between objects.

* Parameter types and return types: gives statisticalinformation on different data types used as methodparameters or return values.

* CPU percentage: measures the amount of the CPU timespends on a particular method.

Since it is not the aim of this paper to design new algorithmsfor image processing or analysis, we focused on onecomputation only, calculating colour histogram, to illustrateour proposal. Different sizes of an image were computed tostudy the effect of the size of an image on the performance ofthe computation. The image sizes are, respectively, 160x120,320x240, 480x360 and 640x480. The Image used is shown inFigure 2.

Figure 1: An overview of the Profiling procedure.

Figure 2: A photo of the Nanu flower as the analysis image [4].

IV. PROFILING RESULTS

A. Classes and Methods ProfilingTable 1 shows the total number of classes and methods

loaded and objects initialised at runtime. In the table, the firstrow represents different sizes of the profiling image. Row 2-4shows the total amount of method codes that are loaded duringcomputation regarding various image sizes. Similarly, Row 5-7 shows the total number of classes constructed at runtime.The last row indicates the number of objects created.

Furthermore, the Classes and Methods may be subdividedinto two categories: system-supplied and user-defined.System-supplied classes or methods are those included in SunJDK, which their names generally start with java, javax orsun. The rest are considered as user-defined classes ormethods. It is obvious that the percentage of system-supplied

316

r--

pTofAnaNser

Authorized licensed use limited to: Murdoch University. Downloaded on June 15, 2009 at 05:18 from IEEE Xplore. Restrictions apply.

classes or methods is significantly higher than the percentageof user-defined ones for all benchmarks because system-supplied libraries had been well tested and documented andtherefore they are generally more efficient and safer to usethan those user defined. In addition, using system libraries canhelp to reduce the complexity of a program.

It is noted that the same measures are almost invariantacross different image sizes. This implies that the memory sizeof the execution machine, i.e. JRE 1.4, was not affected by thesize of the input image as the image is an external static entity.

Table 1: statistical information on runtime classes and methods.

Size 160x120 320x240 480x360Total Methods 4562 4586 4586

System Methods 4161 4168 4168User Methods 401 418 418

Total Classes

System Classes

User Classes

Initialised Objects

851 851 851797 797 797

54 54 54

2358 2362 2363

640x480

45624156406

851797

54

2353

B. Method Calls - Message Passing between ObjectsTable 2 illustrates the statistical summary of various types

of method calls. This differs from the methods describedpreviously. Method codes are functions definitions whilemethod calls are the process to invoke a particular method ofan object by another object. Similar to classes and methodcodes, there are two types of method calls; system calls anduser calls. System calls are those calls to system-suppliedcodes where user calls are calls which invoke user definedcodes. Further, a callee method and a caller method may be inthe same class - inner calls; otherwise, the call is an externalcall.

Table 2 shows that the total amount of method calls isinfluenced by various image sizes. Figure 3 indicates that thetotal method calls is linearly increasing with the size of theimage. In particular, methods such as getRed, getBlue andgetGreen got more accesses, as an obvious result, when itoperates on a larger image. Later, we will present an objectmodel that is able to parallelise the sequential accesses tothese methods.

Table 2: statistical summary of various types of method calls.

4000000

3500000

3000000/

2500000-2000000

1500000 -

1000000500000

00 100000 200000 300000 400000

Pixels

Figure 3: Linear relationship between the total method calls and the size of animage.

C. Parameter types and return typesThe ProfAnalyser utility program provides statistical

information on the parameter types (Table 3) and return types(Table 4) monitored by the profiling program. It is interestingthat the profiling results are roughly the same for all theimages. However, the percentage of different data types usedin parameter and return is not identical as shown in Figure 4and Figure 5. In particular, object types are frequently used,which are implemented using reference pointers.Consequently, it is important to design an efficient referencehandling mechanism to improve the performance of anexecution environment. The next mostly used data types areinteger and boolean, respectively. For the sake of highperformance, operations on integer and boolean have to beoptimised due to their regular usage.

Table 3: Statistical information on parameter types across various image sizes.

SizeBoolByteCharShortIntLongFloatDoubleArrayObjectTotal

160x120 320x240 480x360 640x480261 261 261 2591135361423100545023226724874

1135361443100545023226804902

1135361443100545023226804902

1135361414100545023226774868

Table 4: Statistical information on return types across various image sizes.Size

Total CallsCalls to system

code

Calls to user code 17613

160x120 320x240 480x3601054491 1584892 2425624

1036878 1566622 2407178

640x4803370895

3352953

18270 18446 17942

Inner Calls 385762 806257 1497567 2407492

External Calls 668729 778635 928057 963403

SizeBoolByteCharShortIntLongFloatDoubleArrayObjectTotal

317

160x1204505181937338132910212012248

320x240455S181937838132910312042262

480x360455S181937838132910312042262

640x480448S181937538132910211972244

Authorized licensed use limited to: Murdoch University. Downloaded on June 15, 2009 at 05:18 from IEEE Xplore. Restrictions apply.

Object

Array

Double

Float

Long

Int

Short

CharIByteBool

0 500 1000 1500 2000 2500 3000

Figure 4: Percentage of parameter types for analysis of the 160x120 image.

3.0% 1 + 0 ij.process.ImageProcessor.getBestlndex3.0% 1 + 0 ij.process.ByteBlitter.copyBits3.0% 1 + 0 java.awt.image.IndexColorModel.setRGBs3.0% 1 + 0 sun.awt.windows.WFramePeer.setlconlmage3.0% 1 + 0 java.awt.EventQueue.postEvent90.9% 19 + 11 Total interpreted

Compiled + native Method3.0% 1 + 0 ij.process.ColorProcessor.getHistogram3.0% 0 + 1 ij.gui.Newlmage.createBytelmage6.1% 1 + 1 Total compiled

Thread-local ticks:3.0% 1 Unknown: no last frame

Figure 6: Screen capture of the profiling of the histogram operation.

Table 5: Measures computation time of four sizes of the image.

CPU (percentage)9.10%6.10%3.10%N/A

Size

640x480

480x360

320x240

160x120

Char V. AN OBJECT ORIENTED IMPLEMENTATION OF THE COLOURByte HiSTOGRAm ALGORITHM

Bool Here, we will illustrate how the colour histogram can be

0 200 400 600 800 1000 1200 1400 implemented by using fine-grained objects to parallelise suchexecution.

Figure 5: Percentage of return types for analysis of the 160x120 image. Since the colour histogram algorithm counts of different

D. Measurement ofCPU time colour values, the following pseudo-code realises suchFigure 6 shows the screen capture of the execution algorithm:environment that was performing the histogram operations. Itis noted that there were a lot of methods involved in theprocess, including updating the GUI. However, the method ofinterest is the highlighted method calledij.process. ColorProcessor.getHistogram. The profiler alsogives the percentage of the CPU time that dedicated to theparticular method. Table 5 shows measurements on CPU timefor the getHistogram method. However, the measurement forthe smallest size image is not available since the cost of thefunction is too small to sample.

Flat profile of 0.73 secs (33 total ticks): Histogram

Interpreted + native Method12.1% 0 + 4 sun.awt.windows.WComponentPeer.pShow9.lOo 3 + 0 i1proccss.ColorProc6ssorg6ltHistogram6.1% 1 + 1 java.lang.ClassLoader.findBootstrapClass3.0% 0 + 1 sun.font.FileFont.getGlyphlmage3.0% 0 + 1 sun.awt.windows.WWindowPeer.reshapeFrame3.0% 0 + 1 sun.awt.windows.WComponentPeer.reshape3.0% 1 + 0 java.lang.ClassLoader.defineClassl3.0% 1 + 0 java.awt.Window.setClientSize3.0% 0 + 1 java.util.zip.Inflater.inflateBytes3.0% 0 + 1 sun.awt.windows.WFramePeer.createAwtFrame3.0% 0 + 1 ij.ImagePlus.getStatistics3.0% 1 + 0 sun.awt.AWTAutoShutdown.notifyThreadBusy3.0% 1 + 0 sun.awt.image.BytelnterleavedRaster.verify3.0% 1 + 0 java.awt.Container.validate3.0% 1 + 0 java.util.zip.InflaterlnputStream.read3.0% 1 + 0 java.awt.EventQueue.postEventPrivate3.0% 1 + 0 ij.process.TypeConverter.convertRGBToByte3.0% 1 + 0 ij.plugin.Histogram.run3.0% 1 + 0 ij.gui.HistogramWindow.<init>

Fori = I TomForj = I Ton

color vector = image.getRGB(i, j);End

End

Let us assume that the color vector is an array with length of256. It is obvious that such algorithm has complexity ofO(m*n). Hence, it requires a powerful machine to process verylarge size images such as satellite photos. In suchimplementation, the object image controls the access toinformation of the image such as colour values.

However, it is also possible to create a special object calledPixel. That is, each object stores information on a single pixelof the whole image. Thus, there will be (m n) number ofobjects (pixels) created at start-up. Rather to sequentiallyaccess to individual pixel object, one can issue a broadcastingmessage to the group of pixel objects. That is, a pixel objectadds its value to the color vector that is sent by its neighboursand passes the result array to the next neighbours. Thismechanism can be illustrated by Figure 4. In such model,every object needs to receive up to one message from its topor left neighbour while the other message can be ignored, thenit passes the results to two other neighbour objects. It is notdifficult to see that the complexity of the operation is now

determined by passing messages along the longest path(branch), i.e. O(m+n), which is a significant improvement to

318

Object

Array

Double

Float

Long

Int

Short

lkr--

Authorized licensed use limited to: Murdoch University. Downloaded on June 15, 2009 at 05:18 from IEEE Xplore. Restrictions apply.

the sequential implementation.There is another parallel implementation of image

processing systems using the object model described in [5]. Intheir model, image features such as HSV are implementedusing objects. These objects as primitive objects can be sharedby other software components to eliminate the need ofrecomputation of the same features.

Figure 7: A broadcasting implementation of the colour histogram algorithm.

VI. CONCLUSIONSThis paper studied the characteristics of an image

processing and analysis software program termed ImageJ byusing the profiling technique. Although profiling has beenused in software engineering to identify execution bottlenecks,to our knowledge, it has not been considered as a means ofanalysis object based distributed systems. The papersummaries the characteristics of the program in four aspects:classes and method codes profiling, method calls, parametertypes and return types and CPU percentage. The profilingresults show that the total amount of classes that was loaded atruntime is invariant to the change of the image size. Similarbehaviours are observed for parameter types and return types.

However, increasing in image size leads to increasing thenumber of messages exchanged between objects. In particular,it has significant impact on the methods that are related to thehistogram operations such as getRed, getGreen and getBlue.Therefore, we propose an alternative implementation that

utilise more fine grained objects. The complexity of the newimplementation is O(m+n) comparing to the sequentialimplementation that requires O(m.n) operations. However,such model is still too abstract that it has not consideredoverhead involved in communication. Simulation-basedanalysis on the new implementation, such as how does thenetwork latency affect on the overall performance, will bereported in the near future. It is also expected that theavailable information on the profile of execution will forminput to an intelligent monitoring unit which will carry outautomatic load scheduling and balancing based on theinformation of the objects. This will improve the imageanalysis and retrieval process.

ACKNOWLEDGEMENT

This project is conducted at the Centre for Enterprisecollaboration in Innovative Systems with support receivedfrom a grant awarded by the 2004 Murdoch UniversityResearch Excellence Grant Scheme (REGS). Jia Bin Li is arecipient of the International Postgraduate ResearchScholarship (IPRS) at Murdoch University from 2004.

REFERENCE[1] W. Chu and Y. Li, "An instruction cache architecture for

parallel execution of Java threads," Proceedings ofthe FourthInternational Conference on Parallel and Distributed Computing,Applications and Technologies, PDCAT'2003., 2003.

[2] C. M. Chung and S. D. Kim, "A dual threaded Java processorfor Java multithreading," Proceedings of 1998 InternationalConference on Parallel and Distributed Systems, Taiwan, 1998.

[3] ImageJ, Research Services Branch, [url]http:l/rsbJn/o.nih (accessed date: Jun 2005)

[4] Plants of Hawaii, [url]

u20flo g (accessed date: Jun 2005).[5] K. P. Chung, J. B. Li, C. C. Fung, and K. W. Wong, "A parallel

architecture for feature extraction in content-based imageretrieval system," The 2004 IEEE International Conference onCybernetics and Intelligent Systems, pp. 468-473, IEEE,Singapore, 1-3 Dec., 2004.

319

Authorized licensed use limited to: Murdoch University. Downloaded on June 15, 2009 at 05:18 from IEEE Xplore. Restrictions apply.