A Framework for Interacti veHardwareAcceleratedRemote3D-Visualization

KlausEngel,OveSommer,andThomasErtl

Universityof Stuttgart,IfI, VisualizationandInteractiveSystemsGroup�engel, sommer, ertl � @informatik.uni-stuttgart.de

http://wwwvis.informatik.uni-stuttgart.de

Abstract. In this paperwe presenta framework that provides remotecontrolto OpenInventoror Cosmo3Dbasedvisualizationapplications.A visualizationserver distributesa visualizationsessionto Java basedclients by transmittingcompressedimagesfrom the server framebuffer. VisualizationparametersandGUI events from the clients are applied to the server applicationby sendingCORBA (CommonObjectRequestBrokerArchitecture)requests.The framework providestransparentaccessto remotevisualizationcapabilitiesand allows sharingof expensive resources.Additionally the framework opensnew possibilitiesfor collaborative work and distanceeducation.We presentateleradiologysystemand an automotive developmentapplicationwhich makeuseof theproposedtechniques.

1 Intr oduction

Therapidevolutionof todaysdigital communicationnetworksenablesaccessto ahugeamountof scientificdataandremotecomputationcapabilities,likesharedmemorymul-tiprocessormachinesor specialhigh-endgraphicshardware.Concerningthis develop-mentwe believe that techniqueshave to be developedto enablethe visualizationofscientificdatausingremotelyavailablehigh-endvisualizationarchitecturesfrom anyInternet-connecteddesktopcomputer.

In the developmentof todaysdesktopcomputerstwo contrarytrendscan be ob-served.Ontheonehandthecomputationandrenderingcapabilitiesof modernlow-costPCsarequickly rising.Onetheotherhandthenetwork computer(NC) is averysimpleandinexpensivedevice thatactsasa thin client to morepowerful servermachines.

Up to now hardwareacceleratedrenderinghasrequiredlocal rendering.For exam-ple, in theX-Windowssystema remotelystarted3D applicationuseslocal 3D acceler-ationhardwarefor rendering.Local renderingenableshigh interactionrates.However,therearecertainconditionsunderwhich local renderingis impossibleor undesirable.For example,typicaldatasetsfrom scientificsimulationsandmeasurementsneithercanquickly transferrednor canbehandledon moderndesktopcomputersbecauseof theirimmensesize.On the othersidethereis a classof high-endservers,supercomputersandworkstationswith special3D graphicsaccelerationhardware,numericalcomputa-tion power andhigh-performanceIO bandwidththat provide the necessarymeanstohandlelargescientificdatasets.Furthermorelocal renderingoftenrequiresthetransferof sensitive datafrom serversto clients.This maybe undesirablebecauseof security

Framebuffer pBuffer

Graphics Hardware

OpenGL

OpenInventor /Cosmo3D

Server Application

imageencoder

CORBAapplicationcontrolinterface

imagedecoder

Framebuffer

Graphics Hardware

socket

corba events Client Application

Server Client

CORBAeventgenerator� display window

mouse events

applicationparameters,menu events

CORBAOI/Cosmo3Dcontrolinterface

CORBAeventgenerator�corba events

thread

Fig.1. Theclient-server scenariousedin this framework. Oneor moreclientsremotelyobserveandcontrolanapplicationonaserverby usingtwo CORBA interfacesandasocket connection.

reasons,e.g. for patientdatain medicalapplicationsits moresecureto transfer(en-crypted)imagedatainsteadof originaldata.

Wedevelopeda framework for theremotevisualizationof largescientificdatasets.It is ableto use3D accelerationhardwareof theserver systemandallows to usethesefeaturesfromany Internet-connectedPCinteractively. Thegeneralscenariousedfor theframework consistsof anhigh-endvisualizationserver for renderingimagesandoneormoreclientsto provide a userinterface,displaythe renderedimages,andcontrol thevisualization(Fig. 1). Thereareno specialrequirementsfor the choiceof theclients.Evena PDA with TCP/IPnetwork connectionandadequatedisplayresolutionwouldbesufficient.

OntheserversideanOpenInventoror Cosmo3Dbasedvisualizationapplicationisrenderingeitherimageson-screeninto the framebuffer or off-screeninto thepbuffer.Therebytwo differentscenarioscanbederived:A masterusercontrolsthesessionontheserver machineandusesthevisualizationapplicationlocally. Imagesrenderedon-screeninto the framebuffer areencodedandtransferredto all clientsusinga socketconnection.Thesecondscenariousesa display-lessserver that runsthe server job inthebackgroundandrendersinto thepbuffer. Thepbuffer is aspecialprotectedgraphicsmemoryblockwhichallowshardwareacceleratedoff-screenrendering.Imagesarereadfromthisbuffer into mainmemory, encodedandtransferredto theattachedclientsusinga socketconnection.

TheJavaclientenablestransparentremoteaccessto thevisualizationcapabilitiesofthehigh-endserver from any window systemsupportingplatform.Thereceivedimagedatais decoded,storedasa Java2D bufferedimageanddrawn into the framebuffer.Mouseandkeyboardeventsgeneratedon the client aretransportedto the server ma-chineusingCORBA methodcalls.Theserver providesanOpenInventoror Cosmo3Dinterfacefor theseeventsthatpassestheparametersto objectsandfunctionsthathandletheeventson theserver side.Thereby, whenusingOpenInventor, a manipulatorwid-getcanberemotelypickedanddragged.Applicationparametersaretransferredto theserverusinga secondCORBA eventinterface.

Using our framework, which consistsof several C++ andJava classes,the clientandserver applicationscanbeeasilydeveloped.Moreover the framework enablestheconversionof any existingvisualizationapplicationinto a remotelycontrollableonebyaddingonly a small amountof additionalcode.Multiple clientsareableto sharetheview on thedataandinteractin turn.

In thefollowingsectionwewill describesomerelatedwork in thefieldof webbasedvisualization.Section3 outlinesthearchitectureof our framework. Section4 explainstwo exemplaryapplicationswhich usethe proposedtechniques.Resultsaregiven insection5. We will concludethepaperwith someremarkson futureactivities.

2 RelatedWork

In thepastyearsseveralapproachesfor scientificvisualizationondigital networkswereinvestigated.Oneof thefirst progressiveapplicationsfor volumevisualizationwaspre-sentedby Lippertetal. [7].

Hendinintroduceda VRML basedvolumevisualizationtool [5], which usesthreestacksof perpendicularslices.We introducedtechniquesfor fastvolumeclipping,col-laborativework, anddatasizereduction[2] asanextensionto Hendin’sapproach.

An oppositeapproachwasproposedby Ma etal. Thewebbasedvolumevisualiza-tion systemcalledDiVision allows usersto exploreremotevolumetricdatasetsusinga web-browser[10]. Thesystemcomputesimageson a visualizationserver, which aretransferredto the client and insertedinto a graph.However, the applicationdoesnotsupportinteractivemanipulation.

In the virtual network computing(VNC) systemby Richardsonet al. [11], servermachinessupply applications,data,and desktopenvironmentsthat can be accessedfrom any Internet-connectedmachineusinga wide varietyof machinearchitectures.Itis a remotedisplaysystemwhichallowstheuserto view acomputingdesktopenviron-mentfrom anywhereon theInternet.VNC doesnot supportto useremote3D graphicsaccelerationhardware.

JustrecentlySGI announcedOpenGLVizserver [8]. From the limited amountofinformationwhich is currentlyavailableit is understoodthat OpenGLVizserver willenableasingleOnyx2 workstationto distributevisualizationsessionsto multipleUNIXoperatingsystemdesktopworkstationsby transmittingcompressedimagesfrom theOnyx2 framebuffer. While thesystemonly seemsto work insideorganizationnetworksandonly on SGI machines,our framework allows transparentaccessto any high-endserver from any Internet-connecteddesktopPC.

3 The Framework

Theframeworkconsistsof severalC++classesfor theserversideandJavamodulesthatareusedto build aclientapplication.Becauseof themodularstructureof theframeworkit canbeappliedto any visualizationapplicationandnew codecscanbeadded.

3.1 Server modules

A stand-alonevisualizationapplicationcaneasilybeconvertedinto aremotelyaccessi-bleoneby addingaCORBA interfaceandslightlymodifyingthescenegraph.Currentlytheframework wasadaptedfor OpenInventorandCosmo3D.

Open Inventor: OpenInventortraversesthescenegraphin a fixedorderfrom top tobottomandleft to right. Becauseof this behavior we canadda new scenegraphnodenamedPBufferNode thatswitchesto the pbuffer renderingcontext in front (to theleft) of the contributing nodes,generallyat the leftmostposition.TheSocketNodereadstheframebuffer contentinto mainmemoryandperformstheencodingandtrans-missionof theimagedata.It is insertedbehind(to theright) of thecontributingnodes,generallyat therightmostpositionin thescenegraph.

As soonasthescenegraphhaschangedor new mappingparameters(e.g.modifiedtransferfunctions)havebeenreceiveda renderactionis appliedto therootnodeof thescenegraph.Thenthefollowing sequenceof stepsis performed:

1. Therenderactiontraversesthescenegraphfrom top to bottomandleft to right.2. As soonas the PBufferNode is reached,the rendermethodswitchesto the

pbuffer renderingcontext. If therewasnoPBufferNode nodeaddedto thescenegraphtherenderingcontext remainson-screen.

3. Therenderactioncontinuesto traversethroughthescenegraphandtheshapenodesdraw geometryinto thecurrentrenderingcontext.

4. As soonastheSocketNode is visited,thecontentof theframebuffer is readintomainmemory. It compressestheimagedatausingoneof theavailablecompressionmethodsandprovidestheencodeddatato all connectedclientsvia a givensocketport.

5. Therenderactioncontinuesto traversethescenegraph.

OpenInventorprovidesconvenientmechanismsto convert2D eventsreceivedfroma client to 3D events.Thestepsthatareperformedcanbesummarizedasfollows:

1. Theclientapplicationregistersinterestin particulareventswith its window system.2. Theclientapplicationreceivesaneventfrom its window system.3. Theclient callstheappropriateservermethodusingCORBA anddeliversthenec-

essaryparameters(e.g.mouseposition,mousebutton)to theserver.4. The server applicationtranslatesthe event into a SoEvent (for mouseevents

SoLocation2Event).5. TheSoEventis insertedinto aninstanceof theSoHandleEventAction class.6. Thehandleeventactionis appliedto the top nodeof thescenegraph.This action

traversesthescenegraph.Eachnodeimplementsits own actionbehavior. Whenanode(typically a manipulator)is found,it handlestheevent.

7. If necessarya renderactionis appliedto thescenegraph.

TheCORBA mainloop andtheOpenInventormainloop arerunningin two sepa-ratethreads.As OpenInventoris not thread-save thereceivedclient eventscannot beappliedto thescenegraphimmediately. TheTicker classis anOpenInventorengine(derived from SoEngine) that storesSoHandleEventActions createdby clienteventreceiving CORBA methods.TheTicker classis calledfrom theOpenInventormainloop in fixedtimestepsandtriggersaSoHandleEventAction.

Cosmo3D: Thescenegraphstructureprovidedby Cosmo3Ddiffersfrom thatof OpenInventor. No informationis inheritedhorizontallyin theCosmo3Dscenegraph,whichis traverseddownwardsfrom top to bottomin eachbranch.OpenGLOptimizeroffersdifferentkindsof scenegraphtraversalactions:On the onehandtherearedepth-firsttraversalactionswhich do their work in the sameorderasOpenInventoractionsdo.Ontheotherhandbreadth-firsttraversalactionscanbeappliedfor parallelizationusingmultiple processors.Thus,a new Cosmo3Dscenegraphnode,which implementsthecorrespondingfunctionality of the describedPBufferNode andSocketNode, isderived from the Cosmo3DclasscsGroup. Its methoddrawVisit() containsapre-anda post-traversalsection.Theformerswitchestherenderingcontext to pbufferwhile thelattersectionstartstheencodingandtransmissionof theimagedata.

In contrastto the OpenInventorscenariothe new Cosmo3Dscenegraphnodeisusedasa rootnodewhichenfoldstheoriginalscenegraphasits subgraph.

Alternatively, thescenegraphcanbe left untouchedandjust onefunctioncall hasto beaddedat thebeginningof themethodopXmViewer::swapBuffers(). Thisfunctiondeterminesthecurrentdrawing buffer, callsglReadPixels() andinitiatestheimageencodingandtransmission.

Incomingmouseor key eventsareinterpretedandhandledafter a pre-processingstepby thecorrespondingmethodsin theopXmViewer classasif they hadbeenap-pearedon the local site.For example,mouseeventsaresentto a methodwhich con-vertsthe incomingdataandemulatesthe methodprocessPendingEvents() ofopXmViewer.

3.2 Client modules

TheclientmodulesareimplementedusingtheJAVA2 platform.Weprovidethefollow-ing classes:

RenderArea: The basicrenderareavis.inventor.RenderArea is a drawingareafor framebuffer content,thatwasreceivedfrom thevisualizationserver. For thispurposetheJava2Djava.awt.image.BufferedImage classis used.Thedraw-ing areaalso relaysmouseeventsthat are sentto the server. It is derived from thejava.awt.Panel classandcanbeaddedinto any Java container. Thevis.inventor.FullViewer class,derivedfromthisclass,providesthelook-and-feelandfunctionalityof OpenInventorsSoXtFullViewer, whichincludesadecora-tionaroundtherenderarea.Thisdecorationis makeupoutof thumbwheels,sliders,andpushbuttons.It alsosupportsa context menuthatallows to changetheOpenInventorrenderingtypein severalways.

Decoders: vis.imagedecoder.Decoder is theabstractbaseclassfor all codecswehave implemented.New decoderscanbeintegratedinto theframework by derivingnew classesfrom thisbaseclass.We providethedecodersvis.imagedecoder.ZLIBDecoder,vis.imagedecoder.LZODecoder,vis.imagedecoder.RLEDecoder andvis.imagedecoder.RAWDecoder.

time

event

render encode

apply

read framebuffer

CORBA request image

decode writeframebuffer

latency

Client

Server

Fig.2. Thelatency in betweenmanipulationandimageupdateconsistsof request,event transla-tion, rendering,framebuffer read,encoding,transfer, decodinganddisplaytime.

SimpleViewer: Wealsoprovideaclassvis.viewers.SimpleViewerwhichcanbeusedasaclienttoobserveany remotevisualizationsessionof anapplicationthatwasadaptedusingourframework.For thispurposeonly somelinesof codehavetobeaddedto a stand-alonevisualizationapplication.

3.3 Network Communication

Thetransmissionof imagesandtheremotecontrolof theserverapplicationarestrictlydecoupled.The imagedatais streamedthrougha TCP socket connectionandthe ap-plication,mappingandrenderingparametersareappliedto the server applicationbyusingCORBA methodcalls.Oncenew mappingor renderingparameterswerereceivedby the server they areappliedto the visualization.Then the renderingis performed,the framebuffer is copiedinto main memory, the datais encoded,transferredto theclients,decodedandfinally copiedinto the framebuffer of the client (Fig. 2). Thesestepsdeterminetheoverall latency of theapplicationbetweenmanipulationandimageupdate.

CORBA interfaces: TheCORBA interfaceto theserver is dividedup into two parts:

– Theeventsfor therenderareaof theserverapplicationaresentto theserver usingthe interfaceRenderArea. For a server that usesOpenInventor the interfaceFullViewer extendstheRenderArea interfacewith functionalityof theOpenInventorSoXtFullViewer class.

– Applicationspecificparametersareaccessedvia theApplication interface.Weprovideabaseinterfacewhichcanbeextendedto makethefunctionalityof aserverapplicationaccessible.For example,if theapplicationallowsto addclippingplaneswewouldhaveto addamethodaddClippingPlane to theinterfaceandimple-menttheappropriateservermethod.

Image compressionand transfer: The imagetransmissionis decoupledfrom theCORBA interfacesbecausewe wantedto beableto quickly replacetheimagetransfer

codecwith moresophisticatedoneswithout any changesin theremainingnetworkingcode.It could be aswell integratedinto the CORBA communication,but right nowwe transferthe imagedatausinga TCPsocket connection.Theusercanselectoneoffollowing encodingtypes:

– RAW: Thecodecreturnstheoriginaldata,thusnocompressionis performed.Thismethodcanbeusedin a highbandwidthnetwork to keepthelatency low.

– ZLIB: Performsaloss-lesscompressionbasedontheZLIB library [12]. Thealgo-rithm findsduplicatedstringsin theinputdata.Thesecondoccurrenceof astringisreplacedby a pointerto thepreviousstring,in theform of a pair (distance,length).ZLIB compressionis a standardfeatureof Java sinceversion1.1andis performedvia fastnativecode.

– LZO: Performsa loss-lesscompressionbasedon the LZO library [9]. LZO is ablock compressionalgorithm- it compressesanddecompressesa block of data.Block sizemustbethesamefor compressionanddecompression.LZO compressesablockof datainto matches(aslidingdictionary)andrunsof non-matchingliterals.LZO favors speedover compressionratio. As the LZO codecis not a standardmethodof Java, the LZO decompressionis accessedvia a Java Native Interface(JNI) call.

– RLE: Performsa loss-lesscompressionbasedon run-lengthencoding.

Despiteof the imagecompressionthe amountof datathat hasto be transmittedto the client is still too large for low-bandwidthnetwork connectionslike an ISDNconnection.In orderto allow highinteractionratesonsuchconnectionsweadditionallyapply an imagesize reductionwhile interactingwith the data.For example,whenamanipulatoris pickedanddraggedwetransfertheimageswith half or quarterresolutionandscalethe imagesto full sizeon the client. As soonastheuserstopsdraggingthemanipulatorafull frameis transmitted.Thecombinationof compressionandimagesizereductionprovidessufficient frameratesevenon56k modemnetwork connections.

4 Collaborativevisualization envir onments

In areaswherespecialistsareseparatedby distancethe work-flow efficiency can beimprovedby collaborative applications.For example,suchapplicationsallow userstodiscussthevisualizeddatasharingthesameview. Furthermore,expensive expertscanbeconsultedanddistanceeducationor advancedtrainingcanbeheld.Additionally, ourapproachprovidessimultaneousaccessto a server applicationfor multiple users.Thusthecapabilitiesof expensivehardwarecanbeshardedby low-costclient systems.

In this sectionwe will presenttwo applicationsthat wereextendedby our frame-work to enablecollaborative work. Thefirst oneis anapplicationthatuses3D texturemappinghardwareof high-endgraphicsworkstationsto visualizemedicalvolumedata.Thesecondoneis employed in thecardevelopmentprocessfor visualizationof hugetime-dependentfinite elementmodels.

4.1 Teleradiology

Theuseof 3Dtexturemappinghardwarehasbecomeapowerfulvisualizationoptionfordirectvolumerendering[1,15].Unfortunately, upto now 3D texturemappinghardware

is still restrictedto high-endgraphicsworkstations.Now onecanmake the hardwarecapabilitiesaccessibleto almostany clientsystemby usingour framework.

For this purposean interactive stand-alonetexturebasedrenderingapplicationformedicalvolumedatahasbeenextended(Fig. 3, left). It hasbeenintegratedinto theOpenInventorframework in orderto obtainthewholeflexibility andfunctionalityof-feredby this graphicsAPI. By introducinga new class,thevolumerendererhasbeenrepresentedasa separateobjectwithin the hierarchicalstructureof the scenegraph.This allows convenientuseof built-in manipulators,sensors,editorsandotherprede-finedclasses,methodsandfeatures(light sources,anti-aliasing,perspective/orthogonalprojection,fly, walk, trackball)[4,13].

First a few lines of codewereaddedin orderto extendthe scenegraphusingtheSocketNode we previously introduced.With this smallmodificationit is possibletojoin avisualizationsessionpassivelyusingaweb-browserandobservethevisualization.To allow remotecontrolof theapplicationwe hadto extendthebaseCORBA applica-tion interfacewith additionalfunctionality (e.g.updateof transferfunction,addingofclipping planes,etc.).On theclient sidea viewer applicationwasdevelopedthatpro-videsthesamelook-and-feelasthestand-aloneapplication(Fig. 3, right). We usedthevis.inventor.FullViewer classasthedisplayareain themainwindow, addedsomebuttonsto thedecoration,andreimplementedthemenubarandthedialogsof thestand-aloneapplicationin Java(e.g.thetransfer-functiondialog).

Recapitulating,ourapproachallowsworkinggroupstodiscussmedicalvolumedatasetscollaboratively. 3D texture supportinggraphicsworkstations,which weretoo ex-pensivefor many hospitals,cannow beusedremotelyfrom any desktopPC.No patientdatais transferredthroughthe network and the securityof the imagestreamcan beensuredby usingSSHsocket tunneling.

4.2 Visualization of crash-worthinesssimulations

Anotherexamplewherethe presentedtechniqueis very useful is the visualizationinthe car developmentprocess.In cooperationwith the BMW Group we developedaCosmo3D/ OpenGLOptimizerbasedapplicationwhich is in productiveusein thepre-andpost-processingof crash-worthinesssimulations.

Thecarbodiesarerepresentedby about500.000mainly four-sidedfinite elements.During simulationthefirst 120msarecomputedandthecoordinatesof thedeformingmesharestoredin 60 time stepstogetherwith thetrackedparametersinto a resultfile.Thoseresultfilesoftencontainmorethan1.5GB of data.

Our applicationcrashViewer builds a Cosmo3Dscenegraphthatdescribesthecarbody for eachtime step.We developedan OpenGLOptimizer basedviewer whichallows theengineerto visualizeandanimatethecrash.Furthermore,thecrashperfor-mancecanbeanalyzedby directlymappingthetrackedscalarparametersascolorsontothegeometryor by visualizingtheforceflux by forcetubes[6].

To representthe large scaleGouraud-shadedtime-dependentgeometrywith con-stanttopologyfor eachtime stepin a Cosmo3Dscenegraph,asproposedin [14], ap-proximately700MB of mainmemoryis required.Sinceonly high-endgraphicswork-stationsareequippedwith sucha lot of mainmemorya techniquewassoughtto make

thoseexpensiveresourcesaccessibleto low-endsystems.Hence,weextendedthestand-alonevisualizationapplicationby our approachwhich offersa solutionby transferringimagedatafrom any OpenGLsupportingworkstationto arbitrarywindow supportingclientsystems.

If ameetingof theanalyzingengineersis too time-consumingbecausethey are,forexample,locatedatdistantsites,theimagetransferallowsfor acollaborativediscussionon thecrash-worthinessof thecurrentmodelvariant.Oneengineerstartsthevisualiza-tion applicationwhich is ableto provide the renderedimagesin encodeddatastreamform aspreviouslyoutlined.

Therearetwo scenarios:first, theotherengineerstartsaJavaapplicationthatoffersa minimal setof functionalityof theoriginal C++-basedvisualizationapplication.2Dmouseeventsandkeystrokestriggeredon the client sideare transmittedback to theserverapplicationandinterpretedtherelikedescribedin section3.1.In thesecondsce-nariowhereoneengineerwill communicatesomeresultsto oneor moreengineerswhodo not have to interactwith themodeltheformeroneadvisesanURL to otherpartic-ipants.They startany HTML browser, downloada HTML pagefrom the givenURLwhichincludesaJavaapplet(seeFig.4).ThisappletencapsulatestheSimpleViewerdescribedin section3.2.

Summarizing,ourapproachallowscollaborativeworkinggroupsto discusssimula-tion resultsin distributedheterogeneousenvironments.Therearelow requirementstoparticipatingclient systems.Additionally, in regardto securityaspects,for example,ifthird party engineersof subcontractorsareinvolved,thepuredatawill stayin-house;insteadjust imagedatais transferred.We expect,that this techniquewill facilitatethecollaborationbetweenaccordantworking groupsof BMW andRover, whereit will betestedin thenext few months.

5 Results

In thissectionweshow resultsfor theproposedtechniques.On theserversideall testswererunonaSGIOnyx2 workstationequippedwith two 195MHz R10000processorsand 512 MB of main memory. A SGI O2 workstationwith the sameprocessorand128MB of mainmemoryanda 333MHz CeleronPCequippedwith 64 MB of mainmemorywereservingastheclientsystems.TheO2waslinkedvia a100MBit Ethernetnetwork connectionandthePCwaslinkedusinga 64 kBit ISDN Internetconnection.We usedthe medicalvolumevisualizationenvironmentwith a 512x512x106CT dataset.A typical imagesequencewith framesof 704pixelswidth, 576pixelsheightand24bitsdepthwasused.

First we analyzethe frameratesthat canbe achieved over the local network andthe Internetconnection(Table1). While interactingwith thevolumethe imageswererenderedandtransferredat quarterresolution(176x144pixels),after interactiona fullframewastransmitted.This leadsto fasterrendering,encoding,transfer, decodinganddisplaytimes.Whenusingthestand-alonenon-networkedvisualizationapplicationweachievedan averageframerateof 4.1 for the full framesand25.4 framespersecondfor quarterframes.Using the ISDN connectionno interactive rateswereachievedus-ing full frames.This is negligible during interactionbecausethenwe aretransferring

datasize LOCAL LAN ISDNmethod full quarter full quarter full quarter full quarterRAW 1.2mb 76 kb 1.9 19 - -ZLIB 106kb 6 kb 2.2 14 0.09 1.5LZO 150kb 9 kb

4.1 252.4 16 0.06 1.3

RLE 160kb 10 kb 2.5 16 0.06 1.2

Table 1. The listed framerateswereachieved whenvisualizinga CT dataset (512x512x106)locally, remotely using ISDN and remotely in a LAN network. Quarter resolution frames(176x144 pixels) were sent while interacting with the volume and full resolution frames(704x576pixels) weresentafter interaction.The averageamountof transferreddatafor a fullandquartersizedframeis denotedin theleft partof thetable.

quartersizedimagesonly. Thusinteractive refreshratescouldbeachieved.Only if theuserstopsto interact(e.g.releasesamanipulator)theuserhasto wait somesecondsforthe full sizeframe.BestframerateswereachievedusingZLIB compressionbecausethe network bandwidthis the limiting factor for transmissionvia ISDN. We will seelater that ZLIB compressionhasthebestcompressionratio. For a local network con-nectionbestframerateswereachievedusingRLE compressionbecausetheencodinganddecodingtimesarethecritical values.

Secondly, we comparethe encodingtime andthe compressionratioswhenusingZLIB, RLE andLZO compression.The averageencodingtime was180 millisecondsfor ZLIB, 50millisecondsfor LZO and30millisecondsfor RLE.TheaveragedecodingtimeontheO2was93millisecondsfor ZLIB, 29millisecondsfor LZO and13millisec-ondsfor RLE.Obviously, concerningthecompressionratioZLIB compressionis aheadof theothercompressionmethods(Fig. 5). RLE performsnearlyaswell asLZO com-pression.RLE is thesimplestandthusthefastestcompressionmethodwe investigated.Thecompressionratiosof all methodsof coursedependon thecoveringof thescreenspaceby thevisualization.Thatis why amagnificationleadsto lowercompressionfac-tors.Becauseof theseresultswecurrentlyfavourZLIB compressiononlow bandwidthchannelsandRLE compressiononhighbandwidthchannels.

6 Conclusionsand Future Work

We have presenteda framework which allows remotehigh-endvisualizationfrom anyInternet-connected,Java-enableddesktopPC.Theintroducedtechniquesweredemon-stratedby two applications.A volumerendererfor 3D medicaldata,thatusesspecial3D texturehardware,cannow beusedremotelyfrom any Internet-connectedPC.Wewill effect an applicationstudywith the Departmentof Neurosurgery of the Univer-sity of Erlangen-Nuremberg. Furthermore,a visualizationapplicationfor large scaledatasetsof crash-worthinesssimulationswasextendedby thepresentedframework toenablecollaborationin thecardevelopmentprocess.

As a mainresultof our work it is now possibleto remotelyexplorehugescientificdatasetson specializedserver hardwareusing low-costclients.We showed that thisis evenpossibleusinga low bandwidthchannellike an ISDN connection.The trans-fer of GUI eventsandapplicationparametersrequiresa muchlower bandwidththan

thedownloadof renderedimages.Exactly this scenariois givenwhile usinglow-costbroadbandInternetconnectionslikecablemodemsor satelliteconnections.

An areaof future work involvesthe developmentof specializedimage-streamingcodecsfor computergeneratedimagesequences.First resultsusingvideo streamingcodecswere presentedin [3]. However, currentlyavailable codecsare basedon theneedsof Internetvideostreams.Thosestreamshave differentcharacteristicsthanren-deredones(variableframerate,partial changesin consecutive frames,...). Onepos-sibleapproachwould be to uselossycompressionwhile interactingwith thedataandloss-lesscompressionwhenhaving still images.Also imageencryptiontechniquesarenecessaryfor transferringsensitivedataover theInternet.

References

1. B. Cabral,N. Cam,andJ. Foran. AcceleratedVolumeRenderingandTomographicRecon-structionUsingTextureMappingHardware. In A. KaufmanandW. Kruger, editors,1994Symposium on Volume Visualization, pages91–98.ACM SIGGRAPH,1994.

2. K. EngelandT. Ertl. Texture-basedVolumeVisualizationfor Multiple Userson theWorldWideWeb. In 5th Eurographics Workshop on Virtual Environments, 1999.

3. Klaus Engel,Ove Sommer, ChristianErnst,and ThomasErtl. RemoteVisualizationus-ing Image-StreamingCodecs.In Proceedings of Symposium on Intelligent Multimedia andDistance Education, Baden-Baden,Germany, August1999.

4. P. Hastreiter, H.K. Cakmak,andTh. Ertl. IntuitiveandInteractive Manipulationof 3D DataSetsby IntegratingTextureMappingBasedVolumeRenderinginto theOpenInventorClassHierarchy. In K. SpitzerTh. Lehman,I. Scholl,editor, Bildverarbeitung fuer die Medizin:Algorithmen, Systeme, Anwendungen, pages149–154.Inst. f. Med. Inf. u. Biom. d. RWTH,Aachen,Verl. d. AugustinusBuchhandlung,1996.

5. Ofer Hendin,Nigel John,andOfer Shochet.Medical VolumeRenderingOver the WWWusingVRML andJAVA. In Proceedings of MMVR, 1997.

6. S. Kuschfeldt,O. Sommer, T. Ertl, and M. Holzner. Efficient Visualizationof Crash-WorthinessSimulations.IEEE Computer Graphics and Applications, 18(4):60–65,1998.

7. L. Lippert, M.H. Gross,andC. Kurmann. Compressiondomainvolumerenderingfor dis-tributedenvironments.In Proceedings Eurographics ’97, pagesC95–C107,1997.

8. SGINewsroom.SGIBringsAdvancedVisualizationto theDesktopwith OpenGLVizserver.http://www.sgi.com/software/vizserver/.

9. M.F.X.J.Oberhumer. Lzo. http://wildsau.idv.uni-linz.ac.at/mfx/lzo.html.10. J.PattenandK.-L. Ma. A GraphBasedApproachfor VisualizingVolumeRenderingResults.

In Proc. of GI’98 Conference on Computer Graphics and Interactive Techniques, 1998.11. T. Richardson,Q. Stafford-Fraser, K. R. Wood,andA. Hopper. Virtual Network Computing.

IEEE Internet Computing, 2(1),January1998.12. G. Roelofs.Zlib. http://www.cdrom.com/pub/infozip/zlib/.13. O. Sommer, A. Dietz,R. Westermann,andT. Ertl. An InteractiveVisualizationandNaviga-

tion Tool for MedicalVolumeData. In V. Skala,editor, Proc. 6th International Conferencein Central Europe on Computer Graphics and Visualization ’98, pages362–371,1998.

14. Ove SommerandThomasErtl. Geometryandrenderingoptimizationsfor the interactivevisualizationof crash-worthinesssimultations.In Proceedings of SPIE, Visual Data Explo-ration and Analysis VII, volume3960,January2000.to appear.

15. Rudiger Westermannand ThomasErtl. Efficiently Using GraphicsHardware in VolumeRenderingApplications.In Computer Graphics Proceedings SIGGRAPH ’98, AnnualCon-ferenceSeries,pages169–177.ACM SIGGRAPH,July1998.

Fig.3. Transparentremoteaccessto 3D texturemappinghardwareof a C++ server application(on the left) from a Java client application(on theright). Note that theclient providesthesamelook-and-feelasthestand-aloneapplicationby providing anOpenInventor-like decoration.Thedisplayedmanipulatorscanbepickedanddraggedremotely.

Fig.4. Transferof imagedatafrom theoriginalvisualizationapplication(ontheleft) via asocketconnectionto aHTML browser(on theright).

Fig.5. CompressionRatiosof ZLIB, LZO andRLE compressionfor a typical imagesequence.A rotationof thevolumefollowedby amagnificationwasperformed.