Splatting Illuminated Ellipsoids with Depth Correction

Stefan Gumhold

University of Tubingen,WSI/GRIS

Sand 14, 72076 Tubingen, GermanyEmail: gumhold@gris.uni-tuebingen.de

Abstract

Ellipsoids are important primitives used in visual-ization and modeling, where often a larger num-ber of ellipsoids have to be displayed in real-time.The standard approach of tessellating each ellipsoidinto a smooth polygonal mesh leads to unaccept-able polygon counts that dramatically increase therendering time. In this paper a method is proposedto splat ellipsoids perspectively correct. The splat-ted ellipsoids are illuminated with the accuracy offloating point precision by exploiting the fragmentshader facility of current graphics accelerators. Itis also shown how to correct the depth value of thefragment position such that overlapping ellipsoidsare displayed correctly.

1 Introduction

Ellipsoids have been used as primitives in differ-ent applications for visualization and modeling. Inthe visualization domain, ellipsoids have been usedsuccessively for the splatting of volumetric datasets [15, 8, 9, 13, 10, 17], 3D clouds [3]. and pointsampled surfaces [11, 18, 12]. In all three typesof approaches ellipsoids are the basis functions intowhich the volumetric or surface data set is decom-posed. The ellipsoids represent the density of thedata set, heavily overlap and have to be superposedin order to reconstruct the data set.

In our approach we are not interested in the splat-ting of densities but in splatting the illuminated sur-face of ellipsoids. The application for which we de-veloped our approach is the visualization of sym-metric tensor fields [14, 2, 7, 6, 5, 16]. The mostintuitive approach to visualize a symmetric tensoris to render an ellipsoid. The mathematical relationbetween ellipsoids and symmetric tensors is derivedin section 2. In the simplest tensor field visualiza-

Figure 1: Test scene of a large number of ellipsoidsthat visualize a tensor field.

tion approach one tessellates the three dimensionaldomain of the tensor field with regularly spaced el-lipsoids as shown in Figure 1. The coloring illus-trates the shape of the ellipsoids. Red ellipsoidslook like cigars, green ones like pan cakes and blueones like balls. The magenta ones are not classifiedin one of the three groups. At the moment we areworking on a better placement of the ellipsoids thatillustrate the underlying topology of the tensor field.But for both approaches an interactive visualizationtool demands for fast rendering of a large numberof illuminated ellipsoids.

Guthe et al. [4] have proposed a method to splatsimple geometric shapes such as ellipsoids, arrows,etc, that exhibit a rotational symmetry. The prim-itives are pre-rendered from a large number ofview points and during visualization the best pre-

VMV 2003 Munich, Germany, November 19–21, 2003

rendered version is splat on the screen. All theprimitives were pre-lighted and no depth correctioncould be performed.

Another approach that would benefit from ourmethod was described by Bischoff and Kobbelt [1].They decompose 3D-models represented as trian-gle meshes into a set of overlapping ellipsoids, suchthat the surface of the union of ellipsoids re-samplesthe surface of the 3d-model. They rearrange the el-lipsoids into an order suitable for progressive trans-mission. The representation is very robust to theloss of single ellipsoids. For the reconstruct of the3D model they use the marching cubes algorithmproducing again a triangular mesh. With our ellip-soid rendering approach the ellipsoidal representa-tion can directly be rendered efficiently. The com-bination of both approaches has the potential for anew view dependent rendering system.

There are several approaches to render ellipsoids.One can ray-trace ellipsoids, tessellate them into tri-angles and render the triangles with a graphics ac-celerator or one can splat the ellipsoid. The by farfastest approach is splatting with a graphics accel-erator, i.e. one simply renders a triangle or as we doa quadrilateral that covers the primitive (ellipsoid)and one supplies a texture that represents shape andillumination of the primitive. The shape is typi-cally encoded in the α-channel with a 0 for texelsthat do not belong to the shape and a 1 for texelsthat do. The graphics accelerators allow to discardfragments (rastered pixels) in dependence of the αvalue, such that pixels where the fragments α is zeroare not touched at all.

Most of the newer graphics accelerators comewith an extended texturing facility that can be ac-cessed through a vertex and a fragment shader API,such as CG from NVIDIA or the correspondingOpenGL ARB extensions. The vertex shader APIallows to perform view dependent calculations foreach vertex without the need to re-specify the vertexdata again if the view point changes. The output ofthe vertex shader is a collection of scalar and vectorvalued data that is passed to the fragment shader.This data is dealt with in the same way as texturecoordinates, which are perspectively correct inter-polated over the triangles or quadrilaterals that arerasterized. In world coordinates this means that thedata is linearly interpolated over the triangles andquadrilaterals. For each pixel encountered duringrasterization the interpolated data is passed to the

fragment shader that computes the final color anddepth coordinate of the fragment. The latter infor-mation is used for the α- and z-buffer tests and ifthese succeed for the combination of the fragmentcolor with the current pixel color.

In this paper we first elaborate on some basiccharacteristics of ellipsoids in section 2. Then wederive the necessary formula to compute the cor-ner vertices of a quadrilateral splat that containsthe silhouette of an ellipsoid seen from the cur-rent view point in section 3. In section 4 we solvethe ray-ellipsoid intersection problem for a givenray from the view location to a pixel. The resultis used to compute the surface normal and surfacelocation at the pixel, what is necessary for the il-lumination computation and the depth correction.Section 5 solves some problems arising with lessflexible fragment shaders. We close the paper witha comparison of the proposed rendering approachwith other approaches to render symmetric tensors.

The main contribution of this paper is the deriva-tion of a simple solution of the ray-ellipsoid inter-section problem, which can be used incrementallyand implemented in vertex and fragment shaders ofcurrently available graphics accelerators.

2 Background on Ellipsoids

Figure 2: characteristic quantities of an ellipsoid

An ellipsoid in 3D is given by a center locationc, an orthonormal basis v1,v2 and v3 and for eachbasis direction a radius λi as illustrated in Figure 2.We adopt the convention that λ1 ≥ λ2 ≥ λ3, whatcan always be ensured by a permutation of the ba-sis vectors vi. Any ellipsoid can be generated froma sphere by stretching the sphere by the λi in the

666

major directions vi. If we enter the radii in the di-agonal matrix Λ and the vi as columns in the rota-tion matrix O = (v1v2v3), we can define the setof points on an ellipsoid from the points on the unitsphere

E = {q = OΛp + c| ‖p‖ = 1} .

As a rotation of the points p on the unit sphere re-produces the unit sphere, we can as well replaceOΛin the definition of the ellipsoid by the symmetricpositive definite matrix

Tdef= OΛOT .

We can interpret Tv + c as a transformation froma parameter space of points on the unit sphere intoworld space. We denote points p in the parameterspace with a tilde on top and the definition of anellipsoid becomes

E = {p = T p + c| ‖p‖ = 1} , (1)

which is valid for an arbitrary symmetric, positivedefinite matrix T . We can conclude that a non de-generate ellipsoid is defined by a symmetric, strictlypositive definite matrix T and a center location c.

From equation 1 we can directly derive the im-plicite representation of an ellipsoid in world spaceby inverting p = T p + c

1 = ‖p‖2 =∥∥T−1 (p − c)

∥∥2.

The surface normal can be computed from the im-plicite representation via the gradient operator, re-sulting in the not normalized normal vector n inworld coordinates:

n = ∇[∥∥T−1 (p − c)

∥∥2]

= T−2 (p − c) (2)

= T−1p = T−1n. (3)

In equation 3 we transformed p back to parame-ter space, i.e. on the sphere, where the normalizednormal n is the same as the location vector p. Al-though the equations for the normal are defined allover space they only make sense on the surface ofthe ellipsoid. Equation 3 tells us that the normal istransformed from parameter space to world space

with the inverse T−1 = OΛ−1OT =(T−1

)T.

3 Splatting the Silhouette

For the rendering of ellipsoids we assume a pinholecamera as used in most applications with an eyepoint e, a view look at point and a view up direc-tion. In order to be able to splat an ellipsoid witha planar quadrilateral, the silhouette of the ellipsoidis determined. Here only the eye point is of inter-est. The silhouette of the ellipsoid seen from the eyepoint is given by all points on the ellipsoids wherethe surface normal is orthogonal to the vector to theeye point

S ={p ∈ E|nT (p − e) = 0

}.

If we transform the definition of the silhouette intoparameter space by transforming n via equation 3and p and e via 1 we get the silhouette S in param-eter space

S ={p| ‖p‖ = 1 ∧ nT (p − e) = 0

}, (4)

as the T and T−1 cancel each other out. Thus wecan compute the silhouette in parameter space andtransform it back.

0

s

rm

1

e

e

z

x

.

~~

~

~

~

Figure 3: Computation of the silhoutte in parameterspace. The silhoutte is a circle.

Figure 3 shows the 2D version of the silhouettereconstruction problem in parameter space, wherethe ellipsoid is a unit sphere with center in the ori-gin. It is obvious that the silhouette is a circle ona plane orthogonal to e. Let m denote the centerof the circle and r its radius. By applying twicePythagoras one can derive that m is 1/ ‖e‖ awayfrom the center in direction of the eye point, i.e.with e = ‖e‖

m =1

e2e.

666

The radius of the circle can be computed from

1 = r2 +1

e2=⇒ r2 = 1 − 1

e2. (5)

If we build an orthonormal basis x, y, z in parame-ter space with the z-direction in the opposite direc-tion of e as illustrated in Figure 3, the silhouette isthe circle parameterized through φ

S = {m + r (cosφx + sinφy) |φ ∈ [0, 2π]} .To splat the ellipsoids we simply transform the lo-cation m and the vectors x and y back to worldspace

m = T m + c,x = T x,y = T y

and splat a quadrilateral with the four corners

V±±def= m + r (±x ± y) (6)

resulting from choosing the four possible sign com-binations ++, +−, −+ and −−. If we onlywanted to fill the ellipsoid with a uniform color wecould simply texture the quadrilateral with a texturecontaining a filled unit circle. All the computationsnecessary to compute the four corners of a quadri-lateral can be easily performed in the vertex shaderunits. We will come back to that later on when itwill be clear what further parameters are needed bythe fragment shader.

4 Incremental Ray Tracing of Ellip-soids

4.1 Equation of the Intersection

After we have derived the formulas to compute thecorners of a quadrilateral splat that covers the sil-houette completely, we need to compute the inter-section of the ray

p(λ) = e + λv (7)

from the eye location e in the direction v of the cur-rent pixel, where v is the vector from the eye loca-tion to the pixel location as illustrated in parameterspace in Figure 4. This computation has to be donefor each pixel covered by the splat in the fragmentshader. Therefore, we want to derive a formula assimple as possible, which allows to share as manycomputational results as possible between the pixelscovered by one ellipsoid.

a)

0

v s

r

1

b e

x

e

z

x

n

p(λ)~

~~~

~

~

~

~

b)

v =

x

y

b

e =

(00−e

)

⇒ eT v = −eb

Figure 4: Illustration of the intersection computa-tion between the ray from e in direction of v withthe spherical ellipsoid in parameter space. The in-tersection point p(λ) is equal to the normalized nor-mal n in parameter space.

The intersection of the ray and the ellipsoid hasto be between the plane of the silhouette and theeye location. From this follows that λ has to be inthe interval [0, 1]. The transformation to parame-ter space is an affine transformation and preservesstraight lines. Thus, we can as well compute λ inparameter space from the much simpler ray-sphereintersection

1 = ‖p(λ)‖2 . (8)

Figure 4 illustrates the ray intersection problemwith all the necessary lengths. On the right the co-ordinates of the vectors e and v are shown. We seethat the scalar product eT v is given by −eb. Forthe silhouette length s we get from Pythagoras andwith the equation for the radius r

s2 = e2 − 1 = e2r2 = e2(s2 − b2

). (9)

And by solving for eb and applying s2 = e2 − 1once more, we get eT v = s2. Plugging this into 8yields

0 = s2 − 2s2λ+ v2λ2.

Next we plug in the coordinates of v and devide theequation by s = e2r2:

0 = 1 − 2λ+b2 + x2 + y2

e2r2λ2.

The ray only has an intersection with the sphere ifthe 2D vector formed by the x- and y-coordinatesof v has a length smaller than r. By dividing thecoordinates through r, the in this way normalized

666

x- and y-coordinates need to be inside the easier tohandle unit circle. We therefore define

qdef=

1

r

(xy

)q2 =

1

r

(x2 + y2

)∈ [0, 1].

With the easily derivable equality b2 = e2r4 theequation for λ simplifies to

0 = 1 − 2λ+

(r2 +

q2

e2

)λ2

(5)= 1 − 2λ+

(1 − 1 − q2

e2

)λ2. (10)

We finally define the reciprocal u of e, which hasto be between zero and one for any view locationoutside of the ellipsoid, and the two quantities αand β as follows

udef=

1

e∈ [0, 1]

αdef=

√1 − q2

βdef=

1

eα = uα.

Comparing the definition of β with equation 10gives

0 = 1 − 2λ+ (1 − β2)λ2,

which has only one solution, which is smaller thanone

λ =1

1 ∓ β

λ≤1=

1

1 + β=

1

1 + u√

1 − q2. (11)

4.2 Incremental Implementation

This surprisingly simple formula enables a very fastincremental computation of the ray-ellipsoid inter-section, which is perfectly suited for the implemen-tation in a fragment shader. For this we define cor-responding to the four corners (6) of the splat twolinearly interpolated vertex attributes A[0] and A[1]and one constant C[0]

A[0]±±def= V±± − e,

A[1]±±def= (±1,±1, u)T ,

C[0]def= e,

which encapsulate v, q together with the per ellip-soid constant term u and e. For each pixel we firstcompute q2 and check if it is ≤ 1. If not, the frag-ment is discarded. Otherwise λ is computed and viaequation 7, v = A[0] and e = C[0] the intersectionin world space.

4.3 Per Fragment Lighting

For the lighting computations we need the surfacenormal of the ellipsoid in world coordinates. Asthe surface normal in parameter space is identicalto the ray-sphere intersection p, the normal can becomputed via equation 3 to

n(λ) = T−1p(λ) = T−1 (e + λv) .

If we express the constitutes of p in the correspond-ing quantities in world coordinates we get

n(λ) = T−2 (e − c + λv) .

For the computation of the not normalized surfacenormal we introduce one per ellipsoid constant ver-tex attribute A[2] and one linearly interpolated at-tribute A[3]

A[2]def= T−2 (c − e) ,

A[3]±±def= T−2A[0]±±,

which allow to compute the surface normal incre-mentally.

4.4 Per Fragment Depth Correction

As the x- and y-coordinates in screen space areknown from the rasterization process, only the z-coordinate needs to be corrected. Let M be thetransformation matrix from world space to screenspace, i.e. perspective transformation and modelview transformation and Mi its i-th row. For thedepth correction we have to interpolate the screenspace z- and w-coordinate, compute z and w forthe intersection and finally divide the resulting z bythe resulting w. For this we compute the two 2Dvertex attributes in the vertex shader

A[4]±±def=

(MT

2 A[0]±±,MT3 A[0]±±

)T,

A[5]±±def=

(MT

2 A[1]±±,MT3 A[1]±±

)T.

Figure 5 illustrates the depth correction at theexample of two overlapping ellipsoids. In b) wezoomed onto the intersection curve which is nicelysampled on screen resolution independent of theviewing distance.

666

a)

b)

Figure 5: Illustration of depth correction for over-lapping ellipsoids.

5 Low Precision Implementation

One can even implement the shading of splatted el-lipsoids without depth correction on graphics accel-erators that do not support square root operationsbut at least one dependent texture lookup in thefragment shader, as for example the Radeon 9000.On these graphics accelerators one can implementthe computation of λ by a 3D texture lookup. Forthis one simply scales the coordinates of q to therange [0, 1] and uses u as the third texture coordi-nate.

Two problems arise with this approach. Firstly,does the computation of n consume the only avail-able dependent texture lookup, which would be nec-essary to allow for Phong shading. And secondly isλ for most distances e very close to 1 all over thesplat, what leads to severe numerical problems.

The first problem can be solved by implement-ing Phong shading with a 2D texture lookup forthe diffuse component and a 3D texture lookup forthe specular component. The arguments to the dif-fuse texture map are s = nT n and t = nT l,where l is the direction vector to a directional lightsource. The map simply implements the functiont/√s clamped to [0, 1]. Similarly, does the specu-

lar map take the three coordinates s = nT n · hT h,t = nT h and u = shininess/128, where h is theinterpolated half-vector. The specular map imple-ments the function (s/

√t)128u.

The second problem can be solved by transform-ing λ into the parameter µ that varies for any dis-tance e between zero and one. For this we examine

the range of λ, which only depends on q2. Substi-tuting zero and one for q2 results in the range

λ ∈[

e

e+ 1, 1].

Thus we define µ as

µdef= (e+ 1)λ− e =

1 − α

1 + uα∈ [0, 1],

where the second equation can be derived with sim-ple algebra. The ray-ellipsoid intersection com-putes to

p = e + λv = e +e

e+ 1v + µ

1

e+ 1v.

In a similar way we change the vertex attributesA[2] and A[3] necessary for the illumination cal-culations. One final problem arises as the inversetransformation T−1 can scale the world space sur-face normal to a length exceeding one, which leadsto a problem for the texture lookup in the dif-fuse and specular maps. As the normal in param-eter space is normalized, the transformation back toworld coordinates can scale it no more than

∥∥T−1∥∥,

which is equal to the largest eigenvalue of T−1 orthe reciprocal of the smallest eigenvalue of T . Theworld space normal can therefore be kept shorter orequal to length one, if one divides the modified def-initions of A[2] and A[3] by

∥∥T−1∥∥.

6 Results

We implemented vertex and fragment shaders withthe OpenGL ARB vertex and fragment program ex-tensions and the approach of section 5 with the ATIfragment shader extension. Our API consists of fivefunctions:

1. enableEllipsoidShader(constViewDescr& vd). . . sets up the vertex andfragment programs and creates textures in thefirst call and binds textures and programs insuccessive calls and sets per frame constants.

2. renderEllipsoid(const Pnt&center, const SymMat& T, constSymMat& I). . . renders an ellipsoid at thegiven center location, which is given by asymmetric matrix. Also the inverse of thesymmetric matrix has to be provided.

666

a) b) c)

Figure 6: Visual comparision of three different rendering approaches for symmetric positive tensors: ellip-soids, icosahedra and boxes; all Phong shaded.

3. setMaterial(float ambient,float diffuse, float specular,float shininess). . . sets the differentmaterial coefficients used for Phong shading.

4. setSpecularColor(float r,float g, float b). . . sets the specularcolor.

5. disableEllipsoidShader(). . . turnsoff the vertex and fragment programs.

The color of the rendered ellipsoids can be specifiedvia glColor commands.

Figure 1 shows our test scene of a symmetric ten-sor field visualized on a grid of 403 = 64000 ellip-soids. In Figure 6 we compare a coarser samplingof the test scene tensor field for three different ren-dering approaches, that are typically used for the vi-sualization of tensor fields. In a) the presented ap-proach of ellipsoid visualization is used, in b) tes-sellated icosahedra are used and in c) rectangularboxes are shown. We optimized the rendering oficosahedra and boxes by stripification.

To analyze and compare the performance of ourapproach we rendered the test scene with differ-ent sized ellipsoids. We computed the number ofellipsoids rendered per second and the number ofrastered fragments per second. In the diagrams ofFigure 7 we plotted the number of rastered frag-ments per second over the number of ellipsoidsper second, i.e. the fill rate over the setup speed.We compared the different rendering approaches fortwo graphics accelerators: a GeForceFX 5800 anda Radion 9000. For both cards the box renderingis always faster than the icosahedron rendering andthe ellipsoid rendering is fast than the icosahedronrendering for small ellipsoids, i.e. a low fill rate,and slower for larger fill rates. In the case of the

GeForce card, the ellipsoid rendering is even fasterthan box rendering in case of low fill rates. We canconclude that the setup for ellipsoid splatting is veryfast but the splatting is fill rate limited already forsmall splat sizes (about 102 splats). But splattingis never more than twice slower as icosahedron ren-dering and achieves a much higher image quality.An appropriate tessellation of the sphere consumessurely more than twice as many triangles than anicosahedron and will in most cases be slower thanour ellipsoid splatting approach.

In future work we want to apply our ellipsoidrendering strategy to the ellipsoidal representationproposed by Bischoff and Kobbelt [1]. We want toinvestigate how to render ellipsoid decompositionsof 3D models view dependently. Furthermore, wewant to investigate different texturing approachesfor the ellipsoids such that also the anti-symmetricpart of a tensor field can be visualized by for exam-ple a spiral pattern or a bump map.

References

[1] S. Bischoff and L. Kobbelt. Ellipsoid de-composition of 3d-models. In Proceedings of3DPVT Conference, pages 480–488, 2002.

[2] T. Delmarcelle and L. Hesselink. Visualiza-tion of second order tensor fields and matrixdata. In Proceedings of IEEE VisualizationConference 1992, pages 316–323, 1992.

[3] P. Elinas and W. Sturzlinger. Real-time ren-dering of 3d clouds. Journal of GraphicsTools, 5(4):33–45, 2000.

[4] S. Guthe, S. Gumhold, and W. Straßer. Inter-active visualization of volumetric vector fields

666

a)

GeForceFX 5800

0,00E+00

3,00E+07

6,00E+07

9,00E+07

1,20E+08

1,50E+08

0 50000 100000 150000 200000 250000 300000

boxes icos ellipsoids b)

Radeon 9000

0,00E+00

2,00E+07

4,00E+07

6,00E+07

8,00E+07

0 50000 100000 150000 200000 250000 300000 350000 400000 450000

boxes icos ellipsoids

Figure 7: For two different graphics accelerators the fill rate in fragments per second plotted over the setupup speed in rendered tensors per second.

using texture based particles. In Proceedingsof WSCG Conference 2002, 2002.

[5] G. Kindlmann, D. Weinstein, and D. Hart.Strategies for direct volume rendering of dif-fusion tensor fields. IEEE Transactionson Visualization and Computer Graphics,6(2):124–138, 2000.

[6] G. L. Kindlmann and D. M. Weinstein. Hue-balls and lit-tensors for direct volume render-ing of diffusion tensor fields. In Proceedingsof IEEE Visualization Conference 1999, pages183–189, 1999.

[7] D. H. Laidlaw, E. T. Ahrens, D. Kremers, M. J.Avalos, R. E. Jacobs, and Carol Readhead. Vi-sualizing diffusion tensor images of the mousespinal cord. In Proceedings of IEEE Visualiza-tion Conference 1998, pages 127–134, 1998.

[8] D. Laur and P. Hanrahan. Hierarchical splat-ting: A progressive refinement algorithm forvolume rendering. In Proceedings of ACMSIGGRAPH Conference 1991, pages 285–288, 1991.

[9] X. Mao. Splatting of non rectilinear volumesthrough stochastic resampling. IEEE Transac-tions on Visualization and Computer Graph-ics, 2(2):156–170, 1996.

[10] K. Mueller, T. Moeller, and R. Crawfis. Splat-ting without the blur. In Proceedings of IEEEVisualization Conference 1999, pages 363–370, 1999.

[11] H.-P. Pfister, M. Zwicker, J. van Baar, andM. Gross. Surfels: Surface elements as ren-dering primitives. In Proceedings of ACMSIGGRAPH Conference 2000, pages 335–342, 2000.

[12] L. Ren, H. Pfister, and M. Zwicker. Objectspace ewa surface splatting: A hardware ac-celerated approach to high quality point ren-dering. In Proceedings of Eurographics Con-ference 2002., 2002.

[13] J. E. Swan, K. Mueller, T. Moeller, N. Shareef,R. Crawfis, and R. Yagel. An anti-aliasingtechnique for splatting. In Proceedings ofIEEE Visualization Conference 1997, pages197–204, 1997.

[14] J. J. van Wijk. Spot noise: Texture syn-thesis for data visualization. In Proceedingsof ACM SIGGRAPH Conference 1991, pages309–318, 1991.

[15] L. Westover. Footprint evaluation for vol-ume rendering. In Proceedings of ACM SIG-GRAPH Conference 1990, pages 367–376,1990.

[16] S. Zhang, C. Demiralp, D.F.Keefe, M. J.da Silva, D. H. Laidlaw, B. D. Greenberg,P.J. Basser, and E.A. Chiocca andC. PierpaoliT.S. Deisboeck. An immersive virtual envi-ronment for dt-mri volume visualization ap-plications: A case study. In Proceedings ofIEEE Visualization Conference 2001, 2001.

[17] M. Zwicker, H. Pfister, J. VanBaar, andM. Gross. Ewa volume splatting. In Proceed-ings of IEEE Visualization Conference 2001.,2001.

[18] M. Zwicker, H.-P. Pfister, J. van Baar, andM. Gross. Surface splatting. In Proceedingsof ACM SIGGRAPH Conference 2001, pages371–378, 2001.

666