Real-Time Volume Graphics

Klaus EngelSiemens Corporate ResearchPrinceton

Aaron E. LefohnCenter for Image Processing and Integrated ComputingUniversity of California, Davis

Markus HadwigerVR VIS Research CenterVienna, Austria

Christof Rezk SalamaComputer Graphics andMultimedia GroupUniversity of Siegen, Germany

Joe M. KnissScientific Computing and Imaging Insitute,University of Utah

Daniel WeiskopfVisualization and Interactive Systems Group,University of Stuttgart, Germany

Real-Time Volume Graphics

Klaus EngelSiemens Corporate ResearchPrinceton

Aaron E. LefohnCenter for Image Processing and Integrated ComputingUniversity of California, Davis

Markus HadwigerVR VIS Research CenterVienna, Austria

Christof Rezk SalamaComputer Graphics andMultimedia GroupUniversity of Siegen, Germany

Joe M. KnissScientific Computing and Imaging Insitute,University of Utah

Daniel WeiskopfVisualization and Interactive Systems Group,University of Stuttgart, Germany

Real Time Volume Graphics

Pre-Integration and High-Quality Filtering

REAL-TIME VOLUME GRAPHICSKlaus Engel, Siemens Corporate Research, Princeton, USA

This is what we want …


But often this is what we get …


Volume Rendering Artifacts

Volume Rendering PipelineWhere are errors introduced ?

Low sampling

rate

Linear

filtering

High

frequencies

8 bit

blending

Integration

Quantized

Gradients

Framebuffer

Sampling

Filtering Classification Shading


Sampling Artifacts


Reason:Low sampling rate

Solutions:Increase sampling rate to Nyquist frequency => at least 2 samples per voxel

Adaptive SamplingHigher sampling rate where data contains high frequencies

Sampling Artifacts128 Samples284 Samples


Sampling Artifacts

Remove woodgrain artifact by stochastic jittering of ray-start position

Look-up noise texture to offset ray-position in view direction


Filtering Artifacts

643 volume, object-aligned slices

bi-linear filter bi-cubic filter


Filtering ArtifactsReason:

Internal filtering precision dependent on input texture formatLinear filters do not approximate the ideal reconstruction filter (sinc-Filter) very well

Solutions:Use internal format with higher precisionImplement a HQ filter in a fragment program


Filtering Artifacts / HQ-Filters

Discretized version of convolution integral

g(x) = f[x] * h(x)f[x]

discrete signalh(x)

filterconvolution

Filter width: 2m



Procedural evaluation of kernel and convolutionTexture-based kernel and convolution

FetchInputSamples();ComputeWeights_X();for (i=0; i<3; i++) Convolution_X();ComputeWeights_Y();Convolution_Y();

see for example NVIDIA’sBicubicTexMagnification demo

pre-sampled B-spline kernelfrom [Hadwiger et al. 2001]



Procedural evaluation of kernel and convolutionTexture-based kernel and convolution

FetchInputSamples();ComputeWeights_X();for (i=0; i<3; i++) Convolution_X();ComputeWeights_Y();Convolution_Y();

t0

t1

t2

t3

R G B A

pre-sampled B-spline kernelfrom [Hadwiger et al. 2001]

see for example NVIDIA’sBicubicTexMagnification demo



Transform original kernel into look-up texture

Replicate kernel tile over output grid

kernel tile



Easy when kernel separable

Use same 1D kernel tile for all axesSample two (or three) timesand multiply weights

Separable bi-cubic andtri-cubic filters need onlyone 1D RGBA kernel tile



Distribution instead of gatheringWorks for all filter shapes and sizesWorks for separable and non-separable kernelsMulti-pass evaluation of filter convolution sum



Tri-cubic gradients and second derivatives for isosurfaces possible in real-time!


Filtering Artifacts / HQ-Filterslinear cubic


Classification ArtifactsPre-Classification

Post-Classification

Pre-Integrated-Classification

Filtering Classification

FilteringClassification

Filtering Integral LookupPre-Integration

…

…

…

…

……


Classification Artifacts

High frequencies in the transfer function T increase required sampling rate

x

s(x)

s

T(s)

x

T(s(x))


Classification Artifacts

multiple peaks

50 x more slices

s

T(s)


Reason:Classification can add high frequencies to rendered data

Solution:Pre-Integrated Classification=> split numerical integration into

one pre-integration for the TFone integration for the scalar field

slab-by-slab rendering

Classification Artifacts / Pre-integration


image-order

object-order

slice-by-slice slab-by-slab

sample-by-sample raySegment-by-raySegment



Screen

Slab

Eyesf

sb



Pre-Integration of allpossible combinations

save valuesIn table

sf

sb

sbsf

Pre-processing



project back slice

sf sb

front slice

back slice

Rendering



// define inputs from applicationstruct appin{ float4 Position : POSITION; float4 Normal : NORMAL; float4 TCoords0 : TEXCOORD0;};// define outputs from vertex shaderstruct vertout{ float4 HPosition : POSITION; float4 TCoords0 : TEXCOORD0; float4 TCoords1 : TEXCOORD1;};vertout main(appin IN, uniform float4x4 ModelViewProj,// model-view-projection matrix uniform float4x4 ModelViewI, // inverse of the modelview matrix uniform float4x4 TexMatrix, //Texture matrix for texture 0 uniform float SamplingDistance

)

Cg Vertex Program



{vertout OUT;

OUT.TCoords0 = mul(TexMatrix, IN.TCoords0);

// transform view pos vec and view dir to obj spacefloat4 vPosition = mul(ModelViewI, float4(0,0,0,1));float4 vDir = normalize(mul(ModelViewI, float4(0.f,0.f,-1.f,1.f)));

//compute position of virtual back vertexfloat4 eyeToVert = normalize(IN.Position - vPosition);float4 backVert = {1,1,1,1};backVert = IN.Position - eyeToVert * (SamplingDistance / dot(vDir,eyeToVert));

//compute texture coordinates for virtual back vertexOUT.TCoords1 = mul(TexMatrix, backVert); // transform vertex position into homogenous clip-spaceOUT.HPosition = mul(ModelViewProj, IN.Position);return OUT;

}

Cg Vertex Program



sf sb

slice

utilize sf and sb as texture coordinates forthe dependent texture

sf

sb

fetch sf and sb duringrasterization

Rendering



sf sb

slice

fetch pre-integratedray-segment

sf

sb

texture polygon withpre-integrated value

Rendering


struct v2f_simple { float4 Hposition : POSITION; float3 TexCoord0 : TEXCOORD0; float3 TexCoord1 : TEXCOORD1;float4 Color0 : COLOR0;};

float4 main(v2f_simple IN, uniform sampler3D Volume, uniform sampler2D TransferFunction, uniform sampler2D PreIntegrationTable) : COLOR{ float4 lookup; //sample front scalar lookup.x = tex3D(Volume, IN.TexCoord0.xyz).x; //sample back scalar lookup.y = tex3D(Volume, IN.TexCoord1.xyz).x;

//lookup and return pre-integrated value return tex2D(PreIntegrationTable, lookup.yx);}

Cg Fragment Program



Pre-integration table to large12bit x 12bit x RGBA8 = 4096x4096x32bit = 64MB

Pre-integration table to slow to computeeven with integral functions

Solution: use Integral functionsStore integral functions in 1D floating point texturetwo 1D table lookups



Almost no performance penalty in sw raycasting

Cache last scalar value

Currently slower on graphics hardwareTemporary fragment registers are zeroedTwo lookups into the volume per fragment

=> multiple integration steps @ once (raycasting)

Pre-integrated volume rendering still not “correct”

Assumes linear progression of scalar value along ray=> Gaussian Transfer Functions (Kniss Vis’03)



128 slicespre-

classification

128 slicespre-integrated

284 slicespost-

classification

128 slicespost-

classification



single step



multiple peaks



many peaks



acoustic volumeinside car cabine



homogeneousregion

inhomogeneousregion



Shading Artifacts


Shading ArtifactsReasons:

Gradients are pre-computed and quantizedInterpolation of normals causes unnormalized normals

Solutions:Store gradients in high-precision texture,re-normalize in fragment program too much memory

Compute gradients on-the-fly high-quality gradientmany texture fetches

Sobel gradient


Shading Artifacts

Add offset to texture coordinate of fragment// samples for forward differences

half3 normal;half3 sample1;sample1.x = (half)tex3D(Volume, IN.TexCoord0+offset.xwww).x;sample1.y = (half)tex3D(Volume, IN.TexCoord0+offset.wyww).x;sample1.z = (half)tex3D(Volume, IN.TexCoord0+offset.wwzw).x;

// additional samples for central differenceshalf3 sample2;sample2.x = (half)tex3D(Volume, IN.TexCoord0-offset.xwww).x;sample2.y = (half)tex3D(Volume, IN.TexCoord0-offset.wyww).x;sample2.z = (half)tex3D(Volume, IN.TexCoord0-offset.wwzw).x;

// compute central differences gradientnormal = normalize(sample2.xyz – sample1.xyz);

Cg Fragment Program


Shading Artifacts

Pass in shifted texture coordinates// samples for forward differenceshalf3 normal;half3 sample1;sample1.x = (half)tex3D(Volume, IN.TexCoord2).x;sample1.y = (half)tex3D(Volume, IN.TexCoord4).x;sample1.z = (half)tex3D(Volume, IN.TexCoord6).x;

// additional samples for central differenceshalf3 sample2;sample2.x = (half)tex3D(Volume, IN.TexCoord3).x;sample2.y = (half)tex3D(Volume, IN.TexCoord5).x;sample2.z = (half)tex3D(Volume, IN.TexCoord7).x;

// compute central differences gradientnormal = normalize(sample2.xyz – sample1.xyz);

Cg Fragment Program


Blending Artifacts

8 bit

fixed point

blending

16 bit

floating point

blending

32 bit

floating point

blending


Blending ArtifactsReason:

8 bit blending accumulates error in the framebuffer

Solutions:Blending into a floating point pbufferNv4x support 16 bit floating point blendingNv3x, R3xx and R4xx do not support floating point blending=> implement blending in a fragment programPing-pong blending to prevent read/write race conditions


Blending Artifacts

The graphics hardware does not support floating point blending (NV3x, R3xx, R4xx)=> implement blending yourselfClean: ping-pong

Blend

A B


Blending Artifacts

The graphics hardware does not support floating point blending (NV3x, R3xx, R4xx)=> implement blending yourselfDirty: ping without the pong

Blend

A B


Blending Artifacts

The graphics hardware does not support floating point blending (NV3x, R3xx, R4xx)=> implement blending yourselfDirty: ping without the pong

Blend

A


Conclusions

Artifacts are introduced in various stages of the volume rendering process

Current graphics hardware is flexible and precise enough to remove or suppress artifacts

Real-Time performance can still be achieved in most cases


Questions?

Thank you!

Acknowledgements:NVIDIA and ATI for the demo hardware

For some of the images and figures:Joe Michael KnissMarkus Hadwiger

Real-Time Volume Graphics

Documents

Transcript of Real-Time Volume Graphics