Veronika Solteszova

Interactive GPU-based Segmentation Of Large

Volume Data With Level Sets

Veronika Solteszova (presenting Oliver Klar’s master thesis)

VRVis Research Center

www.vrvis.at

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Overview

• Introduction• Level set method

– Math and theoretical background– GPU Data structure for large volumes– Solution on GPU

• Rendering and user interaction• Results and future work

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Introduction

• Level set main idea– Deformable implicit surface– Evolve simple surface into

target object shape

• Short demo

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Introduction

• Problem statement & motivation– Segmentation of large volumetric data is

expensive– GPU allows direct rendering– Overcome GPU memory restrictions

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Overview

• Introduction• Level set Method

– Math and theoretical background– GPU Data structure for large volumes– Solution on GPU

• Rendering and user interaction• Results and future work

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

LS Method: Math• A level set definition

– Curve represented as a set of roots of an implicit function

– Contour lines = level sets

– C = 0.2 ↔ 0.2 level set

– Contours, iso-surfaces also level sets

Φ > 0.2

Φ = 0.2

Φ > 0.2Φ < 0.2

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

LS Method: Math (cont.)

• In this approach– Φ: f(x, y, z, t) = 0.5– Embedding 2D contour (surface) into 3D

space, 4th dim = time– Discretization: values of Φ in a 3D texture

• Evolution– 3D texture changes → level set moves

• Advantages of such curve representation– No problems with topology changes– Gradient & curvature computation easy– Evolution described by partial differential equation

(PDE)

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

• Convection partial differential equation (PDE), solved for each point of the grid (discrete volume data) – .

– ∆t → time step,– F → gradient magnitude– F → speed function, determines deformation

LS Method (cont.)

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

LS Method (cont.)• Speed function F:

• Density (D) and mean curvature (κ) influence– Only curvature (κ)

driven– Only density factor

(D) driven

α

t

0.0 0.3 1.0

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

LS Method (cont.)

• Density factor:

• T: target density• I : volume density

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Overview

• Introduction• Level set Method

– Math and theoretical background– GPU Data structure for large volumes– Solution on GPU

• Rendering and user interaction• Results and future work

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

GPU Data Structure

• Level set calculation only where necessary• Avoid GPU memory restrictions• Volume subdivision into bricks

– Neighbors in cache ≠ Neighbors in virtual volume– Bricks packed into cache texture

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Cache Layout (cont.)

• Volume subdivision into bricks (34x34x34)– Brick 32x32x32 + 1vox border (filtering)

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Overview

• Introduction• Level set Method

– Math and theoretical background– GPU Data structure for large volumes– Solution on GPU

• Rendering and user interaction• Results and future work

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

LS Solver

• 1. Setup– Signed distance field– Values clamped to:

• 0.0 (exterior)• 1.0 (interior)

– Narrow band within [0.0, 1.0]

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Narrow band width

Narrow Band Culling• Interior: Min = Max = 1.0• Exterior: Min = Max = 0.0

0.5 LS

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

LS Solver (cont.)

• 2. Calculation on HW– PDE solved in fragment programs

• Discretization (with time step ∆t)

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

LS Solver (cont.)

• 3. Sparse volume update– GPU – CPU memory request

• For each brick: min/max determination• GPU Neighbor activation & encode

– CPU – GPU memory allocation• Decode & CPU neighbor activation

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Neighbor Activation

• The level set front may leak out• Activate the bricks touched by the front

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Neighbor Activation

• For each brick– 6 faces ↔ 6 bits state (active/inactive)– State decoded on CPU– Activation on full 26-neighborhood

• Flagged face → activate brick• 2 adjacent bricks → edge brick• 3 adjacent bricks → corner brick

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Overview

• Introduction• Level set Method

– Math and theoretical background– Data structure– Solution on GPU

• Rendering and user interaction• Results and future work

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Rendering

DVR• Rendering 2 different volumes simultaneously

• Combination of DVR and ISO rendering

ISO DVR

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Overview

• Introduction• Level set Method

– Math and theoretical background– Data structure– Solution on GPU

• Rendering and user interaction• Results and future work

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Results’ Gallery

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

• Results of presented level set solver:

Performance

[ms] hydrogen hand brain1 brain2

volume size 64x64x64 244x124x257 192x256x222 512x512x154

cache size 128x128x64 512x256x128 256x256x256 512x512x256

sphere setup 5.95 50.24 61.37 215.18

level-set comp. 13.27 156.17 111.42 403.80

gpu min-max 0.99 1.98 1.56 3.03

gpu neighbors 4.58 17.64 13.66 35.07

cache update 7.52 54.25 59.86 201.08

overall 29.27 234.48 191.14 647.80

Tests performed for 1 iteration on GeForce 8800 GTX

Veronika Solteszova

Demo

• Short demo: brain segmentation process

Interactive GPU-based Segmentation with Level Sets

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Future work

• Other speed functions• Further optimization of LS computation:– On neighbor look-ups– Determining the active region in the cache cross

section

Veronika Solteszovawww.vrvis.at

Agfa, AKH, Kplus

Acknowledgements: Oliver Klar, Markus Hadwiger, Katja Bühler

Thank You for Your attention!

Veronika Solteszova

/*todo*/ performance

• How many iterations for brain2• How much time does it take• Why is seg. with LS better than

tresholding? (proves, illustrations)• Why is the cache by hydrogen 2x bigger

as a hydrogen data set?• Segmentation mask export possible with

this framework??

Interactive GPU-based Segmentation with Level Sets 29

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets

Brick-to-tile Mapping• Address translation requires mapping

– Case 1: a tile has no mapping– Case 2: a tile maps to a brick– Case 3: a brick has no mapping

Brick cache

Brick

Tile

Case 1

Case 2

Case 3 Virtual volume

Veronika SolteszovaInteractive GPU-based Segmentation with Level Sets 31/26

Rendering

• Hit point refinement