The Visible Human Project Today

Director, Michigan Center for Biological Information (MCBI)

UM Office Vice President for Research (OVPR)

Assistant Professor

Director, University of Michigan Visible Human Project

Department of Cell and Developmental Biology

University of Michigan Medical School

Ann Arbor, Michigan

http://vhp.med.umich.edu http://CTAalliance.org

September 26, 2002

Presented by

Brian D. Athey, Ph.D.

2002: NLM/NIH NGI Visible Human CollaboratoryFor Health Sciences Education and Training

StanfordUniversity

Universityof Colorado

Health Sciences

Universityof

Michigan

Pittsburgh SupercomputingCenter (PSC)

National Libraryof Medicine (NLM)

Uniformed ServiceUniversity of the Health Sciences

(USUHS)

Abilene Network

Virtual Environment – Combat Battlefield

Integrated Core – Manikin with Synchronized Anatomy from Visible

Human Project

An Early Test-bed Application: ARPA 1992-1995To Extend the DoD Medical Training Platform from WRAMC to Battlefield

Internet2 Backbone

Virtual Workbench

Dexterous Skills Training

Middleware Needed

Future:Middleware and Applications

Past: Enabling Infrastructure

Old-time (~1995-1997) VH Client-Server Application Concept

Servers

Visible Human

Applications/Tools

Development Focus: Visible Human

Project (courtesy Don Jenkins, NLM)

VISIBLE HUMAN DATASETSVISIBLE HUMAN DATASETS

Natural Biologic Natural Biologic Variation Variation

Disease and PathologyDisease and PathologyNormal VariationNormal Variation

TeachingTeaching ApplicationsApplications

Public Domain Web Public Domain Web SitesSites

National Anatomy ArchiveNational Anatomy Archive

DIGITAL HUMAN SIMULATIONDIGITAL HUMAN SIMULATION

ANCILLARY DATA SETSANCILLARY DATA SETS

MRI, CT, PETMRI, CT, PETRadiologyRadiologyAnatomicAnatomic

Structural ChangeStructural ChangeScale & ResolutionScale & ResolutionBiological Biological

Structure Structure ContinuumContinuum

EmbryologyEmbryologyGross AnatomyGross AnatomyMicro AnatomyMicro AnatomySub MicroscopicSub MicroscopicPhysiologyPhysiologyBiochemistryBiochemistry

DevelopmentDevelopmentAgingAging

Clinical PatternsClinical PatternsSigns & SymptomsSigns & SymptomsPathophysiologyPathophysiology

The Evolving Amphitheater of Human Anatomy

Vision: Medical Student User Product

Interface

Visible Human Database

Animations

Visuals

Text

Learning Materials

Labels

Simulations

Quizzes

Case Inquiry

ConceptU of M Center for Parallel Computing

U of Kiev

Local Server

U of MichiganU of Kiev

Local Server

U of Michigan

iBook

Bejing Med U

Visible Human Infrastructure Development Embraces “Cyberinfrastructure” Vision

Vision of Integrated IT and Computing ArchitectureFor Life and Health Sciences Informatics

Ontology Integration and Biological Context

NLM UMLS Ontology (Linked to 50 Others)

Digital Anatomist

Cell Cell

Proteome

Bio/SPICE (DARPA) and Others

Gene

NLM UMLS Semantic Network + the “Semantic Web”

Bioinformatics Databases and Data Standards e.g. Celera, GO

nM

M

Physiology109

Gene

Organs&

Tissues

High Throughput “Structural Biology”

Geneotype

Phenotype

Behavior(organs,

organisms)SystemsPhysiology(process

engineering)Cellular

Processes and Interactions

Years

Minutes

Seconds

Milliseconds

Microseconds

Nanoseconds

Picoseconds

Molecular Dynamics

Quantum Mechanics

0.1 nm 10nm 100nm 1um 1mm 1cm 1mAdapted from the work of Bill Goddard, Caltech

Need for Multiple Need for Multiple ScalesScales

Visible Human “Data-locator”

Navigable Volume Data

UMich Visible Human Knowledge Base Schematic

Anatomical layer

Semantic layer

Annotation layer• Hyperlinked texts• Movies• Sounds, etc

• Medical lexicons• Semantic relations• Links to regions, volumes, and

annotations

• Labels of structures• Voxel data and segmentation• Volume surfaces• Spatial relationships

System: Skeletal MuscleRegion: PelvicStructure: Sartorius

BoundariesContainsSurface AnatomyCommon AnomaliesIllustrations

ActionOriginInsertionInnervationArterialVenousBounded by

Femoral Nerve SourceBranchesMotorSensory

lumbar plexus (ventral primary rami of spinal nerves L2-L4)

Original Labeling Scenario

Database Development

Over 7000 Terminologia Anatomica(TA) terms are now available via the UMVH database.

Structural relationships (i.e. heirarchy) imported from UMLS/TA and linked to VH content.

Non-English Languages (e.g Chinese). Web-based querying and editing.

Navigable Volume Data

Volume/Structure/Instructional Space

Structure

UMLS MESH TA

Area VolumeUser

Request

Rotation-TranslationParameters

Arbitrary Section

Integration of VH, and Volume/Structure/Instructional Databases

User Skill Instruction Area

Dental Surgery Nursing AnatomyAlliedHealth

Video

Audio

Quizzes

Tests

XML Tagged

XML Tagged

Walter Meixner

Labels

Overview of simulation

3D rendering computation

tissue biomechanics computation

Tool tip position

deform

ation

Force sensed at tool tip

Tissue deformation observed

Force computed 1000 - 10,000 times per sec

Image computed 10 - 30 times per sec

Remote client

Server

Haptics process Display process

Internet

3D model

Parvati DevStanford University

MediaserverAccess to resources

Library of images and models – A mediaserver

Retrieval interface (general purpose)

Interface to rotate and peel through dissection images (specialized)

Interface to control a simulation (specialized)

Search and browse interface

Client Client Client Client

Parvati DevStanford University

National Test-bed Demonstration at NREN (June 2001): Four Parallel VH Streams—

Memory-to-Memory Transfer (UMich to Ames)

University of Michigan Internet Connectivity

vBNS

UM BINGigE

ITCS CAEN

DOE

NASA

DREN

Abilene

MERIT

MCIT

MichGiga PoP

Gig

EO

C-3

CAC MGrid Test sitesATLAS Physics

Michnet

Internet Qwest Cleveland POP

Research Networks Commercial Networks

FW

StarLightChicago

Gig E Gig E

MGrid Test sites

4x 100mbit/sec

UMArbor Lakes

#1 #3

Medical SchoolUM Hospital + MCAREBioinformaticsProteomics

MCBI

FW

MCIT 4x100mbit/sec

UCAIDInstitutions

Europe

Stereo Volume

Java Browser - Model View

UMich Visible Human Project NGI/I2 Network-Enabled Software Tools and Content

Pittsburgh Supercomputing Center (PSC) Segmentation Tool: To facilitate the creation of surface models, our collaboration partner the PSC has been commissioned with the development of a segmentation tool for the production of contours. This tool has been ported to Linux, Mac and Windows platforms. Data Slice - Embedded Model

View

Edgewarp: Created by Dr. F. Bookstein and Dr. W. Green as a tool for analyzing shape variation by extracting invariant geometric operations from volumes and surfaces, this software program has been extensively modified for the UM VH Project. The addition offeatures such as labeling, model display and filmstrip creation has expanded its usefulness to medical education.

Java Browser:Alexander Ade has created a set of Java-based Browsers that allows web-based volume visualization as either slices or volume-rendered scenes. The applications show anatomical labels queried from an database and displays 3-D models in either mono or stereo views.

Java Browser - stereo model view

VH Image Data Representation

• Hierarchical multi-resolution tree

• Each node is an 8*8*8 cube

• 4 levels of representation

• Volumetric compression

Implementation Approach• Avoid disk seek and network bottlenecks

• Use memory based server representations

• Apply volumetric compression techniques

• Serve volume data by client demand

• Take advantage of intelligent clients

• Minimize server computational demands

• Leverage network tuning from Web100

AnatomyDatabase

MeshServer

Voxel IdentityServer

CollaborativeServer

indirect

(NGI)

direct

…or…

Intermediate Server

Router or Hub

System Architecture

Compression Numbers• Lossy simple VQ 9.42:1 = 2.67Gbyte• Lossless TS wavelet VQ 7.03:1 =3.53Gbytes• Normal JPEG @23.3:1 = 1.06Gbytes• JPEG2000 estimated ~40:1 = ~500Mbytes• Lossless SPHIT + arith ~4.22:1• Lossless SPHIT + huffman ~4.10:1 (2x faster)• Lossy SPHIT & other EWZ useful beyond 50:1

Shared & PrivateLibraries

Shared Services

Object Libraries

Version Control

Data Library

Runtime Results

Authoring ToolsSolvers,utilities.

VisualizationUser Interface

Validation

Bug Reporting

Peer Review

A Scalable Test-bed Must Include an Open Source Mechanism Featuring a Standards-based Adaptive Middleware Environment

Middleware and Infrastructure

Users/ Applications

ETF-based Raycasting: The Next Step