llllIIIUilllilflilil1|il tt]tl'

?:: --

&: . i,,rii.tril

==:.:.:==:.:::::: .'..s

n6 winter evening in the early 1860s, German chemistAugust Kekul6 dozed off while sitting before a fire,falling into a remarkably vivid dream. Atoms formed

themselves into undulating strings that morphed into a snakeeating its own tail. Kekul6 contended that this intense imageryhelped him solve the mystery of benzene's ringlike structure, adiscovery that is considered a foundation of modern chemistry.

Nearly 100 years late! research teams on both sides of theAtlantic were vying to be the first to decipher the structureof DNA, the genetic material that is the basic molecule oflife. In the United States, Nobel laureate Linus Pauling foundhimself up against obscure English physicist Francis Crickand his 2O-something American postdoc, James Watson, inCambridge. The upstart British team had a hidden advantage:crystallographic X-rays of DNA taken by colleague RosalindFranklin. This chemically enhanced X-raytechnique revealed that DNA was composedof two complementary strands of nucleicacids linked by chemical bonds on a ladder-like chain. The ability to visualize DNA gave

them insights into the spiral double-helixstructure - and they won the race.

In 1993, Kary Mullis won the Nobel for hisinvention of poll,rnerase chain reaction, thechemical Xerox machine that makes thousandsof copies of tiny strands of DNA, a break-through that jump-started the biotech revolu-tion. The biochemist, then based in Berkeley,

freely admitted he conceptualized this advance

while under the influence of the mind-alteringdrug LSD, which helped him visualize thecomplex chemistry three-dimensionally.

These three examples center on the powerof visualization - that ability to "see" some-thing from a different perspective, a spark of insight that pares

away mountains of extraneous details and distills seeminglyimpenetrable puzzles down to their essence. But now we're inthe era of big data, which harnesses the computing power ofmassive databases with bytes measured in teras (trillions) andpetas (quadrillions), combined with sophisticated algorithmsthat can grapple with problems on a once-unimaginable scale.

While this numbers-crunching ability promises to greatlyaccelerate the pace of scientif,c discovery we're suddenly buriedin an avalanche of information.

Immersive environments - 3-D virtual reality worlds -can help us make sense of this in a tangible way. Big datacollects such a vast amount of information that it's difficultto see patterns. Using computing power to translate datainto something that can be seen and heard makes it easier tounderstand. "Scientists and engineers can work with their data,perceptually and intuitively, the way artists do," says JoAnnKuchera-Morin, creator of the AlloSphere. It is perhaps themost advanced of these immersive environments, housed onthe campus of the University of California, Santa Barbara.

. These electronically simulated worlds of sight and soundcut through a lot of the noise of big data, and they enable

researchers to synthesize, manipulate ar.d aralyze large datasets in a way that is easier to comprehend and digest, providingunparalleled insights into the whole picture and how each

individual piece fits in. "We have so much data that we need

these bigger lenses to get a full picture of what's really goingon," says Andrew Johnson, director of research at the Elec-tronic VisualizatiorLaborutory at the University of Illinois atChicago. "These kinds of environments are lenses to look atdata- the modern equivalent of the microscope or telescope."

Pooling massive amounts of data allows patterns and trendsto emerge that aren't apparent in small, individual studies, andthe applications are virtually infinite - thirtk Moneyball, the2003 best-selling book about how the perennially cash-strapped

Oakland l(s used analytics and baseball stats

to scout overlooked talent. Another example:In2013, it took number-crunching algorithms,sifting through terabytes of data, to spot thedistinctive signature of several Higgs bosonparticles. Physicists could finally identify them.Medical scientists, on the other hand, are

crunching billions of data points culled frommillions of patients about genetic mutationsthat make people more vulnerable to diseases

like diabetes, heart disease and cancer. Theycombine this information with sequences of theproteins those bits of DNA produce. (Proteins

are the body's workhorses that control everyce11.) This information is used to concoctmore targeted therapeutics and more precise

diagnostics using biomarkers - in a patient'sblood, saliva or urine - that signal the pres-

ence of a disease-

Immersive environments like the ones you'll see in thelollowing pages allow scientists to watch a tumor grow, observemolecules binding together - or even see a re-enactment ofthe Big Bang and witness the transformation of the universeover billions of years. Rudimentary versions of these environ-ments have been around since the 1990s, but with today'stechnology, scientists can sink into even greater realism andvisualize more with sharper resolution. This immersion is used

in disciplines as diverse as medicine, physics, neuroscience,green technology, structural engineering and archaeologyat universities, government research agencies and in privateindustry all over the world.

"Originally, we created these as an educational tool forvisualizing concepts and ideas - in place of a blackboardand hand waving - to help people see things they never didbefore," says Thomas DeFanti, a research scientist at UCSan Diego's California Institute for Telecommunications andInformation Technology, and a pioneer of virtual reality sys-

tems. "But the newest technology gives you the feeling of trueimmersion that makes for a completely riveting experience."

t,.l...,t.:,i3:I r{*i.lri

'q$,r,i:lI{*qlr:,,:,,j#{rl

,'irr'li':ll:i3i:l

zE

.. . . ..._ . . . , . ... i - . " . _..--

!I!,|

1

z

52

"Shall we, Matt?"JoAnn Kuchera-Morin instructs her media

systems engineer, Matthew Wright, to fireup the computer cluster that powers theAlloSphere. With a simple keystroke, we'resuddenly plunged into a virtual world of sightand sound that transports us on a fantasticvoyage through a three-dirnensional model ofthe human body. We hurtle down an artery,as if we're sliding down a slippery chute,and nearly collide with the liver and heart.

It feels as if we're propelled, airborne and

hovering in free fall in an onrush of imagesin the darkened chamber.

We're wearing 3-D glasses and stand-ing on a sturdy metallic catwalk suspend-ed at the center of a 33-foot-diametersphere, which seems to be floating insidea 2,000-square-foot room three storieshigh. Dozens of speakers and other audioequipment envelop us in sound fromevery direction, while high-resolutionvideo projectors beam floor-to-ceilingimages in 40-million-pixel detail. This

all creates a unique 360-degree immersiveenvironment that far outstrips the technologyof other virtual reality systems. Here, research-

ers can use all of their senses to uncovernew patterns in the data.

The AlloSphere cost $12 million for thestructure alone and was completed in 2007.

It is the brainchild of Kuchera-Morin, an or-chestrally trained composer turned computergeek who directs the AlloSphere Research

Laboratory at the University of California,Santa Barbara, perched on the rocky shorelineof the Pacific. A gregarious woman clad all inblack with long, straight gray hair that makes

her resemble a hippie grandmother, Kuchera-Morin began dabbling with big mainframecomputers in the 1980s, when traditionalinstruments couldn't translate the sounds she

heard in her head into music."The computer helped me understand all

of the acoustics, vibrations and physics ofinstruments," she says. "And through math-ematical equations, I could transform theminto anything I wanted to."

Her early experiments ultimately evolvedinto the AlloSphere, which converts reams ofdata into moving images and sound that areeasier for researchers to comprehend and di-gest. Sometimes, dozens of scientists in data-rich disciplines ranging from neuroscience andmedicine to green tech, theoretical physics,

materials science and nanotechnology gatheron this bridge. They use special wireless con-trollers and sensors embedded in the railingsto maneuver through the constellation ofimages. Physicists can watch representationsof electrons spinning inside hydrogen atoms,allowing them to actually "see" these invisibleprocesses of nature. while neuroscientists can

seemingly fly through 3-D images inside a pa-

tient's brain. "Everything you see is a numberthat's been crunched," says Kuchera-Morin.

July/August 2014 OtSCOvrn 53

E;:::::::: _.rr, ir. :: r:.:::.j!:ri.

It

,

,t:a1

T'|&

il.:i

"Mathematical algorithms can be translatedinto visual and audio frequencies by map-ping their vibratory spectrum in the light andsound domain - like mapping heat throughinfrared light. The AlloSphere is a virtualinstrument that allows scientists to do simula-tions, which will speed up time to discovery."

On this particular day, we're looking at aproject by Jamey Marth, director of the Cen-ter for Nanomedicine at UC Santa Barbara.Marth is using the sirrulation version of thehuman body to examine the makeup andbehavior of critical cell components, such asproteins, lipids (fats) and glycans (sugars).This particular simulation was built with MRIinformation collected from a living humanbody. Using specialized sottware and compu-tational language to translate mathematicalalgorithms and scientific data into sight andsound, Kuchera-Morin's band of techies f irstintegrated the geometries of the arteries,veins, pancreas and liver, and then scaledthem up like a high-powered digital micro-scope so researchers can better visualize thebiological processes of health and disease.

Right now, Marth's team is simulatingthe transport of chemotherapy directly tocancerous tumors in the pancreas and liverwithout harming healthy tissue. Artif icial

nanoscale particles might provea good trucking device" But first,the researchers have to gauge ifthe organic nanoparticles can suc-cessfully navigate through bloodvessels and then bind with cancercells to deliver their toxic payload,ln the AlloSphere, it's as if Marth'sresearchers are standing insideblood vessels, visualizing data ona human scale that is normallytoo small to see. The next stepis to integrate fluid dynamicsto simulate precisely how bloodflows through arteries and veins.They'll also work with materialsscientists to create a reproductionthat mirrors the composition ofdifferent-shaped nanopa rticles tosee how they navigate throughthe bloodstream so the team canrun virtual tests of new treat-ments in nanomedicine.

"We need to design nanopar-ticles that will, like a lock-and-keymechanism, travel through thebody and interact only with thediseased cell surface," says Marth."Right now, we use MRls andPET scans to visualize theseprocesses, but other imagingapproaches are needed - andthat's where the AlloSpherecomes in. This is the breedingground for the next generationof solutions in medicine."

54 otscovrnnaaGAZtNE.coM

,:,,

The AlloSphere is one of dozens of immersive environments that are now routinely used at universities, government agenciesand key research centers. Here's a sampling of some of the more innovative applications.

t,ft#H"= fl&Bffi#l

F=*u&H*ng €5*pr*ee#m eek

#ruue5pa'$re#m ese Sftw €eetr#ru

At the Electronic Visualization Labora-tory (EVL) at the University of lllinois atChicago, a psychiatric team is mappingthe braint intricate web of neural con-nections, using images from MRI scans,to try to identify regions responsible fordepression. They're in CAVE2 (Cave Au-tomatic Virtual Environment), a 24-loot-wide, 8-foot-tall, 320-degree panoramicroom, with 72 3-D liquid-crystal displaysthat have a 37-megapixel resolution,which is about the limit of human 20i20visual acuity.

The bundles of neural fiber tracts inthis brain visualization are color-codedby their primary direction (green: front-back; red: left-right; blue: up-down).Scientists are scouring this rainbowarray for faulty wiring in the neuralconnections in the white matter; whichtakes up half the brain. lt serves asa neural switchyard with millions of

{l j$ i*Fi#; i.i: iii;::r:

communication cables that operatelike telephone trunk lines connectingdifferent regions of the brain. previousresearch suggests that damage to thesecables in the white matter is associatedwith depression.

"We're hoping this 3-D environmentwill help us spot differences that aren.t

easily detectable in two dimensions ona flat screen," says Olusola Ajilore, theUIC psychiatrist conducting the research."lf we get better at mapping the brainareas responsible, it will lead to moreprecision in the use of technologies thatmay repair these damaged connections,like deep brain stlmulation."

&$+c*ffmrgr**rEE€ g€5€€= c**&* ffi*€ffmn*a sgcfuw #*s€#CAVE2 is able to switch gears from the inner workings of thebrain to Antarctica's West Lake Bonney. peter Doranitandsin front of the EVL's 8-by-24-foot wall of LCD screens at theUniversity of lllinois at Chicago, wearing 3-D glasses andsurrounded by images from the 2lz-mildong pool of waterentombed beneath 15 feet of ice. His glasses are outfittedwith tiny tracking balls and sensors that adjust the projectionson the screen according to his movements. Aturn of his headchanges the scene, accentuating the sense of immersion.

An earth and environmental scientist, Doran was part ofa NASA-funded team that did field trips in 2008 and 2009 inAntarctica's Dry Valleys to explore the underground lake. Eachday, ENDURANCE (Environmentally Non-Disturbing Under-iceRobotic Antarctic Explorer) was lowered through a hole inthe ice and used its sensors to take readings in different partsof the lake - temperatures, light levels, solar radiation and

a

s6

=UIC graduate student Alessandro Febretti stands inside cAVE2, immersed in 3-D images fromAntarctica's west Lake Bonney. The project may lead to more work on extreme envirlnments.

Peter Doran (center) manipulates the visualization of data thatdepicts Antarctica's West Lake Bonney.

dissolved organic matter. The $2.3 million research projectmay be a dress rehearsal for future exploration when a modi-

fied version of ENDURANCE could bedispatched to explore the extremeenvironments.of the solar system, suchas Mars or Jupiter's moons.

The information Doran's teamgathered was relayed to the EVL togenerate 3-D images and maps to helpthem get a better understanding ofthe lake's watery depths. "This is anincredible tool - you feel like you,reflying through the lake, and it,s theclosest thing to actually being there I

could imagine," says Doran. "lf I putall this data on my laptop, l'd miss alot, and l'd have to break it down sliceby slice, which would be grueling andtake years to accomplish-"

The braint neural fiber tracts are shown on 72 3-D LcD screens that line the inside of a24-foot-wide, 8-foot-tall, 320-degree room at the university of illinois at chicago.

July/August 2Ot4 OtSCOVen 55

5€a*y *re*k's E*ru3*ty ffie<k:€5p {$*me fficdS@s#efficp€ $a.e S#us tr€fuaf ea&atrCalled "virtual reality on steroids,,, the $2 million Reality Deckat Stony Brook University in New york is the world,s hig-hestresolution panoramic immersive environment. The facility, whichdebuted in 2012, has a total resolution of more than 1.5 billionpixels (1.500 megapixels). For comparison, high_def TVs have2 million pixels, while 3-D movies have 4 million; the RealityDeck boasts about 500 times more resolution. This surround-viewtheater, a 627-square-foot room tiled floor-to_ceiling with 4,16high-resolution flat-screen monitors, can display a 4-5-gigapixelphotograph of the city of Dubai as a single image. Researchers cansoar overthe city, yet detect such minute detaili as a car,s licenseplate number. No need to pan or zoom _ to see something closeup, just walk up, thanks to the ,,infinite canvas,,, a 360-degreesmart screen that changes images as you walk around the,deck.

The level of detail is remarkable: Flood maps can pinpoint sub_merged areas during Superstorm Sandy, even the debris on some_one! lawn. A shot of the half-million [eople at Barack Obamatpreside.ntial inauguration in 2009 is "so detailed that you canrecognize e.ach and every face in the crowd,,,says Arie Kaufman,chief scientist of Stony Brook,s Center of Excellence in Wireless andlnformation Technology.

This technology can be used for applications as diverse as tour_ism or thwarting terrorists because of its uncanny ability to detect

a suspicious person in vast crowds. lt,s also useful in cosmology andcharting the far reaches of the Milky Way, weather prediction andclimate change modeling or even aerial ieconnaissance missionsfor national defense.

"Hollywood islooking at this .-

,"?* t*,.r 'rt%

.s.ri$gtrs

-e

, u*

technology for thenext generation

of super-high-defTV and 3-D movie

experiences."- Arie Kaufman of the Center of Excellence in Wirelessand lnformation Technology at Stony Brook University

i;t:;fl,::T[li[::,':"fiH',?ffi:isdisplayedaCro55416monitorsasasingle';;

Ia

o

u

=

56 orscovenrr,tecAzrNE.coM

II"-I:?lty_P"* s 627 square feet are tited ftoor-to-ceitins with 416 high-resotution monitors, lts total resolution is more than 1.5 billion pixels.

lP-q:i!i:g or zooming.-- to see something in gtre"t", d"tait, just watkcroser to the wall. The ,,infinite

canvas,,, a 360_degree smart screen,atters images in concert with ,i"*"ri rnoring

"i."r"Jtt " nl"iii, iil.t.

{$ff $n* #E*g*'s St*rf,&y8:

prove cardiac pumps forchildren with heart de-fects. Archaeology teamshave used the StaTCAVEto interact in real timewith ongoing excava-tions at ancient dig sitesin Jordan and to studysatellite images and datagathered during field ex-peditions to Mongolia toaid in the quest to findGenghis Khan's tomb.

tlY'x T*C{ Vigar*Eiea1cf; effln 2008, Hurricane lke had already deci-mated parts of Cuba and the Bahamas,and it was barreling for the U.S. mainland.Disaster management teams had less than72 hours to make decisions that wouldaffect millions. Before it made landfall inGalveston. Texas, lke was a powerful cat-egory 4 storm 450 miles wide with winds ofup to 145 mph. lt ultimately caused billionsof dollars in damage from the Louisianacoastline to Corpus Christi, Texas, shutter-ing oil refineries and forcing more than amillion people to evacuate.

Scientists at the Texas Advanced Com-puting Center (TACC) at the University ofTexas'Austin campus turned to Stallion, a328-megapixel system that uses g0 30-inch

t' .'.:,:;'a,r-"r"1,;:.-. t;i, .'r ....

:::.. -] ]r::.1:.: :' :: ::::]::': l ::::: .:1 :. -:::1i

;!.\rie L-l1trl;l

#scgas6f*d ffm #$rgws# Sr*ff**eslnside the StaTCAVE at the University of California, San Diego, highschool biology students find themselves engulfed in virtual 3_Drenderings of green fluorescent proteins and other structures de_rived from the university,s protein Data Bank. The five-sided virtualreality room, with 70 monitors and nearly 2g7 million pixels, is thethird generation of technology first developed at the University oflllinois' EVL. (See page 55.) SIaTCAVE has been integral to projectsin areas ranging from molecular biology to archaeology and itruc_tural engineering. Neuroscientists, working with archiiects, havebuilt virtual hospitals, allowing them to design workspaces thatensure.better patient care, such as keeping a line of sight from thenurses'station to patients, bedsides. Engineers are devising bloodflow simulations to im-

Above: Researchers can stand immersed in 3-D render-ings of green fluorescent proteins and other structuresderived from UC San Diego,s protein Data Bank.

Left: Research scientist Jiirgen Schulze (left) and doctoralstudent Kyle Knabb traverse an archaeological site insouthern Jordan using a virtual ,,wand,, inlhe StarCAVE.3

2

oE

,Ez

?

=o

s

o'

E

Fz

=Y

=

d2

s

Pii!

p

Lmfu*reteryr ffiduremse ffiwfm#reg ffiF€ #$$$y$#e**$flat-panel monitors covering a single wall,and worked closely with the National Oce-anic and Atmospheric Administration andstate emergency management teams. Bycombining actual photos with satellite im-ages, they devised real-time global and re-gional 3-D simulation models of when andwhere the hurricane would make landfall.

"Think of it as a flow problem, trying toget millions of people out of harm,s way ina couple of days," says Kelly Gaither; TACC,sdirector of visualization. '.you must havean efficient way to evacuate the large coastregions via your pipeline, and they have tohave somewhere to go."

TACC's Stallion has been used in severalresearch projects, including visualizations

TACC scientists worked with NOAA to createreal-time 3-D simulation models of Hurricane lkemaking landfall to help Texas make decisionsabout evacuations when the storm hit.

of the Southwest Power Grid, with imagesstreamed directly from its source in pasa-dena, Calif.; massive oil spills; the spread ofthe flu across the region; and the birth ofthe universe with the Big Bang. ,,Visualiza-

tion technology allows us to leverage theworld's most powerful pattern recognitionengine -the human mind,,, says Gaither."Seeing is a way to gain insights into verylarge data sets that we would have a dif-ficult time understanding otherwise.,, Itr

Linda Marsa is the author of Fevered: Why a

Hotter Planet Will Hurt Our Health and HowWe Can Save Ourselves.

Eighty 30-inch monitors cover a singre wa[, making up the 32g-megapixer stailion iystem.

July/August2Ol4 DTSCOVER 57