Future Challenges for Ensemble...
4
Future Challenges for Ensemble Visualization H. Obermaier and K. I. Joy University of California, Davis Introduction How often have you executed an algorithm only to find that getting reasonable results means changing parameters and restarting? How much time did you spend on finding the ”correct” parameters? Imag- ine going through the same ordeal with unbelievably complex simulation models used for the prediction of physical phenomena. Scientists have fought this battle for many years and have long been sick of the ”sitting, waiting, and restarting” process. Luckily, increased availability of vast computation resources and new computation strategies o↵er a solution to this dilemma: Scientists can now run several alterna- tive parameter settings or simulation models in paral- lel, creating an ’ensemble’ of possible outcomes for a given event of interest (see Figure 1). In our conver- sations with simulation scientists and visualization researchers, the visual analysis of ensemble data has repeatedly come up as one of the most important new areas of visualization and we expect it to have a wide impact on our field in the next few years. The challenge is to develop expressive visualizations of properties of this set of solutions, the ensemble, to support scientists in this challenging parameter-space exploration task. The idea to move away from visualizing a single concrete solution to analyzing a family of outcomes is not entirely novel to the field of scientific visualiza- tion: Since the mid 1990s, visualization researchers have developed methods to visualize uncertainty and errors in data. However, there is a key di↵erence be- tween uncertain data and ensemble data – uncertain data encodes the probability distributions of values throughout a data set, allowing the identification of a ”most likely” or ”most common” output, while con- taining no information about how di↵erent outcomes Figure 1: A set of 15 stirring simulations, showing velocity magnitudes for di↵erent fluid properties. Vi- sual comparison of ensemble members in a list view is difficult. relate. Ensembles, on the other hand, present us with concrete distributions of data, where each outcome can be uniquely associated with a specific run or set of simulation parameters. This discrete character together with the ability to relate outputs to specific inputs is what makes en- sembles so valuable to domain experts. Our chal- lenge is to develop visualization techniques and tools to extract and highlight commonalities, di↵erences, and trends in the set of ensemble members and to allow scientists to discover conceptual drawbacks or 1
Transcript of Future Challenges for Ensemble...
Future Challenges for Ensemble Visualization
H. Obermaier and K. I. JoyUniversity of California, Davis
IntroductionHow often have you executed an algorithm only tofind that getting reasonable results means changingparameters and restarting? How much time did youspend on finding the ”correct” parameters? Imag-ine going through the same ordeal with unbelievablycomplex simulation models used for the predictionof physical phenomena. Scientists have fought thisbattle for many years and have long been sick of the”sitting, waiting, and restarting” process. Luckily,increased availability of vast computation resourcesand new computation strategies o↵er a solution tothis dilemma: Scientists can now run several alterna-tive parameter settings or simulation models in paral-lel, creating an ’ensemble’ of possible outcomes for agiven event of interest (see Figure 1). In our conver-sations with simulation scientists and visualizationresearchers, the visual analysis of ensemble data hasrepeatedly come up as one of the most importantnew areas of visualization and we expect it to havea wide impact on our field in the next few years.The challenge is to develop expressive visualizationsof properties of this set of solutions, the ensemble, tosupport scientists in this challenging parameter-spaceexploration task.The idea to move away from visualizing a single
concrete solution to analyzing a family of outcomesis not entirely novel to the field of scientific visualiza-tion: Since the mid 1990s, visualization researchershave developed methods to visualize uncertainty anderrors in data. However, there is a key di↵erence be-tween uncertain data and ensemble data – uncertaindata encodes the probability distributions of valuesthroughout a data set, allowing the identification ofa ”most likely” or ”most common” output, while con-taining no information about how di↵erent outcomes
Figure 1: A set of 15 stirring simulations, showingvelocity magnitudes for di↵erent fluid properties. Vi-sual comparison of ensemble members in a list viewis di�cult.
relate. Ensembles, on the other hand, present us withconcrete distributions of data, where each outcomecan be uniquely associated with a specific run or setof simulation parameters.This discrete character together with the ability to
relate outputs to specific inputs is what makes en-sembles so valuable to domain experts. Our chal-lenge is to develop visualization techniques and toolsto extract and highlight commonalities, di↵erences,and trends in the set of ensemble members and toallow scientists to discover conceptual drawbacks or
1
Harald Obermaier
for Review onlyAccepted version published in:IEEE Computer Graphics and Applications, vol. 34, no. 3, pp. 8-11, May/June 2014.
the value of simulation models or specific parameterchoices.
Visualization of Ensemble Data
Recently, researchers have started to look into thevisualization of ensemble data. The few existing ap-proaches can be classified as
• Feature-Based: Features are extracted from indi-vidual ensemble members and compared acrossthe ensemble.
• Location-Based: Ensemble comparison is per-formed at fixed locations in the data set.
Due to the existence of a variety of prediction mod-els, weather and climate research is one of the centraldriving forces behind the creation of simulation en-sembles. Predictions of climate events rely on a largenumber of external influences (parameters) and aregenerally associated with a certain probability of oc-currence. This is because there is not only one possi-ble outcome to existing prediction models, but rathera spectrum of possibilities. Sanyal et al. [5] approachthe visualization of ensembles of numerical weathersimulations by extracting sets of isocontour lines anddesigning glyphs that illustrate local variances. Theso-called spaghetti plots created by rendering sets ofisocontour lines allow feature-based comparison be-tween members of the ensemble and give a basic im-pression of how ensembles agree for a given scalarvalue. A sample image of their technique is shown inFigure 2. In three dimensions, slicing can help createan impression of di↵erences in isosurfaces, as demon-strated by Alabi et al. [1] and shown in Figure 3.As you can see, even simple tasks such as renderingspaghetti plots becomes a visualization challenge inthree dimensions.Location-based methods have largely been used
to compute statistical properties of the ensemblethroughout the domain. Common statistical mea-sures then provide insights into outliers, agreementor disagreement between members of the ensemble.Means and variances of scalar quantities in climatesimulations have been employed by Potter et al. [4],
Figure 2: A spaghetti plot of perturbation pressureisocontours along with uncertainty glyphs showingdeviation. Ensembles are particularly relevant inweather and climate prediction. Image courtesy ofSanyal et al. [5].
see Figure 4. Notions of variances can also be em-ployed to detect agreement and disagreement in en-sembles for arbitrary flow simulations [3]. Figure 5shows an example of CFD variance-based coloringand trends querying. This provides a more conciserepresentation of a set of flow simulations, than pro-vided by list-based visualizations as shown in Figure1. Ensemble visualizations can be made especially ex-pressive if (some) ground truth data is available. Thisallows for an estimation of predictive uncertainty ofthe ensemble and can aid in identifying outliers in theset of ensemble members [2], see Figure 6. Gosink etal. also propose a visualization of parameter sensitiv-ity, which is a key component of e�cient parameterspace analysis.
What the Future Holds
We see the need for visualization research to aid sim-ulation scientists in the parameter space explorationtask and to support accurate and robust decision
2
Figure 3: Rendering of sliced isosurfaces from fourensemble members. Feature-based comparison in 3Dis challenging and can su↵er from occlusion and visualcomplexity. Image courtesy of Alabi et al. [1].
making in a complex simulation environment. Theneed for e↵ective visual analysis tools in this area hasthe potential to open and extend a large variety ofresearch directions and application scenarios. In ourcorrespondence with domain scientists we have iden-tified several key requirements of e↵ective ensemblevisualization tools along with conceptual and techni-cal issues. These challenges include
• Conceptual - Finding the Answer: Can we helpdomain experts in finding a ”most likely/bestprediction” made by the ensemble?
• Conceptual - Parameter Space: How can we re-late insights gained by ensemble visualizationwith locations in parameter space?
• Conceptual - Perception: How can we visualizesuch a multitude of data in a precise and easy-to-understand way?
• Mathematical - Features: What are statistical orgeometric feature definitions that are relevant inthe context of ensembles?
• Mathematical - Metrics: How can we comparemembers of the ensemble or their features?
Figure 4: Mean and standard deviation shown asa combination of isocontouring and color-mapping.Correlations of the two variables are conveyed im-plicitly. Image courtesy of Potter et al. [4].
• Algorithmic - Data Complexity: How do we han-dle the immense increases in memory require-ment and data complexity?
• Algorithmic - Exploration: How do we enablegoal-driven exploration and analysis of parame-ter space and parameter sensitivity?
One direction we regard as especially challeng-ing is the visualization and exploration of multi-dimensional parameter spaces. In this directionwe are investigating how techniques from high-dimensional data visualization can help in making theconnection between ensemble and parameter-spaceanalysis. Specifically, the question whether and howrecent work in computational steering, parameter-space exploration (e.g., Waser et al. [6]) and multi-variate analysis may be applied to complex ensemblevisualization problems remains to be answered.
Conclusions
Providing domain scientists with visualization solu-tions for ensemble data will be a key factor in improv-ing analysis performance in complex simulation envi-ronments. Solving the inherent visualization chal-lenges will increase the speed with which scientists
3
Figure 5: Simulation agreement and disagreement inan ensemble of CFD simulations is visualized throughcolor mapped transport variance. Trends are identi-fied as pathline clusters. A large number of ensemblemembers may lead to cluttered pathline renderings.Image courtesy of Hummel et al. [3].
can explore, adapt, and validate simulation models.We expect the visualization community to engage insolving this challenging task, and thereby improve therobustness and reliability of simulation-based predic-tion and decision-making.
Acknowledgements
This work was supported in part by the NSF(IIS 0916289 and IIS 1018097), and the O�ce ofASCR, O�ce of Science, through DOE SciDAC con-tract DE-FC02-06ER25780, and contract DE-FC02-12ER26072 (SDAV).
References
[1] O. S. Alabi, X. Wu, J. M. Harter, M. Phadke,L. Pinto, H. Petersen, S. Bass, M. Keifer,S. Zhong, C. Healey, and R. M. Taylor II. Com-parative visualization of ensembles using ensem-ble surface slicing. In Proc. SPIE 8294, Visual-
ization and Data Analysis 2012, pages 82940U–82940U–12, 2012.
[2] L. Gosink, K. Bensema, T. Pulsipher, H. Ober-maier, M. Henry, H. Childs, and K. I. Joy. Char-acterizing and visualizing predictive uncertainty
Figure 6: Evaluating predictive uncertainty of a sin-gle ensemble member through accuracy classificationwith respect to a probabilistic ground truth model.Ground truth data provides a basis for more complexanalysis and classification of ensemble data. Visualiz-ing complex classifications in 3D proves challenging.Image courtesy of Gosink et al. [2].
in numerical ensembles through Bayesian modelaveraging. IEEE Transactions on Visualization
and Computer Graphics, 19(12), 2013.
[3] M. Hummel, H. Obermaier, C. Garth, and K. I.Joy. Comparative visual analysis of Lagrangiantransport in CFD ensembles. IEEE Transactions
on Visualization and Computer Graphics, 19(12),2013.
[4] K. Potter, A. Wilson, P.-T. Bremer, D. Williams,C. Doutriaux, V. Pascucci, and C. R. Johnson.Ensemble-vis: A framework for the statistical vi-sualization of ensemble data. In IEEE Workshop
on Knowledge Discovery from Climate Data: Pre-
diction, Extremes., pages 233–240, 2009.
[5] J. Sanyal, S. Zhang, J. Dyer, A. Mercer, P. Am-burn, and R. Moorhead. Noodles: A tool for vi-sualization of numerical weather model ensembleuncertainty. IEEE Transactions on Visualization
and Computer Graphics, 16(6):1421–1430, 2010.
[6] J. Waser, H. Ribicic, R. Fuchs, C. Hirsch,B. Schindler, G. Bloschl, and M. Groller. Nodeson ropes: A comprehensive data and control flowfor steering ensemble simulations. Visualization
and Computer Graphics, IEEE Transactions on,17(12):1872–1881, 2011.
4
