Post on 24-Dec-2015
ASCR Scientific Data Management Analysis & Visualization PI Meeting
Exploration of Exascale In Situ Visualization and Analysis Approaches LANL: James Ahrens, Jon Woodring, Joanne Wendelberger, Francesca Samsel
. We explore two in situ approaches at the extreme ends of a spectrum between flexibility and accuracy. We will strive to understand the advantages and disadvantages of both approaches and evaluate their effectiveness. Using the results of this evaluation, we will merge the best of both approaches to produce an optimize exascale
visualization and analysis approach.
Statistics and Sampling of Simulation Data with BitmapsChallenges
Locating the data that a scientist needs is daunting due to the scale of the data and lack of information
Solution: Sample BitmapsBitmap indices provide summary information for a large-scale data set
They also provide distributional data that can be used for samplingStatistics can be extracted from this summary to be able to drill down and extract
information of interestBitmaps accelerate statistics and sampling for faster turn-around in exploration with
lower sample errorPapers
Y. Su, G. Agrawal, J. Woodring, K. Myers, J. Wendelberger and J. Ahrens, "Effective and Efficient Data Sampling Using Bitmap Indices", Cluster Computing, March 2014.
Y. Su, G. Agrawal, J. Woodring, A. Biswas and H.-W. Shen, "Supporting Correlation Analysis on Scientific Datasets in Parallel and Distributed Settings", in Proceedings of the International ACM Symposium on High-
Performance Parallel and Distribued Computing (HPDC'14), June 2014, Vancouver, Canada.Y. Su, G. Agrawal, J. Woodring, K. Myers, J. Wendelberger and J. Ahrens. “Taming Massive Distributed Datasets: Data Sampling Using Bitmap Indices.” In Proceedings of the International ACM Symposium on
High-Performance Parallel and Distributed Computing (HPDC’13), New York, NY, USA, June 2013.Y. Su, G. Agrawal, and J. Woodring, “Indexing and Parallel Query Processing Support for Visualizing Climate
Datasets”, Proceedings of the 41st International Conference on Parallel Processing, Pittsburgh, PA, Sept. 2012.
Bitmaps are used for sampling and statistics for large-scale
data analysisContact: James Ahrens <ahrens@lanl.gov>
Adaptive refinement based on analysis metric
highlighting areas of interest
Reduced Simulation Data Approach Significantly reducing simulation data by storing
sampled and compressed data representations
Adaptive Sampling of Simulation DataChallenges
Simulations and experiments generate more data that can be feasibly stored by the scientist
Solution: Adaptive Sample Data based on Analysis Metrics Treat the exascale data deluge as a sampling problem
Use a variety of metrics to automatically select and triage the important data
Analysis Driven Refinement is a framework that prioritizes and samples using these metrics
Papers B. Nouanesengsy, J. Woodring, K. Myers, J. Patchett, and J. Ahrens, “ADR Visualization: A Generalized Framework for Ranking Large-Scale Scientific Data using Analysis-Driven Refinement”, LDAV 2014, November 2014, Paris, France.
K. Myers, E. Lawrence, M. Fugate, J. Woodring, J. Wendelberger, and J. Ahrens, “An In Situ Approach for Approximating Complex Computer Simulations and Identifying
Important Time Steps”, in submission, arXiv:1409.0909. A. Biswas, S. Dutta, H.-W. Shen, J. Woodring. “An Information-Aware Framework for
Exploring Multivariate Data Sets.” IEEE Visualization 2013, Atlanta, GA, November, 2013.
Image Database ApproachSignificantly reducing simulation data by storing rendered visualization and analysis images into an image database
Sampling in “Visualization and Analysis” SpaceChallenges
Simulations and experiments generate massive datasets that are difficult to store and analysis in a post processing manner
Solution: Generate In Situ Image DatabaseEnables many different interaction modes including: 1)
animation and selection, 2) camera and 3) timeCreates an responsive interactive visualization solution,
rivaling modern post-processing approaches, based on producing constant time retrieval and assembly
Encourages the use of both computationally intensive analysis and temporal exploration typically avoided in post-processing
approaches
Supports Metadata SearchingBy leveraging an image database, our approach allows the
analyst to execute meta-data queries or browse analysis results to produce a prioritized sequence of matching results
Creation of New Visualizations and Content Querying Supports composing of individually imaged operators
Provides access to the underlying data to enable advanced rendering during post-processing (e.g. new lookup tables,
lighting, ...) Makes it possible to perform queries that search on the
content of the image in the database. Using image-based visual queries, the analyst can ask simple scientific questions
and get the expected results
Papers J. Ahrens, S. Jourdain, P. O'Leary, J. Patchett, D. H. Rogers, M. Petersen, “An Image-
based Approach to Extreme Scale In Situ Visualization and Analysis”, Supercomputing 2014, New Orleans.
Interactive visualization and compositing using images from the image database
Using lighting and color mapping, render passes and compositing enable more capable visualization pipelines
such as changing color scale mapping for objects
Queries based on the image content can be used to search for qualitative results like “best view”
LA-UR-15-20106