Advisor: Dr. Sergiu Dascalu Committee: Dr. Valerie Fridland Dr. Fred Harris Dr. Yaakov Varol Dr....

Advisor: Dr. Sergiu Dascalu Committee: Dr. Valerie Fridland Dr. Fred Harris Dr. Yaakov Varol Dr. Yantao Shen Likhitha Ravi

1. Introduction 2. Background 3. Requirements 4. Architecture 5. Research Plan 6. Conclusions

Cyberinfrastructure (CI) developments are part of an NSF EPSCoR project (2008-2013, cca $21.7 million) Focused on climate change (CC) research, education, and policy making in Nevada Six project components: climate modeling (air) water resources (water) ecological change (land) education cyber infrastructure policy, decision making and outreach

The projects major goals: Create research capabilities to add value to the existing R&D resources Establish unique positions in focused research fields Increase inter-institutional and interdisciplinary collaborations

Research focus: The effects of regional climate change on ecosystem resources Major interdisciplinary science questions: How climate changes affect water resources and linked ecosystem services and human systems? How will climate changes affect disturbance regimes (e.g., wildland fires, insect outbreaks, droughts) and linked systems?

Cyber Infrastructure (CI) goals: Facilitate interdisciplinary climate change research, education, policy, decision-making, and outreach by using CI to develop and make available integrated data repositories and intelligent, user- friendly software solutions

Envisioned in the NSF EPSCoR project proposal 2008

CI outputs: Nevada Climate Change Portal (NCCP)NCCP Software tools for climate change research, outreach and education: software frameworks Integration and interaction across project and among CI groups within the 3-State Western Consortium: facilitator of collaboration

NCCP provides the climate data online to help researchers working on climate change all over the globe. Why do we need data visualization? Although most of the climate related data is easily available on the World Wide Web, it is a complex and demanding task to analyze very large datasets without the help of visualization.

Uses of visualization Presenting the results in a comprehensible manner for decision makers, stakeholders and general public. Evolution of climate models. Verification of hypotheses. Data exploration in order to find the trends and patterns.

VISTED mainly helps the climate researchers by visualizing the datasets over the web. The users of the VISTED are researchers, educators, students, policy makers and general public.

Research Questions What specific visualization techniques and displays can increase the efficiency of the environmental scientists? What mechanism for integrating data extraction, conversion and visualization are most beneficial for the environmental scientists work? What are the challenges facing researchers in the field of data visualization?

Significant features of VISTED Data Visualization Data Download Data Extraction Data Conversion Capabilities of VISTED Handling several input data formats such as Network Common Data Form (NetCDF), Comma-Separated Values (CSV), American Standard Code for Information Exchange (ASCII) and Hierarchal Data Format (HDF5). Providing different kinds of visualizations such as line chart, bar chart, bubble chart, and many more.

New capabilities A web based tool for climate researchers, students, educators and general public. Uploading datasets from users machine. Reading input from several data formats such as NetCDF, CSV, ASCII and HDF5. Extracting NetCDF, CSV, ASCII and HDF5 datasets. Converting into different data format. Introducing new visualization techniques to the climate researchers.

Table 1: Matrix representing the features of visualization tools #Tool Name Operating system support Visualization Techniques Programming/ Scripting languages # of variables 1ArcGIS Microsoft Windows, Linux,Sun Solaris Map (MXD), Globe, Geoprocessing, Geocoding, Network Analysis,Geodata, Mobile VBA, VB,.NET, Java, C++, COM, Python, VBScript, JavaScript, ASP, JSP, ColdFusion, Java,.NET, JavaScript, XML, FLASH, PHP Multidimensio nal data 2 AVS/Expres s Windows, Mac OS X, Linux, Solaris, and HP-UX, IRIX and Alph Tru64 2D line field plots, Gamma plot, 3D shaded,contour, and arrow field plots, Animations, particle tracing using stream lines and streak lines, isosurfaces, Volume Visualization C, C++, and FORTRAN. 2D, 3D, univariate,mult ivariate data 3Ferret Unix systems, and on Windows XP/NT/9x Geophysical formatting, symmetrical processing. Ferret Scripts 3D, 4D, Multidimensio nal data 4GGobi Windows, Mac, Unix Histogram, textured dot plot, barchart, spineplot, Scatterplot, parallel coordinates, time series plot Ggobi scripting 3D, Multivariate data 5Google Visualizatio n API Windows, Mac, Unix pie chart, Scatterplot, Guage, geo chart, bar chart, tree map, bubble chart, line graph, stack graph,, combo chart, column chart, area chart, candlestick chart, word cloud generator, and maps. Javascript2D

AVS/Express Terrain and Weather Wind Modeling Source: http://www.avs.com/products/avs-express/gallery.html

ArcGIS Impacts of Sea Level Rise Climate change Source: http://www.esri.com/library/ebooks/climate-change.pdf

Table 1: Matrix representing the features of visualization tools # Tool Name Operating system support Visualization Techniques Programming/ Scripting languages # of variables 6GrADS Linux, Mac OS X, Windows, Solaris, IBM AIX, DEC Alpha, IRIX line and bar graphs, scatter plots, smoothed contours, shaded contours, streamlines, wind vectors, grid boxes, shaded grid boxes, and station model plots FORTRAN, GrADS scripts 5-dimensional 7 Integrated Data Viewer (IDV) Windows, Linux, Solaris (SPARC and x86), Mac OS-X Charts, maps, radar displays, gridded data displays, isosurfaces, volume rendering, globe display, plan view, profiler winds Java 3D, multi- dimensional data 8 Mathemati ca Windows, Mac, Unix polar and spherical plots, contour and density plots, parametric line and surface plots, and vector, stream plots, candlestick charts, quantile plots, box whisker charts, Bode plots, histograms, 2D and 3D bar charts, pie charts, bubble charts, B-spline curves in 2D or 3D C++, Java,.Net, FORTRAN, CUDA, OpenCL 2D, 3D 9Matlab Linux, Microsoft Windows Line, area, bar, pie charts, Histograms, Scatter/bubble plots, Animations, Direction and velocity plots, isosurfaces, Volume Visualization C, C++, and Fortran. 1D,2D, 3D visualizations 10 OpenDXWindows, Mac OS X, Linux, Solaris, and Unix Animations, Direction and velocity plots, isosurfaces, Volume Visualization C, FORTRAN and Visual Basic 2D, 3D, univariate,multiva riate data

Grads Temperature ForecastIDV view of Hurricane Charlie Source: http://wxmaps.org/pix/temp5.html Source: http://www.unidata.ucar.edu/software/idv/docs/userguide/index.html

Table 1: Matrix representing the features of visualization tools # Tool Name Operating system support Visualization Techniques Programming/ Scripting languages # of variables 11Prefuse Windows, Mac, Unix Area chart, Bar chart, Pie chart, scatter chart, line graph, Tree map, network diagram and animations Java2D 12R Windows, Mac OS X, Linux and Unix Graphs, traditional statistical tests, time series analysis, linear & nonlinear modeling, classification, clustering C, Python, Perl 3D 13S-PLUS Windows, Linux, UNIX, Solaris Graphs, linear & nonlinear modeling, classification, clustering FORTRAN,C, S3D 14SPSS Windows, Mac, and Linux Tables, graphs, linear regression, cluster analysis, and non-parametric tests Java, Python, SaxBasic 2D 15TableauWindowsScatterplot, matrix chart, bar chart, area chart, bubble chart, stack graph, pie chart, link map and spatial maps No programming or scripting required 2D, univariate, multivariate data

R-Statistical PackageTableau Gallery Source: http://www.r-project.org/Source: http://www.tableausoftware.com/learn/gallery

Table 1: Matrix representing the features of visualization tools #Tool Name Operating system support Visualization Techniques Programming / Scripting languages # of variables 16UV-CDATMac, Linux multi-view visualization, Direction and velocity plots, isosurfaces, Volume Visualization, and parameter space exploration Python, C/C++,Java, FORTRAN 3D, multi- dimensional data 17VisTrails Windows, Mac, Linux multi-view visualization, Direction and velocity plots, isosurfaces, Volume Visualization, and parameter space exploration Python 3D, multi- dimensional data 18VisIt Windows, Mac, Linux, Unix, AIZ, Solaris, Tru64, IRIZ Contour 3D, Pseudo color plot, Contour 3D, volume plot, vector plot, subset plot, molecule plot, parallel axis plot Python 3D, multi- dimensional data 19Visualizati on toolkit (VTK) Windows, Mac, Unix scalar, vector, tensor, texture, volumetric methods, implicit modeling, polygon reduction, mesh smoothing, cutting, contouring, and Delaunay triangulation C++3D

Vis Trails Gallery VisIt Gallery Source: http://www.vistrails.org/index.php/File:Screen_Shot_2012-01- 12_at_2.50.19_PM.png Source: https://wci.llnl.gov/codes/visit/gallery.html

NASA (National Aeronautics and Space Administration) NASA * Provides data extraction. * Data can be downloaded in several formats. - No data interaction. NOAA ( National Oceanic and Atmospheric Administration) NOAA * Supports data interaction. * Provides data extraction. - Data can be downloaded only in ASCII format. Cal-adapt Cal-adapt * Supports data interaction. - Cannot change visualization technique - Does not support data conversion. Many eyes Many eyes * Supports several visualization techniques. * Allows users to upload data -Supports only CSV and ASCII file formats.

Source: http://mynasadata.larc.nasa.gov/ NASA

Source: http://www.climate.gov/#climateWatch NOAA

CAL- Adapt Source: http://cal-adapt.org/temperature/decadal/

Source: http://www-958.ibm.com/software/analytics/manyeyes/page/create_visualization.html Many Eyes

Less learning time No programming knowledge required ArcGIS, Tableau, Graphpad, Many eyes Programming/Scripting knowledge required AVS/Express, VisTrails, VisIt, VTK, Ferret, UV-CDAT, GrADS, IDV, R, SPSS, Jquery visualize, D3 Open Source Ferret, GrADS, IDV, R, UV-CDAT, VisTrails, VisIt Supporting several input formats ArcGIS, GrADS, VisIt, Ferret, NCL Supporting several visualization techniques VisTrails, UV-CDAT, VTK, IDV, Many eyes Supporting large and complex datasets AVS/Express, IDV, VisIt, VTK, Ferret

Degrading performance while working with large datasets VisTrails, VisIt, XmdvTool, IDV Poor data modeling capabilities VTK, Tableau, Not supporting data interaction ArcGIS, VTK Supporting limited operating systems/ browsers/ hardware UV-CDAT, OpenDX, Many eyes, Ferret

None of the tools fulfill the needs of climate researchers completely. Switching among the tools could be easier if there is a standard input data format. Support of interactive 3D/4D visualizations. Support of several devices such as touch pads, display walls, mobile devices, and desktops. Handling erroneous data and missing data values.

One-Dimensional histograms, normal distributions Two-Dimensional line graphs, bar charts, area charts, pie charts, maps, scatterplots, and stream line and arrow visualizations. Three-Dimensional Isosurface techniques, direct volume rendering, slicing techniques, 3D bar charts and realistic renderings. Multi-Dimensional scatterplot matrices, parallel coordinates, star coordinates, maps, and autoglyphs

Source: http://www-958.ibm.com/software/analytics/manyeyes/page/Visualization_Options.html

VISTED shall allow user to select a climate variable. VISTED shall allow user to select a combination of climate variables. VISTED shall allow user to select a time period. VISTED shall allow user to select a particular location. VISTED shall accept input data in netCDF format. VISTED shall allow user to download data in netCDF format. VISTED shall accept input data in CSV format. VISTED shall allow user to download data in CSV format. VISTED shall accept input data in binary format. VISTED shall allow visualization of datasets that are loaded from users system.

VISTED shall allow user to download data in binary format. VISTED shall allow user to view the selected data. VISTED shall provide the links for the navigation across the website. VISTED shall provide some sample visualizations to the users. VISTED shall allow user to choose a visualization technique. VISTED shall allow user to view data as time series graphs. VISTED shall allow user to pick a location from the map. VISTED shall provide users with frequently asked questions and answers.

VISTED shall be platform independent. VISTED shall support many browsers VISTED shall be developed using competitive technologies like HTML5, jQuery, and CSS3. VISTED shall be extensible and reusable. VISTED shall be fault tolerant. VISTED shall have high performance. VISTED shall have high reliability. VISTED shall support devices like tablets and mobile phones.

Technologies HTML5 D3 JavaScript Library C# IDE Visual studio 2012

D3D3 is the winner! * Provides several visualization techniques. * Provides data interactivity. Source: https://github.com/mbostock/d3/wiki/Gallery

Modeling Output Modeling Output

NetCDF File

Activity diagram

Exploration of the current state-of-the art methods and technologies/tools used for the presentation and visualization of environmental data. Research and design of a new web-based software toolset for processing and visualizing transect data (these activities will lead to advanced data processing capabilities for the NCCP). Development, experimentation, and integration of the new processing and visualization software into the Nevada Climate Change Portal.

Task 1: Survey existing methods and supporting tools used for the presentation and visualization of environmental data. Identify strengths and limitations. Outputs: survey report. Task 2: Elaborate conceptual design and operational approach (method) for a new web-based software toolset dedicated to presenting and visualizing NCCP environmental data. Outputs: conceptual design document; documented method. Task 3: Create software specification and architectural design of the new software toolset. Outputs: Software requirements specification document; design document (high-level sign, detail-level design, data design, user interface design, interface design). Task 4: Implement web-based software solution. Outputs: Implemented software; documented code.

Task 5: Integrate web-based software toolset into the Nevada Climate Change Portal and prepare user manual. Outputs: Integrated software, executable through the NCCP; tutorial and user manual. Task 6: Perform usability tests on the data portal and process results. Output: usability test report. Task 7: Based on user feedback, revise and improve web-based software toolset for data presentation and visualization. Output: improved web-based, NCCP-integrated software toolset for environmental data presentation and visualization. Task 8: Disseminate research and development results. Outputs: Journal or conference paper; one or two poster presentations.

As per GRA tasks Performed survey on existing data visualization tools and techniques for environmental data. (Task 1) Gathered the requirements and created the concept and specification document. (Task 2) Created the detail design of the software toolset. (Task 3) Designed the initial prototype of the toolset. (Task 4)

In addition to GRA tasks Wrote chapters 2 and 5 of the dissertation. Presented a paper at CATA-2013 in March 2013. Likhitha R., Qiping Y., Dascalu M. S., Harris F. C. Jr., A Survey of Visualization Techniques and Tools for Environmental Data, CATA, March 2013. Presented a poster in NSF EPSCOR Annual Climate Change Conference in March 2013. Likhitha R. An overview of visualization approaches for environmental data, Tri-State EPSCoR Climate Change Workshop, March 2013. Coauthor on another paper and poster. Qiping Y., Michael M. Jr., Dascalu S., Harris F. C. Jr., Likhitha R., Community Metadata ISO 19115 Adaptor, CATA, March 2013. Richard k., Michael M. Jr., Eric F., Sohei O., Likhitha R., Ivan G., Jigarkumar P., Adrew D., Ershad S., Shahram., Dascalu., Harris F. C. Jr., Communicating Climate Change on the Web: The Nevada Climate Change Portal, Tri- State EPSCoR Climate Change Workshop, March 2013.

To do Get additional input from scientists. Finalize proposed approach and web-based solution. (Task 4) Integrate with NCCP. (Task 5) Perform user tests. (Task 6) Revise VISTED and compare with related toolsets. (Task 7) Disseminate research. (Task 8) Finalize and defend dissertation. (Task 9)

The main goal of the VISTED is to help the climate researchers in visualizing datasets using new capabilities. It provides a new approach and supporting tools. It gives users the flexibility in choosing the data of their interest. The toolset allows users to upload files for the visualization.

Main contributions New approach that integrates data extraction, conversion, and visualization (with possible extensions for data analysis). Associated web-based toolset for data manipulation and visualization. Support provided for several data formats. Flexible data extraction capabilities. Mechanisms for efficient visualization of climate data.

I would like to thank all my committee members. Dr. Sergiu Dascalu Dr. Valerie Fridland Dr. Fred Harris Dr. Yaakov Varol Dr. Yantao Shen I am also thankful to CSE R&D faculty Mr. Eric Fritzinger Dr. Richard Kelley

Nevada Climate Change Portal, available at http://www.sensor.nevada.edu/NCCP/. Graphical Forecasts, Nation Oceanic and Atmospheric Administration, available at: http://graphical.weather.gov/. UNR Valley Road Weather Station, Western RegionalClimate Center,, available at: http://www.wrcc.dri.edu/weather/unr.html. Snow Pack: Decadal Averages Map, Cal-adapt ExploringCalifornias Climate Change Research, available at: http://caladapt.org/snowpack/decadal/. Pavlopoulos G. A., Wegener A., and Schneider R., "A survey of visualization tools for biological network analysis", BioDataMining, November 2008. Aigner W., Bertone A., and Miksch S., "Comparing Information Visualization Tools Focusing on the Temporal Dimensions," 12th International Conference on Information Visualization, pp. 69 - 74, July 2008 Mozzafari E. and Seffah A., "From Visualization to Visual Mining: Application to Environmental Data", IEEE Confererence on Advances in Computer-Human Interaction, pp.143-148, February 2008. Aigner W., Miksch S., Schumann H., and Tominski C.,Visualization of Time- Oriented Data, Springer, May 2011.

ArcGIS - Mapping and Spatial Analysis for Understanding Our World, ESRI, available at:. ArcGIS QGIS Faceoff, blog.donmeltz.com, available at:. AVS/Express Data Visualization Software, AVS/Express,. GrADS Home Page, Grid Analysis and Display System,. Unidata | IDV, Unidata,. UV-CDAT, UV-CDAT, available at:. VisTrailsWiki, VisTrailsWiki, available at:. VisIt Visualization Tool, visIt, available at. VTK - The Visualization Toolkit, Visualization Toolkit, available at

Questions ?

Advisor: Dr. Sergiu Dascalu Committee: Dr. Valerie Fridland Dr. Fred Harris Dr. Yaakov Varol Dr....

Documents

Transcript of Advisor: Dr. Sergiu Dascalu Committee: Dr. Valerie Fridland Dr. Fred Harris Dr. Yaakov Varol Dr....