enabling effective workflow and data quality management
Transcript of enabling effective workflow and data quality management
Enabling Effective Workflow and Data Quality Management
Tope Bello, Jay Cary, Javier Gutierrez April 11, 2012
Workshop Agenda
• Overview of ArcGIS Workflow Manager
• Overview of ArcGIS Data Reviewer
• Demonstration: Enabling Data Quality-focused Workflows
• Resources
• Questions
Overview of ArcGIS Workflow Manager
Section Topics
• Business Problem • ArcGIS Workflow Manager overview • System architecture • Extending Workflow Manager
Common Business Problems
• Why is managing work important?
- Optimize data production time - Improve data quality and accuracy - Reduce production cost - Resource allocation - Simplify processes
What is ArcGIS Workflow Manager Right work completed the right way, at the right time, by the right person
Standardize
Utilize Improve
Allows users to: • Standardize workflows
quickly
• Utilize workflows efficiently
• Improve performance continuously
Key Features and Capabilities
Anatomy of a Job
Geospatial data
Descriptive Information
Resource
Geographical Area of Interest
Activity Log
Workflow
Job Workflow
• Ensure that work is done the right way!
Concurrent Steps
Looping Workflows
Start other processes
Email Stakeholders
Decision Points
Real-world Workflow
Notifications
• Email notifications on - Job action - Workflow step - GIS data edits
• Fully configurable
Manager
GIS Analysts
Data Collection
Data Processing
Edit Data and Map
Perform Quality Check
Create the Final Output
Provide Project Status
• How long does the average project take to complete?
• What jobs are overdue?
• What areas needs more resources dedicated to them?
Tracking and Reporting Enforces Accountability
• What happened on this job? - Who did what? - When did it happen?
• Central information repository
Process Improvement
• Automate bottlenecks • Reallocate resources • Improve QA process
System Architecture
Workflow Manager System Framework
…Windows only
Workflow Manager
Geodatabase
Desktop
Administrator
Application
…Standard edition or higher
Server Web applications
SOAP/REST API
Why ArcGIS Workflow Manager Server?
Non-GIS Users • Customers • Admin staff
GIS Users • GIS analyst • QA • Data stewards
Applications • Maximo, SAP • In-house apps • SOA
GIS Department
Business Systems
Stake-holders
ArcGIS Workflow Manager Server
• Types of Services - Feature service - Geocode service - Geoprocessing service - Workflow Manager service - Etc.
ArcGIS Workflow Manager Services
SOAP - WSDL is available at
- http://<WebServer>:6080/<ArcGIS Instance>/services/<ServiceName>/WMServer?wsdl
REST - ArcGIS Server : http://<WebServer>:6080/<ArcGIS
Instance>/rest/services/<ServiceName>/WMServer
ArcGIS Workflow Manager Server Architecture
ArcGIS Server
Workflow Manager Server
Workflow Manager Server Applications
SOAP REST
Extending Workflow Manager
How can I extend Workflow Manager?
Desktop Server
• Commands • Toolbars • Tabs • Views • Applications
Workflows and External system integration
• Web Apps
Supported Development Platforms
• Custom Steps - .NET (C##, VB .NET) - C++ - Java
• Web Application - Flex, JavaScript, Silverlight, ASP .NET
Overview of ArcGIS Data Reviewer
Section Topics
• Importance of Data Quality
• What is ArcGIS Data Reviewer
• ArcGIS Data Reviewer for Server
• Enabling Data Quality-focused Workflows
Data Quality Matters
• Do you publish a doc without spell checking? • Do you operate a PC without a virus scanner?
• Cost of poor quality can be extreme
- $$ to fix - Impacts users - Reputation
What is ArcGIS Data Reviewer? Data quality management for ArcGIS
• Provides - Rule based workflows - Interactive tools - Track errors
• For individuals and enterprise - Saves time/money - Less rework
• Standard extension to ArcGIS - Desktop extension since 8.0 - Server extension since 10.1
Trusted data through improved quality management
Enterprise Workflow
Review
Correct Verify
REVIEW
Find & Record Errors
CORRECT
Perform Edits or
Note Exceptions
VERIFY (optional)
Acceptable or
Unacceptable
Managing Quality Control
• Framework for managing QC lifecycle
A
Support for the Complete System Make quality automatic and transparent
• Flexible automation of quality control for the organization
• Enable quality review and feedback from web clients
• Provide open access to data quality information
Cloud
Enterprise
Local
ArcGIS Data Reviewer Mobile
Desktop
Web
ArcGIS Data Reviewer Overview QC Workflows at 10.1
A
QC Workflows at 10
Visual QC
Automated Validation
DQ Reporting
Data Workspace(s)
Reviewer Workspace
Dashboard Reporting
Visual QC
Automated Validation
DQ Reporting Connected
Users
Server-based Adhoc/Scheduled
Data Workspace(s)
Reviewer Workspace
ArcGIS Data Reviewer for Server Capabilities
Rule-based Automated Data Review
• Over 40 data checks • Grouped into 11 categories • Configurable • Additive/Scriptable
www.esri.com/datareviewer
Batch Validation via the Web Leveraging servers for automated validation
• Automated quality control for the organization
- Scheduled and on-demand data validation
- Frees data editors from running checks
- Utilize servers for faster validation processing
Quality Feedback via the Web Expanding the dialog on quality data management
• Extending quality control workflows into other communities
- QC review across ArcGIS platform
- Simple to use tools for error identification
- Deployed via intranet or Internet
Manual quality control Expanding the dialog on data quality
Data Maintainers
A
Internal Feedback
• Consumers • Stakeholders • Volunteers
External Feedback
Internal Feedback
Dashboard Reporting Enabling transparency in data quality
• Better decision making by communicating data quality across stakeholders - Open quality reporting
- Shared across ArcGIS
system
- New tools and methods to communicate quality
DRS System Architecture
• Server Components - Server Object Extensions (SOE) - Geoprocessing (GP) Server - Accessible through REST API
• Web Components - Flex API - JavaScript API - Flex/JS Web applications
A
Dashboard
Flex Web App Flex Web App Flex/JS
Web Apps
Batch Validation
Custom Applications
REST API
Custom Flex/JS Applications
Flex/JS API
Workspace Browser
Enabling Data Quality-focused Workflows
Linking workflows with validation ArcGIS Data Reviewer for Desktop Integration
• Create Reviewer Session - Reviewer Workspace
• Execute Batch Job
- Reviewer Batch Job - Validation Area - Data Workspace - Reviewer Workspace - Reviewer Session Name
• Launch ArcMap - Reviewer Session
Linking workflows with validation ArcGIS Data Reviewer for Server Integration
• Server-based workflow integration emerging
• Future integration options: - Server-compatible WMX custom steps - Geoprocessing models/scripts - WMX Execute URL step
Demonstration: Enabling Data Quality-focused
Workflows
Scenario
• As a Contracting Officer I need to assess the quality of city-provided data before I can provide it to outside consultants for construction estimates.
• Acceptance Criteria: - Enable non-GIS user to identify areas where data is
required to support contractor bids - Automate independent review of data quality - Report errors detected in area of interest
Resources & Training
• Resource Center (resourcesbeta.arcgis.com) - Help and Documentation - Gallery (samples, tutorials, templates) - Forums - Blog
• Training (training.esri.com)
- Instructor Led Training - Web Courses - Training Seminars
Questions?
Please fill out the surveys on your way out.