Post on 14-Apr-2017
Performance and Scalability Optimizations - Xerox Government Healthcare - Health Enterprise Portal Gorkey Vemulapalli, Senior Architect, Xerox; Thomas Hurek, Senior Digital Experience Architect, IBM
Agenda
• About Xerox • Medicaid Management Information System (MMIS) • Architecture • Performance Optimization • AppDynamics
Copyright © 2015 AppDynamics. All rights reserved. 2
Xerox
• Founded 1906 • 140,000+ employees • Active in 190+ countries • 2014 Revenue: $19.5 billion • For more than a half a century, Xerox has been a leader in document
technology and services
Copyright © 2015 AppDynamics. All rights reserved. 4
Solution
• Preferred Medicaid Management Information Systems provider
• End-to-end healthcare program administration services and solutions
• Automated solution for complex data intensive deeply regulated market
• Interfaces with existing systems • Input from web, phone, batch jobs, programmatic
Copyright © 2015 AppDynamics. All rights reserved. 8
Features
• Service Oriented Architecture platform fully aligns with MITA interoperability guidelines and web-based technologies
• Reusable, adaptable core provides common starting point – Reduces implementation time & costs
• Fully aligned with the MITA 3.0, MECT & the Seven Conditions and Standards from the first day of service
• Flexible architecture enables edits and updates to be implemented in days or hours – not weeks or months
• More than 40 years experience with Medicaid and other publicly funded health programs
Copyright © 2015 AppDynamics. All rights reserved. 9
Use Cases
• Providers enter claims and other information • End users can check health care coverage and claims • State employees can approve / deny / administer • Third Party Integration • Batch processing
Copyright © 2015 AppDynamics. All rights reserved. 10
Presentation Layer
Copyright © 2015 AppDynamics. All rights reserved. 13
• Processes the presentation and information dissemination for users
• Web Portal provides the information via a web browser.
• AVRS allows providers and members access to information by using a touch-tone telephone.
• EDI provides connectivity for the flow of data between providers, state agencies & switch vendors
Business Process Layer
Copyright © 2015 AppDynamics. All rights reserved. 14
• Encapsulates high-level business processes exposed as service for consumers
• Uses well-defined execution of services and components
Services Layer
Copyright © 2015 AppDynamics. All rights reserved. 15
• Composite services – Combines granular business
services to reduce some complexities and to reduce coding efforts.
• Business Service Layer – Provides specific business
services which are highly cohesive and atomic functionality
• Infrastructure Service Layer – Supports systems integration
services that are necessary to build a web application
HE System Interaction – Claims Processing
Copyright © 2015 AppDynamics. All rights reserved. 16
PRINTER
SOURCE HEALTH ENTERPRISE ECHO-‐SYSTEM OUTPUT
Call Center
CLAIMS
Flat File
ELIGIBILITY
Paper Claims
PROVIDER
INBOUND
FINANCE
PRAProvider
270271 Flat Files
837
OUTBOUND
Facility
TPs
Member
OCR
CLAIMS
ELIGIBILITY
PROVIDER
FINANCE
B2B
ONLINE
NON-‐EDI BATCH INTERFACES
CONTACT
GLOBAL
SERVICE AUTHORIZATION
PROGRAM
Provider
Facility
TPs
Member
CMS
Bank
NON-‐EDI BATCH INTERFACEAdjudication
ORR
B2B
ELIGIBILITY
CLAIMS
PROVIDERS
Payments
ELIGIBILITY
CLAIMS
PROVIDERS
EFT
EOB
ELIGIBILITY
CLAIMS
PROVIDERS
FINANCE...etc
835
Goals
• Throughput • Response time for different use cases • Batch processing throughput and response time
Copyright © 2015 AppDynamics. All rights reserved. 18
Implementation
• Agile process • Rational Performance Tester for recording & playback • Analysis of performance issues and bottlenecks with AppDynamics software • Tuning and code adjustment • Repeat
Copyright © 2015 AppDynamics. All rights reserved. 19
Major Improvement Items
• Heap size tuning • Connection pool tuning • CPU and RAM assignments • Virtualization changes • Portal 8.0 -> 8.5 • WSRP updates as part of Portal upgrade • Theme optimization • WCM caching
Copyright © 2015 AppDynamics. All rights reserved. 20
Major Improvement Items
• WSRP logout (now part of Portal 8.5 CF6) • Database indexing / adding of new indexes • Thread Pool Tuning
– LDAP – WebSeal – ORB – WSRP
• Increasing number of JVMs where needed for memory reasons
Copyright © 2015 AppDynamics. All rights reserved. 21
Custom Code Adjustments
• Reduction of redundant calls • Optimizing SQL statements • Co-locating BPM and Business Layer • Making calls asynchronous • Reduction of logging • Fixes in WAS, WSRP, JSF, Hibernate, Log4j
Copyright © 2015 AppDynamics. All rights reserved. 22
Application Performance Management
• Auto-discover and monitor end-to-end application performance • Monitor any production app within minutes • Monitor production apps at code-level depth with minimal overhead • Resolve application issues quickly for exceptional end-user experience • Eliminate false alarms with automated baselining • Broadest coverage of languages and technologies • Deploy On-premises, in the cloud, or in hybrid environments
Copyright © 2015 AppDynamics. All rights reserved. 24
Why use at Xerox?
• Easy to setup • Allows code level analysis without large overhead / complicated
instrumentation • Follows transactions through a complicated set of transactions on various
layers – even using different technologies • History based data can be analyzed later – very helpful in multi-shore
development • Agile
Copyright © 2015 AppDynamics. All rights reserved. 25
Why use at Xerox?
• Production Monitoring and Alerting with very low overhead • Flexible views and dashboards • Dynamic detection of different transactions – via URL patterns • Browser based • Delegated Administration
Copyright © 2015 AppDynamics. All rights reserved. 26
Adjustments
• Learning of URL patterns • Database extensions / detail levels for slow SQLs • Need to monitor physical CPUs vs. virtual • Did not catch vmotion performance issues • PMI data is available
Copyright © 2015 AppDynamics. All rights reserved. 27