ex ecutive white paper
Transcript of ex ecutive white paper
®
Best Practices in Siebel CRMPerformance Management:
Monitor, Measure and Manage the End User Experience
exec
utiv
ewh
itepa
per
2Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Contents
Executive SummaryWhy End User Performance Management is needed to optimize Siebel CRM apps
ROI Isn’t Real Until It’s RealizedThe ROI impact of issues with adoption and efficient and effective use
Traditional Performance Management Approaches Aren’t EnoughMonitoring system and application health is no longer enough for service of business apps
Overview of Knoa Experience and Performance ManagerA review of the architecture and functionality of Knoa EPM
Knoa EPM Use CasesKnoa EPM delivers results to the entire application team. This sectiondetails use case examples compiled from Knoa’s customer base
You Can Manage What You Measure The impact that real end user experience and performance metricscan have on the application management team
Could You Benefit From Knoa EPM?Four questions to help you evaluate the potential value of Knoa EPMfor your Siebel deployment
Knoa® Software is
leading the performance
management industry
with end user experience
monitoring solutions
that reach beyond
infrastructure and
application performance
to the monitor, measure
and manage how end
users are utilizing the
application to optimize
business process
execution.
3
5
6
7
9
14
15
3Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Executive Summary
CRM solutions, like Siebel, automate complex business processes where highlevels of application performance are crucial. Yet many frustrated execs findthat despite the small fortune they have spent on products to manage servers,networks and applications, the business constituencies still complain about theperformance, usability and availability of the Siebel CRM application.
What is often overlooked is that application performance is just one of three keydrivers of return on investment (ROI) from your Siebel application. An efficientinfrastructure is critical. It’s important to optimize the Siebel application for yourown unique business environment, user needs and usage patterns. But, it’s allfor naught if you do not build and sustain adoption and efficient and effectiveuse of the Siebel application by your end users.
How can you tell how if your Siebel solution performs for each person who uses it? How can you identify which users have difficulty executing which transactions? What does it mean when your end users say the system is “slow”?How can you pinpoint the actual issues with the real data you need to take corrective action? Knoa can give you the answers.
Enterprises with mission critical Siebel deployments do not have accurate measurements of adoption, efficient use, and effective use of the application by the end users. Without these measurements, companies are unable to identify andresolve critical issues that impact the user’s ability to optimize business processexecution. The result is significantly reduced ROI, and significantly increased costof application support.
Knoa® Software is leading the performance management industry with end user experience management solutions that reach beyond infrastructure andapplication performance to monitor, measure and manage how end users areutilizing the application. Knoa customers not only know if core enterprise applications are delivering an acceptable user experience, but also, if application users are executing key processes effectively and efficiently – the key to achieving business value and ROI.
“An end user performance
management package
from Knoa Software is
capturing the CIO’s
attention for its ability to
boost worker productivity
and optimize application
performance. Knoa’s
Experience and
Performance
Management suite lets
IT departments measure
response times, distinguish
between system and user
error and monitor work
flows – all in real time…”
– Brian GilloolyOptimize IT
4Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Knoa Experience and Performance Manager (EPM) monitors enterprise applications from the perspective of the end user to capture the actual end userexperience. Knoa EPM captures transaction response times as well as ecosystemand application errors – along with a time-synchronized workflow – from allusers, at all locations, at all times, across the entire application. This informationis used for alerting and proactive remediation of end user issues (whether theyare system errors, user errors or performance problems). Knoa EPM also analyzes and aggregates information to provide actionable insight into processcompliance, training requirements, adoption issues and a number of other management dimensions.
Knoa EPM provides a complete picture of end user experience by focusing on both the experience the end user receives from the application and the performance the end user achieves working with the application. You can thinkof Knoa EPM as the “third leg” to the performance management stool – infrastructure, applications, and now, end user metrics. Think about it! The infrastructure is monitored and measured and optimized; application executionacross that infrastructure is monitored and measured. In a recent study conducted by Forrester Consulting an overwhelming 87 percent of IT executiveseither strongly agreed (50%) or agreed (37%) that the best way to manage theservice of business critical applications is by measuring performance at the enduser level and rapidly resolving significant performance problems.
Clearly, there’s a need for a change. In fact, the Forrester Consulting reportfound that change is imminent. Sixty seven percent of the surveyed organizations are either much more focused (27%) or more focused (40%) on the end user experience than they were two years earlier.
Forrester concluded that best practices companies are intent on collecting metrics on the end user experience.
For all the investment
in application
management solutions,
nearly two-thirds of
companies (66 percent)
rely upon help desk calls
as the main indicator
that end users are
experiencing response
time and/or availability
problems.
Forrester Consulting
5Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
ROI Isn’t Real Until It’s Realized
The initial investment in CRM applications is high. Software licensing costs canrange from $1,000 to $2,800 per end user. Implementation services are 2-3times the software costs and there is investment in back-end infrastructure.Ongoing costs are high, as well. Software maintenance averages 20%, annually.Gartner reports that 60% of ongoing costs of a typical deployment are for support staff. ROI is only realized from this investment if the end users adoptthe application and use it efficiently and effectively to improve the execution of core business processes – which is very difficult to achieve.
Consider this all too common scenario that afflicts many CRM deployments.Months after deployment a high percentage of the functionality is not beingused. End users complain that the application is “slow” and “doesn’t work”.Some users find the software cumbersome to adopt into their routines and create their own non-compliant work-arounds. Employee turnover slowlyerodes the user base proficiency.
Flying BlindHere’s how one of Knoa’s customers described the problem that led them todeploy Knoa EPM.
“User adoption of our CRM application has been stalled by a number of factorsaffecting field productivity. There are complaints about poor response time, remoteusers can’t access the system, and it’s not clear if there are system problems or if they are just using the wrong procedure. And across the board there are complaints that things take “twice as long” with the new system. In the absence ofcomprehensive and representative data, many discussions regarding applicationperformance are based on reported user impressions, limited visual observations,and occasional stopwatch timings. This has resulted in extrapolations about theperformance of the whole system that are inaccurate, and lead to decisions thatare not in the best interest of the business.”
Enterprises with mission critical CRM deployments do not have accurate measurements of adoption, efficient use, and effective use of the application bythe end users. Without these measurements, companies are unable to identify,and resolve critical issues that impact the user’s ability to effectively execute.The result is significantly reduced ROI, and significantly increased cost of application support.
One Version of the Truth.
Knoa EPM gives the
enterprise real metrics for
adoption, efficient and
effective use of the
application. With end user
metrics, companies are
able to identify and
resolve critical issues that
impact the users’ ability
to effectively execute, and
hamper ROI realization.
6Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Traditional Approaches to Performance Management Aren’t Enough Anymore
CRM applications like Siebel are complex. It’s the flip-side of the flexibility they provide to meet a wide variety of customerneeds. Each Siebel customer has their unique business environment – user needs, usage patterns, customizations, systemcomponents, architectures and platform environments vary widely. In the face of this complexity, it is becoming increasinglydifficult for measurement and metrics taken at the infrastructure level (system, network or transaction management) to providea true perspective of the end user experience with the application.
In a recent study conducted by Forrester Consulting an overwhelming 87 percent of IT executives either strongly agreed(50%) or agreed (37%) that the best way to manage the service of business critical applications is by measuring performanceat the end user level and rapidly resolving significant performance problems.
Knoa believes that the application is “not performing” – from the end user’sperspective – whenever the end user is unable to effectively execute the keyprocesses required for their role. In our vision, application performance, fromthe end user perspective includes:• Slow transactions• Cumbersome Navigation• System Errors• Non-intuitive user interface which causes excessive user errors
Knoa EPM goes beyond the response-time dimension to monitor and capturemetrics on the entire end user experience – including system and user errorsas well as the user navigation and interaction with the application (workflow).
The monitoring capabilities of traditional performance management products provide a significant amount of insight about application health (e.g. transactions that are failing; aggregate transaction response times, application availability). Knoa EPM extends performance management to a whole new level by monitoring and managing the “health” of the end-userpopulation. Has the application been adopted? Are the end users using the application proficiently. Is the application enablingthe end users to efficiently and effectively? After all, adoption and efficient and effective use are the keys to ROI from theapplication investment.
Snapshot of the End User Experience with online transactions(Source: USA Today, 2006)
Received Error Messages
Site Navigation Issues
Transaction Execution Issues
Login Issues
Content Issues
40%
37%
31%
31%
31%
7Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Knoa Experience and Performance Manager
Knoa EPM gives the Siebel support team a comprehensive "fact-based" perspective about the end user's experience with the application. Who's using it?What's the response time? Who's getting errors? How many transactions arebeing abandoned?
Knoa EPM is the only end user experience management solution that monitors the entire end user experience from the perspective of the end user. Knoa EPMmonitors the Siebel application from the perspective of the end user to capturethe actual end user experience, as well as ecosystem and application errors, and a time-synchronized workflow – from all users, at all locations, at all times.
Knoa EPM uniquely captures a complete picture of the end user experience and behavior:• Actual User-Experienced Response Time for key system transactions,
such as navigating between application screens, save operations, execute operations, etc.
• Complete Quality of Experience, including system and application errors and user created errors
• Comprehensive perspective of application utilization – which transactions are used, in what sequence, for how long, etc.
Knoa EPM is based upon a passive monitoring engine that is deployed to the client workstation along with a template specifically designed for the target application. This client engine monitors end users’ interaction with theapplication. Knoa EPM selectively captures the actual steps, and the sequenceof events in the user’s interaction with the application. In addition to this time-synchronized workflow, Knoa EPM captures a number of metrics around boththe application’s performance and the end user’s performance:• All user errors• All system errors• Transaction response times
This information, called track data, is transferred to the Knoa EPM analysis server which prepares a series of targeted analyses and reports. This information is viewed through the Knoa EPM Console, which presents a series of management dashboards and drill downs for analyzing and querying theperformance data.
“The combination of
the user-base coverage
and application depth
provided by this
approach is tough
to beat. With this
information, companies
can manage the
performance achieved
by end users, as well
as the experience that
they receive.”
– Bernd HarzogCEO, APM Experts
8Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
A Unique Architecture for End User ExperienceManagement
Knoa EPM is based upon a unique architecture that was designed, from theground up for user-centric monitoring and experience management. Key capabilities the architecture delivers are:
Automated DiscoveryThe ‘out of box’ Knoa EPM solution automatically ‘sees’ every Siebel transactionand the steps that the user is taking before, during and after the transaction.Knoa EPM’s self-discovery capabilities eliminate, completely, the need for customers to design a data collection strategy, define transactions to the systemor train the system to recognize new transactions. Automated Discovery means:• No up-front requirements definition or project scoping• Faster implementation using out-of-the-box templates• lower overall cost of ownership • Adaptable technology to easily support the most aggressive release/update
schedules
Global CoverageThe ‘out of box’ Knoa EPM solution gives you global coverage of the entireSiebel application, across all instances and systems in the field. This global coverage also gives you a comprehensive perspective on adoption and usageacross the entire user population.• Application usage• Transaction usage• Screen usage• Function usage• Active/idle time• User behavior
Non-Intrusive to Siebel ApplicationThe Knoa EPM system does not insert tags or instrumentation into the Siebel application, create any system events, or require any back-end processes that impact application performance. The Knoa components do not requireintegration with any components in the application landscape, whether front-end or back-end.
Small Footprint/Low ImpactKnoa EPM uses a Seibel-specific template to selectively record the significantuser events that will produce actionable insight, without any discernable system overhead or impact. It is this light operational footprint that enables you to monitor and measure the user experience across the entire applicationlandscape.
“The insight into actual
end user experience
gives us benefits across
the board, helping
target response time
issues, solve end user
problems faster, identify
usage and adoption
issues and even make
priority decisions on
ongoing application
investments.”
– Stuart Smith, Director, CRM Performance, British Telecom
9Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Knoa Use Case: Performance Management
Improve Siebel Application PerformanceExcessive application response time is one of the most critical performance issues encountered by the users of today’s complexmission critical applications. It’s also a huge support problem. With Knoa EPM you know the response time that was actually experienced by the end-user – for every transaction, for every user, every time.
A leading consumer packaged goods company had put millions of dollars over 3 years in a new Sales Force automation solution.The number one complaint from users was long response times for key functions, such as creating and saving CRM records. TheQA/Performance team was only able to act on these performance issues once they escalated to critical levels.
The company implemented the Knoa EPM system to detect performance fluctuations for key functions in real-time. Using an alert system that notified the IT team whenever response times in the field exceeded set tolerance thresholds, the team was able topreempt the escalation of performance problems in many cases. In addition, by being able to isolate performance problems tospecific areas of the application, the team also reduced the time to resolve critical issues.
See the forest and the trees. Knoa EPM delivers results to the entire Siebel support team. Here are some real use case examples submitted by Knoa customers:
10Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Knoa Use Case: Application Utilization
Link investment and effort to business impactWith Knoa EPM you gain valuable, actionable insights into application utilization: Who is using the application and for howlong? Which areas and functions are most used? Which areas and functions are least used? What usage patterns are creatingstrains on IT infrastructure? Knowing the usage profile for your Siebel application and your user base is the most fundamentalstep towards application and user performance management.
The IT Support and Development team of a leading global financial services company received requests from their field usersto implement a costly and extensive functionality enhancement for their enterprise sales application. The IT team dedicatedconsiderable resources to implement this request and launch the new capability.
After the enhancement was implemented, using the Knoa application usage data, the IT Team discovered that few employeesactually used the new functionality. Based on this fact, and a detailed review of Business Operations based on Knoa data, theymade a decision to discontinue support. Eliminating the module from production saved the organization tens of thousands ofdollars in development time and resources.
11Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Knoa Use Case: End User Proficiency
Drive efficient and effective use by managing errorsWith Knoa EPM solution you can identify 100% of all user-experienced errors, whether they are caused by user actions or byany of the application layers. Knoa EPM also gives you the context in which each error was encountered and the path that theuser took before encountering the error. This information is available in real-time for the end user support team.
By implementing Knoa EPM, the IT department of a leading computer services company realized that a large percentage ofthe errors encountered by users were never reported to the Support Desk. In particular, a single error type, generated whileexecuting searches, accounted for 25% of all errors generated by the company’s CRM application.
The productivity losses and infrastructure impact of repeated and failed queries executed by users amounted to thousands of dollars each day.
Based on the Knoa data, the IT department promptly identified the issue, which was then addressed with a two-fold strategy: 1) provide targeted user training in order to avoid the error, in the short run, and 2) re-engineer the search functionality of theapplication to eliminate the error altogether, in the long run.
12Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Knoa Use Case: End User Support
Accelerate Problem Resolution with end user metrics and workflow Knoa EPM captures a complete record of end user behavior, before, during and after the transaction. This end user workflow datatransforms the end user support process.
A major electronics company was well into their deployment of a CRM solution. An inside sales reps in the Raleigh NC call centerwas experiencing problems. The information being communicated to the support team from the help desk was very sketchy andincomplete. In the past the CRM support team would spend hours on the phone with the inside rep, and often had to make a cost-ly on-site support call to deal with issues like this. However, Knoa EPM enabled a dramatically different approach. Through themanagement console, they reviewed this ISRs’ recorded experience with the application. First the team looked at the metrics andfound that the response time she was experiencing was fine – well within the sub-3 second goal. Next they found that she wasgenerating four times more user errors than her peers. Investing further, by looking at a recorded record of her actual interactionwith the application, they found she was using an incorrect process to validate the customer information (a common process stepin many different transaction types). With this insight, her supervisor was able to instruct her in the proper customer validationprocess and “the problems with the system” were quickly resolved.
13Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Knoa Use Case: Application Support/Engineering
Application Support/Engineering:Key Performance Indicators (KPI) that report user errors on an aggregate level were useful in resolving another vexing issuefor that electronics provider. There were grumblings in the outside sales force that the opportunity management screen wasreally difficult, slow and cumbersome.
When the KPI information for errors was reviewed, the application team noticed that virtually every end user was making thesame errors on these screens. The first instinct was to schedule an e-learning retraining session. However, a closer review of therecorded work flow for a representative sample of users revealed that the errors were being caused when the sales reps weretrying to put data in extraneous fields that were on the screen but not required for the transaction to execute. The fix was simple. No retraining was needed. The application team removed those fields from the screen and the errors were eliminated.The end-to-end transaction time improved by 25% as well.
14Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
You Can’t Manage What You Can’t Measure.
It’s rule #1 in the management handbook, isn’t it? And the rule that is the drivingforce behind the investment in CRM applications in the first place. But, without astrategy to measure end user adoption, and without a strategy to gather insightabout end user’s experience and actual performance with the application, manyIT organizations are left with the reality that they can’t manage and improve enduser adoption and build end user proficiency.
Knoa Experience and Performance Manager accurately determines how eachemployee actually uses the application and provides input to comprehensivereports that you can use to pinpoint training needs, adjust business processes,and ensure that your end users are using the application to deliver all the valueyou expect from the application. As an added benefit, Knoa also measures andreports critical information about how your users experience the application’sactual performance.
• Application Support teams have real-time metrics to proactively addressresponse time issues problems
• Help Desk personnel have immediate visibility user into actual user interaction with the application, so they can quickly resolve issues with minimal disruption to the end users
• Training organizations can pinpoint which specific application areas (andwhich specific users) require additional training
• Application and Process Engineering can spot cumbersome work flows anderrors that are affecting end user satisfaction and efficiency
• The CIO has comprehensive metrics from the entire application landscape to support reporting to the business on SLAs and insight into applicationusage and performance to make intelligent decisions about where to deployresources to improve the application performance and effectiveness
• The business executives have actionable insight to drive the highest possiblelevels of adoption, efficient and effective use of the application which is thekey to a business return on the investment
Questions Knoa Answersfor British Telecom
Is the application performing effectively?
Are end users able to effectively and efficiently execute critical business processes – in otherwords, has the trainingprogram led to realresults?
Can we provide objective,real-time data on theuser experience to allowthe help desk and keyusers to resolve user’squestions rapidly?
Can we maintain an electronic audit trail thattracks the completion ofkey transactions?
What is the impact ofineffective procedures on operations?
15Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Can Knoa Experience And PerformanceManagement System Help You?
Are you certain that all of your end users are actually using the functionsof the application required to execute their role?Knoa EPM captures and reports real metrics about application usage. Who isusing which transactions? How long are they active? How long are they idle?Knoa EPM gives you the ability to pinpoint and analyze user adoption issuesthat are hindering the achievement of business results.
Do your end users ever use system performance as a key reason why critical objectives are not met on time and on budget?One of the benefits of Knoa EPM is that it gives you the ability to correlate business outcomes with efficient, effective application usage. Knoa EPM systemcaptures and reports real metrics about the end users experience with the application, you will now know if the users were experiencing long transactionresponse times or if there were a lot of system errors presented to the end user.You will also know if the end users themselves were causing a lot of errors by their own behavior. You will then know where to apply corrective action.Perhaps it's a user training issue, or perhaps the errors are being caused by acumbersome or confusing user interface that can be corrected.
Do you suspect that there are a lot of problems and issues that are neversurfaced to the help desk? Well, you’re probably right. It is often reported in surveys of the service management process that 70-80% of all end user problems are not reportedbecause the end users ignore the errors, abandon the transaction and/or findworkarounds. In fact, Forrester Consulting recently reported that an end usertypically experiences a problem 6 times before they call the help desk. KnoaEPM gives you the ability to pinpoint and proactively remediate problems whichimpact the efficiency or the effectiveness of the end user population using theapplication to accomplish business goals. An EPM system will make sure youhave visibility into all errors – for all users, in all locations, all the time.
When your end users do report problems to the help desk, do you find they are wasting a lot of time trying to answer questions and document orrecreate the problems?Knoa EPM captures the actual work flow that the user experienced – before,during and after the transaction. This information is immediately available tohelp desk personnel so there is minimal business disruption from issues with theapplication. Employee productivity impact is kept to a minimum when there isan issue (users spend less time on the phone with help desk and problems areresolved faster).
Sixty seven percent of
surveyed organizations
are either much more
focused (27%) or more
focused (40%) on the end
user experience than they
were two years earlier.
Forrester Consulting2006
16Best Practices in Siebel CRM Performance Management: Monitor, Measure and Manage the End User Experience
Expertise and Performance Management systems are becoming a priority forthe business executive and CIO who need to drive the highest possible levels ofadoption and efficient and effective use of core enterprise applications to drivea business return on investment. Only with the insight into end user metrics that an EPM system can provide can you really know if your core enterpriseapplications are delivering an acceptable user experience and if applicationusers have adopted the applications and are executing key processes effectivelyand efficiently – the key to achieving business value and ROI from the application investment.
5 Union Square West4th FloorNew York, NY 10003
212-807-9608
www.knoa.com
Training
Support
Business
IT
About Knoa
Knoa® Software is leading the
performance management
industry with end user experience
monitoring solutions that reach
beyond infrastructure and
application performance to the
monitor, measure and manage
how end users are utilizing the
application to optimize business
process execution. Knoa
customers not only know if
core enterprise applications
are delivering an acceptable
user experience, but also, if
application users are executing
key processes effectively and
efficiently – the key to achieving
business value and ROI. Knoa is
headquartered in New York, NY.
For more information about
Knoa, please visit www.knoa.com.
Copyright, 2007 Knoa SoftwareAll rights reserved.