Post on 01-Oct-2020
Whitepaper
Evidant Inc., www.evidant.com (949) 609-1494
Performance Management for Call Centers
The effort around assuring that the call center
applications satisfy availability and performance
requirements are typically not considered or are
relegated to traditional IT monitoring.
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
1 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
Preface
There has been an explosion in the
channels customers can use to interface with
organizations today. The traditional channel is
the Call Center, but now includes the vast array
of internet services - such as email, blogs,
social media (Twitter, Facebook, et. al.),
websites (corporate and agents), mobile
applications. The increase in the number of
self-service channels is an enabler to meet
market forces and customer expectations while
allowing companies to reduce the costs
associated with servicing these needs. These
self-service channels, however, place greater
pressure on service delivery by the call center,
as customers who have not been able to
accomplish a task via a self-service channel are
now potentially frustrated. These frustrated customers will then reach out to the call center agent
for problem resolution, making exceptional performance delivery critical. This challenge is
further complicated by increasingly complex business processes, agent turnover and the ever
driving need to reduce costs. This leaves call center executives with three “levers” to work with:
• People: The challenge of hiring, training and retaining effective agents.
• Process: The challenge of utilizing consistent processes across the call center – BPM and
CRM applications have helped tremendously in this area, but place a greater burden on IT
performance being linked to call center performance.
• Technology: A lot of money is spent here supporting the other to ‘levers’. However, it is
an area that business operations executives lack sufficient metrics to understand and
manage the impact on productivity.
This last element is typically only thought about when call center agents are complaining, and
if not managed well can result in increased AHT, missed ASA service levels, frustrated agents, and
can ultimately lead to poor customer satisfaction. Everyone has heard a call center agent utter
the words, “my system is a little slow today” yet business operations managers typically have little
insight into this issue until agents complain en masse. This typically leads to call center agents
developing their own ‘work-arounds’ (something we have seen in a majority of projects we have
managed), masking the impact that system performance is having on their business processes. If
this happens enough it becomes and organizational truism where, effectively, call center agents
“over-complain” about technology, describing personally frustrating issues as having a higher
impact than they actually do, leading management to believe IT performance is causing Service
Levels to be missed or ACH times to increase.
This paper discusses the challenge of how to effectively identify and monitor
performance issues using valid, accurate methodologies.
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
2 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
Table of Contents
PREFACE ....................................................................................................................................... 1
TABLE OF CONTENTS ......................................................................................................................2
COMMON PROBLEMS ......................................................................................... 3
APPLICATION PERFORMANCE IS NOT ENOUGH OF A FOCUS .............................................................. 3
CUSTOMER SERVICE APPLICATIONS UTILIZE MULTIPLE “BACKEND” SYSTEMS .................................. 3
DEPLOYING MONITORING BUT NOT CAPTURING THE CALL CENTER AGENT EXPERIENCE .................4
FAILING TO USE APM TOOLS STRATEGICALLY ................................................................................4
THE SOLUTION ................................................................................................... 5
CAPTURING THE MOST IMPORTANT CALL CENTER AGENT TRANSACTIONS ....................................... 5
CAPTURING THE “REAL” CALL CENTER AGENT EXPERIENCE ......................................................... 6
ENABLING DASHBOARD VISUALIZATION OF THE CALL CENTER AGENT EXPERIENCE ......................... 7
Visualizations ......................................................................................................................... 8
BUILD A STRATEGY & ACTION PLAN ............................................................................................... 9
Establish an End-User Experience Team............................................................................. 10
On-Going Operations ............................................................................................................ 10
CONCLUSION ..................................................................................................... 12
ABOUT EVIDANT ............................................................................................... 13
Evidant helps to align business and its technology partners
throughout an organization. This is executed by developing and
implementing a metrics and analysis strategy that utilizes
business context to exemplify application performance.
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
3 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
COMMON PROBLEMS
Application Performance is Not Enough of a Focus
Understanding the effect of application performance on agent
effectiveness is often overlooked when it comes to deploying or
upgrading those applications. Typically, the greater proportion of effort
is spent on business process flow and application usability. These
elements are critical to help assure high compliance with expected
business processes and to help reduce new hire training costs. The
effort around assuring that the customer service application satisfies
availability and performance requirements are typically not considered
until after they are already a problem or are relegated to traditional IT
monitoring which, by its nature, does not take into account user
performance requirements.
By ignoring the impact of application performance upfront, hoped-for workflow improvements
can be negated by users creating ‘work-arounds’ for the screens that are slow. Or worse, it
provides leverage for the seasoned reps to have disdain for the new application they are not used
to and continue using the legacy application that they are more comfortable with.
Customer Service Applications Utilize Multiple “Backend” Systems
The trend toward reducing the number of applications that call center agent’s need to learn and
interface with has clear savings in terms of reducing new hire training time and instances when
agents provide unnecessary or inaccurate information, and improving business process
compliance and AHT. The problem is the “core systems” (usually legacy systems) behind the
application were not typically architected with a BPM or CRM application as a consideration,
resulting in these applications being “bolted on” to the existing platforms and infrastructure. It is
challenging to assure that these “core
systems” and the customer service
application work well together to provide a
seamless environment for the agent.
Poor performance or availability of these
applications can have an immediate impact
on AHT and reduction in service levels. This
impacts customer satisfaction scores, risks
negative feedback on social networking sites
and potentially results in the loss of existing
and potential customers. Internally it can
affect agent moral and turnover rates,
particularly if remuneration is tied to
performance.
“Traditional IT
monitoring often
does not account
for end-user
experience.”
The typical IT infrastructure monitoring fails to identify the impact on call center agents.
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
4 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
Deploying Monitoring but NOT Capturing the Call Center Agent
Experience
IT organizations will typically rely on traditional server-based and/or network-based monitoring
approaches to determine whether the customer service application is performing well. These
approaches, while important, only provide a marginal indication of application performance from
the perspective of the call center agent.
Call center agents are managed, and their performance judged, by a large number of metrics. If
the systems supporting the agent are not performing acceptably, and the organization does not
respond to remedy this, it is natural for the agent to find work-arounds to maintain their own
performance. We have seen organizations with 1 official way to handle an incoming call, and 15
ways agents actually handle the call (one for each team!). The disconnect between system
performance and the standards that agents are measured to will drive this behavior.
To truly understand and quantify the agent experience, a strategy around monitoring this
experience itself must be developed. This means measuring the application from the end-
user’s perspective.
Although this last sentence seems simple it is often misunderstood. To truly capture the end-
user’s experience it is important to satisfy the following criteria:
1. The response time measurements are taken as close to where the work is done. Active or
passive monitoring (discussed more in the solutions section) ideally should reside on an
agent workstation or as close as possible. Metrics that are “representative” or “close” to what
is really occurring should be avoided, unless the network conditions are well understood.
2. An end-user transaction is not equivalent to either a server transaction, an application
transaction, a database query or the time it takes to accomplish a business activity (e.g.,
handling, beginning-to-end, a customer call). An example of an end-user transaction is a call
center agent looking up a client’s information. The transaction begins when the agent clicks
“go”, after entering the client ID, and ends when the screen has updated with the client’s
information – aka “click to glass”. This single end-user transaction will potentially have
multiple IT-transactions associated with it.
In all cases the captured end-user metrics MUST be validated by testing – this means hand timing
individual transactions to validate they match what the tool is reporting.
Failing to Use APM Tools Strategically
There are numerous Application Performance Monitoring vendors that
provide capabilities for measuring end-user performance and
availability. However, the tools are not the issue. The challenge is
having a strategy that drives the action regarding end-user
experiences that can negatively impact call center performance.
“Tools do not
deliver the value –
using the tools
delivers the value.”
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
5 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
THE SOLUTION
Solving these issues requires a collaborative effort between both business and IT partners. In
order to implement a sustainable application performance management strategy the organization
should include the following key components in the plan;
Capture the call center agent experience – monitoring both the transactions that are most
important AND in a way that most closely represents it
Enable reporting visualizations of this experience
Establish an End-User Experience Team to understand, implement and track the end-user
experience needs of business and establish programs to take action when IT performance
negatively impacts the end-users.
Make sure that business leaders responsible for managing the end-users are bought into the
strategy and process. This way they become supporters of IT in these initiatives.
Keep the costs within reason, and proportionate to the business risks identified.
Implementing a strategy this way aligns application performance management with the call
center agent performance requirements (AHT, etc.), identifies how reliant the call center business
objectives are on application performance, and gives your IT partners objective data to direct
their efforts in rectifying identified problems.
An End-User Experience Team would be responsible for developing and delivering this strategy
such that it works closely with business to represent the needs, not “wants”, of end users. We’ve
seen companies attempt to manage this strategy by cobbling together people in a part-time role
(in addition to their full-time job) versus assigning a single team to focus on this effort. In larger,
or competitive, organizations application performance management is too
important not to give it a strong, defined, focus and executive support.
Capturing The Most Important Call Center Agent Transactions
Determining what to monitor is as critical as determining how to monitor
it. Identifying the specific end-user transactions that should be monitored
requires a consistent process based on what the call center agents do most
frequently (i.e., have the greatest impact on call center performance).
Often organizations will ask the application development team to identify
the transactions that should be monitored. Typically, they will select the
transactions that have received the most complaints or that “exercise” the
most backend systems – neither of which may have a real impact on the
call center. In other instances they will ask someone in the call center to
identify the “top 10” end-user transactions for monitoring – a better
approach, but can miss out on the most impactful transactions due to the
human nature of prioritizing by the most recent frustration, rather than
what is done most frequently.
The key to identifying which end-user transactions to monitor is to first identify the business
activities, or ‘call flows’, performed by the call center agents (e.g., member call to change their
address). This process leverages the BPM work already done in many call centers. The business
“Determining what
to monitor is as
critical as
determining how to
monitor it.”
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
6 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
activities should then be broken down into the individual tasks – typically these are associated
with an end-user transaction. A model of each business activity and associated time study should
be built to facilitate the service level needs of the call center agent by application transaction.
This methodical approach, while taking a bit more time initially, enables both IT and Business to
assure that the correct end-user transactions are monitored and what are reasonable service level
needs associated with them.
Capturing the “REAL” Call Center Agent Experience
The key to successful strategy development is capturing the End-User Experience as close to what
it looks and feels like from the call center agent’s point of view. In the world of application
monitoring, this effort centers around two basic methods: Active and Passive monitoring.
Active monitoring simulates what a call center agent would do in the customer service
application utilizing a “robot” to simulate the typical tasks that an agent would perform. This
“robot” goes through a predefined set of end-user transactions that simulate what a call center
agent would perform. Closely simulating the agent experience is critical in capturing whether the
end-to-end network and infrastructure is delivering a satisfactory experience that meets the needs
of the call center. This type of monitoring enables a baseline of the application
performance, because it uses the same set of transactions over and over again; and it also
establishes the availability of the customer service applications from a call center agent
perspective.
Passive monitoring captures the experience of actual users (it’s often referred to as ‘real-user
monitoring’). There are a range of technologies available to accomplish this. Some reside directly
on the end-users desktop; others reside on a network tap, close to the end-user desktop. The
advantages of passive monitoring over active monitoring is the ability to capture:
The application performance call center agents are actually experiencing
The relative volume of these transactions to get a true perspective on how many call center
agents are being impacted
Sensitive “update” transactions (e.g., “Pay a Bill”, “Submit a Claim”, etc.) that can be difficult
to monitor using active monitoring.
There are a range of tools vendors on the market today that can deliver one or both of the
solutions above. Selecting these solutions often becomes the key effort that an End-User
Experience Team will focus on. During this process, it is critical to remember that tools do not
deliver value; using the tools is what delivers value. Call center executives often incorrectly
believe that they have an end-user experience strategy once a “tools vendor” has been selected.
When in fact they should be asking, who will deploy and maintain the tool, how will the
organization be able to visualize what customers are experiencing, and most
importantly, how will the organization take action on this information.
(This paper does not attempt to compare and contrast the various active and passive tools
vendors. There are a number of providers and the selection process, although important,
is the least critical element of executing a well-developed strategy.)
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
7 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
Enabling Dashboard Visualization of the Call Center Agent Experience
The ability for an organization to visualize the end-user experience is the next important piece of
an end-user monitoring strategy. It clearly doesn’t make sense to capture the end-user experience
if it cannot be easily viewed and understood by both business operations and supporting IT
organizations. The following are some of the typical mistakes that organizations will make when
trying to provide customer visualizations:
• Too technically focused. Tools vendors provide technically rich visualizations, which can
be extremely valuable when doing root cause analysis; however, they are typically far too
busy to be valuable for obtaining a rapid understanding of the customer experience.
• Excessive use of “cool” dashboard
graphics. Tool vendors tend to promote a
range of dashboard elements to produce a
“cool looking”, but difficult to quickly
interpret, dashboard. This dashboard is an
example of one with multiple different
presentation elements. This requires the user
to understand and interpret the various
elements before they can be used to make a
decision. Simplicity, as with most things in
life, removes obfuscation (i.e., keep it simple).
The key to presenting a valid, valuable and
accurate view of the customer service
infrastructure is having a clear understanding of
the needs of the agent and then measuring against
those expectations. The following are some of the critical pieces to identify for call centers:
• The key tasks that call center agents perform (e.g. member lookup) and the specific screens
that the agent needs to walk through to satisfy the most important business activities
performed by agents.
• The performance needs of the call center agent for each screen, ideally characterized, in
seconds, into the following ranges:
Acceptable: “Good enough” to meet the NEEDS of the call center
Unacceptable: “Call center agents strained” but able to get work done if kept at
reasonable levels and/or for short periods of time
Critical: Slow enough that some call center agents are at increased risk of missing
AHT or service level targets
Unavailable: So slow that call center agents cannot reasonably handle customers.
Although IT organizations don’t view unavailability in this way, end-users definitely
perceive application availability this way (i.e., too slow to get the job done).
The “Too Cool” to Understand Dashboard
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
8 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
Visualizations
Visualizations typically center around two aspects: Dashboards and Reports. Although
different, there are common issues for both and should cover the following areas:
Be Accessible: The dashboards and reports should be made accessible to any associate who
has a business need to view and understand its contents.
Tailored for the Audience: The dashboards and reports should be tailored to the groups of
associates who will be using the information.
Enable Effective Decision-Making: The dashboards and reports should be easy to understand
and enable quick action. The dashboards should be relatively simple, enabling the consumer
to quickly understand the information presented (i.e., avoid complex presentations) and
present sufficient detail with enough depth so that the user is not mislead.
Represent Business Impact: The information presented in dashboards and reports should
indicate how application performance is impacting the company’s business operations. The
use of comparisons to “averages” should be avoided!
Include Training: Some basic training in end user metrics and business impact goes a long
way. Many times when reporting is first introduced, IT and business leaders overreact to a
situation that has existed for quite a while but didn’t know it existed until the metric exposed
it. This new information needs to be used in proper context to add value and be properly
prioritized into the overall goals of the organization.
Call Center Agent Experience Dashboard that BOTH IT and Business can Understand
Example of simplified visual: Level of Service Delivered, Avg Response, and Availability Delivered over a period of time
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
9 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
Build a Strategy & Action Plan
Once there is an understanding of the gaps in putting together an end-
user monitoring plan, the strategy should then focus on:
What applications and groups of users will be monitored? This
decision should be focused on those with the GREATEST
BUSINESS IMPACT. Business impact should not be confused with
number of users utilizing an application but the impact to the
business if those users are slowed down or cannot perform their job
functions in an acceptable manner.
Communication and interaction between IT and Business
organizations on service delivery
Establishment of an End-User Experience Team (discussed in
further detail below)
Here are some pitfalls that should be avoided:
Trying to do it all versus do what is most important. It is far better to focus on the
20% of tasks that business users perform 80% of the time than attempt to monitor 100% of
what they might do. The 80:20 Rule should drive this effort and will not only speed up the
process but requires less time and money to execute.
Trying to get it right the first time. Time to knowledge is critical and organizations will
often waste time in an attempt to perfect reporting, versus taking an educated guess and then
building off of it. It is not how much you know, but how soon you know it.
Reporting that is oriented toward “baselines” versus business needs. A common
mistake is to use previous application performance as the benchmark for what needs to be
delivered to call center agents. This approach, although valuable from a technical, or even
six-sigma, perspective misses the point. It is critical to understand the business needs of the
call center agents in achieving their AHT objectives, not “what are they are used to getting” or
even “what do they want”. Performance that fails to meet call center agent needs equates to
potential risk of increased AHT and reduced service levels not to mention frustrated
customers. At the other extreme, exceeding business needs, while it sounds terrific,
potentially means you overspent on your customer service infrastructure.
Avoid the use of “averages”. This is a typical IT view that focuses on average response
time. The adage that a man with his feet in a freezer and his head in an oven, “on average” is
just fine highlights why averages should be avoided. Instead focus on the volume of
instances that meet or fail to meet business needs.
“The goal is to have
a Valid, Valuable
and Accurate
representation of
end-user
experience.”
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
10 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
Establish an End-User Experience Team
Establish an End-User Experience Team that is focused on these metrics – this can be a single
person. A successful team will include these aspects:
“Own” the end-user experience metrics. This team will have a deep understanding business
processes, tools and underlying data to provide information on how the business is impacted.
Communicate with business operations managers and key IT stakeholders about how
monitoring information is indicative of the impact on end-users. This role needs to use
metrics to provide an “outside-in” view of performance and end-user experience and does not
need to be from IT.
Maintain the metrics. This may seem obvious, however, it is the single area where
application monitoring tool implementations fail. There is nothing more frustrating
for business operations managers and their IT counterparts to find out when they most need
these metrics that they have fallen out of maintenance and cannot be relied upon. It is
common for organizations to start an end-user monitoring project but fail to assign someone
to maintain the monitoring, or set it aside as a low priority task. It is preferable to have the
team that is utilizing the end-user data to also be responsible for the maintenance (“one neck
to wring”).
Have senior executive support from both IT and the business.
On-Going Operations
Develop “actionable” alerts for the organization. There is an important distinction between
alerts and those that are actionable. Actionable alerts focus on issues that support, or hinder,
those tasks identified as business needs. In addition, as business needs change the alerting must
reflect this or the results will be either the organization not getting alerts, or worse, getting so
many alerts that it isn’t possible to decide when to take action. Here are some keys to developing
“actionable” alerts:
o A best practice is to initially send out too many alerts (i.e., risk the potential of a high
number of “false positives”) to a small group, or individuals, who can then tweak or tune
the alerting to those with a small number of “false positives”.
o The End-User Experience team needs to drive alerts to the team or individual that
it relates to. The idea of sending out all alerts to everyone is flawed and misses the
concept of “actionable” alerts. These alerts, in order to be “actionable”, must go the team
or individual who are most likely to be able to address the issue.
o The alerts must have a sufficient amount of information about the problem so recipients
understand the issue and how it is impacting call center agents, enabling them to take
action.
Review trends on a regular basis. The End-User Experience Team should engage business
operations managers and IT stakeholders in regular reviews of the end-user experience against
the needs of the business that were established. The trend analysis should review the end-user
experience over weeks and even months to identify slow degradations and periodic events that
could negatively impact call center agents. The business operations teams should bring to the
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
11 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
review reports of poor application experiences from call center agents to help identify gaps. The
End-User Experience Team needs to drive this effort working across the organization.
Use the End-User Experience information to make decisions about changes,
improvements, investments and event to request IT to perform root cause analysis. This
information should provide evidence for return-on-investment (ROI) decisions and then
utilized to validate that the ROI was achieved. In addition the organization should use this
information to validate that infrastructure changes do not negatively impact call center
agents by integrating the End-User Experience Team into the organization’s change control
process.
There is often a concern that the establishment of “another team” means more meetings and
potentially less time for “real work” to get done. This is a reasonable concern and makes the
selection of the End-User Experience Team leader, whether internal or external to the
organization, critically important to the success of this initiative.
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
12 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
CONCLUSION
The importance of call centers in customer support hasn’t changed. In fact it is arguable the
importance of delivering quality service has increased as the number of self-service channels has
grown. To support this, call center executives have worked with their technology partners to
implement CRM and BPM systems, to enable business process compliance, reduce training costs
and still maintain service levels and customer satisfaction. The resulting complexity of these
highly integrated systems is now driving the need to move beyond traditional IT ‘five-nines’
metrics to more business focused performance metrics.
To take these common issues and turn them into a strength of the organization, management
teams should employ a sustainable end user experience management strategy. This would enable
the linking of IT performance to the business requirements in a manner that also accounts for
budgetary goals. This strategy also focusses on the business operations in a way that makes sense
to the IT organization that is tasked with supporting it.
BRIDGING THE COMMUNICATION GAP
BETWEEN IT & BUSINESS
13 HOW APPLICATION PERFORMANCE MANAGEMENT AFFECTS CALL CENTERS
Evidant Inc. www.evidant.com 949-609-1494
ABOUT EVIDANT
Since being established in 2002 Evidant has been a pioneer in aligning Application Performance
Management with End-User Productivity requirements.
Evidant's core belief is that the value to the enterprise of Application Performance Management
(APM) does not lie in the capabilities of the tool alone, but in the way it is configured to match
End-User Productivity requirements. This requires working with both the IT and Business units
of an enterprise to form a common metric - and that is what we excel at. We partner with your IT
and business units to define, implement, analyze and report on your APM needs.