Don't Stare at the Matrix: Nine Pitfalls to Avoid in Defining KPIs for the Contact Center

In the movie The Matrix (1999), a character named Cypher whiles away the hours staring at a torrent of raw data scrolling vertically down an old-fashioned video screen. While others see an indecipherable waterfall of symbols, he sees a coherent pattern that makes up the artificial reality known as the Matrix. (He also eventually goes insane.)

There are days when contact center analysts, supervisors and managers must feel like they’re staring at the same indecipherable waterfall of data — “staring at the Metrix,” if you’ll forgive the pun. The many, various and complex systems that make up the modern contact center form a fount of data that never runs dry; they collect information on virtually every measurable aspect of a customer interaction, how it was handled, and by whom. Typically, the number of database records rises into the millions in short order.

To be sure, those contact center systems usually offer an ever-widening array of standard metrics to help managers make sense of it all. But tracking more metrics doesn’t necessarily create better performance, any more than tracking more baseball statistics makes all players into Hall of Famers.

That’s why managers are increasingly turning to Key Performance Indicators (KPIs) — those essential measurements showing how the contact center is impacting the company or organization’s business goals. While the process of choosing and/or defining KPIs is critical to effective performance management, it’s important to be aware of (and thus hopefully avoid) a few common pitfalls. 

Pitfall #1: Metrics & KPIs, Carts & Horses

Many contact center systems already pump out a steady stream of metrics; automatic call distribution (ACD) and interactive voice response (IVR) servers alone are a wellspring of standard reports showing typical baseline metrics such as calls offered and calls handled,  time in queue, and average talk time, along with information about call routing and skill sets.  Many call centers also use workforce management (WFM) or quality monitoring (QM) applications, which generate their own “canned” metrics.

It’s tempting to assume that all the KPIs you need are hidden in this laundry list of standard metrics, and that measuring performance for your particular contact center is a matter of picking and choosing which ones apply to your business.

That assumption is probably the biggest single strategic cart-before-horse mistake in performance management. KPIs are not simply derived from available metrics — KPIs determine which metrics should be measured. Many of the needed metrics might already be at hand, but others will likely need to be calculated, drawing on data from several applications and systems.

Pitfall #2: Looking in the Wrong Place

But where do you begin your search for the KPIs and supporting metrics you need? The starting point isn’t in any particular contact center system, application or report. It isn’t in the contact center at all. It’s upstairs in the executive suite.

We’ve written elsewhere¹ about the need to get buy-in from senior management for better reporting and analysis of contact center performance. The most powerful way to accomplish this is to derive the contact center’s KPIs from strategic targets and goals set for the company or organization as a whole by its leaders. An hour spent with the vice-president of sales and marketing or the chief financial officer — or the CEO — can go a long way towards defining what needs to be measured back in the call center.

This may sound daunting, but you can count on most broad strategic goals boiling down to specific, measurable actions in the contact center. For instance, if a financial institution’s strategic goal is to move into a new market category (say, home insurance), then measuring and improving cross-selling by customer service agents handling mortgage inquiries will serve that strategic goal. Similarly, if an online retailer of computers wants to win market share from competitors based on better post-sale customer service and technical support, then KPIs measuring customer satisfaction will directly impact that strategic goal.

To be sure, the organization’s strategic goals are not likely to be achieved simply by tuning performance within the contact center. But if you are measuring a KPI that can’t be mapped directly to one of those goals, then it’s likely to be of secondary importance. Operational efficiency metrics and KPIs, which help managers operate the contact center at the lowest cost, should always be subordinate to strategic KPIs. At best, a “merely efficient” contact center is doing no harm; at worst, it could be efficiently driving its resources in the wrong direction.

Pitfall #3: Cookie Cutters Don't Cut It

Once you have a set of KPIs rooted in corporate strategy, then it’s time to define the metrics you need to measure them. This is where it’s tempting to go back to the laundry list of standard metrics generated by your various systems. These default, “cookie-cutter” metrics are the system manufacturers’ best guesses at what measures will be needed by contact centers spread across every industry, geography, and organizational type, without regard for which products or services those contact centers offer or support.

Look at average talk time: this out-of-the-box metric is crucial for most customer service centers, who seek to minimize it to reduce cost-per-call. But a sales center focused on revenue-per-call would care less about longer call times and more about shorter speed-of-answer. A customer-retention center would no doubt also care about speed-of-answer, but not as much as they do about various quality scores. 

Pitfall #4: Formula for Confusion

What’s more, the formulae used to create these metrics are by no means standard across the industry. They can vary not only among contact center system manufacturers, but also within a given manufacturer’s product line — a significant problem for mixed-version, mixed-system contact centers.

Take service level, for instance. It’s one of the oldest, most common call center metrics in the business. It’s almost universally understood to be the percentage of calls answered within a defined wait threshold — X percent of calls handled in Y seconds or less. Yet one major ACD manufacturer calculates service level one way for historical reports (accounting for abandoned calls), and offers another option for real-time reporting (omitting abandoned calls). Both metrics are “right” for their particular use. But which is right for your KPIs? 

Pitfall #5: Two Plus Two Equals…Five?

Then there are cookie-cutter metrics that just don’t seem to add up. Take another staple of call center metrics — calls offered. Intuitively, managers expect this to be calls answered plus calls abandoned, and in standard ACD reports, those numbers may add up fine at first. But as IVR systems and more sophisticated skills-based routing techniques are introduced, the arithmetic begins to go awry: calls answered plus calls abandoned may equal less than calls offered. This is because other call-routing-based outcomes won’t be captured by the answered-or-abandoned metrics. These “missing and presumed unresolved” calls will continue to fall off the radar, unless the reporting and analysis environment can account for the impact of call flow routines.

Pitfall #6: Target Locked!

Even when off-the-rack metrics are calculated correctly, the out-of-the-box reports that use them will likely be too inflexible for anything more than simple KPIs. 

Take average handle time, for instance: for a given agent, it’s the average talk time plus the average after-call work time for data entry and other tasks related to calls. Cookie-cutter, dashboard-style  reports from your ACD will likely allow you to set a single target for average handle time for a single agent — which would be fine if that agent dealt with only one kind of product or service requiring only one skill.

But agents usually have more than one skill, and companies usually have more than one product or service. Thanks again to skills-based routing, a given agent will likely be serving several skill queues at once, each demanding different average call length and even post-call wrap-up work. Furthermore, real-time management techniques allow agents to be dropped into (and pulled out of) different queues as needed. 

Under these circumstances, how can you set just one target for average handle time for a single agent, and expect any comparison with the performance of other agents – each with their own unique call mix – to be unchallenged and accepted as fair? You can’t. To say “it all averages out” doesn’t cut it. Metrics have to be more dynamic if the KPIs on which they’re based are to make any sense.

Pitfall #7: The Customer’s Many Faces — and Phone Numbers, Fax Numbers, E-mail Addresses…

One of the most talked-about call center KPIs is first-call resolution (FCR). Its definition is deceptively simple: the percentage of inbound customer inquiries resolved in the first call, without requiring call-backs — or for some call centers, transfers (“one and done”). 

According to the Service Quality Measurement Group (SQM), 15 percent of customers will defect to a company’s competitors if their inquiry is not resolved on the first call. Pursuing better FCR performance is worthwhile; every 1% increase in first-call resolution leads directly to a 1% increase in customer satisfaction — the strongest correlation of any metric in the call center.²

Though defining FCR may be easy, measuring it accurately has proven to be a major challenge for many call centers. 

One issue is tracking the customers themselves; the default method — screening ACD data for duplicate caller IDs occurring within, say, a three-day threshold — is notoriously unreliable. A customer whose first issue was resolved can call back with a completely different inquiry, understating FCR. Conversely, a customer who initially made contact on her business phone can call back later in the day regarding the same unresolved issue using her mobile or home phone, inflating FCR.

The issue is made all the more complex when the definition of FCR is expanded to first-contact resolution, to include all the channels consumers now use to communicate — voice, e-mail, fax, web-chat, web forms, and so forth. Different channels impact FCR in different ways; for instance, a customer service inquiry mediated via e-mail will likely require multiple interactions before being resolved, so FCR measures should focus on resolution within a short time-frame by a single agent rather than the number of interactions required.

To properly gauge FCR, metrics have to draw on data from ACD, IVR, and particularly customer relationship management (CRM) systems to determine which channels a customer used to communicate regarding a given issue. Some contact centers use post-call, menu-driven surveys to ask customers directly whether their issues were in fact resolved. FCR accuracy can further increase when data from call-quality monitoring and screen-capture systems are used in the calculation.

Pitfall #8: Tug-of-War

Another common pitfall is adopting metrics and/or KPIs that pull agents and teams in two different directions. If managers decide that first-call resolution is a paramount customer service KPI, then setting low targets for average talk times will actually hinder agent and team performance. Arbitrary call-time targets will also hamper sales-oriented centers which also value (and compensate) revenue-per-call or conversion rate (the percentage of inbound or outbound calls converted to a sale). 

Again, this highlights the essential difference between KPIs and metrics. KPIs ultimately define how the contact center can improve the overall business, while metrics measure the actions and practices that affect those KPIs. That’s not to say you should stop paying attention to subordinate metrics — if it takes your agents an average of an hour to close sales on an “as-seen-on-TV” fast-fryer, you’ve got a training problem. But it’s vitally important that KPIs and subordinate metrics not compete with one another.

Pitfall #9: The Right KPIs, the Wrong Tools

Though they may have selected the right KPIs and supporting metrics, many contact centers still fail in their performance management efforts because they choose the wrong technologies for reporting and analysis of contact center data.

The most common — and in the long run, most costly — mistake is to attempt to build a contact center reporting solution using the tool most readily at hand: the spreadsheet.  It’s easy to understand why; spreadsheet software (usually Microsoft Excel) is installed by default on most managers’ computers, and it’s the first tool they’ll use to crunch numbers or present data. It’s the tool they know.

Understandably, managers will also use spreadsheets to assemble data from a variety of operational systems (ACD, IVR, CRM, WFM, etc) to create sample KPIs and associated graphs, charts and tables. But once a performance management strategy is adopted, those very spreadsheets are too often put into production as an enterprise reporting solution, a task they are uniquely unqualified to perform in the long term.

The disjointed, disorganized and ever-expanding briar-patches of spreadsheets that inevitably result have been dubbed “spreadmarts” by industry analysts³ — that is, spreadsheets doing the job of a data mart or data warehouse, the architecture best-suited to collect, transform and optimize data from disparate operational systems for reporting and analysis.

Data warehouses and data marts are the foundation of business intelligence (BI), the broader field of gathering, storing, analyzing and distributing enterprise data for better, more informed business decisions. Because the contact center’s key systems are many, various, and complex, BI solutions are increasingly seen as the crucial enabling technology for performance management.

To be effective, a contact-center specific BI solution must:

• Connect to and “understand” the native data schemas of key contact center operational systems
• Extract, transform and optimize those systems’ data for analysis, and resolve apple-and-orange data variations found in complex, mixed-vendor environments
• Manage a unified data store (the data warehouse/mart) containing all the information needed for reporting and analysis of KPIs
• Allow metrics and KPIs to be calculated specifically for a given contact’s center’s unique formulae
• Be capable of storing months or even years of data for long-term trend analysis
• Provide tools to manage data access security, report scheduling, and distribution
• Allow managers to create their own ad hoc reports and “drill” into the data to discover why KPIs are not being met.

Conclusion: Go Forth and KPI 

While we have been examining some challenges and pitfalls in defining and choosing key performance indicators, there is really no more valuable an exercise in contact center performance management. 

Good KPIs help senior executives understand the value of the contact center, and help managers make decisions about hiring, training, scheduling, and compensation that are ultimately rooted in the organization’s business goals. Agent and team reports, dashboards and scorecards are no longer based on off-the-rack, one-size-fits-all metrics; instead, the contact center’s own unique needs are reflected in the measurements everyone is working to meet and improve. Data is not only accurate and relevant, but also seen by all to be accurate and relevant. Confusion over conflicting targets is eliminated. Most importantly, the customer is ultimately better served — and no one has to get migraines staring at the Metrix.

Endnotes

1 Seven Tips for Building a Business Case for Better Contact Center Reporting & Analysis. InAAU Insider, Volume Fifteen, Issue Two (2010)

2 First Call Resolution (FCR) — The Metric that Matters Most. Service Quality Measurement Group.

3 Strategies for Managing Spreadmarts: Migrating to a Managed BI Environment. The Data Warehousing Institute Best Practices Report, First Quarter 2008.