Beyond the Tyranny of the Spreadmart: Business Intelligence in the Contact Center

Once upon a time, there was a grief-crazed millionaire widow who consulted a psychic medium for advice on how to get past her sorrow. The medium consulted the spirits and proclaimed that the widow was under a curse – her departed husband’s livelihood had indirectly caused the deaths of thousands of souls, and she would have no rest until she not only built these tortured spirits a new home, but also kept ceaselessly adding rooms to make space for the ever-expanding legions of the dead. Thenceforth and until the day she died 38 years later, the penitent widow kept her mansion under constant construction, 24 hours a day, 365 days a year.

True story. Well, most of it. 

The widow was Sarah Winchester, whose husband was heir to the Winchester Repeating Arms Company fortune.  And whatever her reasons (the bit about the medium is probably hearsay), she did keep her San José, California mansion under round-the-clock construction from 1884 until 1922. When construction ceased, the “Winchester Mystery House” was seven stories high and featured 160 rooms, two ballrooms, 47 fireplaces, 10,000 window panes, 19 chimneys, two basements and three elevators. Over 20,000 gallons of paint adorned its multi-colored walls.

It’s estimated that Sarah Winchester “invested” over $5 million ($71 million in today’s coin) into the house’s construction over the years.  But because neither she nor her builders were working to any architectural blueprint, the house was festooned with cockeyed features like doors leading into second-storey space, doors opening onto interior walls, and staircases disappearing into ceilings — all passages to nowhere. So random and ramshackle was its design (or lack thereof) that when the Widow Winchester finally died, the house was appraised as worthless.

The object lesson of this story: Just because you’ve sunk a ton of time and money into building something doesn’t mean it’s worth a dime. Sound structure determines utility and value.

The Tyranny of The Spreadmart in Contact Centers

That’s a hard lesson that many contact center analysts, managers and executives are now learning when it comes to reporting and analysis of data in today’s multi-system, multi-channel and multi-vendor contact centers.  

Why? Because many are wrestling with — and yet still perpetually building — an equally unwieldy structure called the spreadmart — an ever-expanding collection of spreadsheets doing the job of a data warehouse or a data mart. Coined by Wayne Eckerson of the Data Warehousing Institute, the spreadmart is the Winchester Mystery House of contact center reporting and analysis; it’s always under construction, but because it has no blueprint or coherent underlying architecture, it’s essentially worthless in the long run. 

Eckerson likened the spreadmart to a pernicious vine, spreading everywhere and strangling accurate reporting and analysis efforts over time. This pest is particularly problematic in the modern contact center. What was once a simple system — operators standing by to take your phone call through a limited number of hardware-switched lines — is now a multifaceted, multi-channel and multi-system architecture that mediates much more frequent and complex interactions between company and customer. Today’s contact center must mediate this relationship by communicating with the customer through the channel of his of or her choice — voice, fax, e-mail, web chat and, increasingly, emerging channels like social media (Twitter especially) and even video. 

By necessity, the contact center is thus managed through an assembly of systems and applications spanning  automatic call distribution (ACD), interactive voice response (IVR), help-desk systems, workforce management (WFM), customer relationship management (CRM), and even enterprise resource planning (ERP) suites. Very often, contact centers also develop custom applications to handle unique needs not captured in packaged applications. 

When it comes to reporting and analysis, each of these operational systems and applications is an island of data; each has a database optimized to help it do its job and only that job. They may feature some kind of simple “canned” reports to help managers understand how they’re performing their specific tasks. However, these reports exist in a kind of vacuum — they don’t play well with data from other contact center systems, and thus offer poor analytical views of the center’s overall performance.

So answering simple management questions like “Is our break schedule affecting our service levels?” means looking at reports from two separate systems (the ACD and WFM systems or scheduling worksheet) and trying to marry the data manually, using the one tool found on every analyst and manager’s desk — the spreadsheet. And it’s just this kind of soil that the spreadmart finds most fertile in its quest to strangle reporting and analysis efforts. 

Oh, I'll Just Build This One Spreadsheet... I Can Stop Anytime

It all begins innocently enough. An analyst sets out to create a single report to answer a simple question, like the one about service levels cited above. The task is not impossible but the process is unwieldy; if she’s lucky, both systems will spit out results in .CSV format for easy loading into the spreadsheet, and if she’s unlucky, she’ll have to cut-and-paste them into the spreadsheet herself — or enter them by hand. 

A few hours pass between the data entry and consolidation, verifying the service level formulas and formatting the results to make them presentable, and now our analyst has a mild headache. But she e-mails the spreadsheet to the supervisory team. Question answered, job done.

Until she gets a reply from the Director of Customer Service. “Great report. Very useful. Send me a daily update, will you? Thx.”

So the tedious process must now be repeated on a daily basis. Then word comes down from the executive suite that a weekly summary is also required. To save time, the analyst forgoes assembling the weekly summary from source data and instead just adds up the daily reports. She now has a spreadsheet based upon another spreadsheet. The next step is to roll all the weekly reports into a monthly one (a spreadsheet based on a spreadsheet based on a spreadsheet), and the monthly reports into an annual one (a spreadsheet based on… well, you get the idea). What was once a single report is now a briar-patch of interconnected spreadsheets. Thus does the spreadmart take root.

The High Cost of Low-Cost Tools

Though ostensibly built with a low-cost, ubiquitous tool, the spreadmart becomes extremely costly very quickly. The most obvious reason is the sheer time-wasting repetitiveness of assembling data, matching it to related data from another source, tweaking formulae, checking for inconsistencies and errors, formatting the results and handling the distribution. Most of this process must be started from scratch every time for every report.

With highly skilled analysts of managers spending thousands of hours every year on mind-numbing data entry, the implicit costs begin to stack up very quickly. Not as obvious —but perhaps much more damaging — are the opportunity costs of such talent wasting valuable time that could be focused on solving actual business problems or improving the contact center’s operations. 

What’s more, there’s little or no data integrity in the spreadmart: its results can’t be easily traced or audited. This often leads to the embarrassing phenomenon of “duelling spreadsheets” where two separate reports for identical metrics yet different data surface during a management meeting. More valuable time is lost in the argument about which report is accurate, when the answer may in fact be unknowable due to the spreadmart’s twisted, untraceable structure. Simple errors in spreadsheets have had multimillion, even multi-billion dollar impacts in other business contexts.

Finally, the spreadmart has no consistent long-term memory. To do any historical trend analysis, the analyst has to forage through an ever-expanding bramble of spreadsheets, each of which is marred by bad data integrity and thus cannot be reliably related to one another. 

Thus, like the Winchester Mystery House, the spreadmart is overlarge, baffling to map or navigate, replete with errors because of its improvised construction, and ultimately worthless despite its huge incremental building costs. As an added bonus, it may be feeding false data to the contact center management, tricking them into taking the wrong action.

No one except the Widow Winchester intentionally sets out to build such a monster. It’s born of necessity; contact center managers need regular, dependable reports with data drawn from an ever-widening array of systems. So what’s the alternative?

Business Intelligence and the Contact Center

As we’ve seen, sound structure determines value and utility, whether in houses or technology. Luckily, there is a proven, time-tested architecture that can not only serve as the foundation for all current contact center reporting and analysis, but can also expand and extend it over time to include new systems and applications we may not even have thought of yet. 

Business Intelligence (BI) is a software architecture designed specifically for reporting and analysis of information from operational data systems of any kind — point-of-sale, manufacturing, banking and finance, sales and marketing, or any other computerized system with an underlying database logging transactional data. BI is actually composed of several technologies that began converging in the late 1980s to solve a pressing problem: transactional data in its raw form is very difficult to analyze. 

Just look at the raw event-level data in a typical ACD system, for instance: it’s almost an indecipherable blur of call identifiers, timestamps, queue assignments, call times, and so forth. The ACD’s database is optimized to write this data quickly so the system as a whole can handle as many calls as possible. Thus, each event in the lifetime of an individual call is logged as it occurs, with no easy way to thread all these events together for a cradle-to-grave account of what happened. If the ACD acts as a call center’s traffic cop, it doesn’t have time to do the job of a forensic accountant as well.

The situation becomes all the more complex when a second system or application comes into the mix. For instance, many contact centers use a separate IVR-based self-help system to screen incoming calls before handing them off to an ACD for routing to an agent. Like the ACD, the IVR logs events in real-time without giving a thought to later analysis. If you’re tasked with mapping a single customer’s call through these two systems, you’re now grappling with two streams of nearly indecipherable data. And if the ACD and IVR are made by different companies, you also have the classic apples-and-oranges problem — call events from each system will use completely different naming and numbering conventions. 

Harmony from Disharmony: How BI Works

Business Intelligence solves these problems by separating the operational and analytical tasks altogether. Operational systems are left do to their job of logging transactions, and a separate BI architecture focuses exclusively on reporting and analysis. Typically, a BI architecture is made up of four layers:

Operational: These include the databases contained within the transactional systems to be analyzed. In the contact center, these include the ACD, IVR, CRM, WFM and other systems and applications that underlie all operations.

Access & Staging: This layer includes services that extract, transform and load data from the operational systems and applications to prepare it for storage. It’s in this layer that apples-and-oranges issues (think of our ACD & IVR example) are resolved to create a “single version of the truth” for reporting and analysis — a complete, cradle to grave view of an entire call, or all calls during a given shift.

Data warehouse/data mart: This is a separate, permanent database structured specifically for the task of reporting and analysis. Whereas the operational systems log relatively simple sets of data (like time stamps, DNIS, etc), the data warehouse contains more complex, calculated dimensions and metrics such as service level, speed of answer, time in queue, and average work time.  In other words, where the operational systems hold raw data, the data warehouse holds contact center management information. The data warehouse can also store months, even years of data for long-term historical trend analysis, while operational systems typically keep only a few weeks’ worth of data.

Presentation & management: Here is where end users — managers, supervisors, analysts and agents — access, interact with and share information in a variety of forms, from scorecards for each individual agent to graphical dashboards showing how the contact center is doing as a whole. This layer also handles security, storage, scheduling and distribution of content through e-mail, the web, and mobile technologies. Because this presentation layer is separate from the logical layer of the data mart, it can easily change over time (changing reports, adding new scorecards, tweaking dashboards, adding new distribution channels like smart phones) without affecting the integrity of underlying analytical data.

Key Drivers of the BI Architecture for Contact Centers

Since its emergence in the late 1980s, Business Intelligence has become a standard technology in most industry sectors (banking, manufacturing, consumer goods, telecommunications) and many functional areas of the enterprise (finance, sales and marketing, operations, customer support). Six factors are driving its increasing adoption in the modern contact center:

1.  Service is now strategic: In a world where “brands” (what we used to call “companies”) compete with one another by building long-term relationships with customers, contact centers have come out of the complaint-desk wings and taken center stage. For a company like Dell, beating competitors doesn’t mean simply convincing you to buy its computers today; it has to win your next computer purchase by executing flawlessly on solving technical support issues you may have with this computer, and fast. Dell’s customer service contact center thus carries much, maybe most, of the burden of its branding promise to be ready with answers and solutions when needed.

2. Channel proliferation: Where dial-tone call centers once handled a single channel (voice), modern IP-based contact centers are expected to manage every channel a customer is likely to use for contact — voice, fax, e-mail, web chat, online collaboration, and now social media. Video is seen on the horizon. Each of these channels tracks very different data about customer interactions. Amplifying the complexity further is the fact that many customers use multiple channels to communicate with the enterprise, sometimes simultaneously (the so-called “double-barrelled” customer interaction via voice and online self-help services).

3. Multiple sites: Better service through more channels means more agents.  Finding good talent in enough supply at competitive wages has driven the contact center beyond its origins at headquarters into often remote markets both foreign and domestic. Whether it’s opening a site in Smallville, USA or in India or the Philippines, new sites mean local systems and applications. Even if these are identical to those at head office, they are islands of data that must be merged into a single, coherent view for performance management.

4. Mixed-vendor environments: When voice was the only channel, contact centers generally purchased contact center systems from the same vendors who provided their telephone systems. The introduction of IP telephony and new contact channels brought new competitors into the market, so that today fewer and fewer contact centers are single-vendor shops.

5. The M&A Mash-Up: Take the complexity created by the first four factors above, then double it – that’s what happens when companies with contact centers merge or are acquired. Simply closing “overlapping” contact center sites is often not an option due to product/service complexity and the need to keep trained, qualified agents. 

6. Inbound & outbound: In olden days, customer service handled inbound calls, sales and marketing made outbound calls, and never the twain did meet. Today, the proliferation of channels has erased that borderline – think of a customer who initiates contact via web chat or e-mail, but needs a follow-up call to help with a thorny technical issue. An agent at a “high-touch” therapeutic pharmacy center might take an inbound call from a chronically ill patient experiencing side effects from a new drug, make a warm transfer to a pharmacist, then make a follow-up call to the patient later to make sure new dosage instructions are understood and followed. 

Advantages of BI in the Contact Center

Business Intelligence is not the only response to these challenges and the menace of the spreadmart. Many contact centers choose to extend existing point-to-point reporting services already offered in one of their key systems, such as their ACD or workforce management application. Others opt for performance management suites which offer data integration and reporting services as part of a much wider array of applications, including workflow management, coaching, contact type profiling, agent recognition and compensation. Finally, some real-time dashboard systems feature historical reporting extensions as well. 

All three of these approaches are viable alternatives to the spreadmart. However, BI has some distinct advantages that make it the best architecture not only for a contact center’s present reporting and analysis needs, but future requirements yet to be discerned.

Depth and breadth: Treating reporting and analysis as a mere “add-on” feature for operational systems greatly undervalues its central role in contact center management. The canned reports and limited analysis services included in these systems are often the focus of managers’ and analysts’ frustrations and lead to the use of spreadmarts in the first place. Because it concerns itself exclusively with the task of reporting and analysis, the BI architecture can offer a much deeper and broader functionality set – ad hoc queries, multidimensional data analysis, and statistical analysis. 

System Neutrality: Unlike point-to-point reporting extensions of existing systems, the BI architecture isn’t tied to a particular contact center technology such as the ACD. It can “roll” with wholesale changes in infrastructure brought on by, say, new signalling technologies like the Session Initiation Protocol (SIP). Whatever operational systems will dominate the contact center a decade from now, a BI solution will be able to connect to them and optimize their data for analysis.

Customizability: The one-size-fits-all performance management suites have one major disadvantage compared to BI – they are monolithic and inflexible. Contact center managers are expected to adapt to a particular suite’s approach to optimal performance lock, stock and barrel; the “standard” reports, scorecards and dashboards are thus considered all that is needed to support the paradigm. However, no single approach to performance management can encapsulate every particular challenge or opportunity faced by every contact center in every industry and market. Conversely, the BI architecture is designed to be customized to measure exactly what an individual contact center needs measured based on its unique needs. A BI solution can not only offer libraries of common reports and key metrics, but also the ability to adapt these to specific requirements, as well as change them over time.

Getting Started: The Good News About Your Spreadmart

One final advantage of Business Intelligence in the contact center is that you can start small. Although the BI architecture is designed to scale to truly huge proportions, implementing one doesn’t have to start with an equally huge planning and needs assessment exercise. 

Despite all our sturm und drang about the spreadmart, if you have one, it may not be entirely worthless. The single virtue of the spreadmart is that it is probably a very accurate reflection of the metrics that matter most to your contact center. As such, it can be a useful report template to begin a migration to a Business Intelligence solution. There is value in a more comprehensive BI needs assessment, but you should first seek the “quick win” of automating the reports currently in circulation. The payoff in saved time (and thus money) will be immediately evident, and will make the best case for expanding the BI environment to include new metrics, reports, scorecards and dashboards.

As for the wider needs assessment, keep it simple. The best way to take advantage of everything your BI architecture can offer is to state in simple business terms what you need, without being constrained by canned measures and metrics you’ve been forced to work with in the past. Say “I need to know how my agents are splitting their time among the various activities that they do within their shifts, assigned or not,” rather than merely asking for a daily report that shows not-ready time, talk time, idle time, and breaks. Your BI solution can support a multidimensional report formatted not only to answer your specific question, but be alerted to abnormalities, drill into specific agent metrics, and take action to correct problems. And that same report can change over time as needs and circumstance dictate. 

Ultimately, it’s this dynamism that’s the defining difference between the spreadmart and the Business Intelligence architecture. Like the Winchester Mystery House, the spreadmart is made up of on-the-spot, jury-rigged reports, each answering a specific question at a specific time, but then forced into regular “production” through constant manual effort. The BI architecture is more like a modular home, with well-designed, purpose-built components made to fit and work together in whatever combination needed. Expansion and reconfiguration are possible at any time, without damaging the utility (and thus the value) of the structure as a whole. 

Maybe such a coherent, logical structure would have done little to assuage the Widow Winchester’s tormented guests – but at least the doorways and staircases would have led somewhere other than dead ends.