The forecast says there’s a 20% chance of rain tomorrow in your city. Does that mean that it will be raining 20% of the time? Or that there’s a 20% chance there will be at least some rain? Or something in between?
And what in the world does this have to with call center metrics?!
The forecast problem illustrates how difficult it can be to work with probability. The most common performance metric used by call centers is the “service level”, which measures how quickly calls are answered. It is defined as a pair of numbers: a percentage value and a time value in seconds. So, for example, an “80/20″ service level means 80% of calls answered in 20 seconds. Like the forecast of rain, this is a probabilistic measurement and can be misleading if not understood properly.
1. Avoiding the Probabilities
The only way to properly model the service level in your call center is to use the Erlang equations. The equations themselves can be intimidating so it’s understandable that many call center managers want to avoid this topic. Today, most call centers have a workforce management tool that does the math for you, but you still need to understand the concepts behind the numbers or you can fall victim to some of the flaws listed below.
To get more comfortable with the concept, you can think of Erlang as a translator box, where you put in values like calls per hour, handle times, and number of agents available. These are “deterministic” values, in that they are easily measured. What you get as an output are the answers to “probabilistic” questions such as, “What are the odds a call will have to wait in a queue?” or “What are the odds a call will wait more than 30 seconds?”
Another important reason to be comfortable with Erlang is that it’s the only way you can do proper analysis of call volume “spikes”. For a walk-through of that kind of analysis, check out this blog post.
2. Picking The Wrong Service Level
The most common service level is “80/20”. Many people assume this standard is based on careful analysis which revealed that 80/20 was a good target to set. But that assumption is wrong and adhering to it blindly is not a wise choice. Sadly, the “myth” of the 80/20 standard is now deeply rooted in the industry, like an urban legend that can’t be debunked.
The obvious danger in picking a service level arbitrarily and then trying to meet it is that resources get allocated the wrong way resulting in more harm than good than good. For more on this read Finding the Right Service Level for Your Call Center.
3. Picking the Wrong Time Period
Like the rain forecast mentioned at the beginning of this post, a service level of “80/20” is underspecified. We have to also select a time period for calculating the average. Are we going to average over calls every hour? Every day?
Call center consultant Rebecca Wise Girson said:
“The bigger the time period, the easier it is to ‘look’ like you’re providing a good customer experience. Measuring the percentage of intervals throughout the day that you meet your SL goal is a more telling metric than measuring only to daily, weekly or monthly averages.”
She goes into more depth in this blog post: The Cost of Having the Wrong Service Level Goal.
Another commenter talked about how organizations can game the system by using time intervals :
[managing] to a monthly or weekly SL goal… allows for many periods of poor performance to be ‘averaged out’. I’ve often seen organizations that play the averages to meet the SL goal by intentionally overstaffing periods (sometimes with overtime) just to make up for periods with low performance. This really just wastes labor dollars and does nothing to change the negative experience of customers who called during a period with low SL.
4. Doing an Average of Averages
So now we’ve picked a time limit (e.g. 20 seconds) and a percentage target (e.g. 80%) and a time period (e.g. 1 hour). Is this now a fully specified metric? Technically yes, but in practice, there’s another step before the metric is actually useful because now we have a set of 8 scores at the end of an 8 hour day. Are we going to average them together? That could hide important variability, as mentioned in the section above. But if we don’t somehow aggregate the data, we will have a hard time talking about performance over longer time scales.
A better approach is to agree on some kind of “compliance rate” where you calculate how many of the time intervals had service levels that met the target. For example: “We met our service level targets in 6 out of 8 time ranges today.” This could be further condensed into saying “We had a 75% compliance rate.”
Now we are finally able to state a fully specified goal for a call center: “We want to answer 80% of all calls within 20 seconds, averaged each hour, 75% of the time.”
5. Ignoring Outliers
Even if you’ve done all the right analysis and settled on a proper fully-specified service level for your call center, there’s another pitfall. The probabilistic nature of this metric means that it hides variability “under the rug”. For example, if your agents successfully met 80/20 all day, you know that 80% of the calls were answered in under 20 seconds. But you know nothing about how bad the other 20% were. Were their wait times 30 seconds or 10 minutes? This variability is critical.
In his new book, The Executive Guide to Call Center Metrics, James Abbott states “In today’s centers, variability is the largest driver of cost… the bigger the variation the more costly the operation.”
How can you incorporate this into your metrics? One way is to have multi-tiered service level. For example “We want to answer 80% of all calls within 20 seconds, and 95% within 40 seconds.” To be complete, this can be combined with the concept of a compliance rate from the section above. (And, of course, don’t forget to specify the time period for averaging!)
Have you found other pitfalls that we didn’t mention? We’d love to hear from you!