Bots Haven’t Truly Arrived Until We See Containment Stats

Call Center

Bots Haven’t Truly Arrived Until We See Containment StatsWhat a huge year it was for bots! A term that was barely on the radar became a sizzling hot trend. Facebook kicked off the mania by opening its Messenger platform and now has 33,000 bots. Competing platforms have high numbers too: 19,000 for Kik and 10,000 for Line. It looks like 2017 will continue the momentum.

But after the hype comes the backlash. Too many expectations were placed on this idea, as I wrote in ChatBot Backlash Boogie Woogie (which includes a list of 4 specific expectations and why they were misguided.)

Now that we’re past the initial crush, it’s time for bots to start the slow climb to a mature technology. One of the clearest sign-posts along that climb is the metrics that are reported.

How Do We Measure Success?

One of the main use cases for bots is as a new channel for customer service. We’ve seen many companies jump on board to try this out including, in the last two months, American Express and EBay.

So we definitely have quantity. But do we have quality? In other words, are these bots accomplishing something?

An Important Step Forward from Facebook

In November, Facebook took us a step closer to answering that question. It announced that its free analytics platform would be extended to cover chatbots on Messenger.

 

Bot Analytics

 

As the leading bot platform, what Facebook does sets the tone for everyone else. It has, after all a billion monthly active users.

 

Facebook Messenger Growth

 

Right now, platforms are boasting “we have 30,000 bots” and companies are boasting “our bot handled 100k chats last month”. These aren’t bad numbers to report; they definitely signal progress. But what I’m really watching for is companies that start reporting success metrics rather than usage metrics.

The Old World Can Teach the New World

You might say, “Isn’t it too soon to already define metrics for a technology that is so new?”

Not if we borrow from the world of customer service analytics. There we have decades of experience and wisdom around defining and using metrics. (See our Call Center Metrics Pitfalls whitepaper for a good starting point.)

The metric that springs first to mind is “containment rate”. It answers the question: “Did the bot fully eliminate the need for an agent conversation (or other channel)?” That’s the main outcome we’re hoping for, right?

It’s not a coincidence that “containment rate” is often the main metric used to grade IVR implementation. In many ways, it’s useful to think of chatbots as the modern-day successor to the IVR. Both offer front-line, automated interaction, in an attempt to resolve issues before escalating to a live agent. When you look at it in those terms, it’s easier to see through the hype and focus on what real success looks like.

Fonolo Resource cover image

Having trouble connecting with Gen-Z?

Download this free guide to learn how to attract more Gen-Z members to your credit union.

Download now