Don't ignore them. Let them highlight your biggest failures and juiciest opportunities.
Companies are increasingly turning to data science to better understand how customers interact with their products and services. And with good reason. A 2018 survey sponsored by SAS, Accenture, and Intel found that 58 percent of responding business leaders said that using customer analytics significantly increased customer retention and loyalty, and nearly half associated analytics with significant revenue growth.
“It’s important to know if you’re providing the experience you want customers to have and making it better,” says Joel K. Shapiro, a clinical associate professor of data analytics at Kellogg. “To do that, you need to measure as much as you can about what happens to them.”
But according to Shapiro, far too many companies are ignoring some of the juiciest data around. These are the data that induce head-scratching, the measurements that don’t quite fit the existing models. These are the outliers. And they can highlight your product’s or service’s greatest weaknesses—as well as where it has the potential to truly shine.
Companies can and should use this knowledge to optimize the customer experience.
“The mere presence of outliers in customer experience data means that really good or bad things can happen to customers,” says Shapiro. “Maybe you can move that [experience] toward something that either increases the number of positive experiences or doesn’t detract from them.”
Keep Your Outliers
When data scientists come across an outlier, their first inclination may be to discard it in favor of “cleaning” or “smoothing out” the data. After all, the data might have been entered incorrectly or appear as the result of a modeling error. Or it may represent a freak accident—a set of circumstances unlikely to replicate itself. Why waste time accounting for the easily discountable?
Resist that urge, Shapiro says. It is always worth examining why the outlier occurred.
Shapiro comes to this insight firsthand. While studying the effectiveness of Spanish language instruction in West Virginia’s middle schools, he found that overall, students who received online instruction scored about the same as those who received it in person. But he also found that face-to-face class instruction produced a wider range of experiences. Investigating further, Shapiro learned that most of the high-performing in-person students came from one specific class.
After he had presented his results, the state’s board of education advised him to exclude that class from analysis because its instructor was known to work well with students. “They said she was the best thing that’s ever happened to the state of West Virginia from an education perspective,” Shapiro says.
But Shapiro realized that, rather than excluding this instructor’s students, the board of education should do the exact opposite. They should find out what this person was doing and try to replicate it.
What this outlier ultimately demonstrated was that, under the right circumstances, the in-person classroom model had the potential to outperform online instruction. This is an important first step toward understanding the potential of these outliers, Shapiro says. “They shouldn’t just be treated analytically and sort of smoothed over for some sort of aggregate analysis. They should be brought to the forefront.”
Get Some Context
It is important to note that Shapiro only learned of the exceptional teacher after disclosing his results to the state’s board of education. This highlights another critical step in utilizing data: business leaders must work closely with data scientists to interpret these outliers, as often the leaders are the only ones with the necessary institutional knowledge or business context.
"[Outliers] shouldn’t just be treated analytically and sort of smoothed over for some sort of aggregate analysis. They should be brought to the forefront.”
[This article has been republished, with permission, from Kellogg Insight, the faculty research & ideas magazine of Kellogg School of Management at Northwestern University]