Using big data: 3 reasons it fails and 4 ways to make it work

Image: Shutterstocks

Most companies already understand the importance of using big data to drive insights and decisions. The problem, instead, is that very few companies know how to integrate data analytics in a sustainable way. Too many rely on occasional, flash-in-the-pan successes from a handful of talented employees, but there is a better way.

Darden Professors Raj Venkatesan and Kim Whitler have found, in their research, seven key lessons that apply to all companies, whether they are “native” in data analytics or have transitioned into it.

When Venkatesan presented their research at the Leadership in the Face of New Technology conference at the HWZ University of Applied Sciences in Business Administration Zurich in Switzerland, he drew on the examples of Netflix, Airbnb, ESPN and CarMax to explain those lessons.

Potential Problems
Cognitive Inertia
If your company has already been successful without using data analytics, be aware that the switch will be harder for you than most. Managers will have to let go of decision-making processes that have worked for them in the past and learn how to use data analytics instead.  Cognitive inertia might also be a problem if data is in a silo — i.e. if only part of a firm uses data.

Uncertain Outcomes
Without clear and consistent Key Performance Metrics (KPMs), management won’t know how to start applying insights gained from data or be able to measure success. Even if you start out with clear KPMs, keep in mind that the process of analyzing the data may need to change. Consider leading television networks such as HBO, Showtime, ESPN and Comedy Central. When these networks introduced their apps, it unexpectedly shifted them from business to business to business to consumer. When your customer is no longer other businesses (cable companies, in ESPN’s case) but individual consumers, your KPMs necessarily change.

Complexity of Problems
In interviews, some managers admitted that they were less likely to use data to drive their decisions when the problem was very complex. However, this is exactly the time Venkatesan says data analytics is most useful. The key is to break your problem down into small enough component parts so that data becomes useful once again.

How to Make Data Analytics Work
Knowledge
Not only do you need employees who are comfortable using data, but you also need to make sure everyone is on the same page with an explicit, company-wide mental model (a mental model is a set of assumptions about how the world — or your industry — works). This mental model should be updated continually, as new data sheds new light on the real dynamics underlying your business. A poll of Venkatesan’s audience showed that while many had formed a mental model for how their industries worked, only one person thought that anyone else in her firm shared that mental model.

Horizontal Embeddedness
Make sure that data is integrated into every department and function, for instance with a “data” person embedded on each and every team — or even better, by having data analytics be part of almost every job.

Vertical Consistency
There need to be firm-wide norms about how to interpret data, and there needs to be a consistent and repeatable set of data that the whole firm uses.

Merchandising
Here, merchandising means using presentations and reports to share progress on projects; however, these should also be treated as public accountability and training sessions. CarMax, for example, holds an “open house” every two weeks. These presentations are open to anybody in the company and many C-suite leaders regularly attend and provide feedback. These sessions not only create a systematic and rigorous approach to managing projects, but also help to create a culture centered on using data to move projects forward.

In Sum
Data analytics isn’t about achieving perfection, Venkatesan stressed. Rather, it’s about the journey of rapid prototyping, in which each iteration adds more data that you can use to inform your mental model. You don’t need vast resources to do this well — just a starting point and a systematic approach.

Darden Professor Rajkumar Venkatesan presented these insights at the Leadership in the Face of New Technology conference, co-hosted by the HWZ University of Applied Sciences in Business Administration Zurich and the University of Virginia Darden School of Business.

[This article has been reproduced with permission from University Of Virginia's Darden School Of Business. This piece originally appeared on Darden Ideas to Action.]