Not All Data is Big Data


It’s really about how we make decisions

I heard author and data analyst Cathy O’Neil on the radio recently. She discussed her new book, Weapons of Math Destruction. Essentially, O’Neil talked about the misuse of data and the incorporation of biases when making decisions. She was especially concerned about using trends to stigmatize individuals. 

I would like to reframe this and relate it to healthcare analytics. I’m renaming a few things, but it’s really about how we make decisions. 

There are at least two ways to use data. 

Big Data

I have heard a common pattern in big data success stories. We have an intent and a lot of data. Professional data analysts look at a very large set of data and discover a pattern, which leads us to an action, though usually indirectly.

This is a story of discovery and subsequent old-fashioned hard work. We don’t know what the problem is, so trained data analysts help us find some clues in the data. Sometimes the data indicates a broad stroke solution, but usually it’s vague. What we really see are trends across large groups. This approach promises, do more of this and you get more of that. It doesn’t promise individual results.

Directed Data

Contrast this with data products that are sold with the purpose and promise of generating value. Healthcare organizations use lots of reports and dashboards. The products promise us, buy this and you will get that. It is by its nature prescriptive. The data and the problem define one another. The value hoped for is usually lower cost and higher clinical quality.

There is nothing wrong with using data in this way. If the hospital data points to the infection rate on Fourth Floor East, or documentation fallouts by Dr. Williams, we know where to focus our efforts. The key is that the data tells us where the problem is. 

We still need to discover more. We need qualitative data about the fourth floor or the practice of Dr. Williams.  We need to understand what the problem is and develop a solution. Chances are good that our data vendor skipped talking about this step.

Both Approaches Have Limits

The promise of Big Data is to help us find and influence broad trends. A big data trend does not predict individual results. It also does not, as O’Neil argues in her book, justify action on an individual. 

In contrast, we use Directed Data to see where attention is needed. We can’t forget that the metric is not the problem. We still need to understand the true problem.

With both approaches, further discovery through qualitative data helps us to understand the problems fully, and to synthesize effective solutions. There has been a lot of attention paid to quantitative data, and there’s been a lot of money poured into it. Recognizing how we intend to use the data, and supplementing it it with deeper understanding, is essential to getting value from that money. 

Answer Me These Questions Three

In Monty Python and the Holy Grail, a character ( “the old man from scene 24” ) guarded “the bridge of death.” He demanded that the heroes answer three questions before they could cross. If they answered wrong, they were “cast into the gorge of eternal peril.”  

A lot of us have been on that project. The one where the offering doesn’t work, it isn’t adopted, and we’re not sure why. Did anyone even ask us the questions?

To cross the “bridge of death” to a successful offering, there are three questions that you must answer. You must answer them so that you can intelligently decide what the offering is. 

  1. Who are the users? All of them.
  2. What work are they trying to do?
  3. What are their needs in context?

These are the questions that I use with my teams at the start of every project. In more detail:

1. We first need to ask who all the customers, providers and stakeholders are. We identify as many as we can at the beginning. We actively seek to discover the rest as we conduct research. These are the people involved.

2. We need to understand what work the people are trying to accomplish. They each have an end goal beyond purchasing an offering (for a consumer) or following a process (for a provider). The goal is not attached to the offering or process. Most people would be happy to get it done with a magic wand instead. So what is that goal?

3. What are the people’s needs in context? What in their lives or in business do they have to contend with? What’s it really like to be them? What work do they have to do to get to that goal in question 2?

Design is the practice I use to answer these questions. Design using human-centered research gets you good understanding. Answer the right questions. Don’t be cast into the gorge of eternal peril.

https://www.youtube.com/watch?v=cV0tCphFMr8