Rethinking the Obvious: How Data Analysis and Diagrams Can Upend Conventional Wisdom
Has it happened to you?
You organize a brainstorming session to begin analyzing your process.
At the kick-off meeting, several people sit with arms crossed, lips pursed, eyes cast downward. Frequently, they’re the ones who’ve worked at the process for most of their professional lives.
“Here we go again. Wasting time to prove the obvious,” their faces say. “I’ve done my job for years. You’re not going to show me anything I don’t already know.”
Yet you bravely push forward. Every now and then you see someone roll their eyes. “When can I get back to my desk and do some real work?!!!” they seem to implore.
It’s a bit daunting to face down smart, experienced skeptics. Because their in-depth, hands-on knowledge—which they’ve painstakingly pieced together over the years—is indeed impressive, profound, and extremely invaluable. More often than not, they know a lot more than you do.
And make no mistake—you want (and desperately need) them on your team!
However, acquired knowledge is a tricky thing. It enlightens us much of the time. But it can also deceive us when we least suspect it.
It's no wonder, then, that human history is replete with examples of how even the most logical and seasoned reasoning of experts can be dead wrong.
A Classic Example: A Simple Diagram Reveals the Transmission Vector for Cholera
London, August 31,1854.
A major outbreak of cholera suddenly erupts in the city. There had been other outbreaks before, but this one was particularly fast and deadly. In just 10 days, more than 500 people were dead. And the mortality rate quickly shot up to over 12% in some areas.
The outbreak was concentrated along the Thames, in low-lying areas inhabited mainly by the poor. Public health experts at the time knew exactly what the problem was: reeking fumes from open cesspools were mingling with the ever-present fog to create a highly noxious vapor (miasma) that caused cholera.
The theory made perfect sense. And it was believed by most highly intelligent, experienced people at the time. So why would anyone question it?
Thankfully, one physician did.
Instead of jumping on the miasma bandwagon, Dr. John Snow decided to take a hard look at the numbers.
First, he gathered weekly statistics on the cholera deaths (compiled by demographer William Farr). To visualize the results, Snow plotted the location where each victim had lived on a map. He also plotted the locations of the city water pumps.
Next, Snow employed a graphical tool of the time—one that had been around since the 1600s—called a Voronoi diagram. He drew a cell around each data point representing a water pump by constructing a series of connecting line segments. Each segment bisected the distance between that data point (pump location) and a neighboring data point (pump location).
This results in a honeycomb pattern, in which all the points within each cell are closer to the central data point than to any other data points outside of the cell.
Snow adjusted his diagram to account for human walking routes, which were not always the straightest distance between points.
When finished, the visual diagram provided a "Eureka!" moment—one that turned the conventional wisdom about cholera transmission on its head. The diagram revealed that nearly all of the cholera victims lived within the cell defined by a single city water pump, located on Broad Street.
Although the diagram itself didn’t prove that cholera was transmitted via water rather than air, it opened the door to subsequent research that eventually did demonstrate this transmission vector. And it showed that even a fairly simple visual diagram of basic data can be a powerful tool for challenging entrenched assumptions.
Thinking Inside the Box in the 1850s
I can’t help but wonder. What if Snow had set up a team of experts in the 1850s to collaborate on creating the Voronoi map instead of making the map by himself?
I imagine his efforts might have elicited skeptical comments like these:
“Why are we wasting time plotting the exact location of each of the victims on a map? This will take forever. We already know the area of the disease outbreak!”
“You want us to sit here and draw line segments between pairs of points on a map? Are you out of your mind?! While hundreds of people are undergoing painful death throes from a contagious disease!?”
But That Was Then. This Is Now. We're a Lot Smarter...(right?)
We know the world’s not flat. We know what causes thunder. We know the Earth orbits the Sun. We know that…[fill in the blanks].
But the sad truth is, we don’t know nearly as much as we think we do. And most of us aren’t John Snows. Most of us are more like the public health officials in London of the 1850s—shrouded in the miasma of our environment and our times.
Problem is, we can’t always know what we don’t know. Our brains tend to quickly latch onto pat explanations, routinely treading well-worn, narrow paths of thinking. It’s how we simplify the complex reality around us.
Personally, I’ve found that with time and experience, my brain gets just a wee bit smarter with each passing year. (I wish it got much, much smarter, much, much faster. I’m running out of time!)
Unfortunately, I’ve also noticed that my brain gets increasingly stubborn with time—more prone to cling to what it deems to be intuitively right or historically correct.
“Of course I know this. I’ve seen it a hundred times before…”
But every brain should come with an operator’s warning:
DANGER! The growing schema of knowledge that empowers your judgment may one day turn around and bite you in the buttocks. And it will probably hurt.
Ultimately, perceptions that go unchallenged will fail us. And the only way to prevent a misconception, is to regularly challenge a preconception.
Properly performed and objectively analyzed, data analysis and visual diagramming can be your strongest allies in helping to keep mind your open to the counterintuitive.
A Modern Day Example: The Airline Boarding Process
One of my favorite recent examples of how conventional thinking can be turned on its head by objective analysis involves the airline boarding process. (Not everyone would call it a process—it often feels more like a cattle herding operation!)
For years, airlines used a “block” method to board the aircraft by sections. An organized procedure like this is bound to be much faster and more efficient than having passengers board a plane randomly, right?
Once researchers began to question the status quo method of boarding planes—using experiments, diagrams, Monte Carlo methods, regression, and hypothesis tests—many found that (surprise!) the organized block method of boarding a plane turns out to be slower than many other boarding methods, including using a random procedure! (Steffan and Hotchkiss, 2011; Steiner and Phillip, 2009; Inman, Jones, and Thompson, 2007).
Of course, some airlines still use the block procedure for boarding, despite the recent hard evidence against it. But that's not surprising. Because in the 1850s London, most public health officials in London didn't initially accept Snow's theory of water-borne transmission for cholera, even after seeing his graphic evidence.
What can you do? Try another diagram. Run another data analysis. Then, take the long view...
Friday, March 15 was the 200th anniversary of John Snow’s birth in York, England. If you're ever in London, you can visit the infamous Broad Street pump that Snow identified as the source of the cholera outbreak of 1854.
- For a great historical account of how Snow analyzed the mortality data and created a Voronoi diagram, read The Ghost Map, by Steven Johnson (Riverhead books). I've summarized some of the information from Steven's book for this post.