You know that using statistical quality analysis to ensure that your product meets requirements is a great way to keep your customers happy. But as you analyze your data, there’s another person you want to keep happy—a statistician.
As we saw in a previous blog, performing a hypothesis test repeatedly without controlling the family error rate increases the likelihood that any statistically significant results you find may be errors. And with each unplanned comparison you test, the frown on a statistician’s face deepens.
How can you turn the statistician's frown upside-down? By using a statistical analysis specifically designed to perform multiple comparisons, such as Minitab’s One-Way Analysis of Variance (ANOVA).
Multiple Flutters, Multiple Cures, Multiple Comparisons
Suppose you’re the world’s leading medical researcher of synchronous diaphragmatic flutter, also known as “hiccups”. You want to compare 4 home remedies for hiccups from among the 78 “cures” you find listed online.
Using a randomized study, you measure the amount of time that elapses until hiccups stop after applying each remedy. The results for each remedy are summarized by the Minitab boxplots below:
There don't appear to be any outliers that could unduly influence the test results. You wouldn't want to include extreme data due to special causes, such as Charles Osborne from Iowa, who fell down while slaughtering a hog and then hiccupped nonstop for almost 70 years, racking up over 430 million hiccups.
Offering money for hiccups appears to stop them the fastest. Drinking pickle juice and yelling "Awoga" seem to produce the slowest results. But are any of these differences statistically significant?
To find out, run Minitab’s Stat > ANOVA > One Way. Here's the output if you perform the analysis using the default settings:
Because the p-value is less than alpha (0.05), at least one mean statistically differs from the others. But which means differ significantly?
To compare each pair of means, click the “Comparisons” button when you set up One-Way ANOVA. Then choose a multiple comparison procedure. Tukey’s, the first choice on top, works well for most situations. (For more information on each procedure, click the Help button on the Comparisons dialog box).
Now when you run the analysis, the Minitab output shows you exactly which means are statistically different.
Here, only the difference between Money (108.93 minutes) and Pickle juice (168.93 minutes) is statistically significant.
Finally, to make a statistician truly happy, think beyond the statistics
The statistical results showed that Money is a significantly faster hiccup remedy than Pickle juice. But the actual difference was only about 60 seconds, on average.
Is one less minute of hiccupping a big deal? And is the faster cure worth all the $20 bills you'd have to dole out to people who couldn't stop hiccupping right away?
“Awoga” is not significantly different than Money. But although yelling doesn't cost anything, it will probably annoy people a lot more than hiccups.
The Pencil Bite remedy also is not significantly different than Money. However, biting a pencil while you swallow water may significantly increase your risk of esophageal graphitus, also known PST (pencil-stuck-in-throat) Syndrome.
Personally, I’d take the pickle juice—or the hiccups.