---
title: Lessons from a Statistical Analysis Gone Wrong, part 1
description: Lessons from a Statistical Analysis Gone Wrong, part 1
image: https://blog.minitab.com/hubfs/B2BML%20-%20NEW%20FILE%20MANAGER%20STRUCTURE/Blog%20Images/Imported_Blog_Media/pharoah-3.jpg
---
English
- [English](https://blog.minitab.com/en/?hsLang=en)
- [Français](https://blog.minitab.com/fr/?hsLang=en)
- [Deutsch](https://blog.minitab.com/de/?hsLang=en)
- [Português](https://blog.minitab.com/pt/?hsLang=en)
- [Español](https://blog.minitab.com/es/?hsLang=en)
- [日本語](https://blog.minitab.com/ja/?hsLang=en)
- [한국어](https://blog.minitab.com/ko/?hsLang=en)
- [中文（简体）](https://blog.minitab.com/zh/?hsLang=en)
- [ Partners](https://www.minitab.com/en-us/company/partners/)
- [ Contact](https://www.minitab.com/en-us/company/contact-us/)
- [ Talk to Minitab](https://www.minitab.com/en-us/talk-to-minitab/)
- [ My Account](https://licensing.minitab.com/account/manage)
[](https://www.minitab.com/en-us/)
- Products
Back
[Products](https://www.minitab.com/en-us/products/)
[All Products](https://www.minitab.com/en-us/products/)
- [Minitab Solution Center](https://www.minitab.com/en-us/products/minitab-solution-center/)
- [Minitab Statistical Software](https://www.minitab.com/en-us/products/minitab/)
- [Minitab Connect](https://www.minitab.com/en-us/products/connect/)
- [Minitab Model Ops](https://www.minitab.com/en-us/products/model-ops/)
- [Minitab Education Hub](https://www.minitab.com/en-us/products/education-hub/)
- [Minitab Engage](https://www.minitab.com/en-us/products/engage/)
- [Minitab Workspace](https://www.minitab.com/en-us/products/workspace/)
- [Real-Time SPC](https://www.minitab.com/en-us/products/real-time-spc/)
- [Prolink Data Collection & SPC](https://www.minitab.com/en-us/products/prolink)
- [Simul8 Discrete Event Simulation](https://info.minitab.com/en/new-to-simul8?hsLang=en)
- [SPM](https://www.minitab.com/en-us/products/spm/)
- Solutions
Back
[Solutions](https://www.minitab.com/en-us/solutions/)
[All Solutions](https://www.minitab.com/en-us/solutions/)
- [Analytics](https://www.minitab.com/en-us/solutions/analytics/)
- [Statistics & Predictive Analytics](https://www.minitab.com/en-us/solutions/analytics/statistical-analysis-predictive-analytics/)
- [Data Science & Machine Learning](https://www.minitab.com/en-us/solutions/analytics/data-science-machine-learning/)
- [Business Analytics & Intelligence](https://www.minitab.com/en-us/solutions/analytics/business-analytics-intelligence/)
- [Statistical Process Control](https://www.minitab.com/en-us/solutions/analytics/statistical-process-control/)
- [Quality Analytics](https://www.minitab.com/en-us/solutions/analytics/quality-analytics/)
- [Live Analytics](https://www.minitab.com/en-us/solutions/analytics/live-analytics/)
- [Reliability & Life Data Analysis](https://www.minitab.com/en-us/solutions/analytics/reliability/)
- [Discrete Event Simulation](https://info.minitab.com/en/new-to-simul8?hsLang=en)
- [Process Mining](https://info.minitab.com/en/new-to-simul8?hsLang=en)
- [Key Capabilities](https://www.minitab.com/en-us/solutions/key-capabilities/)
- [Continuous Improvement](https://www.minitab.com/en-us/solutions/key-capabilities/continuous-improvement/)
- [Data Integration & Data Prep](https://www.minitab.com/en-us/solutions/key-capabilities/data-integration-data-prep/)
- [Diagramming & Mind Mapping](https://www.minitab.com/en-us/solutions/key-capabilities/diagramming-mind-mapping/)
- [Digital Twins](https://info.minitab.com/en/new-to-simul8?hsLang=en)
- [Model Deployment & ML Ops](https://www.minitab.com/en-us/solutions/key-capabilities/model-deployment-ml-ops/)
- [Innovation & Project Management](https://www.minitab.com/en-us/solutions/key-capabilities/innovation-project-management/)
- [Process Excellence: Detect, Correct & Prevent](https://www.minitab.com/en-us/solutions/key-capabilities/process-excellence/)
- [Featured Industries](https://www.minitab.com/en-us/solutions/industry/)
- [Academic](https://www.minitab.com/en-us/solutions/industry/academic/)
- [Energy & Natural Resources](https://www.minitab.com/en-us/solutions/industry/energy-resources/)
- [Government & Public Sector](https://www.minitab.com/en-us/solutions/industry/government/)
- [Healthcare](https://www.minitab.com/en-us/solutions/industry/healthcare/)
- [Insurance](https://www.minitab.com/en-us/solutions/industry/insurance/)
- [Manufacturing & Industrial](https://www.minitab.com/en-us/solutions/industry/manufacturing-industrial/)
- [Services](https://www.minitab.com/en-us/solutions/industry/service-industries/)
- [Software & Technology](https://www.minitab.com/en-us/solutions/industry/software-technology/)
- [Construction](https://www.minitab.com/en-us/solutions/industry/construction/)
- [Featured Roles](https://www.minitab.com/en-us/solutions/role/)
- [Engineering](https://www.minitab.com/en-us/solutions/role/engineering/)
- [Business Analysis](https://www.minitab.com/en-us/solutions/role/business-analysts/)
- [Information Technology](https://www.minitab.com/en-us/solutions/role/information-technology/)
- [Supply Chain](https://www.minitab.com/en-us/solutions/role/supply-chain/)
- [Customer Service & Contact Center](https://www.minitab.com/en-us/solutions/role/customer-service-contact-center/)
- [Human Resources](https://www.minitab.com/en-us/solutions/role/human-resources/)
- [Marketing](https://www.minitab.com/en-us/solutions/role/marketing/)
- [Research and Development](https://www.minitab.com/en-us/solutions/role/research-and-development/)
- Resources & Services
Back
[Resources & Services](https://www.minitab.com/en-us/resources-services/)
[All Resources & Services](https://www.minitab.com/en-us/resources-services/)
- [Resources](https://www.minitab.com/en-us/resources-services/resources/)
- [Case Studies](https://www.minitab.com/en-us/resources-services/resources/case-studies/)
- [eBooks & White Papers](https://www.minitab.com/en-us/resources-services/resources/ebooks-and-white-papers/)
- [Blog](https://blog.minitab.com/blog/?hsLang=en)
- [Data Sets](https://www.minitab.com/en-us/resources-services/resources/data-set/)
- [Webinars & Events](https://www.minitab.com/en-us/resources-services/resources/events/)
- [Education Hub](https://www.minitab.com/en-us/resources-services/resources/education-hub/)
- [Services](https://www.minitab.com/en-us/resources-services/services/)
- [Training](https://www.minitab.com/en-us/resources-services/services/training/)
- [Deployment](https://www.minitab.com/en-us/resources-services/services/deployment/)
- [Consulting](https://www.minitab.com/en-us/resources-services/services/statistical-consulting/)
- [Self-Paced Learning](https://www.minitab.com/en-us/resources-services/services/self-paced-learning/)
- [Continuing Education](https://www.minitab.com/en-us/resources-services/services/continuing-education/)
- Support
Back
[Support](https://www.minitab.com/en-us/support/)
[Technical Support](https://www.minitab.com/en-us/support/)
- [Subscriptions & Activation](https://www.minitab.com/en-us/support/subscriptions-and-activation/)
- [Minitab Quick Start](https://www.minitab.com/en-us/support/quick-start/)
- [Training](https://www.minitab.com/en-us/resources-services/services/training/)
- [Installation Support](https://www.minitab.com/en-us/support/installation/)
- [Support Videos](https://www.minitab.com/en-us/support/videos/)
- [Support Documentation](https://www.minitab.com/en-us/support/documents/)
- [Software Updates](https://www.minitab.com/en-us/support/software-updates/)
- [Product Downloads](https://www.minitab.com/en-us/support/downloads/)
- [Support Policy](https://www.minitab.com/en-us/support/policy/)
- Company
Back
[Company](https://www.minitab.com/en-us/company/)
[Company](https://www.minitab.com/en-us/company/)
- [About Us](https://www.minitab.com/en-us/company/our-company/)
- [Leadership Team](https://www.minitab.com/en-us/company/leadership-team/)
- [Partners](https://www.minitab.com/en-us/company/partners/)
- [Careers](https://www.minitab.com/en-us/company/careers/)
- [Contact](https://www.minitab.com/en-us/company/contact-us/)
- [News](https://www.minitab.com/en-us/company/press-releases/)
[Try/Buy](https://www.minitab.com/en-us/try-buy/)
- [Statistics](https://blog.minitab.com/en/blog/tag/statistics)
- [Data Analysis](https://blog.minitab.com/en/blog/tag/data-analysis)
-
- [Quality Improvement](https://blog.minitab.com/en/blog/tag/quality-improvement)
- [Subscribe](https://info.minitab.com/blog/subscribe?hsLang=en)
## Notice
You are now leaving minitab.com.
Click Continue to proceed to:
Continue
Go Back
# Lessons from a Statistical Analysis Gone Wrong, part 1
Minitab Blog Editor | 7/14/2015
Topics: [Capability Analysis](https://blog.minitab.com/en/blog/tag/capability-analysis), [Statistics in the News](https://blog.minitab.com/en/blog/tag/statistics-in-the-news), [Data Analysis](https://blog.minitab.com/en/blog/tag/data-analysis), [Statistics](https://blog.minitab.com/en/blog/tag/statistics), [Hypothesis Testing](https://blog.minitab.com/en/blog/tag/hypothesis-testing)
- [Share](https://www.linkedin.com/shareArticle?mini=true&url=https://blog.minitab.com/en/blog/understanding-statistics/lessons-from-a-statistical-analysis-gone-wrong-part-3-v2)
- [Share](https://www.facebook.com/sharer.php?u=https://blog.minitab.com/en/blog/understanding-statistics/lessons-from-a-statistical-analysis-gone-wrong-part-3-v2)
I don't like the taste of crow. That's a shame, because I'm about to eat a huge helping of it.
I'm going to tell you how I messed up an analysis. But in the process, I learned some new lessons and was reminded of some older ones I should remember to apply more carefully.
## This Failure Starts in a Victory
My mistake originated in the 2015 Triple Crown victory of American Pharoah. I'm no racing enthusiast, but I knew this horse had ended almost four decades of Triple Crown disappointments, and that was exciting. I'd never seen a [Triple Crown](https://blog.minitab.com/blog/the-statistics-game/triple-crown-odds-ill-have-another?hsLang=en) won before. It hadn't happened since 1978.
So when an acquaintance asked to contribute a guest post to the Minitab Blog that compared American Pharoah with previous Triple Crown contenders, including the record-shattering Secretariat, who took the Triple Crown in 1973, I eagerly accepted.
In reviewing the post, I checked and replicated the contributor's analysis. It was a fun post, and I was excited about publishing it. But a few days after it went live, I had to remove it: the analysis was not acceptable.
To explain how I made my mistake, I'll need to review that analysis.
## Comparing American Pharoah and Secretariat
In the post, we used Minitab's [statistical software](http://www.minitab.com/products/minitab/) to compare Secretariat's performance to other winners of Triple Crown races.
Since 1926, the Belmont Stakes has been the longest of the three races at 1.5 miles. The analysis began by charting 89 years of winning horse times:
Only two data points were outside of the I-chart's control limits:
- The fastest winner, Secretariat's 1973 time of 144 seconds
- The slowest winner, High Echelon's 1970 time of 154 seconds
The average winning time was 148.81 seconds, which Secretariat beat by more than 4 seconds.
## Applying a Capability Approach to the Race Data
Next, the analysis approached the data from a capability perspective: Secretariat's time was used as a lower spec limit, and the analysis sought to assess the probability of another horse beating that time.
The way you assess capability depends on the distribution of your data, and a normality test in Minitab showed this data to be nonnormal.
When you run Minitab's normal capability analysis, you can elect to apply the Johnson transformation, which can automatically transform many nonnormal distributions before the capability analysis is performed. This is an extremely convenient feature, but here's where I made my mistake.
Running the capability analysis with Johnson transformation, using Secretariat's 144-second time as a lower spec limit, produced the following output:
The analysis found a .36% chance of any horse beating Secretariat's time, making it very unlikely indeed.
The same method was applied to Kentucky Derby and Preakness data.
We found a 5.54% chance of a horse beating Secretariat's Kentucky Derby time.
We found a 3.5% probability of a horse beating Secretariat's Preakness time.
Despite the billions of dollars and countless time and effort spent trying to make thoroughbred horses faster over the past 43 years, no one has yet beaten “Big Red,” as Secretariat was known. So the analysis indicated that American Pharoah may be a great horse, but he is no Secretariat.
That conclusion may well be true...but it turns out we can't use *this* analysis to make that assertion.
## My Mistake Is Discovered, and the Analysis Unravels
Here's where I start chewing those crow feathers. A day or so after sharing the post about American Pharoah, a reader sent the following comment:
*Why does Minitab allow a Johnson Transformation on this data when using **Quality Tools > Capability Analysis > Normal > Transform**, but does not allow a transformation when using **Quality Tools > Johnson Transformation**? Or could I be doing something wrong? *
Interesting question. Honestly, it hadn't even occurred to me to try to run the Johnson transformation on the data by itself.
But if the Johnson Transformation worked when performed as part of the capability analysis, it ought to work when applied outside of that analysis, too.
I suspected the person who asked this question might have just checked a wrong option in the dialog box. So I tried running the Johnson Transformation on the data by itself.
The following note appeared in Minitab's session window:
Uh oh.
Our reader *hadn't* done anything wrong, but it was looking like I made an error somewhere. But where?
I'll show you exactly where I made my mistake in [my next post.](https://blog.minitab.com/blog/understanding-statistics/lessons-from-a-statistical-analysis-gone-wrong-part-2?hsLang=en)
Photo of American Pharoah used under Creative Commons license 2.0. Source: Maryland GovPics [https://www.flickr.com/people/64018555@N03](https://www.flickr.com/people/64018555@N03)
## You Might Also Like
[New Minitab Updates Unite Data Collection, Preparation, and Predictive Analytics](https://blog.minitab.com/en/blog/new-minitab-updates-unite-data-collection-preparation-and-predictive-analytics?hsLang=en)
[Data Preparation](https://blog.minitab.com/en/blog/tag/data-preparation) | 5 Minute Read
[Artemis II Heat Shield: How Do You Decide in Uncertain Conditions?](https://blog.minitab.com/en/blog/artemis-ii-heat-shield-how-do-you-decide-under-uncertainty?hsLang=en)
[Data Analysis](https://blog.minitab.com/en/blog/tag/data-analysis) | 4 Minute Read
[6 Trends Shaping Business Decisions in 2026](https://blog.minitab.com/en/blog/6-trends-shaping-business-decisions-in-2026?hsLang=en)
[Oee](https://blog.minitab.com/en/blog/tag/oee) | 8 Minute Read
---
- [Careers](https://www.minitab.com/en-us/company/careers/)
- [Trust Center](https://www.minitab.com/en-us/trust-center/)
- [Suppliers](https://www.minitab.com/en-us/suppliers/)
- [Site Map](https://www.minitab.com/en-us/site-map/)
- [](https://www.linkedin.com/company/minitab/)
- [](https://www.facebook.com/Minitab/)
- [](https://www.youtube.com/user/MinitabInc)
- [](https://www.instagram.com/minitab/)
---
© 2025 Minitab, LLC. All Rights Reserved.
- [Terms of Use](https://www.minitab.com/en-us/legal/terms-of-use/)
- [Privacy Policy](https://www.minitab.com/en-us/legal/privacy-policy/)
- [Legal](https://www.minitab.com/en-us/legal/)
- Cookies Settings