Minitab | MinitabBlog posts and articles about using Minitab software in quality improvement projects, research, and more.
http://blog.minitab.com/blog/minitab/rss
Sat, 20 Sep 2014 07:56:17 +0000FeedCreator 1.7.3Not Getting a No-Hitter? Statistically Speaking, the Best Bet Ever
http://blog.minitab.com/blog/the-statistics-game/not-getting-a-no-hitter-statistically-speaking2c-the-best-bet-ever
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/fe2c58f6-2410-4b6f-b687-d378929b1f9b/Image/ca5dc4e25f623c98b4c0ab10d4eeba50/money_w640.png" style="width: 325px; height: 217px; float: right; margin: 10px 15px;" />The no-hitter is one of the most impressive feats in baseball. It’s no easy task to face more than 27 batters without letting one of them get a hit. So naturally, no-hitters don’t occur very often. In fact, since 1900 there has been an average of only about 2 no-hitters per year.</p>
<p>But what if you had the opportunity to bet that one <em>wouldn’t </em>occur?</p>
<p>That’s exactly what happened to sportswriter C. Trent Rosecrans. He had a friend who kept insisting the Reds would be no-hit his season. And with 24 games left in the season, the friend put his money where his mouth is, betting Mr. Rosecrans <a href="http://www.cincinnati.com/story/redsblog/2014/09/17/bar091714/15767373/">$5 that the Reds would be no-hit</a> by the end of the year.</p>
<p>Even if the Reds <em>do </em>have one of the worst hitting percentages in baseball, would you take the bet that in 24 games there won’t be an event that occurs only twice in an entire year?</p>
<p>Sounds like a no-brainer.</p>
Calculating the odds
<p>Back in 2012, I <a href="http://blog.minitab.com/blog/the-statistics-game/the-odds-of-throwing-a-perfect-game">calculated that the odds of throwing a no-hitter</a> were approximately 1 in 1,548. If you update that number to include all the games and no-hitters that have occurred since 2012, the odds become 1 in 1,562. The numbers are very similar, but we’ll use the latter since it incorporates more data.</p>
<p>So there is a 99.936% chance that a no-hitter does not occur in any single game. But the bet was that it wouldn’t occur in 24 games. What are Mr. Rosencrans' chances of winning the bet?</p>
<p align="center"><strong>24 games without a no-hitter</strong> = .99936^24 = .98475 = approximately <strong>98.475%</strong></p>
<p>I wish <em>I</em> could make bets with a winning percentage that was that high! For Mr. Rosecrans, 98.475% of the time he’ll win $5, and 1.525% of the time he’ll lose $5. For his friend, the opposite is true. We can use these numbers to calculate the expected value for each side of the bet.</p>
<p align="center">Reds don’t get no-hit: (0.98475*5) – (0.01525*5) = <strong>$4.85</strong></p>
<p align="center">Reds get no-hit: (.01525*5) – (0.98475*5) = <span style="color:#FF0000;"><strong>-$4.85</strong></span></p>
Making it a fair bet
<p>Obviously this was just a friendly wager and was not meant to be taken too seriously. If Mr. Rosecrans regularly made bets with expected values close to $5 with all of his friends, he probably wouldn’t have many left. But what if he wanted to be a <em>nice </em>friend? How much money should he have offered in return to make it a fair bet? We’ll simply set the expected value to 0 and solve for the amount of money he’d lose the 1.525% of the time the Reds were no-hit.</p>
<p align="center">0 = (0.98475*5) – (0.01525*X)</p>
<p align="center">0.01525*X = 4.92375</p>
<p align="center">X = $322.87</p>
<p>To make the bet fair, Mr. Rosecrans should offer to pay his friend $322.87 if the Reds get no-hit. And earlier this week the Reds didn’t get their first hit until the 8th inning. Imagine sweating out <em>that </em>game if you had over $300 on the line!</p>
Adjusting for the Reds
<p>One of the reasons the friend bet on the Reds to be no-hit was that they are one of the worst-hitting teams in their league. Their batting average of 0.238 is ranked 28th in baseball. That means, on average, a Reds batter <em>won’t</em> hit the ball 76.2% of the time. So if a pitcher wanted to no-hit the Reds, they would need to face at least 27 batters who didn’t get a hit.</p>
<p align="center"><strong>Probability of having 27 straight batters not have a hit</strong> = 0.762^27 = 0.00065 = <strong>approx. 1 in 1,539</strong></p>
<p>But remember, just because a batter doesn’t get a hit does not mean they’re out. They can get walked, hit by a pitch, or reach on an error. Unless they pitch a perfect game, the pitcher will face more than 27 batters. Let’s look how the probability changes as we increase the number of Reds batters that the pitcher must face without allowing a hit.</p>
<p align="center"><strong>Probability of having 28 straight batters not have a hit</strong> = 0.762^28 = <strong>approx. 1 in 2,020</strong></p>
<p align="center"><strong>Probability of having 29 straight batters not have a hit</strong> = 0.762^29 = <strong>approx. 1 in 2,650</strong></p>
<p align="center"><strong>Probability of having 30 straight batters not have a hit</strong> = 0.762^30 = <strong>approx. 1 in 3,478</strong></p>
<p align="center"><strong>Probability of having 31 straight batters not have a hit</strong> = 0.762^31 = <strong>approx. 1 in 4,565</strong></p>
<p>This was <em>supposed</em> to show that because they are a poor-hitting team, the Reds have a better chance of being no-hit than the average used above. But as you can see, that’s not the case at all. Despite being one of the worst-hitting teams in the league, it appears that it’s <em>harder</em> to no-hit the Reds than the historical average.</p>
<p>Things get even odder when you consider that the average batting average (according to <a href="http://www.baseball-reference.com/leagues/MLB/bat.shtml">Baseball-Reference.com</a>) is 0.263. Using that number, the odds of having 27 straight batters not have a hit is 1 in 3,788. And those odds drop as you increase the number of batters the pitcher has to face. Applying this probability to the number of games played since 1900, we would expect there to be fewer than 100 no-hitters. And how many have there been? <em>241</em>!</p>
<p>This is the same conundrum I encountered when finding <a href="http://blog.minitab.com/blog/the-statistics-game/the-odds-of-throwing-a-perfect-game-part-ii">the odds of throwing a perfect game</a>. The number of perfect games and no-hitters that have occurred is <em>much higher</em> than what we would expect based on historical batting statistics. One explanation could be pitching from the wind-up vs. the stretch. With no runners on base (which is always the case in a perfect game and often the case in a no-hitter), the pitcher can always throw from the wind-up. Assuming pitchers are better when pitching from the wind-up, this would result in a lower batting average than normal, thus explaining the higher number of perfect games and no hitters. This would make for a great analysis using Minitab <a href="http://www.minitab.com/products/minitab">Statistical Software</a>, but since we can’t separate the data on hand into at bats facing pitchers throwing from the stretch vs. the wind-up, we can't test the theory.</p>
<p>Since the Reds have a batting average .025 points lower than the historical average, it’s probably safe to assume they do in fact have a greater chance of being no-hit. The problem is that it’s nearly impossible to quantify how much greater!</p>
Looking ahead to next year
<p>With the season almost over, it’s unlikely the Reds will be no-hit this year. But what if the two friends decided to do their bet again next year, only this time, they do it at the start of the season. Let’s use our original probability of throwing a no hitter (the one we’ve observed) and determine what the odds are that the Reds go 162 games getting at least one hit per game.</p>
<p align="center"><strong>162 games without a no-hitter</strong> = .99936^162 = .9015 = approximately <strong>90.15%</strong></p>
<p>The probability of the Reds getting no-hit is still pretty low, but it’s a lot better than the current bet. I just hope next year the friend gets some better odds than even money!</p>
Data AnalysisFun StatisticsStatistics in the NewsFri, 19 Sep 2014 13:35:15 +0000http://blog.minitab.com/blog/the-statistics-game/not-getting-a-no-hitter-statistically-speaking2c-the-best-bet-everKevin RudyAttendance Awareness Month: A Graphic Look at the Data
http://blog.minitab.com/blog/statistics-and-quality-improvement/attendance-awareness-month-a-graphic-look-at-the-data
<p>You might not have known, but September is Attendance Awareness Month. Specifically, attendance of children at American public schools. The organization Attendance Works recently came out with a report that highlights the learning gap between students with strong attendance and students with poor attendance.</p>
<p><a href="http://www.minitab.com/products/minitab">Statistical software</a> helps us quickly and easily create graphs that make it easier to understand what a set of data is telling us. To encourage everyone to recognize the importance of school attendance, here are some Minitab graphs of <span style="line-height: 20.7999992370605px;">data</span><span style="line-height: 20.7999992370605px;"> shared by </span><span style="line-height: 1.6;">Attendance Works and other researchers that the authors cite. </span></p>
Attendance's association with academic performance
<p>The main point of the Attendance Works report is to point out the association between attendance and the National Assessment for Education Progress (NAEP).</p>
<p>Students who routinely miss school, tend to score more than one grade level below their peers on math and reading assessments in 4th and 8th grade. It looks like missing school makes academic achievement harder.</p>
<p><img alt="Students who miss 3 or more school days in a month tend to be a grade level behind their peers in math and english test scores." src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/ac11e79e894d338e2a2d28480bf205f5/naep_scores.png" style="width: 576px; height: 384px;" /></p>
Well-begun is half done
<p>Data from California in 2011 shows that being chronically absent in kindergarten and first grade has a strong association with whether a child is a proficient reader in 3rd grade. Only 17% of students who were chronically absent in kindergarten and first grade achieved a proficient score on a test of the English language given in 3rd grade. Of students with better attendance records, 64% achieved a proficient score.</p>
<p><img alt="64% of students who were not chronically absent in kindergarten and first grade were proficient on exams in 3rd grade. 17% of students who were chronically absent were proficient." src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/8520e1e15d5b103d377a9f501c297ec2/chart_of_proficient_on_3rd_grd_english_l.png" style="width: 576px; height: 384px;" /></p>
Not just a problem in high school
<p>Data from students in Rhode Island indicate that even when chronically absent students can graduate from high school, they will likely struggle to continue their educations successfully. Only 11% of chronically absent students went on to a second year of college education, giving them a chance to finish an associate’s degree. Over 50% of students who were not chronically absent in high school were able to begin a second year of college.</p>
<p><img alt="Over 50% of students with good attendnace began a second year of college. Only 11% of students who were chronically absent in high school began a second year of college." src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/8d02f9feef24f471b98bae609fe25903/chart_of_made_it_to_a_second_year_of_col.png" style="width: 576px; height: 384px;" /></p>
Go to school
<p>I’ve been fooled by the difference between <a href="http://blog.minitab.com/blog/statistics-and-quality-improvement/correlation-causation-and-remorse-for-my-nba-finals-prediction">correlation and causation</a> before. It’s entirely possible that other factors could impede a child’s learning whether they attend school properly or not.</p>
<p>Attendance Works identifies that some potential causes for chronic absenteeism might include lack of access to health care, community violence, unreliable transportation and unstable housing. These problems could also be responsible for poor school performance. <span style="line-height: 1.6;">But we should be on the lookout for absenteeism as a symptom that other problems need to be addressed and take the initiative to help solve those problems. </span></p>
<p><span style="line-height: 1.6;">For tools for people from all levels of involvement, from parents to city leaders, go to </span><a href="http://www.attendanceworks.org/tools/" style="line-height: 1.6;">http://www.attendanceworks.org/tools/</a></p>
Bonus
<p>For Attendance Awareness Month 2013, the Oakland County School district made a music video featuring Marshawn Lynch. Check that out!</p>
<p></p>
Statistics in the NewsWed, 17 Sep 2014 12:49:48 +0000http://blog.minitab.com/blog/statistics-and-quality-improvement/attendance-awareness-month-a-graphic-look-at-the-dataCody SteeleWhy Kurtosis is Like Liposuction. And Why it Matters.
http://blog.minitab.com/blog/statistics-and-quality-data-analysis/why-kurtosis-is-like-liposuction-and-why-it-matters
<p>The word<em> kurtosis</em> sounds like a painful, festering disease of the gums. But the term actually describes the shape of a data distribution.</p>
<p>Frequently, you'll see kurtosis defined as how sharply "peaked" the data are. The three main types of kurtosis are shown below.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/080ef9180b573d42447b383c589a4bfa/leptokurtosis_basic_w640.jpeg" style="width: 400px; height: 250px;" /></p>
<p><em>Lepto</em> means "thin" or "slender" in Greek. In <em>leptokurtosis</em>, the kurtosis value is high.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/c446e60709702009edb996a8298e4b50/platykurtosis_basic_w640.jpeg" style="width: 400px; height: 236px;" /></p>
<p><em>Platy</em> means "broad" or "flat"—as in duck-billed <em>platy</em>pus. In <em>platykurtosis</em>, the kurtosis value is low.</p>
<p> </p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/c0b09e243506563ccc348f07e212e24a/mesokurtosis_basic_w640.jpeg" style="width: 400px; height: 251px;" /></p>
<p><em>Meso</em> means "middle" or "between." The normal distribution is mesokurtic.</p>
<p>Mesokurtosis can be defined with a value of 0 (called its "excess" kurtosis value). Using that benchmark, leptokurtic distributions have positive kurtosis values and platykurtic distributions have negative kurtosis values.</p>
<p><strong>Question</strong>: Which type of kurtosis correctly describes each of the three distributions (blue, red, yellow) shown below?</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/e245552e7dac99f9ee357a335147443d/variances_kurtosis_w640.jpeg" style="width: 500px; height: 330px;" /></p>
<p><span style="color: rgb(105, 105, 105);"><strong>Answer</strong>: <em>All three distributions are examples of mesokurtosis. They're all normal distributions. The (excess) kurtosis value is 0 for each distribution</em>. </span></p>
<p>OK, that was a mean trick question. You can roast me in the comments field. But it had a good intention—to illustrate some common misconceptions about kurtosis.</p>
<p><span style="color: rgb(105, 105, 105);">Each normal distribution shown above has a different<a href="http://blog.minitab.com/blog/statistics-and-quality-data-analysis/variations-on-a-theme-of-variation-r-v-sd-se-and-ci" target="_blank"> variance</a><em>. </em></span>Different variances can appear to change the "peakedness" of a given distribution when they're displayed together along the same scale. But that's not the same thing as kurtosis.</p>
Think of Kurtosis Like Liposuction
<p>In Nature, there's no such thing as a free lunch—literally. Research suggests that fat that's liposuctioned from one part of the body <a href="http://www.webmd.com/beauty/liposuction/20110503/study-fat-may-return-after-liposuction" target="_blank">all returns within a year</a>. It just moves to a different place in the body.</p>
<p>Something similar happens with kurtosis. The clearest way to see this is to compare probability distribution plots for distributions with <em>the same variance</em> but with different kurtosis values. Here's an example.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/2ca5367dd436fd26f134f2c22a3e4848/lepto_lipo_w640.jpeg" style="width: 640px; height: 426px;" /></p>
<p>The solid blue line shows the normal distribution (excess kurtosis ≈ 0). That's the body before liposuction. The dotted red line shows a leptokurtic distribution (excess kurtosis ≈ 5.6) with the same variance. That's the body one year after liposuction.</p>
<p>The arrows show where the fat (the data) moves after being "sucked out" from the sides of the normal distribution. The blue arrows show that some data shifts toward the center, giving the leptokurtic distribution its characteristic sharp, thin peak.</p>
<p>But that's not where all the data goes. Notice the data that relocates to the extreme tails of the distribution, as shown by the red arrows.</p>
<p>So the leptokurtic distribution has a thinner, sharper peak, but also—very importantly— "fatter" tails.</p>
<p>Conversely, here's how "liposuction" of the normal distribution results in platykurtosis (excess kurtosis ≈ - 0.83).</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/4f0fdf8030ab43ee8ad89d7ababdd638/platykurtosis_lipo.jpg" style="width: 609px; height: 399px;" /></p>
<p> </p>
<p>Here, data from the peak <em>and from the tails</em> of the normal distribution is redistributed to the sides. This gives the platykurtic distribution its blunter, broader peak, but-— very importantly— its thinner tails.</p>
<p>In fact, kurtosis is actually more influenced by data in the tails of the distribution than data in the center of a distribution. It's really a measure of how heavy the tails of a distribution are <em>relative to its variance</em>. That is, how much the variation in the data is due to extreme values located further away from the mean.</p>
Why Does It Matter?
<p>Consider the three normal distributions that appeared to mimic different types of kurtosis, when in fact they had the same kurtosis, just different variances.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/e245552e7dac99f9ee357a335147443d/variances_kurtosis_w640.jpeg" style="width: 500px; height: 330px;" /></p>
<p>For each of these distributions, the same percentage of data falls within a given number of standard deviations from the mean. That is, for all three distributions, approximately 68.2% of observations are within +/- 1 standard deviation of the mean; 95.4% are within +/- 2 standards deviations of the mean; and 99.7% are within +/- 3 standard deviations of the mean.</p>
<p>What would happen if you tried to use this same rubric on a distribution that was extremely leptokurtic or platykurtic? You'd make some serious estimation errors due to the fatter (or thinner tails) associated with kurtosis.</p>
You Could Lose All Your Money, Too.
<p>In fact, something like that appears to have happened in the financial markets in the late 90s according to Wikipedia. Some hedge funds <a href="http://www.investopedia.com/terms/k/kurtosis.asp" target="_blank">underestimated the risk of variables with high kurtosis (leptokurtosis)</a>. In other words, their models didn't take into consideration the likelihood of data located in the "fatter" extreme tails—which was associated with greater volatility and higher risk. The end result? The hedge funds went belly up and needed bailing out.</p>
<p>I don't have a background in financial modelling, so I can't verify that claim. But it wouldn't surprise me.</p>
<p>If you click on the following link to Investopedia, you'll see a <a href="http://www.investopedia.com/terms/k/kurtosis.asp" target="_blank">definition of high kurtosis</a> as "a low, even distribution" with fat tails. Fat tails, yes. But "low and even"?</p>
<p>Hmmm. I hope the investment firm managing my 401K isn't using that definition.</p>
<p>If so, it might be time to move my money into an investment vehicle with a<em> much</em> lower kurtosis risk. Like my mattress.</p>
LearningMon, 15 Sep 2014 10:32:00 +0000http://blog.minitab.com/blog/statistics-and-quality-data-analysis/why-kurtosis-is-like-liposuction-and-why-it-mattersPatrick RunkelAnalysis and Reanalysis: The Controversy Behind MMR Vaccinations and Autism, part 2
http://blog.minitab.com/blog/adventures-in-statistics/analysis-and-reanalysis3a-the-controversy-behind-mmr-vaccinations-and-autism2c-part-2
<p>In my <a href="http://blog.minitab.com/blog/adventures-in-statistics/analysis-and-reanalysis3a-the-controversy-behind-mmr-vaccinations-and-autism2c-part-1">previous post</a>, I described how I was asked to weigh in on the ethics of researchers (DeStefano et al. 2004) who reportedly discarded data and potentially set scientific knowledge back a decade. I assessed the study in question and found that no data was discarded and that the researchers used good statistical practices.</p>
<p><img alt="Scientist at work" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/d76f9ec07b4fe5856bd4d604c60748cf/researchers.jpg" style="float: right; width: 300px; height: 199px; margin: 10px 15px;" />In this post, I assess a study by Brian S. Hooker that was recently published with a blast of many social media stories that accompanied it. Hooker reanalyzed the DeStefano data and concluded that certain African American boys have a 340% increased risk of developing autism after receiving the MMR vaccination.</p>
The Scenario
<p>After the study by DeStefano et al. was complete, the raw data was made available for other scientists to use. Hooker reanalyzed the data after being contacted by William Thompson, one of the authors in the original study. Thompson is a senior scientist at the CDC. Hooker’s study was published in the peer reviewed journal, <em>Translational Neurodegeneration</em>.</p>
<p>Thompson gave this <a href="http://www.cnn.com/2014/08/27/health/irpt-cdc-autism-vaccine-study/index.html" target="_blank">statement to CNN</a>: "I regret that my co-authors and I omitted statistically significant information in our 2004 article. I have had many discussions with Dr. Brian Hooker over the last 10 months regarding studies the CDC has carried out regarding vaccines and neurodevelopmental outcomes, including autism spectrum disorders. I share his belief that CDC decision-making and analyses should be transparent."</p>
<p>Brian Hooker and the social media stories allege that the original researchers deliberately excluded subjects to hide the increased risk for African American boys.</p>
<p>I personally don’t buy this theory because the DeStefano study analyzed the data from <em>all</em> subjects. The study compared the full results to a subanalysis of just the birth certificate sample, which had more complete data that included potential confounding variables. The two analyses agreed that the timing of the MMR vaccination did not affect the risk of developing autism.</p>
<p>Hooker is a scientific adviser for the Focus Autism Foundation, which believes that vaccines have helped cause an autism epidemic. He also has a 16-year-old son whom he describes as "vaccine-injured."</p>
Hooker (2014)
<p>Hooker used the DeStefano data. Where the two studies performed the same analyses, the results were the same. However, in general, Hooker used the data in a different manner and performed different analyses than DeStefano.</p>
<p>To derive his most startling conclusion, Hooker splits the data into two mutually exclusive groups:</p>
<ul>
<li>African-American children</li>
<li>All children <em>except </em>African-American children</li>
</ul>
<p>He then uses <a href="http://support.minitab.com/en-us/minitab/17/topic-library/basic-statistics-and-graphs/tables/chi-square/chi-square-tests-in-minitab/" target="_blank">Chi-square analysis</a> to look for an association between vaccination timing, gender, and the risk of autism within each of these groups. There are no significant results in the non-African American group at all.</p>
<p>Within the African American group, the only significant results are for the boys. The significant results are for African American boys vaccinated between:</p>
<ul>
<li>18 and 23 months, who have a relative risk of 1.73</li>
<li>24 and 35 months, who have a relative risk of 3.36</li>
</ul>
<p>The latter group is the one mentioned in the headlines that cite a 340% increased risk.</p>
My Verdict on the Hooker Study
<p>On the surface, it looks like there might be something to this study. However, there are problems lurking beneath the surface. The two major problems I see with Hooker’s study are:</p>
<ul>
<li>The full data set has been sliced and diced into a small, biased sample.</li>
<li>Low birth weight is an uncontrolled, confounding variable</li>
</ul>
<p><strong>Sample size</strong></p>
<p>To obtain the results, Hooker has to exclude data for all subjects except for the African American subjects. Even then, only African-American boys vaccinated at specific times were statistically significant.</p>
<p>Let’s look at the number of cases in the subgroup behind the shocking result at the heart of those social media stories and allegations of fraud—African American boys vaccinated between 24 and 35 months.</p>
<p>Consider the following:</p>
<ul>
<li>70% of the sample was vaccinated at less than 18 months because the guideline is to get the MMR vaccination between 12 and 17 months. There were no significant results in this group.</li>
<li>Only 7% of the sample was vaccinated during the time frame highlighted in the Hooker study.</li>
<li>Further sample reductions are necessary because we’re only assessing African American kids (37% of the sample), who are boys (80%).</li>
</ul>
<p>While neither study lists the number of autism cases for this super-specific sub-population, using the percentages and the number of cases, I can estimate that the shocking news of a “340% increase” is based on about 13 cases of autism!</p>
<p>This tiny sample size explains why the <a href="http://blog.minitab.com/blog/adventures-in-statistics/when-should-i-use-confidence-intervals-prediction-intervals-and-tolerance-intervals" target="_blank">confidence interval</a>, which measures the precision of the estimated risk, is so wide [1.5 to 7.51]. In this context, a 1 indicates no increase in risk, which is barely excluded from the CI.</p>
<strong>Uncontrolled confounding variable</strong>
<p>DeStefano used <a href="http://blog.minitab.com/blog/adventures-in-statistics/a-tribute-to-regression-analysis" target="_blank">regression analysis</a> to assess and control the effects of potential <a href="http://blog.minitab.com/blog/adventures-in-statistics/confound-it-some-more-how-a-factor-that-wasnt-there-hampered-my-analysis" target="_blank">confounders</a>. With regression analysis, he could study the effect of the various <a href="http://support.minitab.com/en-us/minitab/17/topic-library/modeling-statistics/regression-and-correlation/regression-models/what-are-response-and-predictor-variables/" target="_blank">predictors</a> (e.g., race, gender, birth weight) without having to subdivide the data. Instead, he included the predictors in the model to estimate the effects within the context of the full sample.</p>
<p>Hooker used Chi-squared analysis which cannot control for these confounders. That’s a huge problem for an observational study.</p>
<p>As for confounding variables, DeStefano found that low birth weights are associated with an increased risk of autism. In the original study, this and other potential confounders didn’t influence the uncontrolled results because they were evenly distributed across the groups of data. However, Hooker’s study sliced and diced the data so much that we can’t make this assumption.</p>
<p>Hooker wrote this about his highly pared down dataset: “It was found that there was a higher proportion of low birth weight African-Americans compared to the entire cohort.”</p>
<p>Hooker has a small data set in which a known confounder (low birth weight) is over-represented. Other studies have estimated that low birth weights can increase the risk of autism by 5 times! Because Hooker’s analysis does not control for this factor, we must assume that the estimated risk of autism is positively biased for this group. In other words, the estimated relative risk of 3.36 is likely higher than the true amount.</p>
<p>Thanks to the tiny sample and the uncontrolled confounding variable, Hooker’s results are both imprecise and biased. Consequently, my personal opinion is that Hooker’s results have no scientific value at all.</p>
<p>It turns out that <em>Translational Neurodegeneration</em>, the journal that published Hooker’s study, is having similar thoughts as well. During the course of writing this post, the journal has removed the article from its website and <a href="http://www.translationalneurodegeneration.com/content/3/1/16/abstract" target="_blank">stated</a>:</p>
<p>"This article has been removed from the public domain because of serious concerns about the validity of its conclusions. The journal and publisher believe that its continued availability may not be in the public interest. Definitive editorial action will be pending further investigation."</p>
Closing Thoughts
<p>In a previous post, <a href="http://blog.minitab.com/blog/adventures-in-statistics/why-statistics-is-important" target="_blank">Why Statistics is Important</a>, I wrote that how you perform statistics matters, and there are many potential pitfalls. Dissecting the problems in Hooker’s study is the perfect illustration of this!</p>
<p>The <a href="http://www.cdc.gov/ncbddd/autism/facts.html" target="_blank">CDC has stated</a> that not all risk factors are known and further study is required. Because we’re dealing with our children, an abundance of caution is required. Therefore, it is worthwhile to continue to investigate the possible risk factors for autism. Even though I don’t think Hooker’s study has scientific merit, if race is a potential a risk factor, we should study it further.</p>
<p><a href="http://www.pbs.org/wgbh/nova/body/vaccines-calling-shots.html" target="_blank"><img alt="Nova program, vaccines calling the shots" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/78f3e5ccef9dcec5f0daa0a91f59487d/nova.jpg" style="float: right; width: 300px; height: 168px; border-width: 1px; border-style: solid; margin: 10px 15px;" /></a>As for the larger debate of vaccinations and autism, the consensus of the scientific literature overwhelmingly supports the view that vaccinations do not increase the risk of autism. This finding is evident in many studies that assess vaccinations and the eventual diagnoses of autism. Other studies have found that autism starts in utero, long before a child is given the MMR or any other vaccinations.</p>
<p>Nova, the science television series on PBS in the U.S., will air a new documentary, <a href="http://www.pbs.org/wgbh/nova/body/vaccines-calling-shots.html" target="_blank">Vaccines - Calling the Shots</a>, on Wednesday, Sept. 10, 2014 at 9 p.m. EDT. Diseases that were largely eradicated are returning.</p>
Statistics in the NewsWed, 10 Sep 2014 12:00:00 +0000http://blog.minitab.com/blog/adventures-in-statistics/analysis-and-reanalysis3a-the-controversy-behind-mmr-vaccinations-and-autism2c-part-2Jim FrostAnalysis and Reanalysis: The Controversy Behind MMR Vaccinations and Autism, part 1
http://blog.minitab.com/blog/adventures-in-statistics/analysis-and-reanalysis3a-the-controversy-behind-mmr-vaccinations-and-autism2c-part-1
<p>The other day I received a request from a friend to look into a new study in a peer reviewed journal that found a link between MMR vaccinations and an increased risk of autism in African Americans boys. To draw this conclusion, the new study reanalyzed data that was discarded a decade ago by a previous study.</p>
<p><img alt="syringe" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/b6df9f4dcaeea9fca5b75b78be096f9f/syringe.jpg" style="float: right; width: 300px; height: 160px; margin: 10px 15px;" />My friend wanted to know, from a statistical perspective, was it unethical for the original researchers to discard the data for African Americans? Did they set scientific knowledge back a decade?</p>
<p>To answer these questions, I looked at the 2004 study, their decision to discard the data, and the reanalysis of this data in 2014.</p>
The Scenario
<p>I clicked the link that my friend sent and saw this headline: “Fraud at the CDC uncovered, 340% risk of autism hidden from public.” Yikes! Apparently variations on this story are making the rounds via social media. Here is the gist of the story.</p>
<p>In 2004, a major study by DeStefano et al.1 determined that there was no connection between the timing of measles-mumps-rubella (MMR) vaccinations and autism. This study was conducted by researchers at the Center for Disease Control (CDC). The CDC presents this study as a key piece of evidence in the debate about vaccinations and autism.</p>
<p>In 2014, Brian S. Hooker2 reanalyzed the 2004 data and found a possible link between MMR vaccinations and the risk of developing autism for a very specific group. African American boys who had their first MMR vaccination between 24 and 35 months of age were 3.4 times more likely to develop autism.</p>
<p>The fraud allegation is made by anti-vaccination groups who say that the CDC applied political pressure to discard the data in order to high these explosive findings.</p>
My Perspective
<p>I’m very aware that this specific case is just a part of a larger, very controversial issue. For this post, I assess the statistical validity of these two studies, and the decision to discard data.</p>
<p>I’m in favor of reanalyzing previous studies. I think it’s great if a later researcher can draw more or better conclusions from a data set. However, all studies must be statistically valid, or their results can be misleading.</p>
Observational Studies of Vaccination and Autism
<p>The original dataset was collected for an observational study. Because both studies use the same data set, they’re both observational studies. To evaluate the validity of the two studies, we need to understand the differences between observational studies and randomized control trials (RCTs),</p>
<p>In an RCT, all subjects are randomly assigned to the treatment and control groups. This process helps assure that the groups are similar to each other when treatment begins. Therefore, any post-study differences between groups are probably due to the treatment rather than prior differences.</p>
<p>For observational studies, there is no random assignment, which increases the chances that the treatment and control groups are not equivalent at the beginning of the study. Consequently, differences in the outcome at the end could be due to the preexisting differences (confounding variables). If the analysis does not account for the confounding factors, the results will be biased.</p>
<p>These issues are crucial in evaluating these two studies. For more information, read my posts about <a href="http://blog.minitab.com/blog/adventures-in-statistics/confound-it-some-more-how-a-factor-that-wasnt-there-hampered-my-analysis" target="_blank">confounding variables</a>, <a href="http://blog.minitab.com/blog/adventures-in-statistics/use-random-assignment-in-experiments-to-combat-confounding-variables" target="_blank">random assignment</a>, and an in depth look at an observational study about <a href="http://blog.minitab.com/blog/adventures-in-statistics/statistics-that-affect-you-are-vitamin-supplements-really-harmful" target="_blank">the effects of vitamins</a>.</p>
DeStefano et al. (2004)
<p>In one corner we have the original DeStefano (2004) study that agrees with the scientific consensus and concludes there is no connection between MMR vaccinations and autism risk. Researchers compare 624 children with autism, age 3 to 10, with 1,824 developmentally healthy children. Most of the children were vaccinated between 12 and 17 months of age in accordance with vaccination recommendations.</p>
<p>The first thing that I noticed is that DeStefano et al. did not actually exclude data as the critics claim. The claim is that the study discarded data from the subjects who could not be linked to birth certificates in order to hide the findings about African-American boys.</p>
<p>There are three issues to consider here:</p>
<p>One, the non-birth-certificate group was not based on race, but on whether the subject was matched to a Georgia birth certificate. African-American kids were actually underrepresented in the non-birth-certificate group. The birth certificate group was made up of 37.9% African Americans while the supposedly excluded non-birth-certificate group contained only 32.2%.</p>
<p>Two, while making the distinction between kids based on birth certificates may seem arbitrary, there are good data reasons for doing so. For those kids who could be matched to Georgia birth certificates, the researchers were able to gather more complete information on possible risk factors for autism, such as the child’s birth weight, mother’s age, and education. This information was not available for children who could not be matched to Georgia birth certificates.</p>
<p>Because this is an observational study, it’s crucial to statistically control for these risk factors. Just having more data isn't always a good thing. You need to record and analyze the potential confounders.</p>
<p>Three, the DeStefano study did not <em>exclude </em>the kids who were not linked to birth certificates. Instead, the study performed an analysis on the full sample (subjects linked and not linked to birth certificates) and a separate analysis of those linked to a birth certificate. So, <em>no data</em> was actually excluded. The results of the two analyses were in agreement.</p>
<p>The study found that some of the additional variables in the birth certificate sample are potential confounders because they are associated with autism risk. The risk factor that becomes important for this post is that low birth weight is associated with a higher risk of autism. We come back to that in the Hooker study.</p>
<p>Using <a href="http://blog.minitab.com/blog/adventures-in-statistics/a-tribute-to-regression-analysis" target="_blank">regression analysis</a>, the study concluded that the timing of MMR vaccinations is not associated with the risk of developing autism.</p>
My Verdict on the DeStefano Study
<p>The criticism that the study discarded data from African American subjects just doesn’t hold water. No data was discarded. For the subjects who were linked to birth certificates, the researchers performed additional analyses. In this light, I see a careful observational study that assessed the role of potential confounders.</p>
<p>The biggest weakness that I see for this study is that the researchers could not compare subjects who were vaccinated for MMR to those who were not vaccinated at all. The authors wrote that they “lacked an unvaccinated comparison group.” The truth is that the vast majority of kids are vaccinated. Consequently, this study compared the distribution of vaccination ages for case and control children to see if the timing impacted the risk of autism. It didn’t.</p>
<p>DeStefano et al. cite a large Dutch study that did include an unvaccinated group of nearly 100,000 subjects. This study found no increased risk for those who were vaccinated compared to the completely unvaccinated group.</p>
<p>Stay tuned! Tomorrow I’ll take a look at <a href="http://blog.minitab.com/blog/adventures-in-statistics/analysis-and-reanalysis3a-the-controversy-behind-mmr-vaccinations-and-autism2c-part-2">Hooker’s reanalysis of the DeStefano data</a>.</p>
<p>1. DeStefano, Frank, Tanya Karapurkar Bhasin, William W. Thompson, Marshalyn Yeargin-Allsopp, and Coleen Boyle, Age at First Measles-Mumps-Rubella Vaccination in Children with Autism and School-Matched Control Subjects: A Population-Based Study in Metropolitan Atlanta, <em>Pediatrics</em>, 2004;113;259</p>
<p>2. Hooker, Brian S., Measles-mumps-rubella vaccination timing and autism among young African American boys: a reanalysis of CDC data, <em>Translational Neurodegeneration</em> 2014, 3:16</p>
Statistics in the NewsTue, 09 Sep 2014 12:00:00 +0000http://blog.minitab.com/blog/adventures-in-statistics/analysis-and-reanalysis3a-the-controversy-behind-mmr-vaccinations-and-autism2c-part-1Jim FrostSwitch the Inner and Outer Categories on a Bar Chart
http://blog.minitab.com/blog/statistics-and-quality-improvement/switch-the-inner-and-outer-categories-on-a-bar-chart
<p>Did you just go shopping for school supplies? If you did, you’ve participated in what’s become the second biggest spending season of the year in the United States, according to the National Retail Federation (NRF). <img alt="Kids running in backpacks" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/b8bd8f188623299c5be19c86c99c2433/backpacks.jpg" style="float: right; width: 300px; height: 170px; margin: 10px 15px;" /></p>
<p>The trends and analysis are so interesting to the NRF that they actually add questions about back-to-school shopping to two monthly consumer surveys. The two surveys have different questions, but there’s one case where the allowed responses are the same. In July, the survey asked, “Where will you purchase back-to-school items this year?” In August, the survey asked, “Where do you anticipate you will do the remainder of your Back-to-School shopping?”</p>
<p>Did people give the same answers both times? Let’s use <a href="http://www.minitab.com/en-us/products/minitab/features/">Minitab Statistical Software</a> to find out. Doing so will give us a chance to see how easy it is to change the focus of a chart by switching the inner and outer categories on a bar chart.</p>
<strong>Did people answer the same way in both surveys? Yes.</strong>
<p>Let’s say that your data are in the same layout as the original NRF reports. Each row contains the percentage for a different location. I put the dates in two different columns because the numbers came from two different PDF files (<a href="https://nrf.com/sites/default/files/BTS%207-09-14%20press.pdf">July</a> and <a href="https://nrf.com/sites/default/files/Documents/BTS%20Update%208-2014.pdf">August</a>).</p>
<p><img alt="Percentages of people who said that they would shop at each location." src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/5b7d784b09b8b949c0ca3ebcfebaea72/data_window.png" style="width: 313px; height: 202px;" /></p>
<p>Making a bar chart in Minitab is easy, so follow along if you like:</p>
<ol>
<li>Choose <strong>Graph > Bar Chart</strong>.</li>
<li>In <strong>Bars represent</strong>, select <strong>Values from a table</strong>.</li>
<li>Under <strong>Two-way table</strong>, select <strong>Cluster</strong>. Click <strong>OK</strong>.</li>
<li>In <strong>Graph variables</strong>, enter <em>'7/1 to 7/8 2014' '8/5 to 8/12 2014'</em></li>
<li>In <strong>Row labels</strong>, enter <em>'Where will you purchase?'</em> Click <strong>OK</strong>.</li>
</ol>
<p>From this display, you can quickly determine that the order of the categories is the same in each survey. In both cases, most consumers plan to shop the most at discount stores and the least from catalogs. In fact, the popularity of where consumers planned to shop and where they planned to finish shopping has a constant order.</p>
<p><img alt="With month outermost, you can see that the popularity of the categories is the same in both surveys." src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/c79ca1158307cd23045fd40ebb404645/outermost_month.png" style="width: 576px; height: 384px;" /></p>
<strong>Did people answer the same way in both surveys? No.</strong>
<p>The order of popularity might not be all that you want to know from this data. Minitab makes it easy for you to get another view of the data. You can quickly switch which category is inner and which is the outer category.</p>
<ol>
<li>Press CTRL + E.</li>
<li>In <strong>Table arrangement</strong>, select <strong>Rows are outermost categories and columns are innermost</strong>. Click <strong>OK</strong>.</li>
<li>Double-click one of the bars in the graph.</li>
<li>Select the <strong>Groups</strong> tab. Check <strong>Assign attributes by graph variables</strong>. Click <strong>OK</strong>.</li>
<li>Double-click one of the category labels on the bottom of the graph.</li>
<li>Select the Show tab. In the <strong>Show Labels By Scale Level</strong><strong> </strong>table, uncheck <strong>Tick labels</strong> for <strong>Graph variables</strong>. Click <strong>OK</strong>.</li>
</ol>
<p><img alt="With categories outermost, you can see which locations have the biggest change between the two surveys." src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/e35c857c8b737666102ca6e14e7f3647/location_outermost_2.png" style="width: 576px; height: 384px;" /></p>
<p>In this display, you can easily see the change for each location between the two questions. For every location, the number of people who reported that they planned to shop there on the first survey is higher than the number who planned to finish shopping there on the second survey.</p>
<p>This result seems reasonable. One possible explanation is that people finished their shopping at some locations. In terms of the difference in the percentages, those who plan to shop for school items at clothing stores and electronics stores changed the most. Customers who finished shopping at a location seem to have finished at those types of locations the earliest.</p>
<strong>Wrap up</strong>
<p>When you’re looking at data, discovering what’s important often involves looking at the data from more than one perspective. Fortunately, Minitab’s bar chart makes it easy for you to change the focus of the categories so that you can dig deeper, faster. It’s nice to know that the information that you need is so readily available!</p>
<p><strong>Bonus</strong></p>
<p>I set up my data as values from a table today. Want to see what the other two options do? Check out <a href="http://blog.minitab.com/blog/quality-data-analysis-and-statistics/bar-charts-decoded">Choosing what the bars in your chart represent!</a></p>
The image of the children running in backpacks is from healthinhandkelowna.blogspot.com and is licensed under this <a href="https://creativecommons.org/licenses/by/2.0/">Creative Commons License</a>.
Statistics in the NewsFri, 05 Sep 2014 16:04:43 +0000http://blog.minitab.com/blog/statistics-and-quality-improvement/switch-the-inner-and-outer-categories-on-a-bar-chartCody SteeleWhy Are There No P Values for the Variables in Nonlinear Regression?
http://blog.minitab.com/blog/adventures-in-statistics/why-are-there-no-p-values-for-the-variables-in-nonlinear-regression
<p>Previously, I showed why there is <a href="http://blog.minitab.com/blog/adventures-in-statistics/why-is-there-no-r-squared-for-nonlinear-regression" target="_blank">no R-squared for nonlinear regression</a>. Anyone who uses nonlinear regression will also notice that there are no P values for the predictor variables. What’s going on?</p>
<p>Just like there are good reasons not to calculate R-squared for nonlinear regression, there are also good reasons not to calculate P values for the coefficients.</p>
<p><img alt="No P values in nonlinear regression" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/742d7708-efd3-492c-abff-6044d78e3bbd/Image/19bcdad63d238044a6742e3643841b4f/no_p.png" style="float: right; width: 225px; height: 225px; margin: 10px 15px;" /></p>
<p>Why not—and what to use instead—are the subjects of this post!</p>
Why are P values possible for linear regression?
<p>This may be an unexpected question, but the best way to understand why there are no P values in nonlinear regression is to understand why you <em>can </em>calculate them in linear regression.</p>
<p>A linear regression model is a very restricted form of a regression equation. The equation is constrained to just one basic form. Each term must be either a <a href="http://blog.minitab.com/blog/adventures-in-statistics/regression-analysis-how-to-interpret-the-constant-y-intercept" target="_blank">constant</a> or the product of a <a href="http://support.minitab.com/en-us/minitab/17/topic-library/basic-statistics-and-graphs/introductory-concepts/basic-concepts/parameters/" target="_blank">parameter</a> and a <a href="http://support.minitab.com/en-us/minitab/17/topic-library/modeling-statistics/regression-and-correlation/regression-models/what-are-response-and-predictor-variables/" target="_blank">predictor variable</a>. The equation is constructed by adding the results for each term.</p>
<p style="margin-left: 40px;">Response = constant + parameter * predictor + ... + parameter * predictor</p>
<p style="margin-left: 40px;">Y = b o + b1X1 + b2X2 + ... +bkXk</p>
<p>Thanks to this consistent form, it’s possible to develop a hypothesis test for all parameter estimates (coefficients) in any linear regression equation. If you enter a coefficient of 0 into any term, and multiply it by the predictor value, the term always equals zero and indicates that the predictor variable does not affect the response value.</p>
<p>Given this consistency, it’s possible to set up the following hypothesis test for all parameters in all linear models:</p>
<ul>
<li>H0: bi = 0</li>
<li>HA: bi <> 0</li>
</ul>
<p>The p-value for each term in linear regression <a href="http://blog.minitab.com/blog/adventures-in-statistics/how-to-correctly-interpret-p-values" target="_blank">tests this null hypothesis</a>. A low p-value (< 0.05) indicates that you have sufficient evidence to conclude that the coefficient does not equal zero. Changes in the predictor are associated with changes in the response variable.</p>
<p><a href="http://blog.minitab.com/blog/adventures-in-statistics/how-to-interpret-regression-analysis-results-p-values-and-coefficients" target="_blank">How to interpret P values and coefficients in linear regression analysis</a></p>
So, why are P values impossible in nonlinear regression?
<p>While a linear model has one basic form, nonlinear equations can take many different forms. There are very few restrictions on how parameters can be used in a nonlinear model.</p>
<p>The upside is that this flexibility allows nonlinear regression to provide the most flexible curve-fitting functionality.</p>
<p>The downside is that the correct null hypothesis value for each parameter depends on the expectation function, the parameter's place in it, and the field of study. Because the expectation functions can be so wildly different, it’s impossible to create a single hypothesis test that works for all nonlinear models.</p>
<p>Instead of P values, Minitab can display a confidence interval for each parameter estimate. Use your knowledge of the subject area and expectation function to determine if this range is reasonable and if it indicates a significant effect.</p>
<p>To see examples of nonlinear functions, see <a href="http://blog.minitab.com/blog/adventures-in-statistics/what-is-the-difference-between-linear-and-nonlinear-equations-in-regression-analysis" target="_blank">What is the difference between linear and nonlinear regression equations?</a></p>
Regression AnalysisStatisticsStatistics HelpThu, 04 Sep 2014 11:00:00 +0000http://blog.minitab.com/blog/adventures-in-statistics/why-are-there-no-p-values-for-the-variables-in-nonlinear-regressionJim FrostHas the College Football Playoff Already Been Decided?
http://blog.minitab.com/blog/the-statistics-game/has-the-college-football-playoff-already-been-decided
<p>A mere 10 seasons ago, USC and Oklahoma opened the college football season ranked #1 and #2 in the preseason AP Poll <em>and</em> the Coaches Poll. They remained there the entire regular season, as neither lost a game. But as chance would have it, they weren’t the only undefeated teams that year. Both Auburn and Utah went undefeated, but neither could crack the top 2, and Oklahoma and USC went on to play in the BCS Championship game.</p>
<p>That’s right, it was only 10 years ago that an <em>undefeated</em> SEC Champion was <em>left out</em> of the BCS title game.</p>
<p>If you’ve only been following college football for 8 years, you probably just fell out of your chair and had a heart attack.</p>
<div style="float: right; width: 325px; margin: 0px 0px 15px 15px;"><img alt="BCS" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/4242d511914c82c9abfdb784212974d5/bcs.jpg" style="width: 310px; height: 242px; float: right; border-width: 1px; border-style: solid; margin: 10px 15px;" /><em>No, that's not what I meant when I said the playoffs have already been decided!</em></div>
<p>But if you think about it, the BCS Championship game was already decided before the season started. In the preseason poll, voters determined that USC and Oklahoma were better than Auburn and Utah. There was nothing either team could do to change that thinking. They were dependent on the teams ranked above them to have more losses than they did.</p>
<p>Is this typical in college football? If the preseason poll has Team A ranked higher than Team B, and they finish the season with the same number of losses, will Team A still be ranked higher? If so, where a team is ranked in the preseason could have a major impact on whether or not they get selected to participate in the playoffs! </p>
Collecting the Data
<p>I took the top 5 ranked teams in the AP Poll that had the same number of losses. The key is making sure they have the same number of losses. Last year Florida State was the only undefeated team, so of course they were going to be the top-ranked team regardless of their preseason ranking. But there were 5 teams ranked right below them with 1 loss (in total there were nine 1-loss teams, but I only used the 5 highest-ranked teams to keep the number of teams consistent from year to year). I recorded the order in which those 5 teams were ranked in the last<em> regular season</em> AP Poll (since the college football playoff will be decided before the bowl games, I’m ignoring the final AP Poll), and the order they were ranked in the preseason poll. For example, here is the data from 2013.</p>
<p align="center"><strong>Team</strong></p>
<p align="center"><strong>Preseason Rank</strong></p>
<p align="center"><strong>Postseason Rank</strong></p>
<p align="center">Auburn</p>
<p align="center">5</p>
<p align="center">1</p>
<p align="center">Alabama</p>
<p align="center">1</p>
<p align="center">2</p>
<p align="center">Michigan State</p>
<p align="center">3</p>
<p align="center">3</p>
<p align="center">Baylor</p>
<p align="center">4</p>
<p align="center">4</p>
<p align="center">Ohio State</p>
<p align="center">2</p>
<p align="center">5</p>
<p>Before the season started, Auburn was the lowest ranked team of the 5. However, voters changed their mind by the end of the season, ranking Auburn the highest out of the 5. Perhaps the Tigers finally got some justice from 2004! But 2 years' worth simply isn’t enough data. So I went back and collected similar data for the last 10 college football seasons. </p>
<p>For most years I used a group of five 1-loss teams, since there weren’t enough undefeated teams to compare. And there were two years I wasn’t able to get a group of five, so I have 47 total teams. You can get the data I used <a href="//cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/File/7ff939dcb7e86c5dde589b16e9f5a76a/preseason_college_football_rankings.MTW">here</a>.</p>
Comparing Preseason and Postseason Ranks
<p>The first thing we’ll look at is an individual value plot showing a team’s preseason and postseason rank. If teams stay in the same order from the preseason to the postseason, we would expect most of the data points to fall along the diagonal from bottom left to top right.</p>
<p><img alt="Individual Value Plot" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/fe2c58f6-2410-4b6f-b687-d378929b1f9b/Image/bf1c1c9de9bd005e8b1772dc4cf6f607/individual_value_plot_of_preseason_rank_w640.jpeg" style="width: 640px; height: 427px;" /></p>
<p>Most of the points do fall along the diagonal. In the bottom left corner, you’ll see that of the 10 teams that held the top spot in the final postseason rankings, half were the top-ranked team in the preseason too. A majority of the points continue going up the diagonal, indicating top-ranked teams with the same number of losses stay in the same order from preseason to postseason.</p>
<p>But we should quantify this relationship to make sure it truly exists. First let’s look at the correlation between the two variables. To calculate the correlation between ordinal variables in Minitab, go to <strong>Stat > Tables > Cross Tabulation and Chi-Square</strong>. Then click <strong>Other Stats</strong> and select <strong>Correlation coefficients for ordinal categories</strong>. </p>
<p><img alt="Correlation coefficients" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/fe2c58f6-2410-4b6f-b687-d378929b1f9b/Image/41bc0c72c69764856a4b1f04418f505d/correlation.jpg" style="width: 257px; height: 58px;" /></p>
<p>The correlation coefficient can range in value from -1 to +1. The larger the absolute value of the coefficient, the stronger the relationship between the variables. An absolute value of 1 indicates a perfect relationship, and a value of zero indicates the absence of relationship. Minitab displays two different coefficients, Pearson’s r and Spearman’s rho. For these data, they both equal about 0.44, indicating a positive association preseason rankings and postseason rankings. Teams ranked higher in the preseason tend to be ranked higher going into the postseason.</p>
Measuring Concordance
<p>We can take this analysis one step further by looking at the concordant and discordant pairs. A pair is concordant if the observations are in the same direction. A pair is discordant if the observations are in opposite directions. This will let us compare teams to each other 2 at a time. For example, let’s go back to our five 1-loss teams from 2013.</p>
<p align="center"><strong>Team</strong></p>
<p align="center"><strong>Preseason Rank</strong></p>
<p align="center"><strong>Postseason Rank</strong></p>
<p align="center">Auburn</p>
<p align="center">5</p>
<p align="center">1</p>
<p align="center">Alabama</p>
<p align="center">1</p>
<p align="center">2</p>
<p align="center">Michigan State</p>
<p align="center">3</p>
<p align="center">3</p>
<p align="center">Baylor</p>
<p align="center">4</p>
<p align="center">4</p>
<p align="center">Ohio State</p>
<p align="center">2</p>
<p align="center">5</p>
<p><br />
Look at Auburn and Alabama. In the preseason, voters had Alabama ranked higher. But at the end of the season, they ranked Auburn higher. So this pair is discordant. Now look at Alabama and Ohio State. In the preseason, Alabama was voted higher. And even though the ranking of both teams fell during the season, Alabama still ranked higher at the end of the season. This pair is concordant. More concordant pairs means voters are keeping the same order as the preseason, while more discordant pairs means they’re switching teams.</p>
<p>When I did this for the other 9 seasons, I ended up with a total of 89 pairs. Of those, 61 were concordant. That’s over 68%! In fact, there were only two seasons with more discordant pairs than concordant pairs (2013 and 2010, though my group in 2010 only had 3 teams).</p>
<p>This give us further indication that if a team is thought to be better in the preseason, that thinking will not change throughout the season, given that the teams lose the same number of games.</p>
Which Teams Are Able to Move Up?
<p>We did see that voters change their minds on teams every now and then. After all, there <em>were </em>28 discordant pairs. So which are these teams that are able to impress voters so much that they move ahead similar loss teams that were ranked ahead of them in the preseason? To answer this question, I’m going to show the individual plot again. But this time I’m going to add labels to the teams that moved up from their preseason ranking by at least 2 spots.</p>
<p><img alt="Individual Value Plot" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/fe2c58f6-2410-4b6f-b687-d378929b1f9b/Image/642ba6a3d93ae11e3ac0b11604a573cc/ivp_sec_w640.jpeg" style="width: 640px; height: 427px;" /></p>
<p>Spoiler alert: Voters love the SEC! Five of the 6 teams that were able to improve their rank by at least 2 spots are from the SEC. In other words, the preseason rank for SEC teams does not matter. If they have a great season (much like Auburn did in 2013), they’ll be one of the top-ranked teams at the end of the year regardless of who was ranked ahead of them before the season.</p>
Which Teams Get Passed Over?
<p>Can we find a similar pattern with teams that start the season ranked high but then drop in the view of the voters eyes? Here is the individual value plot with teams labeled that fell at least 2 sports from preseason to postseason.</p>
<p><img alt="Individual Value Plot" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/fe2c58f6-2410-4b6f-b687-d378929b1f9b/Image/bf8e448dfd0528fef5603a93070b4147/ivp_boise_st_w640.jpeg" style="width: 640px; height: 427px;" /></p>
<p>It’s no surprise to see Boise State on this list, as they played in a small conference at the time. The year 2009 was incredibly unlucky for them. They started the season ranked a respectable 14th and went undefeated. However, they were only able to move up to 6th, as 4 other teams also went undefeated, two of which (TCU and Cincinnati) passed Boise State in the rankings along the way. To add injury to insult, a 1-loss Florida team (not included in the data) was also ranked ahead of undefeated Boise State. </p>
<p>What is really surprising is what happened to USC. In 2007, they started the season ranked #1 overall in the AP Poll. They lost two games, but so did just about everybody else in college football that season, including LSU, who played in and won the BCS Championship Game. But LSU wasn’t the only team to pass USC in the rankings with the same number of losses. Oklahoma, Georgia, and Virginia Tech did too. For crying out loud...<em>Virginia Tech</em>!?!?</p>
<p>If that weren’t bad enough, the <em>same thing</em> happened in 2008. USC started the season ranked #3 in the AP Poll, and the #1 and #2 team would go on to lose multiple games. No BCS team went undefeated that year, and USC only lost a single game. So you would think their high preseason ranking would put them in one of the top 2 spots, and they would be playing the BCS title game. Except that’s not even close to what happened.</p>
<p>Despite all of them being ranked lower in the preseason and losing a game, Florida, Oklahoma, Texas, and Alabama all jumped USC in the rankings. </p>
<p>The takeaway, I suppose, is don't play in a small conference. And, uh, don't play on the west coast?</p>
Will This Trend Carry Over to the College Football Playoff?
<p>The selection committee won’t select the top 4 teams based on the AP poll. But the committee will be free to think on their own (like the AP voters do) instead of being locked into using certain criteria like the BCS did with using computers. It would be folly to think the committee doesn’t have any preconceived notion about which teams are the best, and I bet they are similar to the top ranked teams in the preseason AP Poll. And those notions may just decide who gets into the playoff!</p>
<p>For example, we know that the SEC Champion is getting in (even if they start the season ranked low). And as the preseason #1 team, even a 1-loss Florida State team is a safe bet to get in. That leaves the champion of the other 3 major BCS conferences (Big 10, Pac-12, and Big 12) left to fight over 2 spots.</p>
<p>Oregon and Oklahoma appear to be in the best spot, as they are ranked #3 and #4 in the preseason. If they win their conference, even a single loss shouldn’t hurt them assuming a Big 10 team doesn’t go undefeated. Meanwhile, to be on the safe side, Michigan State or Ohio State should hope one of those two teams ends up with more losses than them. If you’re coming out of nowhere to win one of those conferences, like maybe Nebraska, Kansas State, or Arizona State, you’re probably going to need some chaos to get in the playoffs. And considering there isn’t a single non-BCS school in the AP Top 25, I think it’s safe to say Cinderella isn’t going to make it to the ball this year.</p>
<p>Now, the committee claims that there will be a more deliberative evaluation system and that there could be volatile swings from week to week, with lower-ranked teams moving ahead of higher-ranked teams without either team losing. Whether that actually happens or not (especially with non-SEC teams) remains to be seen. So at the end of the season we’ll compare the final rankings of the committee to the preseason AP Poll. Until then, enjoy the games! Just know that the order for which teams get into the playoffs may already have been decided!</p>
Fun StatisticsStatistics in the NewsFri, 29 Aug 2014 12:46:15 +0000http://blog.minitab.com/blog/the-statistics-game/has-the-college-football-playoff-already-been-decidedKevin RudyEnhancing Weekly Productivity with Staged Control Charts
http://blog.minitab.com/blog/statistics-in-the-field/enhancing-weekly-productivity-with-staged-control-charts
<p><em>by The Discrete Sharer, guest blogger</em></p>
<p>As Minitab users, many of us have found <span><a href="http://blog.minitab.com/blog/starting-out-with-statistical-software/setting-the-stage-accounting-for-process-changes-in-a-control-chart">staged control charts</a></span> to be an effective tool to quantify and demonstrate the “before and after” state of our process improvement activities.</p>
<p>However, have you ever considered using them to demonstrate the effects of changes to compensation/incentive plans for your employees? </p>
<p>Here's an example of how a mid-sized commercial bank used Minitab’s staged control charts to better inform senior leaders about the benefits of an pilot incentive plan program, in the hopes of having it implemented nationwide.</p>
Background
<p>Because most retail banking operations do not generate revenues or profits for the organization, there is a significant focus on reducing costs. For most organizations, this is most easily accomplished by offering starting salaries that are not much higher than minimum wage. While this may make sense initially, it does not properly account for the impact of high turnover in those jobs—and the associated costs of needing to frequently replace the people who work in those positions.</p>
<p>To address that, one organization designed an incentive compensation plan that “paid for itself,” and tested it at one of its processing sites over a period of 18 months. That test window included the baseline period followed by three iterations of the incentive plan, where the potential rewards increased from each previous level.</p>
Production Environment and Incentive Plan Structures
<p>The incentive process had engineered standards that deemed “on-target performance” to be between 315 – 385 items per hour. This reflected a +/- 10% range to the mean of 350 items per hour. For the initial 20 weeks of the pilot, the baseline weekly productivity mean was 349.90 items per hour, with a standard deviation of 5.8.</p>
<p>As had been the common business practice for many years, no additional incentive or compensation rewards were offered to employees for the first 20 weeks of the pilot program. <span style="line-height: 1.6;">For the subsequent three 20-week periods, increasing incentive awards were developed and implemented. They were structured as follows:</span></p>
<p><strong>Single Plan: </strong><br />
Any employee who showed a 5% or greater improvement to the initial 20-week period received a $25 gift card each month. This award also included a “tax gross-up” so that the employee would get the full value of the award without incurring a tax penalty.</p>
<p><strong>Compounded Plan: </strong><br />
About half of the employees showed some slight degradation in their work quality. To combat this unintended consequence, employees could earn additional rewards in the second phase by maintaining their gain (if they had met the initial 5% goal) or improving to that threshold while keeping their quality levels at the same rate or better. For those who met these objectives, the gift card amount was increased to $50 per month.</p>
<p><strong>Increased Compounded Plan: </strong><br />
This was similar to the Compounded Plan, except for the amount of the gift card increased to $100 per month.</p>
Assessing the Impact of the Incentive Plan
<p>By offering employees a way to “earn more by doing more” through the incentive plan, the site saw improved performance that exceeded initial estimates. The table below provides some basic data on the results of each stage.</p>
<p><img alt="incentive plan data table" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/08576df5868512cee57fdf50a8c0318d/data_table_w640.jpeg" style="width: 640px; height: 159px;" /></p>
<p>Shown graphically on a staged control chart created in Minitab <a href="http://www.minitab.com/products/minitab">Statistical Software</a>, the results looked like this:</p>
<p><img alt="staged control chart for weekly productivity" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/0c32611c070918ba2868962cf81aecba/staged_control_chart.jpg" style="width: 600px; height: 403px;" /></p>
Using the Staged Control Chart to Influence Management
<p>Despite the fact that the incentive plan paid for itself through increased employee productivity, the business line manager was reluctant to embrace the results.</p>
<p>That is, until she was shown the Staged Control Chart seen above.</p>
<p>The manager's newfound support for the program was due to the visual impact of seeing variation reduced over the life of the pilot. She hadn't been educated as an engineer, but she did understand the importance of reducing variation within a process.</p>
<p>For her, knowing that the process was seeing reduced variation meant that the unit could do a better job predicting the labor needs for the upcoming quarters.</p>
<p> </p>
<p><em><strong>About the Guest Blogger</strong></em></p>
<p><em>The Discrete Sharer's identity and biographical information is confidential. </em></p>
Lean Six SigmaProject ToolsStatisticsWed, 27 Aug 2014 12:00:00 +0000http://blog.minitab.com/blog/statistics-in-the-field/enhancing-weekly-productivity-with-staged-control-chartsGuest BloggerAnalyzing NFL Ticket Prices: How Much Would You Pay to See the Green Bay Packers?
http://blog.minitab.com/blog/the-statistical-mentor/analyzing-nfl-ticket-prices3a-how-much-would-you-pay-to-see-the-green-bay-packers
<p><span style="line-height: 1.6;">The 2014-15 NFL season is only days away, and fans all over the country are planning their fall weekends accordingly. In this post, I'm going to use data analysis to answer some questions related to ticket prices, such as:</span></p>
<ul>
<li>Which team is the least/most expensive to watch at home? </li>
<li>Which team is the least/most expensive to watch on the road? </li>
<li>If you are thinking of a road trip, which stadiums offer the largest ticket discount for your team?<img alt="Football stadium crowd" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/5781b1d52907305361bd13535983580b/stadium.jpg" style="float: right; width: 350px; height: 269px; border-width: 1px; border-style: solid; margin: 10px 15px;" />
<ul>
</ul>
</li>
</ul>
<p>For dedicated fans, this is far from a trivial matter. As we'll see, fans of one team can get an average 48% discount on road-game tickets, while fans of two other teams will pay, on average, more than double the cost to see their team on the road.</p>
Gathering and Preparing NFL Ticket Price Data
<p>The data I'm analyzing comes from Stubhub, an online ticket marketplace owned by ebay. You'll find a summary of the number of Stubhub tickets available and mimimum price on Stubhub for each NFL game in 2014 on the ESPN website: <a href="http://espn.go.com/nfl/schedule/_/seasontype/2/week/1">http://espn.go.com/nfl/schedule/_/seasontype/2/week/1</a></p>
<p><img alt="snapshot of NFL data from ESPN.com" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/0844140f9ada17966cf7a63eda771c6a/nfl_data.jpg" style="width: 600px; height: 384px;" /></p>
<p>I did a quick copy-and-paste from ESPN into Excel to put each variable nicely into a column, and then another copy-and-pasted the data into Minitab <a href="http://www.minitab.com/products/minitab">Statistical Software</a> to prepare it for analysis. I used the <a href="http://blog.minitab.com/blog/understanding-statistics/three-ways-to-get-more-out-of-your-text-data"><strong>Calc > Calculator</strong></a> commands Left() and Right() in Minitab to extract the minimum ticket price, the first few letters of the away team name, and the first few letters of the home team name. (Since the summary on ESPN.com only shows the minimum price, the analysis below is based only on the minimum ticket price available for each game.)</p>
Which Is the Most Expensive Team to See on the Road?
<p>The Bar Chart below shows that Green Bay is the most expensive road team to watch play with a 2014 average price of $145 per road game. This is noticeably higher than the other NFL teams. The next closest is San Francisco with an average price of $128 per road game. But catching a Jacksonville road game is a fraction of those costs, averaging $48. </p>
<p><img alt="Bar Chart of Average Minimum Price for Away Team 2014 NFL Season" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/b7a4e021aea84b3bf1b37ee619afc93c/avg_min_price_away_team_2014_season.jpg" style="width: 586px; height: 390px;" /></p>
Which Is the Most Expensive Team to See at Home?
<p>The Bar Chart below shows that Chicago is the most expensive team to watch play on their home turf, with a 2014 average price of $175 per home game. Seattle is a close second with an average price of $171 per home game. Seeing Dallas or St. Louis in a home game is a fraction of those costs, averaging just $35. </p>
<p><img alt="Bar Chart of Average Minimum Price for Home Team 2014 NFL Season" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/27f5c44c660d02e27573ca4cd4366632/avg_min_price_home_team_2014_season.jpg" style="width: 580px; height: 387px;" /></p>
Is It Cheaper to See Your Favorite Team on the Road?
<p>Finally, I compared the average home game ticket price to the average road game ticket price for each NFL team.</p>
<p>The road team discount award goes to the Seattle Seahawks. You'll save, on average, 48% watching their games on the road. But if you're a fan of Dallas or Miami, you'll be financially better off watching your team at home—their average price increases more than 110% when they're on the road. One factor that drives this result is the popularity of Dallas and Miami across the country: the higher demand supports their higher road-game price. Also, Dallas' enormous home stadium (AT&T) offers cheap Party Pass seats (which aren't really seats at all, but rather a standing room section). </p>
<p><img alt="Is It Cheaper to See Your Favorite NFL Team on the Road? " src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/8063637683d8b39c1fcc4083965e7428/cheaper_on_the_road.jpg" style="width: 582px; height: 387px;" /></p>
<p>One drawback with this analysis is it doesn't take into account the opponent that each team faces. For example, Chicago may happen to be playing some very popular teams at home in 2014, which drives their home-game ticket prices up for this season.</p>
<p>In a future post, I'll discuss how to adjust for opponents and other variables such as game day and game time.</p>
Tue, 26 Aug 2014 12:00:00 +0000http://blog.minitab.com/blog/the-statistical-mentor/analyzing-nfl-ticket-prices3a-how-much-would-you-pay-to-see-the-green-bay-packersJim ColtonHow Could You Benefit from Plackett & Burman Experimental Designs ?
http://blog.minitab.com/blog/applying-statistics-in-quality-projects/how-could-you-benefit-from-plackett-burman-experimental-designs
<p>Screening experimental designs allow you to study a very large number of factors in a very limited number of runs. The objective is to focus on the few factors that have a real effect and eliminate the effects that are not significant. This is often the initial typical objective of any experimenter when a <span><a href="http://blog.minitab.com/blog/understanding-statistics/getting-started-with-factorial-design-of-experiments-doe">DOE (design of experiments)</a></span> is performed.</p>
Table of Factorial Designs
<p>Consider the table below. In Minitab, you can quickly access this table of factorial designs by selecting <strong>Stat > DOE > Factorial > Create Factorial Design...</strong> and clicking "Display Available Designs." The table tells us the number of runs in a 2k standard factorial design, its resolution, and the number of factors to be analyzed. If you need to study 8 factors, or 7 factors, or even 6 factors, a 16-run design might could be a good choice, because it balances the number of runs with the ability to effectively interpret the experimental results.</p>
<p><span style="line-height: 1.6;">Confounding is the price we pay for reducing the number of runs: the effects of different factors or interactions of factors can't be evaluated individually, so interpreting the results becomes more difficult and riskier. </span>The yellow color indicates that this design is acceptable in terms of confounding/resolution. Designs with the green color have limited or no confounding, but a larger number of runs. On the other hand, any experimenter should refrain from using designs located in the red region due to extensive confounding. Red means that some main factors are confounded with two-factor interactions.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/8989a57ae25694f17802c93b3058539c/factorial_designs_table.jpg" style="width: 466px; height: 366px;" /></p>
<p><span style="line-height: 1.6;">According to the table, studying</span> more than 8 factors means you'll need to perform 32 in order to remain in the yellow region. But when experiments are costly and time consuming, that may be too many.</p>
Plackett & Burman Designs: An Alternative to Factorial
<p>Fortunately, another solution is available: Plackett & Burman designs may be used to analyze a larger number of variables more economically. For example, to study 9 factors you need only conduct 12 runs, rather than the 32 runs needed in <span style="line-height: 1.6;">a</span><span style="line-height: 1.6;"> </span><span style="line-height: 1.6;">standard</span><span style="line-height: 1.6;"> </span><span style="line-height: 1.6;">2</span>k<span style="line-height: 1.6;"> fractional design. In the Minitab <a href="http://blog.minitab.com/blog/understanding-statistics/why-is-the-office-coffee-so-bad-a-screening-experiment-narrows-down-the-critical-factors">Assistant</a>, for example, Plackett and Burman designs are suggested whenever the number of factors to be studied is larger than five.</span></p>
<p><img alt="" spellcheck="true" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/31b80fb2-db66-4edf-a753-74d4c9804ab8/Image/1b9f5ff6a9ca261e0cfd1a9722312914/assistant_plan_and_create_w640.jpeg" style="width: 566px; height: 338px;" /></p>
<p>The main disadvantage of this type of screening design is that two-factor interactions cannot be studied. In Plackett and Burman designs, interactions are partially confounded or "aliased" with all main effects. Sometimes the interactions appear with a positive sign, meaning they are summed up to the main effect. In other cases they have a negative sign, indicating they are subtracted from the main effect. One third of each interaction is added to or subtracted from any main effect. For example, in the experimental aliasing table below, the effect of the A factor is partially confounded with all interaction effects:</p>
<p style="margin-left: 40px;"><strong>Alias / confounding structure in a Plackett & Burman design - Aliases :</strong></p>
<p style="margin-left: 40px;">A - 0.33 BC - 0.33 BD - 0.33 BE + 0.33 BF - 0.33 BG - 0.33 BH + 0.33 CD - 0.33 CE - 0.33 CF + 0.33 CG - 0.33 CH + 0.33 DE + 0.33 DF - 0.33 DG - 0.33 DH - 0.33 EF - 0.33 EG - 0.33 EH - 0.33 FG + 0.33 FH + 0.33 GH - 0.33 BCD + 0.33 BCE - 0.33 BCF + 0.33 BCG + 0.33 BCH + 0.33 BDE + 0.33 BDF + 0.33 BDG - 0.33 BDH + 0.33 BEF - 0.33 BEG + 0.33 BEH - 0.33 BFG + 0.33 BFH - 0.33 BGH + 0.33 CDE + 0.33 CDF + 0.33 CDG + 0.33 CDH - 0.33 CEF - 0.33 CEG + 0.33 CEH + 0.33 CFG + 0.33 CFH + 0.33 CGH + 0.33 DEF - 0.33 DEG - 0.33 DEH + 0.33 DFG + 0.33 DFH + 0.33 DGH + 0.33 EFG - 0.33 EFH + 0.33 EGH + 0.33 FGH</p>
An Added Bonus for Plackett and Burman Designs
<p>But there is an added benefit to consider when using Plackett and Burman designs. Suppose that the effects that were not significant have been gradually eliminated from the model, and that only three main effects remain. With Plackett-Burman, you do not need to perform an additional 23 = 8-run full factorial design in order to estimate the two-factor interactions. The initial Plackett & Burman design already contains all the tests you need for this 23 full factorial design—and you will even get four replicates in addition to the full design!</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/31b80fb2-db66-4edf-a753-74d4c9804ab8/Image/f49870ff200fc6a02d35e7aeff0d8e2f/effects_pareto_for_y.jpg" style="width: 576px; height: 384px;" /></p>
<p>The Pareto diagram above from a Plackett and Burman design shows that only factors A, D & H have statistically significant effects.</p>
Transforming a Plackett & Burman Design to a Full Factorial
<p>To transform your Plackett & Burman design into a full factorial 23 design that will allow you to study all two-factor interactions in Minitab, go to <strong>Stat > DOE > Factorial > Define Custom Factorial Design...</strong> Select your three significant factors and click on Low/high to specify the factor settings. Now you can study the two-factor interactions.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/31b80fb2-db66-4edf-a753-74d4c9804ab8/Image/0b88d0749035cd973d191a9189d79344/effects_pareto_for_y_factorial.jpg" style="width: 576px; height: 384px;" /></p>
<p>The initial Plackett and Burman design has been transformed into a full factorial design, the interactions between the three factors that have been previously selected can now be studied, and as we can see in the Pareto, graph one interaction appears to be statistically significant.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/31b80fb2-db66-4edf-a753-74d4c9804ab8/Image/1756842f34e86e5460892ef1dbbf5c29/interaction_plot_for_y.jpg" style="width: 576px; height: 384px;" /></p>
<p>Of course, we can use Minitab to graph the significant interaction effect to get a better understanding of it.</p>
Choosing the Right Experimental Design for Your Needs
<p>Experimenters need to identify the best trade-off between running a limited number of tests and getting as much information as possible to improve processes / products. Plackett and Burman designs act as a funnel, enabling a quick reduction in the number of potential factors.</p>
<p>That is the reason why, in the <a href="http://www.minitab.com/products/minitab/assistant/">Minitab Assistant</a>, Plackett and Burman screening designs are considered when the number of potential factors is greater than five. After the analysis phase of the DOE results, if the number of significant factors in a 12-run Plackett & Burman design is equal to or smaller than 3, the initial design may easily be transformed into a full factorial DOE enabling you to study all two-factor interactions.</p>
Data AnalysisDesign of ExperimentsQuality ImprovementStatisticsStatsThu, 21 Aug 2014 13:17:29 +0000http://blog.minitab.com/blog/applying-statistics-in-quality-projects/how-could-you-benefit-from-plackett-burman-experimental-designsBruno ScibiliaUse a Line Plot to Show a Summary Statistic Over Time
http://blog.minitab.com/blog/statistics-and-quality-improvement/use-a-line-plot-to-show-a-summary-statistic-over-time
<p><img alt="Terrorist Attacks, 2013, Concentration and Intensity" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/2ecbcca8429152afb73d991b4a532f5a/start_globalterrorismdatabase_2013terroristattacksconcentrationintensitymap_w640.png" style="width: 500px; height: 216px;" /></p>
<p>If you’re already a strong user of Minitab Statistical Software, then you’re probably familiar with <a href="https://blog.minitab.com/blog/starting-out-with-statistical-software/investigating-starfighters-with-bar-charts3a-function-of-a-variable">how to use bar charts to show means</a>, medians, sums, and other statistics. Bar charts are excellent tools, but traditionally used when you want all of your categorical variables to have different sections on the chart. When you want to plot statistics with groups that flow directly from one category to the next, look no further than Minitab’s <a href="http://www.minitab.com/en-us/Support/Tutorials/Minitab-s-Line-Plots/">line plots</a>. I particularly like line plots when I want to use time as a category, because I prefer the connect line display to separated bars.</p>
<p>I like to illustrate Minitab with data about pleasant subjects: <a href="https://blog.minitab.com/blog/statistics-and-quality-improvement/practicing-data-analysis-get-some-fun-data-into-minitab-v1">poetry</a>, <a href="https://blog.minitab.com/blog/statistics-and-quality-improvement/gummi-bear-measurement-systems-analysis-msa-the-gage-randr-study">candy</a>, and maybe even <a href="https://blog.minitab.com/blog/statistics-and-quality-improvement/process-capability-statistics-cp-and-cpk-working-together">the volume of ethanol in E85 fuel</a>. Data that are about unpleasant subjects also exist, and we can learn from that data too. We’re fortunate to have both the <a href="http://cpost.uchicago.edu/">Chicago Project on Security and Terrorism</a> (CPOST) and the <a href="http://www.start.umd.edu/">National Consortium for the Study of Terrorism and Responses to Terrorism</a> (START) working hard to produce publicly-accessible databases with information about terrorism.</p>
<p>START has been sharing <a href="http://www.start.umd.edu/news/majority-2013-terrorist-attacks-occurred-just-few-countries">analyses of its 2013 data</a> recently. The new data prompted staff from the two institutions to engage in an interesting debate on the Washington Post’s website about whether the Global Terrorism Database (GTD) that Start maintains “<a href="http://www.washingtonpost.com/blogs/monkey-cage/wp/2014/08/15/global-terrorism-data-show-that-the-reach-of-terrorism-is-expanding/">exaggerates a recent increase in terrorist activities</a>.” For today, I’m just going to use the GTD to demonstrate a nice line plot in Minitab, which will give a tiny bit of insight into what that debate is about.</p>
<p>When you <a href="http://www.start.umd.edu/gtd/contact/">download the GTD data</a>, you can open one file that has all of the data except for the year 1993. Incident-level data for 1993 was lost, so that year is not included, although you can get country-level totals for numbers of attacks and casualties from the <a href="http://www.start.umd.edu/gtd/downloads/Codebook.pdf">GTD Codebook</a>. Those who maintain the GTD <a href="http://www.start.umd.edu/gtd/using-gtd/">recommend</a> “users should note that differences in levels of attacks and casualties before and after January 1, 1998, before and after April 1, 2008, and before and after January 1, 2012 are at least partially explained by differences in data collection” (START, downloaded August 18th, 2014).</p>
<p>The GTD is great for detail. One column it contains records a one if an event was a suicide attack and a 0 if an event is not a suicide attack, which makes it easy to sum that column so that you can see the number of suicide attacks per year. Absent from the data is a column that references the changes in methodology, but we can easily add this column in Minitab. Without a methdology column, it’s easy to end up with the <a href="http://www.washingtonpost.com/blogs/monkey-cage/wp/2014/07/21/government-data-exaggerate-the-increase-in-terrorist-attacks/">recently-criticized</a> graph that started the debate between the staff at the two institutions. The graph shows all of the data in the GTD for <a href="http://warontherocks.com/2014/06/infographic-suicide-terrorism-past-and-present/">the number of suicide attacks for each year since 1970</a>. It looks a bit like this:</p>
<p><img alt="The number of suicide attacks increases dramatically in the past two years." src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/87aeffb3b5c3391d19070bc9fbc6f717/all_gtd_data.jpg" style="width: 576px; height: 384px;" /></p>
<p>The message of this graph is that the number of suicide attacks has never been higher. The criticism about the absence of the different methodologies seems fair. So how would we capture the different methodologies in Minitab? With a calculator formula, of course. Try this, if you’re following along:</p>
<ol>
<li>Choose <strong>Calc > Calculator</strong>.</li>
<li>In <strong>Store result in variable</strong>, enter <em>Methodology</em>.</li>
<li>In <strong>Expression</strong>, enter:</li>
</ol>
<p><em>if(iyear < 1998, 1, iyear < 2009, 2, iyear=2009 and imonth < 4, 2, iyear < 2012, 3, 4)</em></p>
<ol>
<li value="4">Click <strong>OK</strong>.</li>
</ol>
<p>Notice that because the GTD uses 3 separate columns to record the dates, I’ve used two conditions to identify the second methodology. With the new column, you can easily divide the data series trends according to the method for counting events. This is where the line plot comes in. The line plot is the easiest way in Minitab to plot a summary statistic with time as a category. You can try it this way:</p>
<ol>
<li>Choose <strong>Graph > Line Plot</strong>.</li>
<li>Select <strong>With Symbols</strong>, <strong>One Y</strong>. Click <strong>OK</strong>.</li>
<li>In <strong>Function</strong>, select <strong>Sum</strong>.</li>
<li>In <strong>Graph variables</strong>, enter <em>suicide</em>.</li>
<li>In <strong>Categorical variable for X-scale grouping</strong>, enter <em>iyear</em>.</li>
<li>In <strong>Categorical variable for legend grouping</strong>, enter <em>Methodology</em>.</li>
</ol>
<p>You’ll get a graph that looks a bit like this, though I already edited some labels.</p>
<p><img alt="The last two years, which are dramatically higher in number, have a new methodology." src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/22959d1fc87b72c4ed837c82ef3f1b7a/number_of_attacks_divided.jpg" style="width: 576px; height: 384px;" /></p>
<p>One interesting feature of this line plot is that there are two data points for 2009. Because we’re calling attention to the different methodologies, it’s important to consider that the first quarter and the last 3 quarters of 2009 use different methodologies. In this display, we can see the mixture of methodologies. The fact that the two highest points are from the newest methodology also lend some credence to the question of whether the numbers from 2012 and 2013 should be directly compared to numbers from earlier years. The amount of the increase due to better data collection is not clear.</p>
<p>Interestingly, a line plot that shows the proportion of suicide attacks out of all terrorist attacks presents a different picture about the increase related to the different methodologies. That’s what you get if you make a line plot of the means instead of the sums.</p>
<p><img alt="By proportion, the increase in suicide attacks in the last two years does not look as dramatic." src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/22791f44-517c-42aa-9f28-864c95cb4e27/Image/7cd0512a55b4876dac769278eba9d90c/proportion_of_attacks_divided.jpg" style="width: 576px; height: 384px;" /></p>
<p>Considering which statistics to compute and how to interpret them in conjunction with one another is an important task for people doing data analysis. In the final installment of the series on the Washington Post’s website, GTD staff members note that they do not “rely solely on global aggregate percent change statistics when assessing trends.” The flexibility of the line plot to show different statistics can make the work of considering the data from different perspectives much easier.</p>
<p>We do like to have fun at the Minitab Blog, but we know that there’s serious data in the world too. Whether your application is <a href="http://www.minitab.com/en-us/Case-Studies/Bridgestone/">making tires that keep people safe on the road</a> or <a href="http://www.minitab.com/en-us/Case-Studies/Northern-Sydney-Central-Coast-Health-Service/">helping people recover from wounds</a>, our goal is to give you the best possible tools to make your process improvement efforts successful.</p>
<p> </p>
Statistics in the NewsWed, 20 Aug 2014 15:48:18 +0000http://blog.minitab.com/blog/statistics-and-quality-improvement/use-a-line-plot-to-show-a-summary-statistic-over-timeCody SteeleUsing the G-Chart Control Chart for Rare Events to Predict Borewell Accidents
http://blog.minitab.com/blog/statistics-in-the-field/using-the-g-chart-control-chart-for-rare-events-to-predict-borewell-accidents
<p><em>by Lion "Ari" Ondiappan Arivazhagan, guest blogger</em></p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/ac11ba7bc8daa85327ad905ba5dc5f96/borewell_screencap.jpg" style="margin: 10px 15px; width: 400px; height: 283px; float: right;" />In India, we've seen this story far too many times in recent years:</p>
<p>Timmanna Hatti, a six-year old boy, was trapped in a 160-feet borewell for more than 5 days in Sulikeri village of Bagalkot district in Karnataka after falling into the well. Perhaps the most heartbreaking aspect of the situation was the decision of the Bagalkot district administration to stop the rescue operation because the digging work, if continued further, might lead to collapse of the vertical wall created by the side of the borewell within which Timmanna had struggled for his life.</p>
<p><a href="http://timesofindia.indiatimes.com/city/mysore/8-days-on-boys-body-pulled-out/articleshow/40082590.cms?" target="_blank">Timmanna's body was retrieved from the well 8 days after he fell in</a>. Sadly, this is just one of an alarming number of borewell accidents, especially involving little children, across India in the recent past.</p>
<p>This most recent event prompted me to conduct a preliminary study of borewell accidents across India in the last 8-9 years.</p>
Using Data to Assess Borewell Accidents
<p>My main objective was to find out the possible causes of such accidents and to assess the likelihood of such adverse events based on the data available to date.</p>
<p>This very preliminary study has heightened my awareness of lot of uncomfortable and dismaying factors involved in these deadly incidents, including the pathetic circumstances of many rural children and carelessness on the part of many borewell contractors and farmers.</p>
<p>In this post, I'll lead you through my analysis, which concludes with the use of a G-chart for the possible prediction of the next such adverse event, based on Geometric distribution probabilities.</p>
Collecting Data on Borewell Accidents
<p>My search of newspaper articles and Google provided details about a total of 34 borewell incidents since 2006. The actual number of incidents may be higher, since many incidents go unreported. The table below shows the total number of borewell cases reported each year between 2006 and 2014.</p>
<p><img alt="Borewell Accident Summary Data" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/9e60f3b9c08b0125a38b30d717e1acb8/borewell_g_chart_table_2.jpg" style="width: 189px; height: 289px;" /></p>
Summary Analysis of the Borewell Accident Data
<p>First, I used Minitab to create a histogram of the data I'd collected, shown below.</p>
<p>A quick review of the histogram reveals that out of 34 reported cases, the highest number of accidents occurred in the years 2007 and 2014.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/23338847757384f399eb013afe81191f/borewell_histogram_of_accidents.jpg" style="width: 500px; height: 334px;" /></p>
<p>The ages of children trapped in the borewells ranged from 2 years to 9 years. More boys (21) than girls (13) were involved in these incidents.</p>
<p>What hurts most is that, in this modern India, more than 70% of the children did not survive the incident. They died either in the borewell itself or in the hospital after the rescue. Only about 20% of children (7 out of 34) have been rescued successfully. The ultimate status of 10% of the cases reported is not known.</p>
Pie Chart of Borewell Incidents by Indian State
<p>Analysis of a state-wise pie chart, shown below, indicates that Haryana, Gujarat, and Tamil Nadu top the list of the borewell accident states. These three states alone account for more than 50% of the borewell accidents since 2006.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/8466766e4788ea2d73b7d8672692be4d/borewell_pie_chart.jpg" style="width: 500px; height: 334px;" /></p>
Pareto Chart for Vital Causes of Borewell Accidents
<p>I used a <a href="http://blog.minitab.com/blog/michelle-paret/fast-food-and-identifying-the-vital-few">Pareto chart</a> to analyze the various causes of these borewell accidents, which revealed the top causes of these tragedies:</p>
<ol>
<li>Children accidentally falling into open borewell pits while playing in the fields.</li>
<li>Abandoned borewell pits not bring properly closed / sealed.<br />
</li>
</ol>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/8012effc2a1aa662d5a276d487e55954/borewell_pareto_chart_w640.jpeg" style="width: 500px; height: 335px;" /></p>
Applying the Geometric Distribution to Rare Adverse Events
<p>There are many different types of control charts, but for rare events, we can use <a href="http://www.minitab.com/products/minitab">Minitab Statistical Software</a> and the G chart. Based on the geometric distribution, the G chart is designed specifically for monitoring rare events. In the geometric distribution, we count the number of opportunities before or until the defect (adverse event) occurs.</p>
<p>The figure below shows the geometric probability distribution of days between such rare events if the probability of the event is 0.01. As you can see, the odds of an event happening 50 or 100 days after the previous one are much higher than the odds of the next event happening 300 or 400 days later.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/1587c05dd9a8d77bcda5be87bb2a748b/borewell_distribution_plot.jpg" style="width: 500px; height: 333px;" /></p>
<p>By using the geometric distribution to plot the number of <a href="http://www.minitab.com/support/tutorials/monitoring-rare-events-with-g-charts/">days between rare events</a>, such as borewell accidents, the G chart can reveal patterns or trends that might enable us to prevent such accidents in future. In this case, we count the number of days between reported borewell accidents. One key assumption, when counting the number of days between the events, is that the number of accidents per day was fairly constant.</p>
A G-Chart for Prediction of the Next Borewell Accident
<p>I now used Minitab to create a G-chart for the analysis of the borewell accident data I collected, shown below.</p>
<p>Although the observations fall within the upper and lower control limits (UCL and LCL), the G chart shows a cluster of observations below the center line (the mean) after the 28th observation and before the 34th observation (the latest event). Overall, the chart indicates/detects an unusually high rate adverse events (borewell accidents) over the past decade.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/7571156e97822d68efe18af3225902e5/borewell_g_chart_date_between_events.jpg" style="width: 500px; height: 332px; border-width: 1px; border-style: solid;" /></p>
<p>Descriptive statistics based on the Gaussian distribution for my data show 90.8 days as the mean "days between events." But the G-chart, based on geometric distribution, which is more apt for studying the distribution of adverse events, indicates a Mean (CL) of only 67.2 days as "days between events."</p>
Predicting Days Between Borewell Accidents with a Cumulative Probability Distribution
<p>I used Minitab to create a cumulative distribution function for data, using the geometric distribution with probability set at 0.01. This gives us some additional detail about how many incident-free days we're likely to have until the next borewell tragedy strikes: </p>
<p style="margin-left: 40px;"><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/479b4fbd-f8c0-4011-9409-f4109cc4c745/Image/77a56196f91723fca7f7e7222a815573/borewell_output.jpg" style="width: 290px; height: 640px;" /></p>
<p>Based on the above, we can reasonably predict when next borewell accident is most likely to occur in any of the states included in the data, especially in the states of Haryana, Tamil Nadu, Gujarat, Rajasthan, and Karnataka.</p>
<p>The probabilities are shown below, with the assumption that the sample size and the Gage R&R / Measurement errors of event data reported and collected are adequate and within the allowable limits.</p>
<p><strong>Probability of next borewell event happening in...</strong></p>
<ul>
<li>31 days or less: 0.275020 = 27.5% appx.<br />
</li>
<li>104 days or less = 0.651907 = 65% appx.<br />
</li>
<li>181 days or less = 0.839452 = 84% appx.<br />
</li>
<li>488 days or less = 0.992661 = 99% appx.</li>
</ul>
<p> </p>
<p>My purpose in preparing this study would be fulfilled if enough people take preventive actions before the possibility of occurrence next such an adverse event within next 6 months (p > 80%). NGOs, government officials, and individuals all need to take preventive actions—like sealing all open borewells across India, especially in the above 5 states—to prevent many more innocent children from dying while playing.</p>
<p> </p>
<p><strong>About the Guest Blogger:</strong></p>
<p><em>Ondiappan "Ari" Arivazhagan is an honors graduate in civil / structural engineering from the University of Madras. He is a certified PMP, PMI-SP, PMI-RMP from the Project Management Institute. He is also a Master Black Belt in Lean Six Sigma and has done Business Analytics from IIM, Bangalore. He has 30 years of professional global project management experience in various countries and has almost 14 years of teaching / training experience in project management and Lean Six Sigma. He is the Founder-CEO of International Institute of Project Management (IIPM), Chennai, and can be reached at <a href="mailto:askari@iipmchennai.com?subject=Minitab%20Blog%20Reader" target="_blank">askari@iipmchennai.com</a>.</em></p>
<p><em>An earlier version of this article was published on LinkedIn. </em></p>
Data AnalysisStatistics in the NewsTue, 19 Aug 2014 12:00:00 +0000http://blog.minitab.com/blog/statistics-in-the-field/using-the-g-chart-control-chart-for-rare-events-to-predict-borewell-accidentsGuest BloggerAngst Over ANOVA Assumptions? Ask the Assistant.
http://blog.minitab.com/blog/statistics-and-quality-data-analysis/angst-over-anova-assumptions-ask-the-assistant
<p>Do you suffer from PAAA (Post-Analysis Assumption Angst)? You’re not alone.</p>
<p>Checking the required assumptions for a statistical analysis is critical. But if you don’t have a Ph.D. in statistics, it can feel more complicated and confusing than the primary analysis itself.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/cbbb702484d42f5b000f132cbe62f7c5/angst.jpg" style="width: 250px; height: 265px; float: right; margin: 10px 15px;" />How does the <a href="//cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/File/c38943bb6bcf5aca859630a0fa00ce64/cuckoo_egg_data.MPJ">cuckoo egg data</a>, a common sample data set often used to teach <a href="http://blog.minitab.com/blog/statistics-and-quality-data-analysis/cuckoo-for-quality3a-a-birdseye-view-of-a-classic-anova-example" target="_blank">analysis of variance</a>, satisfy the following formal assumptions for a classical one-way ANOVA (F-test)?</p>
<ul>
<li>Normality</li>
<li>Homoscedasticity</li>
<li>Independence</li>
</ul>
Are My Data (Kinda Sorta) Normal?
<p>To check the normality of each group of data, a common strategy is to display probability plots. In Minitab, choose <strong>Graph > Probability Plot > Multiple</strong>. Enter the response (‘<em>Length of Cuckoo Egg</em>) as the Graph variable and the grouping variable (<em>Nes</em>t) as the categorical variable. Click <strong>Multiple Graphs</strong> and check <strong>In separate panels of the same graph</strong>. </p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/615059cce608b31fb5c1b2ac4af8e429/probability_plot_of_length_of_cuckoo_egg.jpg" style="width: 576px; height: 384px;" /></p>
<p>If the data are normally distributed, the points in each plot should fall along a straight line within the curved confidence bands on each side. In the graph above, the points in most plots are “kinda sorta” straight and fall <em>mostly</em> within the confidence bands—except for the meadow pipit group.</p>
<p>The output table at the bottom right of the graph includes the results of a normality test for each group. Not surprisingly, the meadow pipit group “fails” the normality test. (Its p-value is <0.005, so you reject the null hypothesis that the data are normally distributed.)</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/b97c831ec1860f4b6c7b71e2b13033b0/normal_adler_quote.jpg" style="float: right; width: 278px; height: 113px; margin: 10px 15px;" />Hmmm…now what?</p>
<p>One commonly overlooked factor when interpreting results of a normality test is sample size. Generally speaking, a larger sample size gives the normality test more power to detect departures from normality. When the sample size is small, the test may not have enough “oomph” to catch nonnormality. So you really can’t be sure.</p>
<p>For the cuckoo egg data, sample size for all of the groups is about 15—except for the meadow pipit group, which has 45 data values. It’s probably not a coincidence that the biggest group is the only one flagged as being nonnormal.</p>
<p>Another way to evaluate the normality assumption for ANOVA is to display a normal probability plot of the errors. To do this in Minitab, just click <strong>Graphs</strong> in the ANOVA main dialog box and check <strong>Normal probability plot of residuals</strong>.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/6fb048e236dee771be042b052fa8b620/normal_probability_plot_cuckoo_egg.jpg" style="width: 578px; height: 383px;" /></p>
<p>Hmmm. The data kinda sorta fall along the line. But the circled errors on the bottom left indicate <em>some</em> skewness in the left tail. Is it important?</p>
<p>ANOVA is <em>robust</em> to the normality assumption—it can handle “kinda sorta” normal data—<em>if your sample sizes are large enough</em>.</p>
<p>But how large, exactly? It's starting to feel like one fuzzy result leads to another fuzzy result.</p>
<p>You decide to have a bowl of ice cream and move onto the next assumption. Maybe you'll get lucky with...</p>
Homoscedasticity? Say What?
<p>If the term homoscedasticity makes your flesh crawl, <a href="http://blog.minitab.com/blog/statistics-and-quality-data-analysis/dont-be-a-victim-of-statistical-hippopotomonstrosesquipedaliophobia" target="_blank">read this post</a>. Basically, all it means is that the data in each group should vary by about the same amount (i.e. have roughly equal variance).</p>
<p>ANOVA is also fairly robust to the equal variances assumption, <em>provided the samples are about the same size for all groups</em>. For the cuckoo egg data set, the samples have very different sizes, so we can’t use that “get-out-of-jail-free” card.</p>
<p>To visually assess the variation among the groups, you can look at boxplots. The lengths of the boxes and the whiskers should be about the same for each group. (<strong>Graph > Boxplot > With Groups</strong>).</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/bd51ea3b4d6dd127618fdb9744c72ec6/boxplot_of_egg_length.jpg" style="width: 576px; height: 384px;" /></p>
<p>The boxplots indicate that the variation of the data in each group is “kinda sorta" the same. Some whiskers are definitely longer and some are shorter. The lengths of the boxes vary somewhat as well.</p>
<p>Is it enough to matter? The boxplots can't tell us.</p>
<p>A more rigorous way to compare the variation among the groups is to perform an equal variances test (<strong>Stat > ANOVA > Test for Equal Variances</strong>). Here are the results for the cuckoo egg data:</p>
<p style="margin-left: 40px;"><strong>Method Test Statistic P-Value</strong><br />
Multiple comparisons — 0.476<br />
Levene 0.64 0.670</p>
<p>For both tests, the p-value is not less than the alpha level of 0.05. Therefore, there’s not sufficient evidence to conclude that the variances are different. These data seem to satisfy the equal variances assumption.</p>
<p>But wait. Before you pop that champagne cork. Don't forget the small sample sizes (n = 15) in this data set.</p>
<p>The equal variances test, like the normality test, may not have enough power to detect differences in variance among the groups. You could be assuming the variances are equal only because you don't have enough data to prove they're not.</p>
<p>At this point, you may feel an Excedrin headache coming on.</p>
Using the Minitab Assistant to Perform ANOVA
<p>If you feel like you’re falling into a rabbit hole, struggling to evaluate the assumptions of the assumptions, you may want to drop by the Minitab Assistant to check what condition your condition is in.</p>
<p>To perform one-way ANOVA, choose <strong>Assistant > Hypothesis Tests > One-Way ANOVA</strong>. Then, click on the Report Card.</p>
<p>First, look at the Equal Variances check. </p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/5e2b496a3eed589b4ccc12a59c63eb2a/report_card_equal_variances_w640.jpeg" style="width: 640px; height: 254px;" /></p>
<p>Say so-long to the homoscedastic heebie-jeebies! The report card informs you don’t need to worry about equal variances at all—<em>even when your sample sizes are different</em>. Sweet! That’s because the Assistant uses <a href="http://blog.minitab.com/blog/adventures-in-statistics/did-welchs-anova-make-fishers-classic-one-way-anova-obsolete">Welch’s method rather than an F- test</a>. Welch’s method doesn’t require that the groups have equal variances.</p>
<p>Next, look at the Normality check.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/a17e7dc9f4f6a93a4a004d854a05d929/report_card_normality.jpg" style="width: 640px; height: 240px;" /></p>
<p>Here, the Assistant cautions that the small samples could be an issue for the normality assumption. But it also lets you know that normality can't be reliably checked with small samples—saving you time from pondering the imponderable.</p>
<p>More important, it gives a clear definitive answer to the question “What size samples do I need for robustness to the normality assumption?” </p>
<p>The minimum requirement for robustness to normality depends on the number of groups in your data. The sample size recommendation is <a href="http://support.minitab.com/en-us/minitab/17/assistant_one_way_anova.pdf">based on tens of thousands of simulations</a> performed by Minitab research statisticians to examine the Type I error rate when using Welch’s method to evaluate small and moderate size samples from nonnormal distributions.</p>
<p>There are two other checks on the report card—one for <strong>Sample Size</strong>, and one for <strong>Unusual Data</strong>. The <strong>Unusual Data</strong> check flags any outliers that could severely bias your results. The <strong>Sample Size</strong> check lets you know if you’ve collected enough data to provide the test with sufficient power to detect a difference in means.</p>
<p>These aren’t formal ANOVA assumptions, but they’re critical issues that can affect your results—and they’re often overlooked.</p>
Independence
<p>If you’re a careful reader, and you’re still with me, you’ve probably noticed I haven’t covered the formal ANOVA assumption of independence. It’s not because it’s not important—of all the assumptions, ANOVA is least robust to violations of independence.</p>
<p>But independence of the observations is affected by how you collect your data. To evaluate this assumption with 100% certainty, the Assistant would have to peek over your shoulder and watch you. The Assistant can’t do that.</p>
<p><em>Yet.</em></p>
<p>But it does provide a graph of the data points graphed in worksheet order. If you’ve entered your data in the worksheet in the sequence they were collected, this alerts you to any dependence in the data points collected close together in time or space. That’s one of most common types of dependence.</p>
<p><img alt="" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/ba6a552e-3bc0-4eed-9c9a-eae3ade49498/Image/7955854840800489de5c5327dcc0b91f/data_worksheet_order.jpg" /></p>
<p>These data don't seem to show a time-order effect. However, notice the outliers flagged in the meadow pippit group. Those could be an issue.</p>
<p><strong>Note:</strong> If you use classic One-Way ANOVA in Minitab, you can evaluate this assumption by looking at the residuals vs. order plot. If there’s a time-dependent pattern in the data points, there will be a time-dependent pattern in the errors.</p>
Key Points
<ul>
<li>Sample size is critical not just for providing power to the primary analysis, but for establishing robustness to some formal assumptions.<br />
</li>
<li>If you don't know the formal assumptions for an analysis, or if you run into trouble trying to interpret them, consider using the Assistant.<br />
</li>
<li>If your data are "borderline" for satisfying the ANOVA requirements, consider your application and its potential consequences. You may want to use a nonparametric test, such as Kruskall-Wallis or Mood's median test, to compare the medians of the groups rather than the means. Just remember—those tests have formal assumptions as well!</li>
</ul>
<p> </p>
Data AnalysisLearningStatisticsStatistics HelpMon, 18 Aug 2014 13:33:00 +0000http://blog.minitab.com/blog/statistics-and-quality-data-analysis/angst-over-anova-assumptions-ask-the-assistantPatrick RunkelHow Accurate are Fantasy Football Rankings? Part II
http://blog.minitab.com/blog/the-statistics-game/how-accurate-are-fantasy-football-rankings-part-ii
<p>Previously, we looked at how accurate fantasy football rankings were <a href="http://blog.minitab.com/blog/the-statistics-game/how-accurate-are-fantasy-football-rankings">for quarterbacks and tight ends</a>. We found out that rankings for quarterbacks were quite accurate, with most of the top-ranked quarterbacks in the preseason finishing in the top 5 at the end of the season. Tight end rankings had more variation, with 36% of the top 5 preseason tight ends (over the last 5 years) actually finishing outside the top 10!</p>
<p><img alt="Cheat Sheat" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/fe2c58f6-2410-4b6f-b687-d378929b1f9b/Image/14edab962b5c1df587e395a75459439b/2014_fantasy_football_cheat_sheat.jpg" style="float: right; width: 275px; height: 157px;" />Now it’s time to move our attention to the running backs and wide receivers. Just like before, I went back the previous 5 seasons and found ESPN’s preseason rankings. For each season I recorded where the top preseason players finished at the end of the season, and also where the top players at the end of the season were ranked before the season started.</p>
<p>With quarterbacks and tight ends, I only looked at the top 5 players. But since more running backs and receivers are drafted, I’ll look at the top 10 players. Now let's analyze the data using <a href="http://www.minitab.com/products/minitab/">Minitab Statistical Software</a>. </p>
How did the top-ranked preseason RBs and WRs finish the season ranked?
<p>Let’s start by looking at how the top-rated preseason players fared at the end of the season. I took the top 10 ranked preseason RBs and WRs for each season from 2009-2013 and recorded where they ranked to finish the season. </p>
<p><img alt="IVP" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/fe2c58f6-2410-4b6f-b687-d378929b1f9b/Image/f2db50de83552939b23088e7cb196a7b/ivp_rbs_wrs_preseason_w640.jpeg" style="width: 640px; height: 427px;" /></p>
<p><img alt="Describe" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/fe2c58f6-2410-4b6f-b687-d378929b1f9b/Image/85fe1626bed20238726f327e27bb78af/describe_presason_rb_wr_w640.jpeg" style="width: 640px; height: 109px;" /></p>
<p>At first glance, the individual plots show that the spread for running backs and wide receivers appears to be about the same. But the descriptive statistics tell a different story. The 3rd quartile value (Q3) is the most telling. 75% of preseason top 10 running backs finish in the top 18, while that number rises all the way to 28.75 for wide receivers! In fact, 32% of wide receivers ranked in the top 10 in the preseason finished the season outside the <em>top 20</em>, while the same was only true for 24% of running backs. Running backs do have the biggest outlier (when Ryan Grant had a season ending injury in his first game of 2010 and finished as the 126th ranked running back), but injuries like that are random and impossible to predict. Overall, preseason ranks for running backs are more accurate than for wide receivers.</p>
How were the top-scoring RBs and WRs ranked in the preseason?
<p>Let’s shift our focus to later in the draft. How often can you draft a lower-ranked running back or wide receiver and still have them finish in the top 10?</p>
<p><img alt="IVP" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/fe2c58f6-2410-4b6f-b687-d378929b1f9b/Image/6b08619fded42ff700cb050fa03f0033/ivp_wrs_rbs_postseason_w640.jpeg" style="width: 640px; height: 427px;" /></p>
<p><img alt="Describe" src="http://cdn2.content.compendiumblog.com/uploads/user/458939f4-fe08-4dbc-b271-efca0f5a2682/fe2c58f6-2410-4b6f-b687-d378929b1f9b/Image/c7bc6b2d68572eff76d8a22edfeda563/describe_postseason_rbs_wrs_w640.jpeg" style="width: 640px; height: 108px;" /></p>
<p>Wide receivers have had more players come out of nowhere to be top 10 scorers at the end of the season (Victor Cruz in 2011 and Brandon Lloyd and Stevie Johnson in 2010 were all ranked 87th or worse, yet finished in the top 10). But the descriptive statistics indicate a pretty even distribution otherwise. About half of the top 10 scoring RBs and WRs were <em>not</em> ranked in the top 10 to begin the season. And 25% of players were ranked outside the top 25, yet were still able to finish in the top 10. For both positions, there are frequently lower ranked players that exceed expectations and finish in the top 10.</p>
<p>But if you want one of the <em>best</em> players, say top 3...can you afford to wait or do you need to select a top ranked player early? The following table shows the 3 highest scoring players for each year, with their preseason rank in parentheses.</p>
<p align="center"><strong>Year</strong></p>
<p align="center"><strong>Top Scoring RB</strong></p>
<p align="center"><strong>2nd Highest Scoring RB</strong></p>
<p align="center"><strong>3rd Highest Scoring RB</strong></p>
<p align="center"><strong>Top Scoring WR</strong></p>
<p align="center"><strong>2nd Highest Scoring WR</strong></p>
<p align="center"><strong>3rd Highest Scoring WR</strong></p>
<p align="center">2013</p>
<p align="center">Jamaal Charles (6)</p>
<p align="center">LeSean McCoy (10)</p>
<p align="center">Matt Forte (12)</p>
<p align="center">Calvin Johnson (1)</p>
<p align="center">Josh Gordon (42)</p>
<p align="center">Demaryius Thomas (6)</p>
<p align="center">2012</p>
<p align="center">Adrian Peterson (10)</p>
<p align="center">Arian Foster (1)</p>
<p align="center">Doug Martin (27)</p>
<p align="center">Calvin Johnson (1)</p>
<p align="center">Brandon Marshall (12)</p>
<p align="center">Dez Bryant (15)</p>
<p align="center">2011</p>
<p align="center">LeSean McCoy (6)</p>
<p align="center">Ray Rice (5)</p>
<p align="center">Arian Foster (4)</p>
<p align="center">Calvin Johnson (5)</p>
<p align="center">Wes Welker (22)</p>
<p align="center">Victor Cruz (110)</p>
<p align="center">2010</p>
<p align="center">Arian Foster (23)</p>
<p align="center">Adrian Peterson (2)</p>
<p align="center">Peyton Hillis (63)</p>
<p align="center">Dwayne Bowe (20)</p>
<p align="center">Brandon Lloyd (123)</p>
<p align="center">Greg Jennings (11)</p>
<p align="center">2009</p>
<p align="center">Chris Johnson (7)</p>
<p align="center">Adrian Peterson (1)</p>
<p align="center">Maurice Jones-Drew (3)</p>
<p align="center">Andre Johnson (2)</p>
<p align="center">Randy Moss (4)</p>
<p align="center">Miles Austin (68)</p>
<p>Since 2009, nine different receivers finished the season in the top 3 despite being ranked outside the preseason top 10. <em>That’s 60%</em>! And two of those players were ranked outside the top 100 in the preseason! But amongst all the inconsistency is Calvin Johnson. He’s the only wide receiver that is listed more than once. And he’s finished as the #1 ranked receiver 3 times in a row!</p>
<p>Meanwhile only 4 running backs (27%) were able to finish in the top 3 despite being ranked outside the preseason top 10. Right now in ESPN’s average draft position, the 10th running back is being drafted with the 19th overall pick. So before the 2nd round of the draft is even over, there is a good chance that the top 3 running backs have already been selected. Compare that to wide receivers, where the 10th receiver is being drafted with the 34th overall pick. So in the middle of the 4th round, a top 3 wide receiver (or even two) could still be on the board!</p>
<p>You can definitely wait to draft a wide receiver. The same can’t be said of running backs.</p>
<p>So how should you use this information in your fantasy football draft?</p>
Focus on Running Backs Early
<p>It’s not that the running back you pick is guaranteed to have a great season, but we just saw that, on average, 10 running backs are being selected before the end of the 2nd round! After that, your chances of picking a top running back start to diminish. At least one of your first two picks should be a running back, if not both!</p>
<p>However, keep in mind that selecting RB/RB with your first two picks can be a high-variance strategy. Consider that last year, in a 10-team league you could have taken Jamaal Charles and Matt Forte with the 6th and 15th pick respectively. Those players finished as the #1 and #3 RB, and if you didn’t win your fantasy league you definitely made the playoffs. Of course, you could have just as easily picked C. J. Spiller and Stevan Ridley, who finished 31st and 26th. Unless you got really lucky with your later picks, you could say hello to the consolation bracket.</p>
<p>If you want to play it more conservative, this data analysis pointed out a few other options. We know that quarterbacks are the most consistent position (Aaron Rodgers in 2013 aside), and this year Peyton Manning, Aaron Rodgers, and Drew Brees are the top 3 ranked quarterbacks. Spending an early pick on one of them should give you a consistent scorer who is much less likely to be a bust than an early running back.</p>
<p>Calvin Johnson and Jimmy Graham are also two very consistent players at two very inconsistent positions. Both players have finished in the top 3 at their position for the last 3 years (with Johnson finishing #1 all 3 years). You should feel just fine using your first two picks on one of these players and a running back. But use caution on selecting a different TE or WR with an early pick.</p>
Wait on Your Wide Receivers
<p>Wide receivers have the least accurate preseason rankings. Half of the preseason top 10 finish outside the top 12, and 25% finish <em>outside the top 28!</em> Because of this, there is value to be found later in the draft for wide receivers. Try to identify some wide receivers you like in later rounds, and focus your early picks on other positions.</p>
<p>This example is a bit extreme, but last year in a fantasy draft I spent 4 of my first 5 draft picks on running backs (with Jimmy Graham being the non-running back pick). I was able to do so because I was fine getting Eric Decker (preseason #20) and Antonio Brown (preseason #24) in the 6th and 7th rounds. They finished as the 8th and 6th ranked wide receivers. Obviously I got a little lucky that they were <em>that</em> <em>good</em>, but that’s kind of the point. I like to think of fantasy football picks as lottery tickets. You could hit the jackpot with some players, win a decent amount with others, and have some that are busts. After the first few rounds, wide receivers have a better chance of being winning lottery tickets than other positions.</p>
<p>Now, you don’t have to <em>completely</em> neglect the WR position before the 6th round like in the example above. Just know that you’re putting the odds in your favor by waiting to draft the bulk of your wide receivers.</p>
Who Needs a Backup QB?
<p>One last thing while we’re on the lottery ticket analogy. Let’s say you draft one of the top quarterbacks (Manning, Rodgers, or Brees). Don’t draft a backup quarterback! We already saw quarterbacks have the most accurate preseason rankings. By the time you draft a backup, it’s unlikely that lower-ranked player you choose will rise into a star that you will start each week or be able to use as trade bait. And on your QB’s bye week, you can easily pick somebody up off the waiver wire.</p>
<p>So why waste that pick on somebody with very little upside? Even if you’re picking in the 100s, there is still value to be had! Josh Gordon, Alshon Jeffery, Knowshon Moreno, and Julius Thomas were all ranked outside the preseason top 100 last year, and all turned into great fantasy players! </p>
<p>Want to take this idea to the (slightly crazy) extreme? If you have a late first round pick, try and use your first two picks on Jimmy Graham and one of Manning, Rodgers, or Brees. With your QB and TE position locked up, spend your next 12 picks on nothing but RBs and WRs. Then use your last two picks on a defense and kicker! I know this goes against the advice of focusing on running backs early, but I <em>did </em>say it was a slightly crazy and extreme strategy! If you can get lucky and find a winning lottery ticket with a lower-ranked running back or two (maybe Montee Ball, Ben Tate, Andre Ellington), it <em>could</em> even be a winning strategy. </p>
<p>If you decide to try that draft strategy, let me know how it goes! And whatever strategy you use, good luck with your 2014 fantasy football season!</p>
Fun StatisticsFri, 15 Aug 2014 15:48:00 +0000http://blog.minitab.com/blog/the-statistics-game/how-accurate-are-fantasy-football-rankings-part-iiKevin Rudy