P charts and U charts have been a valuable tool in the quality engineer's toolbox for decades. But, as David Laney found out, when sample sizes are very large, the control limits become too narrow and the data can spill out over the control limits.
Where some saw chaos, Laney was inspired to put the teachings of Fisher, Deming, Wheeler, and others to bear on the problem and ended up changing how we think about P charts and U charts.
We recently had a chance to talk with Mr. Laney about his inspirations and about Minitab’s new features, the P’ Charts and U’ Charts that bear his name.
Minitab: You introduced the Laney P’ and U’ charts in a 2002 paper. In a nutshell, what are they for and what is innovative about them?
Laney: What they’re for is to correct an inherent error in the P chart and the U chart, which are—in a word—wrong. They’re wrong because they assume that all the variation in the entire process is within-subgroup variation, or sampling variation.
It’s important to not blindly look at a control chart whose limits are very close together, and with data all over the page and say, “We’ve got utter chaos here.” No, you don’t. What you’ve got is a lot of variation between subgroups. This variation is not explained by the binomial or Poisson assumptions alone, yet needs to be accounted for. That’s what the P’ and U’ charts do.
Minitab: What got you thinking about these issues with the P and U charts?
Laney: At BellSouth, we started doing total quality in a big way in 1990. I was teaching lots of people how to use control charts. Then something interesting happened. We had a project to look at Emergency-911 calls in Florida each month and track the proportion of calls that did not get through. That's a pretty important statistic—you know, lives are at stake!
So I looked at the chart for this project. It was a P chart, but the upper and lower control limits were about a millimeter apart. And the data were all over the page. This was the first time I ever saw that. I said, “Now, how is it possible for every point to be out of control?” So I went to the AT&T handbook and looked it up, and learned that yeah, this can happen when you've got large, large samples.
Minitab: That’s called overdispersion?
Laney: Right. If the samples are large enough, the sampling variation is driven practically to zero; the P chart puts the control limits too close together and the data seem to be escaping. So the handbook says to just use an XmR chart. Okay.
Minitab: When you say XmR chart, just for clarification, it’s what we call an I-MR chart in Minitab?
Laney: Exactly, or an "individuals and moving range" chart. The control limits are based on moving ranges of size 2, which measure short-term variation.
Dr. Shewhart taught us that short-term variation is the key. Because if you use long-term variation you may be allowing trend and seasonality to interfere with your attempt to differentiate between special causes and common causes.
Not long after the 911 project, I was in Knoxville, Tennessee, sitting in a hotel conference room listening to Dr. Don Wheeler. He was talking about analysis of variance, detecting the difference among several means, and so on. Well what does that use? That uses the principles introduced by Fisher -- a comparison of within-group variation and between-group variation. And it makes a careful distinction between those two in order to assess what’s going on. Where does the P chart do that? The P chart doesn’t do that.
Minitab: Why should the P chart do that?
Laney: Well, in the real world, problems, defects and defectives, can be dependent on things that change from day to day—like rain, or temperature, or phases of the moon. Telephone system problems vary a lot depending on how many thunderstorms you have that month.
I got to thinking about all this. What bothered me about using the I-MR chart shortcut was that its control limits were, by definition, flat -- despite the substantial difference in sample sizes from month to month. The number of calls to 911 in Florida tend to be very high in the summer, when there are more cases of heat-related illness. There are fewer calls in the winter. Well, larger subgroups mean more statistical certainty. We would expect a P chart, with varying sample sizes, to have "wiggly" control limits, with wider limits for smaller subgroups and narrower limits for larger subgroups, right?
Minitab: Right.
Laney: How many individuals charts have you ever seen with wiggly control limits? [laughs]
Minitab: None.
Laney: At some point it hit me—instead of an I-MR chart I could use a Z chart.
Minitab: A Z chart has wiggly control limits?
Laney: Well, no, but the Z chart does account for differences in sample size. To create the Z chart, you convert the p-values into z-values using the formula:
zi = pi – pbar / Sigma of pi
where zi is the z-value for a subgroup, pi is the proportion of defectives for that subgroup, pbar is the average proportion of defectives. Sigma of pi is the standard error for the subgroup.
The control limits on the Z chart are always straight, but Sigma of pi is smaller for larger subgroups. For the same pi, a large subgroup size will cause zi to be farther from the center line, which is always at zero. So in both the P chart and the Z chart, extreme values for the proportion of defectives are more likely to fall outside of the control limits if they occur in larger subgroups.
Now, in the classical Z chart we know that 3 standard deviations encompass 99.73 % of the data so therefore we’re going to set our control limits at +/-3. Right? No we’re not! Because I remember what Wheeler said, “Why assume the variation when you can measure it?”
Why would we sit there and just blindly assume, “Well, the upper limit must be 3.” Why don’t we use moving ranges of size 2, like the I-MR chart does, and find out what it is?
So I came up with what I called the Z’ chart. I didn’t know what else to call it. I wanted it to have a DNA linkage to the Z chart, but I wanted it to be sufficiently different. I used the moving ranges of size 2 to estimate the standard deviation of the z-scores, and called that “Sigma Z”. Then I set my control limits at +/-3 times Sigma Z.
Minitab: Did the 911 data look different on the Z’ chart?
Laney: Absolutely. All of a sudden the limits went out to where the data were. Instead of the P chart where almost every point was out of control, on the Z’ chart, only the usual 1 or 2 points out of 20 were out of control. What it showed was that the upper limit wasn’t really 3. It was more like 15, because there was 5 times more variation from month to month than the binomial formula alone could predict.
So we started using the Z’ chart with great success, but then, as you might suspect, a number of our clients would say, “What’s a Z? What does this mean?”
So now we have a new problem. I’ve got something that I know is right, but I can’t sell it because nobody can spell it; "lay" people can't seem to twist their thinking around into the Z-plane and understand what’s going on.
The solution was to convert the Z data back into P data, to turn it into a type of P chart again.
Minitab: A P’ chart?
Laney: Exactly. In the P’ chart, Sigma Z is used to adjust the values so you have realistic upper and lower control limits. In a regular P chart, the control limits are:
+/- 3 x Sigma of pi
In a P’ chart the control limits are:
+/- 3 x Sigma of pi x Sigma Z
Now I had a chart that showed the actual recorded proportions of defectives, rather than contrived z-values. So we had something that worked and was easy for everybody to understand.
One of the first people I told about this was Forrest Breyfogle, who’d invited me to attend his new Advanced Black Belt course. On the side, I told him about what I had just discovered, and he was very interested. I mean, I could tell he was struck by this simple but incredibly powerful concept, because it was correcting something that had been an issue for 70 years.
Not long after that, out came a fabulous book called Implementing Six Sigma, by Forrest W. Breyfogle. And on page 177 he gave me credit for inventing the new Z' chart, which he called the "Z&MR chart" He didn’t take it all the way back to the P’ chart. Ever since, a growing number of people have become early adopters of this method.
Minitab: Of the P’ chart?
Laney: Yes. One of the first was Scott Wise, the first Master Black Belt at Dell Corporation. Other early supporters included Bill Woodall at the University of Alabama (now at Virginia Tech), Tom Pyzdek, Don Wheeler, and Roger Hoerl. And then there was a fellow named Mohammed Mohammed, a professor at the University of Birmingham, England. (I’m in Birmingham, Alabama!) I have no idea how he heard about me, but I got an email from him and he had a problem that was a perfect one for the P’ chart. And so I explained it to him, he got me to help him write a little paper for a British medical journal, and it's caught on pretty well over there.
People kept bugging me, saying, “You know, you really ought to write this up.” Well, I’m not in academia. I don’t publish or perish. But I had about 30 years’ service in industry by that time and I was looking at what to do next. So I wrote an article and sent it in to Quality Engineering. They published it in 2002, just one month before I left BellSouth and started teaching at Samford University.
Minitab: Clearly, they saw a lot of value in what you wrote.
Laney: Here’s the thing I want people to take away from this discussion, and I say this with almost religious fervor. Fisher taught us that there’s more to life than short-term, or within-group variation. There’s also between-group variation, and if we don’t come to grips with that fact, we’re doomed. We can’t understand data if we don’t look at it that way.
Minitab: Have there been any reservations about using your charts?
Laney: Well, the only reason that I can imagine that they haven’t had immediate acceptance is the law of inertia. Engineers are supposed to be innovative, but sometimes old habits die hard.
Minitab: How does that play out in a practical situation?
Laney: Well, people naturally ask, “How might I know when I should use a P’ chart?” Well now Minitab has a test they can run to see whether they need to use the P’ chart or whether they could just rely on the P chart.
Minitab: Yes, the P Chart Diagnostic.
Laney: If your data do not overwhelmingly argue in favor of the P’ chart, then the diagnostic says you can use the P chart. But why would you want to settle for that? For example, suppose a Sigma Z of 1.20 doesn’t trip your test. Why would you want to use the P chart when there is arguably 20% more variability in your data than the binomial can predict? You can end up dealing with more false alarms just because the diagnostic test has a low alpha and wants to be convinced beyond a shadow of a doubt before it recommends the P’ chart.
Even if Sigma Z is 1.01, I’d still rather use the P’ chart! Why wouldn’t you? Now that Minitab is doing the calculations for you, it’s just as easy to click on P’ chart as it is on P chart. Now, the P’ chart won’t make any noticeable difference if Sigma Z really is 1.01 But again, “Why assume the variation when you can measure it?” I would also say, “Why do your analysis in a way that could be wrong when you could do it in a way that’s always right?” Then you don’t have to worry about the diagnostic test.
There are going to be people who are hard to change, skeptics. Deming said that change occurs on a generational basis. That’s why it was good that he lived to be in his 90s so he could actually see some change taking place in this world. We don’t change quickly, so after 70 years of using a P chart there are going to be people who don’t necessarily understand or believe that the P’ chart is better. I’ll be happy knowing that if they at least run your test, and let that tell them they should be using the P’ chart, then they’ll use it and save themselves a whole lot of unnecessary busy work chasing false alarms.
Minitab: The P’ chart and U’ chart seem like powerful and versatile tools. And it’s no small feat to come up with such an elegant solution to a problem that has plagued the quality community for decades.
Laney: Thank you. Really, I revere the giants on whose shoulders I stand: Wheeler, Breyfogle, Woodall, Pyzdek, Hoerl,... I can’t help but think that if those people are behind me I can’t possibly be wrong.
Minitab: Are there any conditions under which you would not want to use the P’ chart instead of the P chart?
Laney: No. Categorically, positively, no. For the same reason I brought up before: there’s no such thing in the real world as a perfect normal, binomial, or Poisson distribution. In the same spirit that a statistician can reasonably say that the normal assumption is always wrong, we can quote George E.P. Box: “All models are wrong. Some models are useful.” And I would be willing to stake my reputation on the statement, “The blind reliance on the binomial or Poisson distribution embodied in classical attributes control charts is also always wrong.” Because there is variation in everything. Say, isn’t that Chapter 1, Page 1 of every SPC text—“There is variation in everything”?
The binomial assumption is never, ever exactly right. So why would you not want at least the slightest little nudge in the right direction?
Part of me still laments that when the time comes that I’m down there smiling up at everybody…I got that from George Carlin …there will always be a bit of regret that in my lifetime there was never a time that everybody just automatically used the P’ chart and the U’ chart.
Minitab: We couldn’t help but notice that when you wrote your Quality Digest article, you closed by saying “My life’s goal is to get this into Minitab.”
Laney: That’s right, exactly. One of my sons is a quality engineer. He’s going to continue the "family business" throughout his career and I hope he will continue to sell this idea. But he will be able to do something I couldn’t do; he’ll be able to call these charts up in Minitab and show his clients, “See, they’re the real deal.”
From now on I can point to this day on the calendar and say “There, right there, is where it changed.” And I can’t tell you how happy that makes me. Because now I don’t have to push anymore. With Minitab taking over the reins of this stagecoach, I know that the strongbox is going to be delivered.