When I talk to quality professionals about how they use statistics, one tool they mention again and again is design of experiments, or DOE. I'd never even heard the term before I started getting involved in quality improvement efforts, but now that I've learned how it works, I wonder why I didn't learn about it sooner. If you need to find out how several factors are affecting a process outcome, DOE is the way to go.
Somewhere in school you probably learned, like I did, that when you do an experiment you need to hold all the factors constant except for the one you're studying. That seems simple enough, until you hit a situation where you have many factors that you want study at the same time. Not only would studying each factor one at a time be very expensive and time-consuming, but you also wouldn't get any information about how the different factors might affect each other.
That's where design of experiments comes in. DOE turns the idea of needing to test only 1 factor at a time on its head by letting you change more than a single variable at a time. This minimizes the number of experimental runs you need to make, so you can obtain meaningful results and reach conclusions about how factors affect a response as efficiently as possible.
Depending on what you want to discover, and how much detail you need, a designed experiment may be very simple or very complex. Some experiments might include only one or two factors—others might look at a few dozen.
One of the most common types of designed experiments is a simple screening experiment, which is used to determine the factors have the greatest influence on an outcome. For example, an auto manufacturer might use a screening experiment to see which of seven or eight factors have the biggest effect on the drying time of paint on a new car.
Once the manufacturer has identified the two or three most important factors, quality engineers can use a more complex, multi-level designed experiment to identify the optimal settings for those factors. Obviously, the same experimental design wouldn't work for both cases.
It's a little bit like sandpaper: sheets with a large grit will let you sand off a big area quickly, while you'll need a finer grit to achieve total smoothness. Similarly, some designed experiments are great for broad, exploratory investigations, while others will give you tremendous precision and certainty.
Let's say you work for an electronics firm that has recently received a large number of complaints about defective mp3 players. Quality engineers have identified up to five different factors that could be to blame. You know that a designed experiment is needed, but how can you be sure you collect the right amount of data, under the right conditions, with the right factor settings, in the right order?
There's good reason to be concerned when starting a designed experiment. If you're setting up even a simple designed experiment by hand, it can be very difficult and leaves plenty of room for error. Fortunately, we can use statistical software to customize factorial designs. These tools make it easy to create experiments that are as detailed as they have be, but also as simple as they can be.
For example, Minitab's Create Factorial Design creates a data collection worksheet for you, indicating the factor combinations to run, as well as the random order in which to collect your data. You can also print the worksheet to simplify data collection.
The right design for your experiment will depend on the number of factors you're studying, the number of levels in each factor, and other considerations. Minitab offers two-level, Plackett-Burman, and general full factorial designs, each of which may be customized to meet the needs of your experiment.
You must have at least two factors and two levels for each (if you're doing a general full factorial design, you can have more than two levels). Factor levels, or settings, can be text (such as high and low) or numeric (such as 100° and 200°). Factors also can be categorical or continuous.
Your goals may demand greater or less statistical power. Are you doing a very sensitive adjustment for a critical process, or an early screening analysis to find out what factors even affect your outcome? If you're on a tight budget, the type of experiment you select might be influenced by how many experimental runs you can afford to do. A good design-of-experiments tool will let you quickly compare power and sample size assessments for 2-level factorial, Plackett-Burman, and general full factorial designs to help you choose the design appropriate for your situation.
If you'd like to learn more about DOE and you're using Minitab, the built-in tutorial (Help > Tutorials > DOE) will lead you through a factorial experiment from start to finish; it's a pretty painless way to get your feet wet. And if you're not already using Minitab, you can get the free 30-day trial to check it out.
Are you using DOE in your work yet?