Last time, we talked about why to use designs with just two levels. Now it’s time to discuss the two-level options in design of experiments, starting with Plackett-Burman designs.
Plackett-Burman designs exist so that you can quickly evaluate lots of factors to see which ones are important. Plackett-Burman designs are often called “screening designs” because they help you screen out unimportant factors.
This fits in with our previously-stated goal for design of experiments: learn as much as possible from the smallest amount of data. If we’re not sure a factor is important, we don’t want to spend time and money collecting data about it.
An example of when to use a Plackett-Burman design
For example, consider this lengthy, but less-than-comprehensive, list of factors that can contribute to vehicle accidents, based on James Harris’ article:
- Weather
- Lighting
- Roadside devices, like traffic signs
- Roadside activities, like construction
- Surface deposits
- Road damage
- Road deterioration
- Road age
- Vehicle condition
- View obstructions
- Distractions
- Instruments
- Signaling devices
- Control sensation
- Comfort
- Automatic controls
- Vehicle Weight
- Performance
- Vehicle Width
- Vehicle Height
- Vehicle stability
- Vehicle speed
- Sensory capabilities
- Driver knowledge
- Driver judgment
- Driver attitude
- Driver alertness
- Driver health
- Driver skill
- Driver age
- Driver habits
- Driver weight
- Driver strength
- Driver freedom of movement
It’s likely that all of these factors contribute to accidents, but that some are much more important than others. Within design of experiments methods, the Plackett-Burman design is a good choice for figuring out which factors are most important. After the Plackett-Burman experiment was done, you could use a different experiment to answer questions like “Does the effect of vehicle speed depend on how damaged the road is?”
It's important to realize just how good Plackett-Burman designs are at reducing the amount of data we have to collect. For example, suppose that you want to find out which are the important factors when you have 15 to consider. For a Plackett-Burman design, you can estimate the effects of the different factors with as few as 20 data points. A General Full Factorial design for 15 factors, even if the factors have only 2 levels, would have 32,768 data points.
That’s a big difference.
Downsides of the Plackett-Burman designs
There are some downsides to this efficiency that you should know about when you prepare to use a Plackett-Burman design. Plackett-Burman designs won’t tell you if the effect of one factor depends on another factor. Also, if you run the smallest design you can, it doesn’t follow that you collect enough data to know what those effects are very precisely. In design of experiments, a Plackett-Burman design is a starting point, and you should use it intending to follow-up with more detailed experimentation.
Quick summary: Plackett-Burman designs are helpful when you have a lot of potential factors, and need to figure out the vital few that will help you solve problems and make decisions. Plackett-Burman designs are great when we want to use design of experiments to learn as much as possible from the smallest amount of data.