What factors significantly affect how quickly my couch-potato pooch obeys the “Lay Down” command?
The cushiness of the floor surface? The tone of voice used? The type of reward she gets? How hungry she is?
I created a 1/8 fraction Resolution IV design for 7 factors and collected response data for 16 runs. Now it’s time to analyze the data in Minitab, using Stat > DOE > Factorial > Analyze Factorial Design.
After removing insignificant terms from the model, one at a time, starting with the highest-order interaction, here's the final model:
Of the original 7 factors in the screening experiment, only three are statistically significant at the alpha level of 0.05.
- Procedure: Sit first, then Lay Down vs Lay Down Only
- Hand signal: Point to the floor while giving the command, or not
- Reward: Dry biscuit vs honey cured ham.
Even though it’s not statistically significant, the Voice term is included in the final model because it’s part of the statistically significant 2-way interaction Voice*Hand signal.
To visualize your results, you can display a normal plot of the standardized effects.
The statistically significant effects are shown in red. The stronger the effect, the farther away from the blue line it is.
The Reward factor (G) has the greatest effect, followed by the Hand signal (D).
Analyzing the Main Effects
How do the different settings of each factor actually affect the response? To find out, display a main effects plot of the factors in your final model.
The vertical scale shows the mean response—here, the number of seconds it took the dog to lie down after being given the Lay Down command.
The two bottom plots show just what you’d expect: Using a hand signal and a reward of fresh honey-cured ham are both associated with a quicker mean response. In fact, they were the two strongest main effects.
But the two upper plots surprised me. They’re a good reminder that you can’ t always trust your “pat assumptions.”
I expected that giving my dog the Sit command first, and then a treat, before giving the Lay Down command, would focus her and make lay down more quickly. But as the Procedure plot shows, she responded faster, on average, when the Lay Down command was given by itself.
I also expected she would respond more quickly to an authoritative tone of voice. But the Voice plot suggests that she’s less motivated by a cold, steely “Nurse Ratched” tone than she is by a perky Mickey Mouse voice.
Or is she?
Warning 1: Beware of a main effects plot for a factor that’s part of a significant interaction. What you see may not be what you get.
Analyzing Interactions
In this model, there’s a significant interaction that involves the Voice factor (Voice*Hand signal). To accurately interpret the effect of this factor, I need to examine an interaction plot.
Now you can see why the main effects plot for Voice was misleading.
When combined with a hand signal, an authoritative tone actually does result in a quicker mean response than the positive tone. And that makes perfect sense—the direct hand signal could serve to emphasize the tone of authority, and vice versa.
But wait. Before I rush to publish my results in the Journal of Applied Canine Psychology, the world's leading paw-reviewed journal, there’s something else that needs to be considered.
Warning # 2: Beware of interpreting a significant interaction in a fractional factorial experiment without considering the aliasing structure.
Assessing Confounded Terms
Remember this experiment was a 1/8 fraction Resolution IV design. To reduce the number of required runs from 120 to 16, several interactions were confounded.
The alias structure of the original design indicated that the significant AD interaction (Voice*Hand signal) was confounded with two other 2-way interactions: EF and CG.
Let’s take a look at those confounded 2-way interactions. This is where you interpret your statistical results with a large dollop of good old-fashioned common sense.
Could an Empty Stomach Affect the Dog's Reaction to the Floor Surface?
EF is the 2-way interaction between the Meal factor (which indicated whether the command was given before or after the dog’s daily meal) and the Floor factor (which indicated whether the floor surface was carpet or wood).
How would hunger influence my dog’s response to the softness of the floor surface?
That interaction just doesn’t make any sense to me—so I decided to nix it as a possibility. (If I’d decided EF was a valid interaction, I’d need to keep factors E and F in the final model).
Could the Reward Given Affect the Dog's Reaction to the Procedure Used?
The other 2-way interaction confounded with AD was CG, the interaction between the Procedure and the Reward. Here’s what the interaction of those two factors looks like:
This interaction seems more plausible.
After the dog sits first and gets a dry biscuit, she may not be as motivated to lay down afterwards as she is when she gets the fresh ham after sitting first.
“A crummy old dry biscuit? Why, that’s hardly worth the extra effort!” she might think.
But when a reward of fresh ham is used, both procedures have a similar effect.
So both interactions AD and CE are plausible—and the final model accommodates both of them.
If I wanted to delineate the effects of these two interactions more clearly, I could perform a follow-up experiment with the 4 factors in the reduced model, using a design that doesn't confound AD and CE.
The Last Step: Checking Assumptions
In the first post of this 3-part series, I mentioned that my doggy DOE screening experiment was meant to be an illustrative example, rather than a bona fide DOE application.
Why? Because in a real DOE experiment, an important assumption is that each run in your experiment is independent. Because I measured the response of the same dog in each run, the runs in my experiment are dependent.
To make the runs independent, I'd need to randomly sample from a population of dogs, and then perform each run on a different dog. (You'd use a similar technique to conduct a DOE experiment to evaluate customer response, by randomly sampling customers).
That being the case, I fully expected see a run order effect in my experiment, as my dog began to respond more quickly to the Lay Down command with each run. But that didn't seem to happen:
Which just goes to show that you can't teach an old, lazy dog new tricks. At least, not without a fridge full of fresh, honey-cured ham.