Many Six Sigma and quality improvement tools could be applied in other areas. For example, I wonder whether my son's teachers could benefit from a little attribute agreement analysis.
He seemed frustrated the other day when I picked him up at school. He'd been working on a presentation that needed to be approved by his teachers. (My son attends a charter school, and each class is taught by a two-person teaching team.)
"What's wrong?" I asked when he clambered into the car with a big sigh.
My son explained that he'd given the presentation to teacher Jennifer that morning. A few minor suggestions aside, she thought it was fine. Jennifer told him the presentation was ready to deliver.
But when he gave the presentation to Jeff in the afternoon, the feedback was very different. Jeff felt the content of the presentation was too vague, and that my son needed to do more research and add more information. Jeff told him the presentation wasn't acceptable.
And because Jennifer had already left for the day, there wasn't a chance to reconcile these very different opinions.
No wonder my son felt frustrated.
The Challenge of Judging Attributes Consistently
We all need to make judgments every day. Some are fairly inconsequential, such as whether you think a given song on the radio is good or bad. But judgments we make at work can have profound impacts on customers, coworkers, clients, employees...or students.
Inspectors classify parts as good, or bad. Employment screeners select applicants they think are worth interviewing. And instructors decide whether a student's work is acceptable. In each case, judgments are made about one or more attributes, which can't easily be measured objectively.
That's where the problems start. One synonym for "judgment" is "opinion," and peoples' opinions don't always match. That's not always a problem: If I like a song and you don't, it's not a big deal. But when two or more people have contradictory assessments of critical things, disagreement can cause real problems. The quality of a business' parts or service can vary from day to day, or even from shift to shift. Customers' experiences can be very inconsistent from one day to the next.
As if different judgments from different people aren't problematic enough, we also have a great capacity for disagreeing with ourselves. And in many cases we're inconsistent without even recognizing it: if you're inspecting parts that all look the same, are you sure you'd judge the same part the same way every time? And can you be sure you're inspecting parts consistently with your fellow inspectors?
Or, in the case of my son's teachers, how can you be sure your assessment of a student's work is consistent with your own judgments, and with those of your fellow instructors?
Benefits of Attribute Agreement Analysis
These situations can be illuminated by Attribute Agreement Analysis. Attributes are difficult to measure -- that's why we rely on judgments instead of objective measurements to assess them -- but we can collect data that reveals whether different people assign attributes to an item consistently, and whether an individual makes the same judgment when assessing the same item at different times.
Attribute Agreement Analysis can tell you whether and where you're getting it wrong. Knowing this helps everyone in the process to make better and more consistent judgments.
The results of an Attribute Agreement Analysis may indicate that your team judges attributes very consistently, and that you can be confident in how you're evaluating items. Alternatively, you may find that one or two team members make very different judgments than others, or that you don't always rate the same item the same way.
Identifying those issues gives you the opportunity to make improvements, through training, developing clearer standards, or other actions.
If my son's teachers did an Attribute Agreement Analysis, they might find they're not on the same page about what makes a good presentation. If they knew that was the case, they could then develop clearer and more consistent standards so they could more fairly assess their students' work.
How to Do an Attribute Agreement Analysis
There are two main steps in an Attribute Agreement Analysis:
- Set up your experiment and collect the data
- Analyze the data and interpret the results
You can use the Assistant in Minitab Statistical Software to do both. If you're not already using it, you can try Minitab free for 30 days.
The Assistant gives you an easy-to-follow Attribute Agreement Analysis worksheet creation tool and even lets you print out data collection forms for each participant and each trial:
Collect your data, then use the Assistant to analyze it and give you clear interpretations of what your results mean.
See a step-by-step breakdown of how it's done in this QuickStart exercise for Minitab, in which a family uses Attribute Agreement Analysis to discover the source of their disagreements about dinner. The example includes instructions, a quick video summary, and a downloadable data set so you can try the analysis yourself.
Where could you use Attribute Agreement Analysis in your work or personal life?