dcsimg
 

Who Is the Better "Mocker", Mel Kiper or Todd McShay?

It’s a common occurrence in any quality improvement situation: You have 2 (or more) suppliers that offer you the same product. How do you decide which one to choose? You could just flip a coin. But that wouldn’t be very sensible, would it? No, instead it’s probably best to do some data analysis to help you make your decision.

So what does this have to do with the NFL draft? Well, everybody and their brother are drawing up mock drafts right now. There are different suppliers offering me the same product! How do I know which one to use when I fill out my NFL Mock Draft Office Pool (that’s a thing, right)? Well, that’s where Minitab comes in. I’m going to look back at some mock drafts from previous years and see just how accurate they were.

For simplicity’s sake, I’m going to stick with mock drafts from 2 different people, Mel Kiper and Todd McShay. When it comes to the NFL draft, those are two of the biggest names in the business, so let’s see which one is better!

For the last 3 years, I collected every first-round pick and the spot they were selected at. After that, I collected the spots at which Kiper and McShay predicted they would be selected. Then I took the absolute value of the difference between where the player was picked and where they were predicted. If you want to follow along in Minitab yourself, you can get my data here.

Now it’s time to compare the results and see who was closer! We'll use Minitab's 2-Sample t-Test to do it. The results appear below:

Neither was closer! The difference between the two is a mere 0.29 places. And the high p-value of 0.797 drives this point home further. Because it’s higher than our significance level of 0.05, we can’t conclude that one person’s mock draft is better than the other!

But we can go a little further. Of the 96 picks, the two disagreed on 58 of them. Maybe Kiper is a lot closer with picks 1-16, while McShay is closer with picks 17-32. Let’s use a Time Series Plot to see the trend of each person as the picks progress in the 1st round of the draft.

Well those look exactly the same! We see that Kiper has predicted the first 6 picks perfectly each of the last 3 years. McShay isn’t too far behind, missing only the 5th pick. After that, their predictions get much less accurate, but their plots don’t separate much. Both of the lines seem to go up and down together, giving more evidence to the fact that there is no difference between the two mock drafts.

So how do we decide now which one to pick? Well, first of all you should examine the mean of both writers. On average, Kiper and McShay (our "suppliers" of draft predictions) are off on their picks by about 6 spots. Now, is this level of accuracy good enough for your office pool...er, I mean, quality improvement process? That depends on your requirements. If neither meets your minimum requirements, maybe you should shop around and find another supplier.

You could also look at cost. If there is no difference in the quality, go with the less expensive one. In our situation, both McShay and Kiper require an insider subscription to ESPN, so the cost is exactly the same.

Hmmmm.  Well, maybe the best thing to do here is to flip a coin.

Photograph by Zennie Abraham. Images licensed under Creative Commons Attribution-NoDervis 3.0.

Comments

Name: Tim • Wednesday, April 25, 2012

I'm curious why you went with a two-sample t test instead of a paired t test here? Aren't these 96 observations paired by pick? Though it doesn't make a difference in your conclusion (p=0.687). Very interesting analysis and a fun read!


Name: Kevin • Thursday, April 26, 2012

You're correct, I should have done a paired t test. The reason I avoided it is because if Kiper's picks alternate between being over by 1 and under by 1 (1, -1, 1, -1....) and McShay's alternate by 10 (10, -10, 10, -10), then the paired t test will say there is no difference. But really, Kiper was much closer. However, because I took the absolute value of the difference for each person, that scenario wasn't even possible! So yes, a paired t test would have been appropriate.


blog comments powered by Disqus