Does the NFL Preseason Matter? Regression Analysis Says "No."

Admit it—if you follow NFL football, both of the following statements are likely true:

1. When talking about the preseason with friends, you say that the preseason doesn't matter and doesn't mean anything for the regular season, so you're not really worried or excited about your team's performance.
    
2. You are worried or excited about your team's performance.

All of us say it doesn't matter, but after so many months without football we are desperate for anything meaningful. We want to see how the recent draftees perform, or how the defense is doing under a new coordinator, or a hundred other things. When our team gets blown out of the water by another team, we panic a little.  And when the Jaguars go 4-0 in the preseason, I and the other four Jaguars fans feel a little bit of hope.

So is it true that the preseason doesn't matter?

Regression Analysis of Preseason vs. Regular Season Winning Percentage

I pulled data for the past three seasons to investigate.  First I looked at the preseason winning percentage versus the regular season winning percentage:

Well, that bit of hope for the Jaguars has quickly dissipated.  That P-value of 0.571 indicates that a team's winning percentage in the preseason shows absolutely no evidence of predicting the regular season.

Regression Analysis of Score Differential for Preseason and Regular Season

But maybe there's hope to learn something from the preseason still...how about using the score differential instead of winning percentage?

Statistical significance!  The P-value in this analysis indicates we have some.  Practical significance, however, may be another story.  The R-squared value shows we've only explained 6.4% of the variation, so while there may be some relationship we shouldn't get too excited or distressed about score differential in the preseason.

Regression Analysis of "Points Against" Measure of Defense

So next I thought I'd break it up between defense and offense—first up, we'll look at "points against" as a measure of the defense:

Not much going on there, although we do have a high-leverage point.  The 2009 Colts—who had a 14-2 regular season and made it to the Super Bowl—gave up 160 points in just four preseason games!  But once the regular season came around, they were only slightly below average defensively.  Concerned about the high leverage of this point, I removed it—but that made no significant improvement to the model.

Regression Analysis of "Points For" Measure of the Offense

So finally, we look at "points for" as a measure of the offense:

As with the score differential, we have statistical significance but not much in terms of practical significance (which is not surprising since "points for" makes up half of that score differential number). So while a high-scoring offense in the preseason means you are likely to score more points in the regular season, the relationship just doesn't predict very accurately.

So back to our statements, which I know are true of you whether you admit it or not:

1. When talking about the preseason with friends, you say that the preseason doesn't matter and doesn't mean anything for the regular season so you're not really worried or excited about your team's performance.
    
2. You are worried or excited about your team's performance.

I hope that after reading this, you are convinced that #1 is the right way to feel about the preseason, and that you can say it with more confidence than ever—and maybe even cite some statistics to back up your case.

But I'm guessing #2 is as true as ever.