This summer, I created a model to determine the correct 4^{th} down decision. But whether it’s for business or some personal interest, creating a model is just the starting point. The real benefits come from applying your model. And for the Big Ten 4^{th} down calculator, the time to apply the model is now!
On Saturday night, Penn State and Rutgers officially kicked off conference play for the 2015 Big Ten football season. So let’s see what the model thinks about each team’s 4^{th} down decisions.
One caveat before we begin. In hypothesis testing, it’s important to understand the difference between statistical and practical significance. A test that concludes there is a statistically significant result doesn’t imply that your result has practical consequences. You should use your specialized knowledge to determine whether the difference is practically significant.
The same line of thought should be applied to the 4^{th} down calculator. The decision the calculator gives isn’t meant to be written in stone. It simply gives a good estimate of what an average Big Ten team would do against another average Big Ten team. Coaches should also consider other factors, like the game situation and the strengths and weaknesses of their specific team. But the 4th down calculator will still give a very strong starting place for the decision making!
Okay, enough of the pregame show, let’s get to the game!
4^{th} Down Decisions in the First 3 Quarters
In the first 3 quarters of a football game, coaches should be making decisions that maximize their points (in the 4^{th} quarter they should be maximizing their win probability, but more on that in a bit). The following graph shows approximately when coaches should go for it and when they should kick.
Let’s start by seeing how Rutgers’s 4^{th} downs decisions lined up with the previous graph.
Rutgers 4^{th} down decisions in the first 3 quarters
Distance |
Yards to End Zone |
Model Decision |
Coaches Decision |
Agree or Disagree |
Expected Points Punt |
Expected Points Go for it |
Next Score |
7 |
95 |
Punt |
Punt |
Agree |
-3.3 |
-4.3 |
-7 |
5 |
37 |
Go for It |
Punt |
Disagree |
0.1 |
0.4 |
-7 |
8 |
73 |
Punt |
Punt |
Agree |
-1.6 |
-2.8 |
-7 |
8 |
55 |
Punt |
Punt |
Agree |
-0.3 |
-1.4 |
-7 |
10 |
63 |
Punt |
Punt |
Agree |
-0.9 |
-2.2 |
-7 |
6 |
81 |
Punt |
Punt |
Agree |
-2.2 |
-3.1 |
3 |
6 |
55 |
Punt |
Punt |
Agree |
-0.3 |
-1.2 |
3 |
Rutgers’ 4^{th} down decisions show how ineffective their offense was against the Penn State defense. They never had a 4^{th} down distance less than 5 yards. The result was a lot of punting, which was the correct decision most of the time. Keyword, most. On their 2^{nd} drive, Rutgers punted on 4^{th} and 5 from the Penn State 37 yard line. The statistics say that on average, they would have scored 0.3 more points by going for it. But remember our analogy about practical and statistical significance? When you factor in more information, I think the decision to go for it is even stronger.
The 4^{th} down calculator assumes a net punt of 40 yards when calculating the expected values. But once a team is in their opponent’s territory, it assumes the punt is downed at the 10 yard line. But what if the punt goes into the end zone for a touchback? If the punt results in a touchback (which is exactly what happened in the game), Rutgers’s expected points on a punt decrease to -0.7. Now instead of a difference of 0.3 points, it’s over a full point! As a 10-point underdog, that’s exactly the type of decision Rutgers couldn’t afford to make. And sure enough, on the very next drive, Penn State went 80 yards and scored the first touchdown of the game. The extra 17 yards Rutgers gained didn't even matter.
Now let’s look at Penn State's 4^{th} downs.
Penn State’s 4^{th} down decisions in the first 3 quarters
Distance |
Yards to End Zone |
Model Decision |
Coaches Decision |
Agree or Disagree |
Expected Points Punt |
Expected Points Go for it or FG |
Next Score |
11 |
41 |
Punt |
Punt |
Agree |
0.7 |
0.5 |
7 |
10 |
35 |
Field Goal |
Punt |
Disagree |
0.8 |
0.5 |
7 |
6 |
63 |
Punt |
Punt |
Agree |
-0.3 |
-1.1 |
7 |
8 |
60 |
Punt |
Punt |
Agree |
0 |
-0.9 |
7 |
1 |
41 |
Go for it |
Go for it |
Agree |
0.7 |
2 |
-3 |
1 |
52 |
Go for it |
Punt |
Disagree |
0.6 |
1.1 |
-3 |
On Penn State’s 2^{nd} possession, they decided to punt from the Rutgers 35-yard line rather than attempt a 52-yard field goal. But remember that thing about statistical and practical significance? These long field goals are absolutely a case where a coach needs to use their knowledge about their specific kicker. To calculate the field goal probability, I used data from the previous 3 Big Ten seasons. The resulting model predicts that a kicker will make a 52-yard field goal about 48% of the time. But for the data I collected, a coach is only going to attempt a 50+ yard field goal with a kicker they feel confident has a chance of making it. For many college kickers, a 50+ yard field goal is out of their range. So the coaches should use their knowledge of their specific kicker.
For Penn State, they had kicker Joey Julius attempt both a 49 yard and 50 yard field goal the previous week (in the pouring rain too!). So I think it’s safe to assume that Coach Franklin thinks he has a strong enough leg. So we’re going to stick with the 4^{th} down calculator’s decision that kicking a field goal is what Penn State should have done.
Late in the game, Penn State had a pair of 4^{th} and 1 decisions to make. In college football, the conversion rate on 4^{th} and 1 is so high (68% is the probability the model uses) that the 4^{th} down calculator is going to say to go for it no matter where you are the field. And on the first 4^{th} and 1, Penn State correctly followed suit. However, they went with an empty backfield, clearly indicating to Rutgers that the play would be a quarterback sneak. Sure enough, Rutgers stopped quarterback Christian Hackenberg on the play. Rutgers then used all their “momentum” from the big stop to go 3 and out on offense and punt the ball back to Penn State.
On the next possession, Penn State decided to punt on 4^{th} and 1. The difference in expected points is half a point, but when you add the fact that running back Saquon Barkley averaged 9.3 yards per rush, the decision to go for it is even stronger. The Nittany Lions were already up 21 points, so the decision to punt didn’t have a large impact on the outcome of the game. But in the future, Penn State will certainly have another 4^{th} and 1 at midfield with a much closer score. Hopefully they go with their first 4^{th} and 1 decision, and not the latter.
4^{th} Down Decisions in the 4th Quarter
In the final quarter, coaches should be making decisions that maximize their win probability. To calculate win probability, I’m using this formula from Pro Football Reference. It uses the Vegas spread, the standard deviation of the Vegas spread (which is 15.53 for college football), the time remaining in the game, the current score, and the expected points the team with the ball currently has.
The Penn State Rutgers game only had one 4^{th} down decision in the 4^{th} quarter that we’ll discuss. Rutgers had a 4^{th} and 9 at the Penn State 11-yard line. To maximize their expected points, Rutgers should kick the field goal. But with only 10 minutes left they should be maximizing their win probability. Down by 21 with 10 and a half minutes left, here is their resulting win probability for going for it and kicking a field goal.
- Win Probability Go for it: 0.71%
- Win Probability Kick FG: 0.57%
Ok, so Rutgers wasn’t likely to win either way. Both decisions result in a win probability of less than 1%. But the win probability for going for it is higher. And down by 21, you need to score 3 times. By kicking a field goal, you go from needing 3 scores to...oh wait, you still need 3 scores.
If a player were to give up and walk off the field with 10 minutes left in a game, their head coach would be irate. The story would probably even make Sports Center later that night. But when it comes to the coaches themselves, apparently giving up is fine. Because by kicking a field goal down 21 with 10 minutes left, giving up is exactly what Rutgers did.
Summary
Each week, I’ll summarize the times coaches disagreed with the 4^{th} down calculator and the difference in expected points between the coach’s decision and the calculator’s decision. I’ll do this only for the 1^{st} 3 quarters since I’m tracking expected points and not win probability. I also want to track decisions made on 4^{th} and 1, and decisions made between midfield and the opponent’s 35 yard line. I’ll call this area the “Gray Zone.” These will be pretty sparse now, but will fill up as the season goes along! Then we can easily compare the actual outcomes of different decisions in similar situations.
Have anything else you think I should track? Let me know and I’ll add it!
Team Summary
Team |
Disagreements |
Difference in Expected Points |
Penn State |
2 |
0.8 |
Rutgers |
1 |
0.3 |
4^{th} and 1
Yards To End Zone |
Punts |
Average Next Score After Punt |
Go for It |
Average Next Score after Go for it |
Field Goals |
Average Next Score After FG |
75-90 |
0 |
0 |
0 |
0 |
* |
* |
50-74 |
1 |
-3 |
0 |
0 |
* |
* |
25-49 |
0 |
0 |
1 |
-3 |
0 |
0 |
1-24 |
0 |
0 |
0 |
0 |
0 |
0 |
The Gray Zone (4^{th} downs 35-50 yards to the end zone)
4^{th} Down Distance |
Punts |
Average Next Score After Punt |
Go for It |
Average Next Score after Go for it |
Field Goals |
Average Next Score After FG |
1 |
0 |
0 |
1 |
-3 |
0 |
0 |
2-5 |
1 |
-4 |
0 |
0 |
0 |
0 |
6-9 |
0 |
0 |
0 |
0 |
0 |
0 |
10+ |
2 |
7 |
0 |
0 |
0 |
0 |