Packers-49ers Stat of the Week: Jerome Boger’s awfulness can now be quantified

Let’s talk about officials for a moment, because to some extent, people who complain about officials are often labeled as cranks, conspiracy theorists, or sore losers. You’ll often here analysts say something to the effect of “had they played better, they would not have left themselves vulnerable to officiating mistakes.”

Over in baseball, a funny thing happened recently. Because of our ability to quantify ball and strike calls, we now know which umpires are good, which are terrible, and how much they impact a game:

What used to be the purview of cranks has been conquered by nerds.

Football is no different. Because of Expected Points Added (EPA) and Win Percentage Added (WPA) models, we can quantify the impact of penalties and just how much influence officials have over games. It’s not quite as simple as baseball, as often the lack of a penalty is just as important as a penalty itself, and of course, many penalties are absolutely justified. But what we can do now is get a good picture of just how much an official influenced a game, and whether one team was favored.

In Sunday night’s thriller between the Green Bay Packers and San Francisco 49ers, Jerome Boger’s crew of officials made several calls that made for huge swings in the momentum of the game.

To examine how much they affected the game, I’m going to use WPA, though you could do the same exercise with EPA. I like WPA because it captures certain clutch elements a bit better, and in this game, penalties were huge, often resulting in first down conversions or big yardage. Stats are courtesy of Ben Baldwin’s RBSDM.com, which puts a nice user interface over the game’s play by play data, and also lets you download all of the underlying fields as a .csv. Please note there are two versions of WPA available in the underlying data. One is strictly model-based, and one is adjusted for Vegas odds. I used the model here.

I’ve removed false start and delay of game penalties because they are small potatoes in terms of WPA, and they’re also almost always objectively correct. Officials rarely miss on the 5-yard penalties.

The Results

Jerome Boger and his crew butchered this game, but on the most impactful penalty call they were (mostly) correct. Adrian Amos was flagged for tripping on 2nd and 11 with 5:32 remaining in the game, and the call granted San Francisco an additional 10 percentage points of WPA, which actually understates the case, as WPA is calculated from the starting position, not the alternative outcome. Jimmy Garoppolo was sacked on the play, which would have set up 3rd and 21 and put the 49ers behind the eight ball, but in the grand scheme of things, there are two good reasons to ignore this play.

First, Adrian Amos did in fact trip Garoppolo, doing so very obviously and in front of an official. You can argue that he was held on the play, which should have resulted in offsetting penalties, but you simply can’t count on holding being called in the NFL.

The second reason is that Jimmy Garoppolo did his hilarious backward pass fumble thing two plays later, and this call didn’t ultimately hurt the Packers at all.

The next-largest referee decision was on the lack of a call on Jimmie Ward for his dirty hit on Davante Adams. This one is more difficult to quantify, but picking up an additional 15 yards and a first down would have moved the Packers out to their own 40 with about 7 minutes left and a three point lead. The Packers lost about 2% of WPA on the play itself, and would have gained something like 5-6% had a penalty been called. More importantly, they lost Adams for their second down play, an incompletion to MVS, and the offense seemed out of sorts until their next drive. Had the penalty been correctly called, the Packers would have been about 80% to win it. After the play, they were at roughly 72%.

This was hardly the only big, questionable call that went against Green Bay. The controversial roughing the passer call on Jaire Alexander with 6:39 remaining in the 4th also robbed Green Bay of at least 8 points of WPA. The result of the play would have left San Francisco with a 3rd and 4.

Eric Stokes was also hit with a pair of big, questionable PI penalties, and I’m surprised his 35-yard DPI call in coverage against Brandon Aiyuk with 11:54 remaining in the game isn’t higher on this list. At the time the Packers led by 10 points, it was 3rd and 14, and the 49ers chances of winning had fallen all the way to 11%. Without this PI call, I suspect the 49ers are essentially dead, and it’s fair to say that on top of the boost in WPA from 11% to 18%, it kept the 49ers in the game. Stokes’ other DPI call, a 5-yarder that led directly to the 49ers’ end-of-half touchdown, cost the Packers another 5 points of WPA.

Conclusion

If you’re getting the impression that the Packers faced an uphill battle due to the officials, you would be correct. It’s not as if the Packers didn’t benefit from any calls, as they did have three significant PI calls totaling 19 points of WPA go their way, but that was kind of it. In the aggregate, the 49ers picked up about 25 points of WPA based on officiating, and even if you exclude the tripping call on Amos, and the “hands to the face” penalty on Yosh Nijman, also called correctly, they still had a significant 12-point advantage.

Football officials have unusual sway over their sport, and given how many penalties against the defense grant a new set of downs, those penalties often serve as an “anti-turnover.” These tend to be forgotten by most after the game, but a defensive penalty on 3rd down is almost as impactful as a turnover. A deep PI penalty is as impactful as a bomb. A holding penalty is often more impactful than a sack. Officials hand these out routinely, drastically changing the course of games, and we should recognize this as a basic fact of football and be more willing to blame the striped shirts when things get out of hand.