Super Bowls, the Packers, and Aaron Jones all illustrate DVOA’s usefulness

Here at Acme Packing Company, we have been discussing DVOA and what it is even mre than usual since “The Demovsky Dustup” last Friday. As one of the longest-tenured APC writers to make frequent use of DVOA, I wanted to weigh in on why it’s a good and valuable stat.

And also try to get you to buy one of our cool shirts, with all proceeds going to charity in support of STEM education.

The Weakness of Old Stats

All stats require context to accurately convey the intended information. What’s interesting about DVOA (and most “advanced stats”) is that built-in, additional context is precisely what makes them “advanced.”

Let’s start with what the NFL refers to as “Total Defense.” I’ve been annoyed by this forever because “Total Defense” is really just “yards allowed,” and yards allowed is a pretty terrible defensive metric. It’s entirely possible for very good defenses to allow a ton of yards for a variety of reasons, from having a higher percentage of “garbage time” per game than other defenses to simply playing on the same team as a fast-paced offense, which in turn provides both teams with more possessions. If you use “Total Defense” without understanding pace of play or game situation, you will frequently get the wrong impression.

I’m particularly annoyed by Total Defense because the name that the NFL has bestowed upon it lends additional gravitas for no good reason, but you could make the same criticism of basically any raw counting stat, from touchdown passes to passing yards to interceptions, and even to sacks.

The analytics community generally starts with the basics and asks how they can be better. Take “sacks.” Sacks are fine, but in terms of predicting who will be a better pass-rusher going forward, hurries are better, and so we started focusing on hurries. DVOA was created because just looking at yards to judge a team isn’t going to tell you the entire truth. Aaron Schatz of Football Outsiders thought he could do better, and he did.

The primary reason DVOA a good metric is in how it defines success. Not all yards are created equal, and getting 15 of them on 3rd and 25 isn’t as valuable as getting 2 of them on 3rd and 1. This is DVOA’s first bit of additional context: the simple realization that in football, what constitutes a good play result is entirely context-dependent.

The second important contextual element is that D (“Defense-Adjusted”). Who you play in football matters more than in any other sport. Given that there are only 16 games, a tough schedule can make even very good teams seem mediocre. Ultimately winning games is the most important thing a team can do, and you have to play the schedule assigned to you, but if you want to know who is a pretender and who isn’t, you have to understand the schedule. The “Defense-adjusted” part of DVOA really sets it apart for this reason. (Note: Football Outsiders doesn’t actually make this adjustment until week 4 games are concluded, because prior to that, there is not enough information about each team.)

Last but not least, DVOA uses the success and failure of every play, which means that it accumulates sample size very quickly, especially at a team level. All statistics are bad over small sample sizes, and football is just one big small sample size due to its compact nature, but DVOA will become relevant more quickly than just about anything else. Indeed, one of the big advantages of almost all advanced football analytics (like DVOA and those based on EPA, “Expected Points Added”) is the per-play aspect.

DVOA and Green Bay

I find that there are two types of people who dismiss advanced statistics out of hand. One is “Old Men” who have been involved in the sport forever, and think they know everything there is to know. The other is a fan who has, at some point, been personally offended by some advanced metric. Here, the analytics community has failed the Packers, as many of their chief ambassadors have made a cottage industry of telling us that Aaron Rodgers isn’t as good as he once was, using a tone that isn’t going to win any friends.

(Note: Aaron Schatz, who created DVOA, is an excellent ambassador and generally doesn’t stoop to any of this.)

Much of that criticism was warranted, by the way. It’s completely true that Rodgers’ production recently hasn’t matched what he did from 2010-2014, and you don’t need any advanced metrics to tell you this, but messaging matters too. DVOA is agnostic as to how it feels about teams, and in reality, it’s treated Green Bay quite well over time because Green Bay has been excellent for most of the DVOA era. DVOA is also accurate, and has plenty to offer in terms of insight, if you know where to look. For instance…

Super Bowl Winners

The best team doesn’t always win the Super Bowl, but DVOA does a great job of telling us who the contenders are. DVOA goes back to 1985, and since then, 12 of the 35 Super Bowl champions led the league in DVOA.

Considering that (for most of that time) 12 total teams made the playoffs per season, being right 35% of the time is pretty impressive. Specifically regarding Green Bay, in 2010 the Packers were a 10-6 wild card team that finished with a worse record than Chicago, Atlanta, and New Orleans. Even though they squeaked in by the skin of their teeth, DVOA saw them for what they truly were: as the best team in the NFC, ranking only behind the Steelers, who they defeated in the Super Bowl, and New England.

The 1996 Super Bowl team was unsurprisingly #1 in DVOA, a country mile ahead of the second place 49ers. That team also gets its due as a defensive juggernaut and a dominant team all around, ranking 3rd on offense, 1st on defense, and 2nd on special teams. Indeed, according to DVOA, the 1996 Packers are secretly one of the best teams ever to win a Super Bowl with a 45.8% mark.

The only Super Bowl winners outside of the top 10 DVOA teams are all of the weird one-offs you would expect, from Eli’s two wins, to Flacco getting hot for a month. The stat does an excellent job of telling us who the best teams actually are.

Hidden Gems

You all know who Aaron Jones is now, but I was into Aaron Jones before it was cool. That’s because way back in 2017 Jones had a 31.3% DVOA, second only to fellow rookie Alvin Kamara. Jones essentially repeated the feat in 2018 by ranking 7th, at which point I was convinced he was a stud. One of the great things about DVOA is its ability to pick out greatness in guys who aren’t necessarily getting the most opportunities, and here it did exactly that.

It was similarly high on Allen Lazard last season as a high-efficiency prospect just waiting to bust out. Lazard led all Packer receivers in DVOA last season, finishing an impressive 14th, and he’s continued to produce at a high level, currently ranking 2nd overall in 2020.

Lazard’s blend of high catch rate and big play ability make him a DVOA superstar, and I expect he’ll continue to be one. This also highlights one of DVOA’s weaknesses, as it tends to be low on Davante Adams, but even in being wrong it tells you something important. Adams tends to run a high percentage of inefficient routes (the flat/bubble screen types) which drags him down. He is also Rodgers’ go-to in desperate situations, which increases the level of difficulty. Schatz has mentioned this as an issue on teams with a clear #1 who take on a disproportionate share of targets, and Adams is often better served by DVOA’s cousin, DYAR, which gives credit for opportunities. By DYAR he currently ranks 9th.

Specific Strengths and Weaknesses

DVOA can also be applied to several more granular pieces of information. The Packers face New Orleans next Sunday night, and if you want to know how they’ll match up, you can dig pretty deep. The Saints are better at defending the run, where they rank 9th, than against the pass, where they are a middling 15th and in fact rank worse than the Packers’ defense. Josh Jacobs of the Raiders had a tough time against the Saints on Monday, gaining just 88 yards on a whopping 27 carries, and DVOA reflects that in the Saints’ ranking. Aaron Jones and company may find it harder going than people think. But on pass defense, it’s a whole different story.

Marshon Lattimore is good, and the Saints have the #13 DVOA against #1 receivers so far, but on the opposite side, Janoris Jenkins has had issues. Jenkins is a gambler and has a pick-6 to his credit, but he’s also prone to getting burned and the Saints are dead last against #2 outside receivers as a result. In fact, outside of Lattimore they don’t really excel anywhere on pass defense, ranking 20th or lower against all receiver types. They’re particularly bad against pass-catching running backs, where they rank 26th. Jones may be bottled up on the ground, but expect the Packer backs to continue to do damage through the air.

And that’s where DVOA can be extra helpful. If something doesn’t make sense at a high level, you can usually drill down a bit and find the explanation. Many advanced stats are specialized for one position or purpose. For instance, CPOE (Completion Percentage Over Expected) is really only useful for quarterbacks. Because DVOA is built of a framework defining success or failure per play, it can be applied to a wide range of positions, position groups, or entire teams, which means if you care to find out, you can always tell why it says what it says.

DVOA has been around for a long time now, and despite a few tweaks, the general reasoning behind it is largely unchanged. It has an outstanding track record, and you can tell a lot about your team from it, if you care to learn. It’s respected in the industry, and if you don’t have at least a passing familiarity with it while continuing to write about football, you are doing your readers a great disservice.