Scouting Through Stats: Using QBOPS to assess the 2020 NFL Draft’s QB class

Many, many years ago, Football Outsiders, the grand daddies of football analytics, did a study on the stats that best predict success for quarterbacks about to enter the NFL draft. They settled one three factors:

1. Completion percentage
2. Games played
3. Being drafted in the first two rounds

That last one may sound like kind of a cheat, but it’s really just a clever way to incorporate scouting, as most QBs taken outside of even the first round are mediocre at best. “Games played” is mostly based on selection bias, as better QBs will start earlier in their college careers (and also not be benched). The real answer from Outsiders, oh so long ago, was simply completion percentage among those players who we believe are good. Given the data available at the time, it’s not a bad way to go.

College football changed dramatically shortly after they published this, with QB-friendly, simple-read offenses taking over. It quickly became common among the bigger programs running sophisticated offenses to have receivers routinely running wide open. Mediocre QBs were suddenly generating lofty completion percentages based on gameplans, yards after the catch, and those wide, wide college hashes that make half the field so much more difficult to defend.

The Outsiders’ formula essentially stopped working, but the insight was still a good one. Completion percentage is, fundamentally, the most important “stat,” it just needed some tweaking to account for all of the things quarterbacks don’t control. In February of 2019 Josh Hermsmeyer of FiveThirtyEight.com wrote a piece that provided that tweaking. With the popularization of EPA-based statistics and greater tracking data, newer and better stats became available, and one of those is “Completion Percentage Over Expected,” or CPOE.

CPOE takes normal completion percentage but adjusts it for quality of defense and, most importantly, how far down the field QBs tend to throw. We would “expect” a checkdown artist to have a very high completion percentage and “expect” an aggressive gunslinger to be lower. CPOE rewards the QB for completing a higher percentage of passes than expected in that context. CPOE, as it turns out, is also highly predictive of NFL success, though there is a downside: CPOE is not publicly available for college players.

Just because CPOE isn’t available doesn’t mean we have to stay in the dark. We have QBOPS. Just as OPS isn’t as good as something like WRC+ in baseball, QBOPS isn’t as good as CPOE. That said, OPS is still useful, and most importantly, freely available. It doesn’t adjust for quality of opponent, and it doesn’t account for things like YAC. QBOPS isn’t going to be the be-all, end-all just as CPOE isn’t, but it can lead us in the right direction, and help us find the occasional diamond in the rough.

What is QBOPS, and why is it good for this?

CPOE is trying to tell us how much better (or worse) a QB is given how far (or short) down the field he generally attacks. While CPOE does so with more precision, QBOPS is attempting the same thing. QBOPS began as a bit of a joke to compare football players and baseball players, but despite that fact, the weighting works surprisingly well. I also built a version to adjust for those players who throw too many picks (QBOPS+), and we’ll take a look at that too, but when examining college players there are a few things to keep in mind.

1. QBOBP (completion percentage) is still the most important stat. We should look at this first, and then use QBSLG (Y/C) to inform our judgement.
2. High QBSLG QBs with low QBOBP are generally terrible, even if their standalone QBSLG is fantastic.
3. That said, a very high QBSLG is still worth looking at.

The 2012 Draft Class

CPOE did a nice job of identifying Russell Wilson and Kyler Murray as potentially good NFL quarterbacks, and because so many people missed on Wilson, and because the 2012 draft class is just a fun group of guys all around, it’s a nice place to test things out.

I took a look at 126 QBs who had at least 150 pass attempts in the 2011 NCAA season. I then made a sub-list of only those quarterbacks who had a QBOPS of at least 1.000, with a minimum .400 QBOBP and a minimum .600 QBSLG. Here’s the list:

That’s not a bad list. Out of 126 guys it picked out not just Wilson, Luck and RG3 (who was sensational until suffering an injury), but Foles and Keenum as well. Foles was a relatively late pick (a 3rd-rounder) who has succeeded beyond all expectations, especially in the playoffs. The undrafted Keenum once led the NFL in DVOA, and is at least an average signal caller. Even Brandon Weeden and Matt Barkley managed to stick around the league for awhile, even if they are busts. Moreover, there were non-statistical reasons to be skeptical of Weeden, who entered the league as a 29-year-old rookie, and Barkley, who appears low on the list with some other non-prospects.

You may be curious about Terrance Owens. He had a sensational career as a dual-threat quarterback for Toledo, and ended up as a late-round pick of the San Diego Padres, but his tape wasn’t up to snuff as far as the NFL was concerned.

If we just look at the top 5 in QBOPS overall, we see a few more familiar names:

Ryan Lindley led the NCAA in Y/C that season, but his lackluster completion percentage should have raised plenty of red flags. He made the NFL but never amounted to anything. Mike Glennon, he of the long neck, had a decent enough career as an occasional starter and frequent clipboard holder.

And then we have Teddy Bridgewater, who posted the 5th-best QBOPS of 2011. While .381 falls short of our .400 cutoff, it’s in the ballpark, and his .750 QBSLG is very good. By the way, Keith Price and Seth Doege had cups of coffee in the league as well, with Doege landing in the CFL for a bit.

QBOPS, as previously stated, is far from perfect, but if nothing else, it gives us a few good pools to choose from.

Recent QBs

The 2011 class was extremely good, but we’ve had a few stinkers in recent years. I put together QBOPS for all 531 NCAA quarterbacks with over 150 pass attempts in a season since 2016. QBs can and do appear on this list more than once as it is organized by season. Here’s the .400/.600 club for the last 4 years:

The hype on Joe Burrow appears to be completely justified. His .450 QBOBP is unprecedented, and he was still able to consistently attack down the field. There are no can’t miss prospects in football, but there are no weaknesses with Burrow.

Utah’s Tyler Huntley is considered a non-prospect by most, but he was a dangerous and surprisingly accurate dual-threat passer in college. Huntley’s line was frequently overmatched and in many cases he really was the offense. It’s easy to see why scouts may not like him as his footwork is a mess of the highest order, and he occasionally throws a wobbly ball, but he is also accurate from the pocket throwing conventional passes, and extremely dangerous out on the edge.

Kedon Slovis just completed his freshman season with USC, and is worth keeping an eye on. That’s very impressive.

Tua’s mix of plus-.420 QBOBP and amazing QBSLG would have him on par with Burrow if not for the injury to his hip. If his medicals check out, someone will be getting a steal. I continue to hope he falls to Green Bay.

Everyone knows Baker Mayfield, but it’s worth noting that QBOPS picked out Gardner Minshew as well. The 6th-rounder played much of the 2019 season as Jacksonville’s starter, and occasionally played well, inspiring a mustachioed, “Uncle Rico”-based fandom.

Dwayne Haskins hasn’t worked out yet, but he also plays for the worst organization in football. Kyler Murray made the cutoff as well, as he should. Finally, keep an eye on Jalen Hurts, who outperformed Tua in many ways, though he threw too many picks.

What about the 2020 prospects it doesn’t like?

Numbers in parentheses are in order of (QBOBP/QBSLG/QBOPS/QBOPS+)

Justin Herbert - Oregon (.394/.595/.989/.919)

Oregon’s 4-year starter has flirted with the .400/.600 club a few times but never quite cracked it. Excluding him based on last season’s .394/.595 split may seem a bit harsh, but if you’re not well over .400, that under .600 number starts to look like the red flag of a guy who checks down too much. Herbert is a giant Brock Osweiler-esque physical talent, and I personally like him about as well.

Jordan Love - Utah State (.365/.569/.934/.755)

In 2018, Love ranked 2nd overall in QBOPS, but that was built on an absurd .773 QBSLG. His 64% completion percentage in his best season is a huge red flag, as is his .755 QBOPS+ in 2019, where he threw 17 picks. Love can move and he has an extraordinary arm. I hate when those guys don’t produce in college, especially in a lesser conference.

Jacob Eason - Washington (.379/.590/.969/.870)

Another in the Brock Osweiler vein, the big, strong-armed guys who can sling it downfield should probably be hitting more big plays.

Young QBs to keep an eye on…

Of the 136 qualifying quarterbacks in 2019, Arizona State’s Jayden Daniels stands out as having the smallest difference between his QBOPs and QBOPS+, meaning, basically, he never throws interceptions. This is remarkable partially because Daniels did so while still attacking downfield (with a QBSLG of .703), and partially because he is the first true freshman to ever start for Arizona State.

USC’s Kedon Slovis was just as impressive as a freshman. Slovis’ was also aggresively recruited by Arizona State, but ultimately opted for the Trojans. Also of note, his high school QB coach was Kurt Warner, and it shows. They should have a fun PAC-12 rivalry for several seasons.

---

Up Next: The Receivers and WROPS