Debunking the Continued Criticism of Aaron Rodgers' "Clutch-Ness"

As I've mentioned before, I've been a baseball stat nerd for a good long time and I tend to view other sports through that prism, and while not every sport is perfectly analogous to baseball, understanding the basic concepts of what kinds of information you need to know is always useful.

I have been a Baseball Prospectus subscriber for a very long time. Probably close to a decade at this point. I have a copy of Baseball Between the Numbers (copyright 2006) autographed by Nate Silver. Back in the old days when bad baseball writers were still legion, one of my favorite pastimes was ridiculing the use of truly bad baseball stats: batting average (not totally useless, but frequently misused), the save (which at least has the side benefit of having cost the St. Louis Cardinals a trip to the World Series this year), and quite possibly the worst stat of all, the pitcher win.

Assigning an entire win to an individual player in a team sport is, quite simply, ridiculous. It almost always provides an inferior picture of that player compared to simply focusing on things he controls. Pitchers, especially in the American League, have no control over their offenses and because of that they actually have very little control over whether they get "wins" or not. This was all low-hanging fruit among the baseball stat crowd many years ago and we've basically won the battle against pitcher wins. Even the Cy Young award, which had famously rewarded pitchers with the most "wins" for basically its entire existence, now routinely goes to a truly worthy recipient. Anyway, that's enough about baseball. The point is that whenever you assign an entire win or loss to a player, you should probably take a step back and reassess your entire analysis. It's a terrible stat for pitchers, and it's at least as bad of a stat for quarterbacks.

I was, therefore, quite surprised to see this "quarterback win" based analysis on, of all places, Nate Silver's own FiveThirtyEight.com. In this piece, Benjamin Morris attempts to measure Aaron Rodgers' clutch ability (which baseball stat people hate almost as much as the pitcher win) by analyzing his comeback wins. This is sort of like measuring the balance of your humors with a magic eight ball.

Morris is trying to make the case that Rodgers is too conservative when trailing big, and attempts to make the case that you can in fact throw too few interceptions. The reasoning is this: if you're down big, you need to take more chances than normal, which will result in more picks but also will occasionally result in a big comeback. Rodgers barely throws any interceptions (15 touchdowns versus one pick so far in 2014), but, as you read in the piece, he has had 21 opportunities to rally the Packers to a win from a deficit of 9 or more points in the 2nd half and has converted exactly zero of those into victories. There is nothing about any of that that is factually untrue, but there are a ton of issues with the methodology. Here is the first problem:

Morris used four games in his sample in which Aaron Rodgers did not start.

As part of his 21 game sample, Morris included the following four games:

Packers at Ravens, 12-19-2005

The Ravens defeated the Packers (who would finish 4-12) by a score of 48-3. The rookie version of Rodgers entered a blowout in relief of Brett Favre at the start of the 4th quarter, trailing 34-3. He was somehow unable to rally them.

Packers vs. Patriots, 11-19-2006

Favre was knocked out on a sack by Tully Banta-Cain and Tedy Bruschi with 1:51 remaining in the first half and the Packers trailing 21-0 to the 12-4 New England Patriots.

Packers at Eagles, 10-2-2006

With the Packers trailing 31-9, Rodgers entered the game with 4:23 remaining to play a few garbage time minutes in another blowout loss.

Packers at Cowboys, 11-29-2007

I have talked to people who have made the case that this game should be included. Favre was knocked out pretty early and Rodgers played the majority of the game. However, I think a lot of the desire to include this game comes from the fact that Rodgers actually played well. He was 18/26 for 201 yards and one touchdown, and he rallied Green Bay from a 27-10 deficit to within 27-24, generally keeping it within one score.

That said, I would argue it's not a good idea to include any game where he came in as a backup. He didn't take first team reps in practice leading up to the game, he wasn't the same Aaron Rodgers he is now, and before he even saw the field the team trailed by 17 points.

Including these four games was a weird decision, but what may be weirder is the fact that...

Morris did not include any playoff games.

I did my own count of Aaron's games in which the Packers trailed by 9+ points sometime in the second half and only came up with 20 instead of 21. The difference is that I used playoff games but not Rodgers' games when he was backing up Favre. There are 3 playoff games that fit this criteria: the January 15th, 2012 loss to the Giants (37-20), the January 12th, 2013 loss to the 49ers (45-31), and the amazing 51-45 loss to the Cardinals on January 10th, 2010. The Arizona game is a good example of why we should not blame wins on individuals. Rodgers could have done a bit more in this game as he did miss a wide open Greg Jennings at the start of overtime, but after trailing 31-10 in the third quarter, he also led the offense to 35 more points and a 45-45 tie at the end of regulation. If they play any kind of defense they win the game.

But in any case, 21 games (or 20, or 17) is a very small sample size.

I talked about this with Morris on Twitter on Thursday and I was actually kind of floored by this:

@BadgerNoonan Also, sample size 21 isn't huge, but it isn't tiny. Here are all QBs with 21 or more games: pic.twitter.com/UGmL4nYRDj

— Benjamin Morris (@skepticalsports) October 16, 2014

21 games is a miniscule sample size, especially if you are only looking at wins versus losses. 21 games worth of pass attempts (which would be around 600 or so) is a large sample size. 21 games of losses is just 21 games. Football is by its nature a game of small sample sizes, but that fact doesn't mean that football stats suddenly become reliable predictors of anything. A lot can go right or wrong over 21 games that has no relationship at all to the contributions of a single player.

Moreover, the sample for Rodgers, especially the sample used by Morris, is just weird. Using his numbers, the Packers under Aaron Rodgers have trailed by nine or more points only 6 times since the start of the 2010 season. That is almost four and a half seasons, which leads us to our conclusion:

Morris's conclusions about Rodgers are unfounded.

The vast majority of this sample (15 out of the 21 games listed) involved 25- or 26-year old Aaron Rodgers in his first two full years as a starter, plus the aforementioned 4 games in which he was not a starter. Most of these failures took place over 5 years ago! The fact is that the MVP-caliber Aaron Rodgers whom we all know and love simply doesn't allow his team to get into this situation very often, and when he does, it's frequently not his fault when the team fails to come back. The Packers have had the occasional great defenses, but they've just as frequently had a bottom-5 defense, and it's very difficult to come back when you can't stop anyone. Let's take a look at a few games that count against Rodgers.

Top five games that Rodgers is being irrationally blamed for in this analysis.

I'm only doing the five most egregious, and perhaps you will complain that all QBs have games like this, where they play brilliantly and still lose, and Rodgers is no different. Please keep in mind that five games is basically 1/4 of the entire sample size here.

V: Carolina at Green Bay, November 30th, 2008. Panthers won 35-31

Rodgers' stats: 29/45, 298 yards, 3 TDs, 1 INT, 26 yards rushing

The Packers trailed 21-10 to start the second half. They immediately cut the deficit to 21-13 when a long Brandon Jackson run got them into field goal range to start the half. The defense held and Rodgers drove them right down the field for a 95 yard touchdown drive, hitting Donald Lee for a 5-yard score and Greg Jennings for a two-point conversion, tying the game at 21. They then forced a three-and-out and Rodgers took them on another 76-yard scoring drive, culminating in a 21-yard strike to Jennings and giving Green Bay a 28-21 lead.

On the ensuing kickoff, Mark Jones took Mason Crosby's kick back 51 yards. Starting at midfield, the Panthers were in good shape, but on his first pass Jake Delhomme was picked by Tramon Williams - except that Tramon was called for pass interference, which moved the ball to the Green Bay 36. Delhomme then hit Steve Smith for a 36-yard score to tie the game. The Packers took over on their 20-yard line and went on a 79-yard, 9:10 drive. The Packers had first and goal on the Carolina 7-yard line with 3:45 left, but for some reason ran 3 straight times with Brandon Jackson and John Kuhn, getting to the one but no further. Crosby kicked a 19-yard field goal to regain the lead at 31-28. Crosby's kickoff was then returned 45-yards by Mark Jones because the Packers had horrid special teams that year, and Steve Smith caught another Delhomme bomb down to the one yard line. DeAngelo Williams took it in on the next play.

Rodgers had one more opportunity, but ended up throwing a game-ending pick to Jon Beason. You know...the kind of interception that would fall into the "taking chances to try to win a game" category. Shockingly, the pick did not increase their chances of winning.

IV: December 20th, 2009. Green Bay at Pittsburgh. Steelers won 37-36

Rodgers: 26/48, 383 yards, 3 TDs, 0 INT, 22 yards rushing, 1 TD

The Packers started the fourth quarter down 24-14, but an 11-yard score to Jermichael Finley made it 24-21 Steelers. Pittsburgh would add a field goal to push the lead back to 27-21, but a 24-yard Ryan Grant touchdown run put the Packers up 28-27. Pittsburgh responded with another field goal with 4:03 remaining in the game to go back on top, 30-28. The Steelers then tried a surprise onside kick and the Packers recovered. Taking advantage of the short field, Rodgers would eventually hit James Jones for a 24-yard touchdown and Brandon Jackson for a 2-point conversion to go back up 36-30. This was, unfortunately, they last time they would see the ball as Ben Roethlisberger ran the 2-minute drill to perfection, taking the Steelers 86 yards and hitting Mike Wallace in the extreme back corner of the end zone for a game-winning touchdown on the final play of the game.

III: December 30th, 2012, Packers at Minnesota. Vikings won 37-34

Rodgers: 28/40, 365 yards, 4 TDs, 0 INT

The Packers started the second half down 20-10. Rodgers would start the half with an impressive 80-yard drive, hitting future Viking Jennings for a 5-yard TD and cutting the deficit to 3 points. The Vikings would get a touchdown on a drive featuring Adrian Peterson and defensive holding calls to go back up by 10, but the Packers would respond with another short score to James Jones which followed a 73-yard bomb to Jordy Nelson. They would then tie the game at the start of the fourth quarter on a 40-yard Crosby field goal.

Unfortunately in the final quarter, the Packer defense was unable to stop Christian Ponder and Jarius Wright, who caught a 65 yard bomb of his own to set up a short Michael Jenkins TD. Rodgers would respond again, leading a 78-yard drive and culminating in a Jordy Nelson TD pass with 2:57 remaining to tie the game. This was the last time the Packers would get the ball, as Adrian Peterson would run the Vikings into field goal range and Blair Walsh hit a chip shot as time expired.

II: October 5, 2008, Atlanta at Green Bay. Falcons won 27-24

Rodgers: 25/37, 313 yards, 3 TDs, 1 INT

The Packers started the second half down 17-7 but quickly tied it up with a field goal and a 25-yard touchdown to Jennings. With about 10 minutes left in the game, Jerious Norwood took Crosby's kickoff back 54 yards, leading to an easy Jason Elam field goal to put the Falcons back in front. With 4:42 remaining, Rodgers was called for intentional grounding, and on 3rd and 19 he threw an interception. This bit of risk-taking led to a Michael Turner touchdown run and a 10-point Atlanta lead with 3:39 left. Rodgers would lead yet another touchdown drive, throwing a scoring pass to Donald Lee just before the 2 minute warning and cutting the deficit to 3, but the Falcons would recover the onside kick and Michael Turner picked up a first down on the ground, ending the game.

I: November 9th, 2008. Packers at Minnesota. Vikings won 28-27

Rodgers: 15/26, 142 yards, 0 TD, 0 INT, 1 yard rushing, 1 TD.

You've seen so many huge Aaron stat lines, so why is this rather pedestrian game from Aaron on the list? The Packers trailed 21-10 in the 3rd quarter, but on the strength of a Nick Collins pick-six and a Will Blackmon punt return touchdown, the Packers stormed into a 24-21 lead. Part of the reason Rodgers' numbers aren't that impressive is simply because the special teams and defense pitched in. With 6 minutes to play the Packers actually led by 6, but Adrian Peterson basically single-handedly led a 69-yard touchdown drive culminating in a 29-yard score with 2:19 to go. Will Blackmon set up the next Packer drive nicely with a 31-yard return and Rodgers hit Driver on the first play for 19, down to the Minnesota 40. It was at this point that Mike McCarthy basically decided to settle for a 50+ yard field goal try. After two Ryan Grant runs gained a total of 3 yards, Rodgers hit Donald Driver in the middle of the field for 3 yards. Mason Crosby missed a 52-yarder with 26 seconds on the clock. It was ridiculous.

Bonus. January 10th, 2010, Green Bay at Arizona, Arizona won 51-45

Rodgers: 28/42, 423 yards, 4 TDs, 1 INT, 13 yards rushing, 1 TD.

Morris didn't count this game as it was a playoff game. Say what you want about Rodgers in this game, taking chances on offense was not a problem. He missed Jennings deep on the first play of overtime, he threw for over 400 yards, and his defense was atrocious. The Packers trailed 31-10 at one point in the third quarter, but rallied to tie the game 45-45 at the end of the fourth before the infamous fumble return by the Cardinals in overtime provided the winning points.

Conclusion

I don't believe that Morris thinks Rodgers is a bad quarterback at all, but there is a bit of a cottage industry around denigrating the clutch performance of Aaron Rodgers and it's just frankly strange. Morris is saying the Aaron Rodgers - the current Aaron Rodgers - should take more chances when trailing big, but that is not supported at all by his numbers.

Most baseball projection systems rely on weighted data from either the previous 3 or 4 seasons. They use multiple seasons to give themselves a large sample, but it's important to note that they do not go back further than that, even though doing so would yield an even bigger sample. The reason for this is because players change over time. Fast guys get slower, skinny guys develop more power, and the like. Projection systems exist partially to tell you this, to pick up on these changing trends, but at some point, old data is just noise.

The fact that most of the data in this chosen sample is from 5+ years ago should set off alarm bells. The fact that he chose "9 points or more" should set off alarm bells. 9 is a weird number to pick, even if we acknowledge that a touchdown and two-point conversion totals to 8. When you see a 9 out there, perhaps someone was looking for data to support a conclusion they already had: that maybe Rodgers should throw more interceptions or take more risks when he's down big. Then again:

Teams recent Aaron Rodgers lost to when trailing by 9+ points in the 2nd half, 2010 to present:

2011 Kansas City Chiefs
2011 New York Giants (won Super Bowl that season, elite defense)
2012 New York Giants (Fresh off Super Bowl win)
2012 49ers (Lost in Super Bowl, elite defense)
2012 49ers (Again, this time in the playoffs)
2012 Vikings (See game III above)
2014 Seahawks (Fresh off Super Bowl win, elite defense)
2014 Lions (Currently #1 in defensive DVOA and DAVE)

Maybe modern Rodgers just can't come back from a multi-score deficit against a sample made up almost exclusively of Super Bowl teams with elite defenses. I'll let you decide.

Finally, there was one other QB that Morris looked at with 0 wins in 21 or more games like this. The guys with the most comebacks are Tom Brady and Peyton Manning, so there's greatness at the top of his chart, but look who's with Aaron on the bottom:

@BadgerNoonan Bubble size is games, not drives. Rodgers is lower left bubble. Only other QB with 0 wins is Kurt Warner. Top 2 are TB&PM, obv

— Benjamin Morris (@skepticalsports) October 16, 2014

Yes, that Kurt Warner.

Not too shabby. Perhaps the trick is to stay out of the middle of the pack.