Did the Prime Time Combine schedule actually hurt players’ testing results?

The NFL is a business. The entire purpose of businesses is to provide a good or service to people, and in return those people provide you currency. If you have a successful, or at least a sustainable business, the revenue you get will exceed the expenses you endure to a level at which you are willing or able to continue the business going forward.

Part of what a business must do is innovate. Innovation can be messy. It can be costly. Sometimes it doesn’t work out. Other times it increases your profit margins and makes your business more successful. These relatively simple rules of business apply to all business. From your mom-and-pop store down the street to the billion-dollar behemoths of sports leagues.

Why am I going over the first few units of Business 101? Because the NFL made a change to one of its biggest events this year in the pursuit of more dollars and cents. The NFL decided to make their NFL Combine drills run during Prime Time television viewing hours. This is a stark difference to schedules in prior years where prospects would largely work out during the day.

The primary, and really only, reason for doing this was to get more eyeballs onto the NFL Combine. More eyeballs means more ad revenue. But if the alteration to the schedule leads to a worsened experience for players and teams, then the NFL must ask the most important question: is it worth it?

As I discussed in my last piece, the number of agility drill participants plummeted in 2020. As a refresher, 56% of players at Green Bay’s “3-cone positions” didn’t run a 3-cone. In total, nearly 54% of participants didn’t run one. But what about those who did? How did they fare? To answer this, I constructed a comparison of this year’s 3-cone results to historical 3-cone results.

I used Kent Lee Platte’s RAS database as the source for all the NFL Combine’s 3-cone data. The purpose of this “study” is to determine if players fared noticeably better or worse as a group with the Prime Time testing versus the traditional time slot. To determine this, I compiled all the 3-cone times run by players in 2020. I then divided them up by position. (The position designations that were used were the ones used by the NFL Combine.) Once players were separated by positions, I calculated the average 3-cone time for each position in 2020.

To get the historical context necessary, I utilized RAS’ 50th percentile 3-cones as the historical average. For most positions, this was straightforward. However, for the positions on the lines, and at defensive back, there are multiple positions that are within those groups. To try and counteract this issue, I took the 50th percentile 3-cone times at each respective position and averaged that time for the group. The combined positions were OL (OT, OG, OC), DL (DE, DT), and DB (CB, FS, SS). The assumption here is that the distribution of participants in the 3-cone should be similar across years. Given that this is a one-year sample, this may not hold true, so that is something to consider during the conclusion.

The results for each position are listed below:

Obviously, in this analysis, faster times are better, so negative numbers indicate that the 2020 group performed better in this drill than the average, while positive numbers indicate worse-than-average performance.

There are certainly some interesting results here. Most positions don’t seem to see a large difference, but there are a few that stand out. WRs ran nearly a full tenth of a second slower than average, linebackers were more than one tenth faster than average, and defensive backs were nearly a tenth faster than average.

Over a small sample like this, surely selection bias can come into play. In a year where fewer players ran the 3-cone than normal, it is possible that only those players who expected to post very good times would run it. However, we would expect other positions to have had similar effects.

The next thing I wanted to test was what the class looked like on a per-player scale. To do this I took the average time differential per position and summed those values together with the total number of participants at each position to get an overall average difference vs. average. It’s a rather crude way of trying to try and look at larger trends, but it also is one of the simpler ways.

Somewhat surprisingly, the 2020 NFL Combine had an ever-so-marginally faster average 3-cone time than historical averages. And when I say ever-so-marginally, I really mean it. The average 2020 NFL Combine 3-cone time was .007 seconds faster than the historical average.

Now there are circumstances to consider here. The primary one appears to be the linebacker position. As the NFL has become a more passing-focused league over the past decade, linebackers have begun to look more like strong safeties. With smaller linebackers, we would expect them to move better than the historical average, and we saw that in 2020. What happens if we removed the linebackers from the sample?

Remember, the linebacker group was .12 seconds faster than their historical counterparts. Despite this quite large difference, entirely removing the group doesn’t change the results too drastically. The new differential shows that 2020 NFL Combine 3-cones were just .008 seconds slower than the historical average when we remove the linebacker position from the equation.

The differences between 3-cone times this year and in past years appears to be negligible. However, there are some issues with this study. First and foremost, we don’t have a true control group. We did not get any of the players at the 2020 NFL Combine running their 3-cones at traditional timeslots. It is possible that this class would be expected to perform better than historical averages, whether that be due to the NFL’s increased emphasis on speed and quickness, perhaps being a more athletic class, or due to better drill training. The sample for the 2020 class is also smaller than in classes before, particularly those of the past ten years.

This study is merely one piece of the puzzle, and by no means a be-all-end-all. I do think the findings from it are a bit surprising, but probably rather informative. We think that variables will change things drastically, but we should remember that in any good study our default hypothesis should be that it has no effect.