In my last post I looked at the propensity for coaches to disproportionately use players drafted highly. Higher draft picks play slightly more than their underlying ability (i.e., performance over the next few seasons) would predict. In this post I want to take a quick look at whether this same effect is visible at the margin between 7th round draft picks and undrafted free agents.
The underlying assumption here is that 7th round picks are not all that different from undrafted free agents who get a look from teams. While I would love to be able to validate that assumption with some data, we don’t exactly have those populations in our available data. What we can compare is 7th round picks who have made NFL rosters to undrafted free agents who have made NFL rosters. As I noted in my previous post on this topic, this obscures the most likely place for this bias to manifest itself – decisions on who makes the roster – but can’t really be helped.
From 1994 through 2010 we have 2,043 undrafted free agents and 540 7th round picks who made NFL rosters in at least one season. This analysis will compare their playing time – games started are equal to 1 and games played but not started vary by position as a proportion of games started – with two factors: whether they were drafted or not and how well they played over the next 3 seasons. Performance over the next 3 seasons serves as a proxy for underlying skill. I am using the square root of that performance because I want to weight the player who has a 3 year line like 1-2-13 close to the player whose line is 9-10-8. I am assuming that both have a similar level of skill but the 1-2-13 player may have been blocked from starting or overlooked because he was undrafted.
As with the other analysis, it’s important to note first of all that the relationship here is not that meaningful (R = 0.36, R^2 = 0.13). For players who never play another season in the NFL, an undrafted one is expected to play the equivalent of 2.69 games while a 7th round pick would be expected to play 3.01. Being drafted alone moves the expectation by 0.32 games (p-value 0.03), more than 10% of the baseline. Compared to underlying skill, however, being drafted is much less meaningful. For the hypothetical “1-2-13” player above, underlying skill adds 3.29 games to the expectation (coefficient is 0.82 per unit, p-value 0.00).
Moving to players in their 2nd season the effect of being drafted goes away completely (p-value 0.99) while underlying skill becomes more powerful (coefficient is 1.12 per unit, p-value 0.00).
Based on this analysis I am pretty confident that there is a weak positive effect of being drafted on playing time for rookies. Given the way it evaporates in the second season I would not be surprised if it is strongest early in the first season on a per-game basis. I still believe there is a larger effect that is hidden by lack of data in terms of roster decisions. If anyone has any idea how to get at this question, feel free to let me know.