Searching for sunk costs: UFAs vs. 7th round picks

In my last post I looked at the propensity for coaches to disproportionately use players drafted highly. Higher draft picks play slightly more than their underlying ability (i.e., performance over the next few seasons) would predict. In this post I want to take a quick look at whether this same effect is visible at the margin between 7^th round draft picks and undrafted free agents.

The underlying assumption here is that 7^th round picks are not all that different from undrafted free agents who get a look from teams. While I would love to be able to validate that assumption with some data, we don’t exactly have those populations in our available data. What we can compare is 7^th round picks who have made NFL rosters to undrafted free agents who have made NFL rosters. As I noted in my previous post on this topic, this obscures the most likely place for this bias to manifest itself – decisions on who makes the roster – but can’t really be helped.

Approach

From 1994 through 2010[1] we have 2,043 undrafted free agents and 540 7^th round picks who made NFL rosters in at least one season. This analysis will compare their playing time – games started are equal to 1 and games played but not started vary by position as a proportion of games started – with two factors: whether they were drafted or not and how well they played over the next 3 seasons. Performance over the next 3 seasons serves as a proxy for underlying skill. I am using the square root of that performance because I want to weight the player who has a 3 year line like 1-2-13 close to the player whose line is 9-10-8. I am assuming that both have a similar level of skill but the 1-2-13 player may have been blocked from starting or overlooked because he was undrafted.

Results

As with the other analysis, it’s important to note first of all that the relationship here is not that meaningful (R = 0.36, R^2 = 0.13). For players who never play another season in the NFL, an undrafted one is expected to play the equivalent of 2.69 games while a 7^th round pick would be expected to play 3.01. Being drafted alone moves the expectation by 0.32 games (p-value 0.03), more than 10% of the baseline. Compared to underlying skill, however, being drafted is much less meaningful. For the hypothetical “1-2-13” player above, underlying skill adds 3.29 games to the expectation (coefficient is 0.82 per unit, p-value 0.00).

Moving to players in their 2^nd season the effect of being drafted goes away completely (p-value 0.99) while underlying skill becomes more powerful (coefficient is 1.12 per unit, p-value 0.00).

Based on this analysis I am pretty confident that there is a weak positive effect of being drafted on playing time for rookies. Given the way it evaporates in the second season I would not be surprised if it is strongest early in the first season on a per-game basis. I still believe there is a larger effect that is hidden by lack of data in terms of roster decisions. If anyone has any idea how to get at this question, feel free to let me know.

---

[1] For this analysis only players with at least 3 subsequent seasons possible (whether played or not) are eligible

Buyer Beware: Are players on better college teams more likely to be busts in the NFL?

Since I’m dipping my toe back into the water with this post, I figured I should stay in safe territory and look at some NFL Draft-related stuff. Enjoy!

With the recent release of Trent Richardson, it seems pretty safe to say that he has been a bust in the NFL. His performance has underwhelmed at a position that is held in low regard in the league. This is especially surprising given that Richardson was held in extremely high regard coming out of college.

As I was reflecting on this I saw an opportunity to test one of my theories on the draft – that players on good teams, specifically those on good units, are overdrafted relative to those from worse units. A player from a good unit, the theory goes, benefits by his skilled teammates taking away attention (no double teams for the second best DE or second best WR) and a better team will execute better in general, making all players look better.

Methodology

I’ll be keeping things pretty simple for this one. For 1994-2010 (I don’t have 2014 data yet and want to use 4 years of data for each player) each player’s first 4 years AV will be compared against the log regression for their draft position. Then I will check to see whether the sum of draft value spent on other players from the same school/same unit in that year or the next explains the over or underperformance.

Results

It does not. 

The overall regression shows no relationship at all (R=0.01) between players from the same unit in the same year (p-value=0.73) or the following year (p-value=0.69). When I tried to splice it by position the results were similarly underwhelming.

While there is a slight uptick in R and R-squared for QBs and offensive linemen, it is extremely slight. It's possible this is related to the draft combine effect I noted a few years ago. QBs and tackles were among those positions for which predictions actually got worse after the combine, guards stayed in place and there weren't enough centers included in Mel Kiper's Big Board (typically just the first round) to include in the analysis. Since these positions are relatively less influenced by raw physical skill than WR, DB and others, teams are more dependent on game film where the quality of teammates could confuse things more. This is all very speculative because, as noted above, this is a very slight effect.

At least the way I approached it with this analysis it appears that playing on a good team isn’t the reason Trent Richardson was overdrafted, he’s just a bust.

Why didn’t the Browns draft a wide receiver?

Less than 24 hours after trading down to grab some extra picks and then trading back up to grab Johnny Manziel the Factory of Sadness got cranked up again and word leaked that Josh Gordon, Cleveland’s All Pro wide receiver, had failed a drug test and would be facing a full season suspension. As someone who grew up in Cleveland let me just say that none of this was surprising.

What WAS surprising to some people was that the Browns, holding the 3rd pick in the 2nd round, didn’t draft a WR there. Not only did they not draft one at the 35th pick, they didn’t pick one through the rest of the draft.

In the medium- to long-term the Browns will certainly need some help at WR now that it seems they can’t count on Josh Gordon. For help with this season, however, the draft pick would have almost certainly been a non-factor – especially after they passed on Sammy Watkins and Mike Evans in the first round. 

Average 1st Season AV by Round Drafted

Position	1	2	3	4	5	6	7
C	4.0	2.9	1.2	1.9	1.2	0.4	0.6
DB	4.3	3.1	2.0	1.4	1.1	1.0	0.9
DE	4.7	2.1	1.6	1.5	1.2	0.5	0.7
DT	4.0	2.1	1.5	1.1	1.1	0.8	0.5
G	4.1	3.0	2.1	1.6	1.1	0.9	0.9
LB	5.9	3.6	1.9	1.4	1.0	0.8	0.8
QB	3.8	2.3	0.6	1.2	0.3	0.4	0.2
RB	6.7	4.2	2.7	1.9	1.1	1.2	0.4
T	5.2	4.6	2.0	1.4	0.8	0.7	1.2
TE	4.1	3.3	1.9	1.8	0.9	0.9	0.7
WR	5.2	3.1	1.6	1.4	0.7	0.3	0.5
Total	4.9	3.1	1.8	1.5	1.0	0.7	0.7

 
The fact that the Browns drafted a T at the 35th pick seems even better with the immediate need for help on the line and ability of tackles to contribute right away. WRs in the first round show strong performance, but those in later rounds underperform the average 1st year AV.

For reference, the averages for different levels of performance are:
 

Highest Performance	Avg AV
AP	13.6
PB	11.1
Starter	6.6
Roster	1.4

 
Finally, the list of wide receivers to deliver more than 10 units of AV since the 1994 draft is relatively short and heavily weighted to round 1. For most of the 20 years in the data set (11, to be precise) there are 0 receivers who reach 10 AV. There are 11 picks in the other 9 seasons who hit that threshold (Josh Gordon had 11 in 2013):

 

Year	Pick	Player	Tm	AV1
2013	76	Keenan Allen	SDG	10
2011	4	A.J. Green	CIN	10
2011	6	Julio Jones	ATL	10
2009	22	Percy Harvin	MIN	12
2006	252	Marques Colston	NOR	10
2003	54	Anquan Boldin	ARI	10
2001	36	Chad Ochocinco	CIN	16
1998	21	Randy Moss	MIN	17
1996	7	Terry Glenn	NWE	13
1996	18	Eddie Kennison	STL	10
1995	8	Joey Galloway	SEA	12

None of this means that Cleveland doesn’t need some help at WR. The Browns’ GM Ray Farmer knows it’s an issue and is likely looking at a number of different options including free agents. The fact that he passed up the crowd-pleasing WR pick to stay true to where he and the Browns’ front office team had players slotted should encourage fans that he has a strong process for finding the necessary support.