Sunk cost and the NFL Draft

I’ve looked at the NFL Draft a lot since starting this blog. As the draft was here in Chicago this year, I found myself running into a number of jerseys on the street when I went out for lunch on Thursday and Friday. Even more surprising than the fact that people had travelled – in some cases from pretty far away according to the jerseys – was the fact that a lot of them were wearing jerseys of players who were disappointments if not outright busts. It got me thinking about sunk costs and whether teams are any better than their fans about cutting their losses.

To try to get at this we’ll need to know how much teams value their draft picks – conveniently we do know this via the Jimmy Johnson-popularized draft value chart – and then compare this to how much those players are used. Usage is a bit tricky but I’m going to approximate it with games started (1 full game) plus games played (2014 avg snaps non-starter / 2014 avg snaps starter, by position).

Before even getting to questions of usage, there is a significant disparity in the proportion of players from each round who end up making a roster.

Round	% on Roster Year 1
1	97%
2	94%
3	83%
4	81%
5	70%
6	62%
7	52%

I am guessing that most of this comes down to talent disparity, but there is certainly some aspect of sunk cost at work here. Lots of later round picks – to say nothing of undrafted players – never make it onto a roster to get into the rest of this analysis. They are, however, not the topic of this analysis. I want to see if a player’s draft value still impacts playing time even after making a roster.

The first cut of this is simply to look at draft weight and usage, checking how much the former impacts the latter. The regressions for each of a player’s first 6 seasons are below:

Usage vs Draft Weight
			Draft Weight
Year	R^2	Intercept	Coefficient	P-Value
1	0.22	4.53	15.87	0.00
2	0.16	6.88	13.87	0.00
3	0.10	8.10	10.71	0.00
4	0.08	8.84	9.18	0.00
5	0.05	9.49	6.58	0.00
6	0.04	9.88	6.14	0.00

The draft weight is a significant variable throughout the first 6 years of a player’s career, but the strength of that relationship declines over time. The 1^st year model explains 22% of the variation in usage while the 6^th year model explains just 4%.

Luck vs Skill - Is anyone good at picking football players?

A couple of weeks ago I took a look at the randomness in NFL Draft results, prompted by articles from Brian Burke and Chase Stuart on advancednflstats.com and footballperspective.com, respectively. I found some slim evidence for skill in drafting – the streaks of drafts with outperformance were slightly longer than expected. Interestingly there was much stronger evidence that some teams are not good at drafting, but you can read that post to get the details there.
 
The year-over-year data, however, has a major issue that I mentioned in the post: it’s not the same people making the decisions. Looking at the picks made by the Browns in 2012 and those made in 2013 provides no insight because the regime had turned over in the interim. Above and beyond that, the competitors do not behave similarly year to year so the decisions of each team – the variable we are trying to evaluate – are mixed in with changes in the way that other teams decide.
 
In response to these problems it seems like a study of the picks within a single year would be the place to look. The premise is that any team that has made a good pick should be more likely to make a good pick with their selection. If the initial pick reveals anything about their skill level, the subsequent success rate will exceed that of teams with a miss on their previous selection.
 
Using my data set of draft performance in the salary cap era[1] and my analysis of what a draft pick is really worth, we can quickly look at which picks were successful by identifying the ones that delivered more than the expected value. The percentage of picks that are successful varies widely by round due to scouting focus and the binary outcomes in later rounds[2]. If a player makes it and stays in the league for a couple seasons, they probably generate enough value to be a success because the expected return is so low in later rounds. Most of the picks in these rounds wash out: 269 of the 569 7th round picks play 1 season or less, and a grand total of 0 of them delivered value in excess of their expectation.

Leave the running back, take the quarterback (or center, or tackle)

For a number of reasons, drafting NFL players is a crapshoot. There are a wide variety of ways for a team to end up getting less than they bargained for – and less than the draft slot could have provided. A player could have a career ending injury before they even hit the field or they could even turn out to be not very good. I can’t help with those things.

What I can help with, however, is the third way a team could end up with a sub-optimal choice in the draft: ignorance of positional value.

Looking back over the past 19 drafts and using career value in terms of % of the salary cap, we can rank the actual value delivered and compare it to the draft slot where the player was selected. For example, Tim Couch was selected first overall by the Cleveland Browns in 1999 and delivered a career value of 18% (units are % of the salary cap to normalize across seasons). The actual most valuable player to date from that draft is Donovan McNabb with a career value of 98%. By those two numbers, Tim Couch’s selection would be rated as an unfavorable error of 80%. McNabb was drafted second and outperformed the actual second most valuable player in the draft (Champ Bailey with 76%) so he gets a favorable error of 22%.

Why should teams keep taking quarterbacks early?

Outcome of first round picks since 1994

Because that’s where the money is! Wait, that’s why people rob banks. Being by far the most valuable position on the field, QB is the position where even an average starter can justify the expectations of a first round pick. A starter at QB is worth the same as a pro bowl-caliber running back. Speaking of running backs, they perform… not well at this metric. Combining the tendency to outright disappoint and the tendency toward short careers leaves teams with an asset that underperforms the expectation for the draft slot.

What to do with the rest?

Note first of all that all errors here are negative. Because these are first round picks and this method is a zero sum game, if any fail to end up in the top 32 of their draft’s actual value they will result in a negative (unfavorable) error. While individual picks can be positive – if the player drafted 4th turns out to be the 2nd best in reality – the effect of 2nd-7th rounders outperforming drags down the first round numbers.

QBs may be roughly split between those two underperform and those who either meet expectations (the best you can do at the first pick) or exceed them, but they have a huge standard deviation that speaks to the boom or bust nature of high QB picks. Players like Tim Couch, Akili Smith and Cade McNown highlight the risks – and that’s just from the 1999 first round.

Offensive linemen turn out to be the most reliable bets in the first round, with relatively low errors and standard deviations. Receivers and running backs, on the other hand, represent the highest errors but have a glimmer of hope in their standard deviations. Despite the average error there are still numerous examples of receivers and running backs who work out in the league.

NFL Draft value charts for everyone!

ESPN.com’s Mike Sando, writer for the NFC West blog, put together an interesting series of posts last Thursday and Friday about the draft capital available to each team. The Thursday post looked at the capital available according to the NFL Draft value chart in wide use within the league, created by the Cowboys organization in the early 1990s, which we’ll call Chart Classic from here on out. On Friday, Sando analyzed each team’s position according to a revised version of the chart created by Kevin Meers in 2011 and featured on the Harvard Sports Analysis Collective site. As you can see below, the capital of each team varies widely between the two[1].

 

Total draft capital for 2013 draft (based on picks as of 3/26)

Being a fan of updating the draft value chart myself, I thought I would be helpful and provide a crack at this with my version of the chart and throw in Chase Stuart’s from FootballPerspective.com as well. Chase acknowledges the crucial point that his chart does not specifically account for the value of concentration of skill. One dollar is better than four quarters on the football field. While using Career Salary Cap Value rather than Career Approximate Value should help mine reflect the excess value of star players, it is a point of uncertainty in my chart as well.

One of the major things that sticks out from looking at the comparison of each chart against the original is how much additional capital is floating around. Picks 2-254 in the draft are worth 19.2x the number 1 pick according to Chart Classic. The comparable numbers for my chart, Football Perspective and HSAC are 37.0x, 44.1x and 54.1x. In more tangible terms Chart Classic is saying that only one team, Jacksonville, has the capital to put together a worthy trade offer for Kansas City using only this year’s picks. My chart would say that 22 teams have enough – though this would cost Green Bay all of their picks just to equal the value so that’s not much inducement for KC to do the deal. Football Perspective has it at 28 and the HSAC chart shows each team having sufficient capital to acquire the pick.

The relative improvement of teams is remarkably similar between the three alternative methods. 

This makes sense, as all are based on roughly the same logarithmic function.

A final chart to highlight the differences between the four charts shows all four valued as a percentage of the number one overall pick.

All four charts with each pick shown as a percentage of the value of the top pick

In this one the differences come out a bit more strongly. The switch from logarithmic to linear halfway through the draft is clear in my chart, while the chart from the Harvard Sports Analysis Collective is an outlier by not reaching near-zero value for the last pick.

---

[1] My totals vary ever so slightly from those featured in Sando’s ESPN.com post because I had to recreate the updated version of the Meers chart that he used. Through trial and error I found it to be the log trendline from the posted 1^st round values on Meers’ original post, ignoring the posted values for subsequent picks. Once I had this I could also extend out to the 254^th pick to account for the compensatory picks. Apologies to Kevin if I misrepresent his data through errors in my recreation.