Monday, May 11, 2015

Sunk cost and the NFL Draft

I’ve looked at the NFL Draft a lot since starting this blog. As the draft was here in Chicago this year, I found myself running into a number of jerseys on the street when I went out for lunch on Thursday and Friday. Even more surprising than the fact that people had travelled – in some cases from pretty far away according to the jerseys – was the fact that a lot of them were wearing jerseys of players who were disappointments if not outright busts. It got me thinking about sunk costs and whether teams are any better than their fans about cutting their losses.

To try to get at this we’ll need to know how much teams value their draft picks – conveniently we do know this via the Jimmy Johnson-popularized draft value chart – and then compare this to how much those players are used. Usage is a bit tricky but I’m going to approximate it with games started (1 full game) plus games played (2014 avg snaps non-starter / 2014 avg snaps starter, by position).

Before even getting to questions of usage, there is a significant disparity in the proportion of players from each round who end up making a roster.

Round
% on Roster Year 1
1
97%
2
94%
3
83%
4
81%
5
70%
6
62%
7
52%

I am guessing that most of this comes down to talent disparity, but there is certainly some aspect of sunk cost at work here. Lots of later round picks – to say nothing of undrafted players – never make it onto a roster to get into the rest of this analysis. They are, however, not the topic of this analysis. I want to see if a player’s draft value still impacts playing time even after making a roster.

The first cut of this is simply to look at draft weight and usage, checking how much the former impacts the latter. The regressions for each of a player’s first 6 seasons are below:

Usage vs Draft Weight



Draft Weight
Year
R^2
Intercept
Coefficient
P-Value
1
0.22
4.53
15.87
0.00
2
0.16
6.88
13.87
0.00
3
0.10
8.10
10.71
0.00
4
0.08
8.84
9.18
0.00
5
0.05
9.49
6.58
0.00
6
0.04
9.88
6.14
0.00

The draft weight is a significant variable throughout the first 6 years of a player’s career, but the strength of that relationship declines over time. The 1st year model explains 22% of the variation in usage while the 6th year model explains just 4%.

Wednesday, April 8, 2015

Buyer Beware: Are players on better college teams more likely to be busts in the NFL?

Since I’m dipping my toe back into the water with this post, I figured I should stay in safe territory and look at some NFL Draft-related stuff. Enjoy!

With the recent release of Trent Richardson, it seems pretty safe to say that he has been a bust in the NFL. His performance has underwhelmed at a position that is held in low regard in the league. This is especially surprising given that Richardson was held in extremely high regard coming out of college.

As I was reflecting on this I saw an opportunity to test one of my theories on the draft – that players on good teams, specifically those on good units, are overdrafted relative to those from worse units. A player from a good unit, the theory goes, benefits by his skilled teammates taking away attention (no double teams for the second best DE or second best WR) and a better team will execute better in general, making all players look better.

Methodology

I’ll be keeping things pretty simple for this one. For 1994-2010 (I don’t have 2014 data yet and want to use 4 years of data for each player) each player’s first 4 years AV will be compared against the log regression for their draft position. Then I will check to see whether the sum of draft value spent on other players from the same school/same unit in that year or the next explains the over or underperformance.

Results

It does not. 

The overall regression shows no relationship at all (R=0.01) between players from the same unit in the same year (p-value=0.73) or the following year (p-value=0.69). When I tried to splice it by position the results were similarly underwhelming.


While there is a slight uptick in R and R-squared for QBs and offensive linemen, it is extremely slight. It's possible this is related to the draft combine effect I noted a few years ago. QBs and tackles were among those positions for which predictions actually got worse after the combine, guards stayed in place and there weren't enough centers included in Mel Kiper's Big Board (typically just the first round) to include in the analysis. Since these positions are relatively less influenced by raw physical skill than WR, DB and others, teams are more dependent on game film where the quality of teammates could confuse things more. This is all very speculative because, as noted above, this is a very slight effect.

At least the way I approached it with this analysis it appears that playing on a good team isn’t the reason Trent Richardson was overdrafted, he’s just a bust.

Sunday, December 7, 2014

Stay tuned...

I realize things have been a bit quiet around here for the past couple months. A few conclusions stand out to me upon reflection.

1. Having a new job negatively impacts frequency of writing
2. Having a new baby negatively impacts frequency of writing
3. Having both a new job and a new baby ensures that no writing will take place

Needless to say the last few months have been exciting and I am happy with both the job and the new addition to the family. I wouldn't trade it for anything even though this blog, which has been thoroughly enjoyable for me, has found itself a notch or two lower in priority. I hope you'll all give me the benefit of the doubt when I say that new research will show up here eventually.

The good news is that I have the data and analysis portions of my next piece (probably a 3-4 part series) complete. This will be a look at how much being drafted impacts playing time while attempting to control for underlying player skill and will breeze through some of my favorite armchair behavioral economist ideas.

The bad news is that I have almost no words of thoughtful, insightful, slightly wordy blog post composed. When that comes - in the next week or month - you'll see it here first. In the meantime, continue to enjoy the site and lots of old posts while keeping up the lively commenter community for which this site is known.

Monday, September 22, 2014

No league for old men


Back in December of last year I went through some calculations with the data set I stitched together including both performance and salary data. You can check the post out here for a series of simple graphs that go a long way toward explaining team behaviors and why certain things are the way they are in the NFL, particularly as they relate to player tenure and performance over time.

As I was reading through that post again recently, however, I was struck that it makes the implicit assumption (explicit above) that things are the way they are. What I mean is that this data is the average over a period of time, while there may in fact be some trends to observe by looking at the period year-by-year.

Charts



The average age of all NFL players[1] fluctuated between 26.5 and 27 for most of the post-1994 period (the salary cap era) before dropping from 2008 through 2013 almost without interruption. Applying a weight based on games played or games started, the other two lines on the chart, indicate that the pattern is consistent for the starters as well as the backups. NFL rosters are roughly half a year younger than they were from 94-08.

Wednesday, August 20, 2014

Are NBA teams getting better at drafting?



Looking back at the NBA Draft one can find some pretty egregious mistakes. Bill Simmons recently took a look at every NBA draft since 1995 and reordered them based on where his evaluation of the players’ careerssuggests they should have gone.

While I’m interested in this exercise – indeed I spent many minutes of my life reading his post on the subject – I have something a bit more rigorous in mind. I want to evaluate whether NBA decision makers have become more skillful at drafting.

Chase Stuart recently looked at this topic in the NFL, finding very little improvement, and my own research on the NFL Draft suggests that the relatively uniform level of skill means that extreme outcomes (e.g., a great draft class, the Browns) tend to be the result of luck. 

Methodology 

I’m trying to keep this as simple as possible while still being representative. Looking at entire careers is out because every single draft in my 1998-2013 data set features active players. At the risk of underrating a few picks in which the player was a late bloomer AND that late blooming was captured by the drafting team, I will look at the first four years for each player[1][2].

I am going to use win shares as a proxy for performance. This is due primarily to availability but also because they are a reasonable approximation of performance (though one with known flaws).

Now for the actual measurement methodology, which I feel is refreshingly straightforward. With the benefit of hindsight we know the best player on the board at any given pick. The proportion of the best available pick’s win shares delivered by the actual pick give us the score of that pick.

As an example, Derrick Favors (3rd pick, 2010 draft) accrued 16.1 win shares in his first 4 seasons. The best available player on the board – Paul George, 10th pick – had 29.7 win shares so Favors has a score of 54%. The Pacers' selection of George at number 10 would have a score of 100%. The higher the average score is for a given draft, the better the decision makers are doing selecting the best available player. 

Results 

We’ll look at the results in three different cuts to see if teams are getting better.

First, are they getting better in the lottery? This is the equivalent of a first round NFL pick, where players are expected to step in and start immediately with a realistic chance of an all-star appearance or two during their rookie deal.

Second, are they getting better in the first round? This has a certain elegance in that each team (barring trades) gets to make their one pick so even if a team is picking 28th, it’s their first shot in the draft and we could expect them to maximize the value of that pick.

Finally, are they getting better across all picks in the draft? It seems logical that we should look at the overall performance of teams at picking players.




  
Interestingly, the further we go into the draft, the more improvement we see from year to year.

A possible explanation here is that the rigor that used to be applied only to lottery picks has been extended further into the draft. In support of this explanation one could cite the technological progress that allows scouts to see innumerable prospects via YouTube rather than scouting them individually.

Alternatively, the data could be telling us elite organizations continue to pull away from the rest of the league. Those teams that put the best team on the court – and thus pick last in the draft – are now the same teams that have the best process in place to evaluate draft picks. This could come from better evaluation techniques or merely the ability to delay gratification in the case of selecting foreign players who cannot come to the NBA immediately.

I’m inclined to put more stock in the first explanation over the second, but distinguishing between the two is a topic for another day (or for the comments section).

-- For those of you who made it to the end, have a visualization of all picks from 1998 to 2013 with a reference line for best available player:



[1] This is also based on the current structure of rookie contracts. There are team options for the 3rd and 4th year, as well as a qualifying offer that can give the team the right to match any offer made to the player for a new contract beginning in their 5th season. To make sure we still have a usable, relatively current data set, we’ll use the first 4 seasons.  
[2] It’s probably not ok, from a style guide perspective, to go with double footnotes, but this is about a different thing. An acknowledgement here that several foreign players selected in the draft in the years we’ll be using are either still not in the NBA or have not yet finished their first 4 seasons. Not much can be done about it but it likely makes some late 1st round and 2nd round picks look worse than they are. Those teams took a defensible shot with their picks and either couldn’t convince the player to come over or knew there would be a couple years before they would move to the NBA.