Are NBA teams getting better at drafting?

Looking back at the NBA Draft one can find some pretty egregious mistakes. Bill Simmons recently took a look at every NBA draft since 1995 and reordered them based on where his evaluation of the players’ careerssuggests they should have gone.

While I’m interested in this exercise – indeed I spent many minutes of my life reading his post on the subject – I have something a bit more rigorous in mind. I want to evaluate whether NBA decision makers have become more skillful at drafting.

Chase Stuart recently looked at this topic in the NFL, finding very little improvement, and my own research on the NFL Draft suggests that the relatively uniform level of skill means that extreme outcomes (e.g., a great draft class, the Browns) tend to be the result of luck. 

Methodology 

I’m trying to keep this as simple as possible while still being representative. Looking at entire careers is out because every single draft in my 1998-2013 data set features active players. At the risk of underrating a few picks in which the player was a late bloomer AND that late blooming was captured by the drafting team, I will look at the first four years for each player[1][2].

I am going to use win shares as a proxy for performance. This is due primarily to availability but also because they are a reasonable approximation of performance (though one with known flaws).

Now for the actual measurement methodology, which I feel is refreshingly straightforward. With the benefit of hindsight we know the best player on the board at any given pick. The proportion of the best available pick’s win shares delivered by the actual pick give us the score of that pick.

As an example, Derrick Favors (3^rd pick, 2010 draft) accrued 16.1 win shares in his first 4 seasons. The best available player on the board – Paul George, 10^th pick – had 29.7 win shares so Favors has a score of 54%. The Pacers' selection of George at number 10 would have a score of 100%. The higher the average score is for a given draft, the better the decision makers are doing selecting the best available player. 

Results 

We’ll look at the results in three different cuts to see if teams are getting better.

First, are they getting better in the lottery? This is the equivalent of a first round NFL pick, where players are expected to step in and start immediately with a realistic chance of an all-star appearance or two during their rookie deal.

Second, are they getting better in the first round? This has a certain elegance in that each team (barring trades) gets to make their one pick so even if a team is picking 28^th, it’s their first shot in the draft and we could expect them to maximize the value of that pick.

Finally, are they getting better across all picks in the draft? It seems logical that we should look at the overall performance of teams at picking players.

  

Interestingly, the further we go into the draft, the more improvement we see from year to year.

A possible explanation here is that the rigor that used to be applied only to lottery picks has been extended further into the draft. In support of this explanation one could cite the technological progress that allows scouts to see innumerable prospects via YouTube rather than scouting them individually.

Alternatively, the data could be telling us elite organizations continue to pull away from the rest of the league. Those teams that put the best team on the court – and thus pick last in the draft – are now the same teams that have the best process in place to evaluate draft picks. This could come from better evaluation techniques or merely the ability to delay gratification in the case of selecting foreign players who cannot come to the NBA immediately.

I’m inclined to put more stock in the first explanation over the second, but distinguishing between the two is a topic for another day (or for the comments section).

-- For those of you who made it to the end, have a visualization of all picks from 1998 to 2013 with a reference line for best available player:

Learn About Tableau

---

[1] This is also based on the current structure of rookie contracts. There are team options for the 3^rd and 4^th year, as well as a qualifying offer that can give the team the right to match any offer made to the player for a new contract beginning in their 5^th season. To make sure we still have a usable, relatively current data set, we’ll use the first 4 seasons.  

[2] It’s probably not ok, from a style guide perspective, to go with double footnotes, but this is about a different thing. An acknowledgement here that several foreign players selected in the draft in the years we’ll be using are either still not in the NBA or have not yet finished their first 4 seasons. Not much can be done about it but it likely makes some late 1^st round and 2^nd round picks look worse than they are. Those teams took a defensible shot with their picks and either couldn’t convince the player to come over or knew there would be a couple years before they would move to the NBA.

NFL Draft Trade Machine

Welcome to the Sports + Numbers NFL Draft Trade Machine, a handy resource to see what teams can do with the assets they have and to evaluate trades as they happen. While I am biased toward my own value chart, I included several others so that individual users can select the one they prefer or look at the differences.

Make edits to the blue-shaded cells to customize for any scenario you want to look at and see where it lands on the graph (with actual 2012 trades overlaid as black diamonds).




I've given in to my recent fixation on the NFL draft and decided to focus exclusively on it for the next few weeks. For those of you just stopping by for the first time, check out a few of my NFL-themed posts:

• Draft-themed
• A summary of the various draft value charts 
• Detailed introduction of my own draft value chart and analysis of whether it fits the results on the field
• Analysis of whether the NFL combine helps or hurts accuracy of draft picks
• The top ten drafts of the salary cap era (split into 10-6 and 5-1)
• Other NFL
• A look at injury rates for running QBs and non-running QBs
• Two articles about how frustrating forecasts are as a Browns fan (actually analyses of future power rankings and beginning-of-season forecasts)

Additional notes on the trade machine:

• Sources for each chart
• Existing Chart: Here's a version at ESPN
• Sports + Numbers 
• Football Perspective
• Harvard Sports Analysis Collective
• Discount Rate - Cade Massey and Richard Thaler's paper, The Loser's Curse, provides the 136% number
• Mean Reversion - I've found this to be roughly 58% in the last 20 years
• 2012 trade data
  

What are NFL draft picks really worth (and do they help teams win)?

One of the biggest moments in the history of the NFL Draft took place almost as far from it as possible. On October 12, 1989 – 171 days after Everett Ross became Mr. Irrelevant as the last pick of the 1989 Draft and 192 days before Jeff George kicked off the 1990 edition – the Dallas Cowboys sent Herschel Walker and a handful of picks (two 3^rd rounders, a 5^th and a 10^th) to the Minnesota Vikings in exchange for five players, three 1^st round picks, three 2^nd round picks, a 3^rd round pick and a 6^th round pick. Dallas’ multiple 1990s Super Bowl runs were powered by players such as Emmitt Smith, Russell Maryland, Alvin Harper and Darren Woodson who either came directly from the picks or as a result of further trades involving those picks.

Mike McCoy, who owned approximately 5% of the team, had been a business partner of majority owner Jerry Jones in the oil business. The team was looking for a way to systematically value their cache of draft picks, and McCoy was the one to do it. According to a 2004 Dallas Morning News article, McCoy spent two days graphing the actual trades that had taken place over the past four years. He found that the trades appeared to fit a trendline overlaid on the graph. This trendline became the basis for the Draft Value Chart[1].

The Chart provides a value for each pick in the form of unitless “points” assigned decreasing from 3000 for the number one overall to, depending on which source you consult, 0.4 points for the 256^th pick or 2 points for the 224^th pick[2]. The decrease in point values is extremely steep at the top of the draft with the value dropping by 50% to the 7^th pick and by another 50% to the 24^th pick, leaving it only 25% as valuable as the number one selection for trade purposes.

Since the early 1990s the Draft Value Chart has made its way through the NFL and become the basis for draft pick value on nearly every team. The assistant coaches and assorted front office employees from the Cowboys took it around the league when they left the team. Research conducted by Cade Massey and Richard Thaler plotting actual trades found that prices aligned closely to The Chart with the deviation from the chart dropping significantly and trade volume increasing in the years after it became well known[3]. In other words, once the chart became widely accepted teams did not vary from the assigned values.

3 Picks to Advance 1 Slot??!!

For just a taste of my ongoing project, here is my (still in process) grade of the Vikings-Browns trade last night that sent the #4, #118, #160 and #211 picks to Minnesota in exchange for the #3 pick.

The "Value" field pulls in the log regression of the average % of salary cap value provided by those picks in excess of the first undrafted free agent. That is to say the last pick of the 7th round may provide 0.7% of the cap value in an average season of their career, but they only provide 0.003% excess value over an undrafted free agent. Only the 0.003%, in that case, matters for the trade because that is the extra value provided over what the other team can achieve for free with the next best option (e.g., a UFA).

That will have to be it for the explanation as I need to get back to writing about 6,000 more words for my project on this very topic.

One more graph on the NFL Draft Value Chart

After re-reading my post from last night, I realized that I left out one more graph that may be of interest to readers. The normalized chart showing the change in the salary cap and guaranteed money to number one overall picks from 1994 to 2009 came up but then was never revisited to see the impact of the new CBA. With the final data point with Cam Newton’s $22 million and the new salary cap number of $120 million the graph below shows the full set (2010 salary cap excluded because it did not exist):

Guaranteed money (red) and salary cap (blue) 1994=100

In terms of relative difference between the change in the cap and guaranteed money from 1994, this puts 2011 on a level with 2003, 2000 and 1995. The big unknown now is whether the straitjacket the owners placed themselves into will hold or if salaries will somehow creep up faster than the cap as they did from 2003 to 2009.

Thoughts on the NFL Draft Value Chart

Edit - Take a look at my expanded thoughts on the NFL Draft Value Chart - and a proposal for revising the Chart - in this post from November 2012.

Let’s say, hypothetically, that you had just traded the best player on your team (one of the best in the league) for a whole mess of draft picks. If you had just done this, wouldn’t you want to convince people of the wisdom of your choice? Wouldn’t the best way to do that be to convince everyone that draft picks, particularly of the type you received, were highly valuable assets? I am not suggesting in that the Herschel Walker trade was anything short of brilliant for the Cowboys (this was not a hypothetical), but in order to convince people of the wisdom of the trade it certainly helped to establish a baseline valuation for draft picks that made the Cowboys’ haul look even better before those picks even had a chance to play out.

Since that time the Draft Value Chart developed by Jimmy Johnson’s organization in Dallas has become the gospel for a great number of NFL teams. At various times much of the NFL has used the chart and reports suggest that a number of teams still do. At a minimum it is still the go-to for reporters speculating on potential trades. Once the chart became widely accepted, draft picks became easily exchangeable as teams agreed on the value and constructed trades accordingly.