The value of a Major League Baseball general manager (Review)

Lewis Pollis, a recently graduated senior at Brown University and past/future intern for several MLB teams, put together an analysis of the value of a general manager in Major League Baseball for his senior thesis in economics. He also had a summary version posted to Regressing, Deadspin’s analytics-oriented sub-site.

The paper estimates the value of the top-performing GMs to be tens of millions of dollars higher than that of middling GMs, who are likewise tens of millions of dollars more valuable than the worst performers. From there the paper links this value differential to the current narrow band of salaries and advances the notion that “the best baseball operations employees are paid substantially less than they are worth to their teams.”[1]

What is being measured in this research is how well teams perform. This performance is measured across a narrow set of activities (signing free agents, making trades) and attributed to the specific GMs who conducted the activity with some caveats about what can and can't be included as well as how it is measured. I am bought-in on this approach. These are hard things to measure and this seems to be a good way to get at it.

The outcome of this performance measurement is a distribution of value generated from best to worst and the conclusions noted above – the value that a good GM provides over a bad GM – flow from this distribution. This is where I have a couple of issues.

A couple of issues

For one thing, Pollis notes that the specific GM is not an accurate predictor of how a trade or signing will work out[2]. If a GM is not a predictor – or more accurately if making good picks in the past doesn’t make them any more likely in the future – then the value difference may be coming from luck rather than skill.

In the posts I did looking at the NFL draft I found no evidence for excess skill in selecting players. In the year-over-year data there were a few more streaks of bad selections than a purely random model would suggest, but the streaks of good picks were well within the expectations from the random model.

Another point offered in support of the model is that “subjectively speaking, the individual rankings seem more closely aligned with how well the GMs’ teams performed than with outsiders’ views of their decision-making processes.”[3] I would suggest that this is somewhat worrisome in that it implies the model may simply be a more complicated way of measuring success for teams – the model is drawn from wins above replacement, wins above replacement are highly correlated with team performance, team performance is what tends to drive GM reputations. Now I’m not sure there is a good way around this but I would be interested in any counter-intuitive results – maybe some GM has had bad injury luck in the draft but free agent signings and trades show a skilled evaluator.

Finally, the paper notes that the correlation between GMs’ measured ability in trades and measured ability in free agent signings is effectively 0[4]. If the model were measuring the combination of skill in identifying good players as well as skill in paying them salaries advantageous to the team – and this is what I understand the model is attempting to measure – we should expect those skills to be common across trades and free agent signings. I can’t think of a skill-based explanation for why a GM would excel at one and not the other. I can think of a non-skill-based explanation: luck.

Conclusions

I enjoyed this paper, and I think this is a great start at separating good from bad in GM performance. What I would like to see is more effort put into demonstrating that good or bad performance relates to the specific GM.This is far easier said than done and might require a lot more observations in the data set.

In the absence of additional data, the fact that skill across trades and free agent signings is not correlated leads me to suspect the answer is that GMs who measure highly were unusually lucky in these activities while those who measure poorly are unusually unlucky. This fits squarely into the paradox of skill – the importance of luck in determining outcomes rises as the overall group becomes more skilled. 

In baseball, as in football and investments, the decision-makers tend to be very smart people and the organizations have developed sophisticated infrastructure to evaluate talent. The result of this is that individual teams are not likely to be significantly more skillful in their evaluation than others and the outcome will increasingly defined by luck as the skill level rises. See here for a fuller explanation.

My biggest disappointment with all of this is that the next iteration is unlikely to be available for public consumption with Pollis now working in-house for the Cincinnati Reds. I hope he still manages to publish occasionally, and I wish him and the Reds good luck.

---

[1] Page 62

[2] Pages 47-48

[3] Page 49

[4] Page 49

Returns to inequality in sports

Now that the last of my posts on returns to income inequality is up seems like the time for a quick reflection on the concept overall and how well it explained the success of teams.

The returns to inequality

The NBA is where the inequality of a team appears to make a difference in the expected success. This fits with the narrative that teams need to have a star (or several) rather than a surplus of role players. In all of the other leagues analyzed it does not make a significant difference. The NFL and MLB show a negative coefficient. Inequality harms a team in those two leagues. The NHL – most similar to the NBA in salary structure and individual player leverage – is the only other league to show a positive correlation between inequality and team performance.

This whole analysis is necessarily limited. The cumulative build-up of a team’s salaries can only tell us so much (R-squared values MLB=0.13, NBA=0.32, NFL=0.07, NHL=0.13) about the way they perform on the field/ice/court. It is a prediction, sometimes made years before, and made either under duress as part of a bidding process for a free agent or dictated by the terms of the collective bargaining agreement to a draft pick.

Still, it is interesting that one of the coefficients was significant while two others were close (p-value 0.2) after controlling for overall team spending. Even if it just confirmed what people already “knew” it was interesting enough for me.

Looking at a metric more-strictly focused on performance like WAR for baseball or Win Shares for basketball is problematic because end-of-season numbers incorporate the ups and downs of actual performance, so the team’s sum total matches to the performance. For 2012 (or 2012-13 for basketball) the WAR correlation with run differential is 0.89 while the Win Shares correlation with point differential is 0.997. These metrics are very good at allocating out the runs (points) to match their actual totals after the fact.

Unfortunately for us, the effects of a transcendent star making others better – or of a well-balanced team attacking weak links in opposing defenses – are already baked into these backward-looking metrics. To be useful we would need to look at the pre-season expected totals. Perhaps in a future post.

Returns to inequality in MLB

Take a look over here if you want to get the background for this series, otherwise read on.

Sports + Numbers Prediction: "This is anyone’s guess. The returns to inequality – after controlling for the wide distribution in overall team salary – might be strong or they might not. I don’t have a good feel for it so this will be more of a fact finding mission."

The data

To see the impact of inequality we will look at each team’s Gini coefficient against their winning percentage, controlling for team spending. The resulting equation gives us an r-squared value of 0.13 with only salary spending being significant (P-value of 0.00001) while the Gini coefficient comes in at a P-value of 0.18.

Payroll vs Winning % - MLB 2008-2012

For every million dollars in team spending the expected increase in winning percentage is 0.000654. For a team that spends $10 million more than a comparable team – all else equal – we would expect them to win an additional game.

Gini vs Winning % - MLB 2008-2012

On inequality the - insignificant - coefficient is -0.14. Within the range of Gini coefficients in baseball (0.35 to 0.66) this would mean a difference of 7 wins from the most equal to the least equal (more wins to the most equal). Not nothing but not exactly a huge impact. The gap in payroll ($19 million to $206 million) projects to a gap of nearly 20 wins.

Payroll vs Gini (color-coded by winning %) - MLB 2008-2012

Returns to Inequality: Introduction and predictions

As a follow up to my data dump post on league-level and individual player inequality in sports, I want to go a level deeper in each league and see where inequality makes a difference on the field (or court, or ice). This series of posts will look at each league and run a simple regression of winning percentage against overall payroll and team Gini coefficient.

I expect there will be relatively low correlation between spending and winning in the harder-capped leagues (NHL and NFL) while the NBA and MLB should show some.

The real interesting point will be whether prominent current teams that are more unequal (stars and minimum-salary guys: the Miami Heat or New England Patriots) are successful as a rule or as an exception.

A few predictions before I get started:

NBA – My guess is the NBA will have the biggest returns to inequality as a proxy for teams having stars. With those stars they are not able to afford middling salaries for role players and drop quickly down to minimum salary or exception-level players.

NFL – I would think the returns to inequality are high here, but not exactly as a proxy for having stars. The NFL’s cap structure essentially forces teams to play rookies and younger, pre-contract extension, players heavily and supplement them with selected veterans. The catch is that nearly all teams have a big young player population so the difference between one that works out and one that doesn’t might not be visible in the salary distribution.

MLB – This is anyone’s guess. The returns to inequality – after controlling for the wide distribution in overall team salary – might be strong or they might not. I don’t have a good feel for it so this will be more of a fact finding mission.

NHL – I am guessing that returns to inequality are strong here too, with relatively high leverage of the individual players resembling the NBA more than the NFL or MLB.

Sports Gini: Inequality within major sports leagues

The Gini coefficient is a way to measure the level of income equality in a country. It is calculated by plotting cumulative incomes in ascending order and measuring the gap between the resulting curve and the straight line that results from taking the average income in each instance. This sounds much more complicated than it is (but you can read more about it here).

Here is an example where total income is 100. The red area is the cumulative income. The blue area represents the gap between cumulative income and perfect equality of income.

A country with a Gini coefficient of 0 would have no blue area visible, as the income for each individual is the same so the cumulative income function looks the same as the straight line average income function. Perfect inequality, on the other hand, would have virtually no red visible as all but one of the people earns nothing and the other person earns something.

In the real world, countries tend to fall between 0.25 or so on the low (equal) side and 0.6-0.7 on the high (unequal) side. The lower countries tend to be Scandanavian or Eastern European while the highest are often African or Latin American countries.

Let’s take a look at the Gini for team revenue in the Big 4 US sports leagues[1]: NFL, MLB, NBA and NHL.

Why doesn't paying win like it used to?

For a while in the late 1990s it seemed like the final standings in baseball were a foregone conclusion. The relationship between payroll and winning topped out in 1998 and 1999 with a correlation of .68 and 0.71. In 2000 and 2001 the relationship dropped to .32 and .31 as the Moneyball A’s racked up impressive win totals on a low budget. While this is a topic that has been addressed at length, I would like to add one more dimension to the discussion: locking in free agents.

Note that correlations differ somewhat throughout - whether calculated on opening day salary or end of year and all related variations. I am going by Dave Studeman's via the next link below.

Free agency only arrived in baseball in the 1970s so data on this phenomenon are already constrained. The collusion of the owners in the late 80s further clouded the issue and drove correlation down to extremely low levels. Dave Studeman at Hardball Times has the explanation:

In the first few years of free agency—the latter half of the 1970s—teams did take advantage of new opportunities by signing top talent to big bucks. It's no coincidence that this period coincided with the Steinbrenner Yankees' return to glory and the introduction of two bottom-dwelling, low-pay expansion teams (the Mariners and Blue Jays). These developments exacerbated the differences between the have's and have-not's.

Beginning around 1980, however, the picture changed as young, lower-paid talent began to make an impact on the pennant races. Players such as Eddie Murray and Cal Ripken in Baltimore, Rickey Henderson in Oakland and George Brett in Kansas City changed their team's fortunes before changing their payrolls. The Mets developed a gaggle of phenomenal, "cheap" young talent. This influx of top young talent helped change the picture in the early part of the decade. At the same time, bad contracts started appearing. The Angels became the first team known for its bloated, underperforming contracts.

Something else happened in the 1980s: collusion. In 1985, 1986 and 1987, free agents such as Andre Dawson, Tim Raines, Jack Morris and many others found no market for their services. It turns out that commissioner Peter Ueberroth had convinced major league owners that they should work together to refuse expensive, long-term contracts. The owners reportedly established standards of no more than three years for position players and two years for pitchers. As a result, average payroll actually declined in 1987.

The impact on the economics of winning was stark, and the correlation between wins and payroll reached two of their lowest points in 1986 and 1987 (0.17 and 0.15, respectively). Money was losing its power and competitive balance seemed possible. Trouble was, this was illegal. In three different cases, arbitrators ruled that the owners had colluded and eventually ordered them to pay damages.

What changed?

What I’m interest in is why the post-collusion, post-strike period of high correlation broke down. After the A’s (and others) pushed correlations down in 2000 and 2001, they popped back up into the 0.50 range by 2004 and hovered there for a while. The last several years, however, have drifted down to a 0.18 correlation in 2012.

Talent Markets in Sports – The value of the Franchise tag

The following is adapted and expanded from my exceptionally and exasperatingly long read on the NFL Draft Value Chart - I'm not sure anyone has made it to the end so I am excerpting key parts when I am too lazy to write a new post.

 

After seeing Andrew Brandt (a must-follow on Twitter @adbrandt) refer to an espn.com article he wrote last summer on the Franchise tag, I thought I would dust off a portion of my NFL Draft behemoth and jump on the bandwagon. 

Each of the three major American sports leagues (with apologies to the Raptors and Blue Jays, and hockey) has a particular way of dealing with their markets for talent. By looking at the comparison we can see some of the sources of value for players and owners - and the massive negotiating advantage that is the Franchise tag.

Baseball – Good for veterans

Baseball allocates the first six years of a player’s Major League career to the team. Arbitration means that the player has some leverage to improve their situation – particularly in later years – but they still receive a salary below their market rate during this period. Once they have completed their first six seasons a player is a free agent in the truest sense. Any team can offer him a contract for any amount or length of seasons. The result of this structure is that the Winner’s Curse tends to play out for in-demand free agents and surplus value to the team is not likely to be found in players outside of their arbitration years.

The Winner's Curse

As the funstravaganza (not a real word) that is NFL free agency winds down, I am reminded again that there is a structural reason that free agent contracts in every sport pay more than players are “worth.” It turns out that the reason is slightly more complicated than: The owners are all crazy (though it still explains why Daniel Snyder, in particular, is crazy). The exception to this is the late 1980s in baseball, when owners agreed to simply not offer contracts to free agents from other teams. This actually happened, you can look it up. It bears mentioning that if this were attempted today, the internet would melt.