Pages

Thursday, August 30, 2012

Preseason polls are worth exactly what you pay for them


It’s college football season again, and I am excited about the possibilities. Week one brings an excellent out-of-conference matchup between Alabama and Michigan played at a neutral site in Dallas that, according to preseason polls, features two of the top ten teams in the country. But are Alabama and Michigan really in the top ten, or are they coasting on their considerable reputations?


Methodology

In evaluating whether teams are properly rated, the first thing that comes to mind is to simply compare preseason and postseason polls. The difference in rankings would show up as a positive (underrated) or negative (overrated) change from preseason to postseason. The weakness with this method is that it treats all spots as the same. A team rated 20th going into the season that finishes 24th would be a -4, just like a team that rated 1st going into the season and finished 5th. To me, the second miss should be more significant because the gap between the 1st and 5th best team in the country should be much larger than that between the 20th and 24th best teams (think of the far right of a bell curve vs. the area closer to the middle). Because of the system that adds weight to movements at the top it will be extremely hard for a team to show up as overrated without being in the top portion of the polls, but that seems like a reasonable proposition as a team starting in the high teens cannot be considered highly rated. Now we just have to think about the best way to systematically account for errors in different ranges of the poll.

In a previous post, I used the average postseason Sagarin points to account for the varying differences between teams from the top (larger gaps) to the bottom of the poll (smaller gaps). While using votes for each team is another option, it relies on the polls themselves – which we are examining for flaws – rather than a reasonably reliable third party. The Sagarin rankings are postseason but, in theory, that’s what the preseason rankings are striving for. Check out the demonstration of the dropoff in points from 1 through 25 displayed nearby (this will help provide context for the numbers in the tables below). Now that we have a methodology, let’s put it to work.


1950s

Most Underrated
Most Overrated
Syracuse (39)
Under: 4 seasons
Equal: none
Over: none
Notre Dame (-55)
Under: 2 seasons
Equal: none
Over: 8 seasons
Mississippi (25)
Under: 5 seasons
Equal: none
Over: 2 seasons
SMU (-24)
Under: none
Equal: none
Over: 4 seasons
Clemson (18)
Under: 5 seasons
Equal: none
Over: none
Ohio State (-24)
Under: 3 seasons
Equal: none
Over: 7 seasons
Iowa (18)
Under: 4 seasons
Equal: none
Over: 3 seasons
Michigan (-23)
Under: 3 seasons
Equal: none
Over: 4 seasons
Auburn (16)
Under: 5 seasons
Equal: none
Over: 1 season
Oklahoma (-18)
Under: 3 seasons
Equal: 2 seasons
Over: 5 seasons

Notre Dame stands out as being hugely overrated in this decade. The 5.5 Sagarin points per year could represent the difference between unranked and 13th, 10th and 4th or 1st and 3rd (it’s a sliding scale!). The pattern that emerges from the two sides of the table is that second- or third-tier teams fill up the underrated side while the overrated teams are almost exclusively top-tier.

1960s

Most Underrated
Most Overrated
Arkansas (35)
Under: 6 seasons
Equal: none
Over: 3 seasons
Syracuse (-30)
Under: none
Equal: none
Over: 5 seasons
Missouri (33)
Under: 4 seasons
Equal: none
Over: none
Texas (-26)
Under: 4 seasons
Equal: none
Over: 5 seasons
Minnesota (30)
Under: 3 seasons
Equal: none
Over: 2 seasons
Oklahoma (-23)
Under: 2 seasons
Equal: none
Over: 5 seasons
LSU (16)
Under: 3 seasons
Equal: 1 season
Over: 2 seasons
Washington (-19)
Under: none
Equal: none
Over: 4 seasons
Navy (15)
Under: 2 seasons
Equal: none
Over: 1 seasons
Ohio State (-17)
Under: 2 seasons
Equal: 1 season
Over: 5 seasons

Two other patterns of note appear in this decade. First, Ohio State and Oklahoma are holdovers. This will continue throughout as they appear in the top five four (OSU) and five (Oklahoma) times, the highest two teams. Second, Syracuse goes from being the most underrated team to being the most overrated team. No other schools go from first to worst, but several make the jump between sides of the table in future decades.

1970s

Most Underrated
Most Overrated
Houston (25)
Under: 5 seasons
Equal: none
Over: 3 seasons
Ohio State (-32)
Under: 1 season
Equal: 2 seasons
Over: 7 seasons
Oklahoma (18)
Under: 4 seasons
Equal: 5 seasons
Over: 1 season
Michigan (-20)
Under: 3 seasons
Equal: none
Over: 7 seasons
Miami (Ohio) (15)
Under: 3 seasons
Equal: none
Over: 1 season
Texas (-19)
Under: 3 seasons
Equal: none
Over: 7 seasons
Pittsburgh (12)
Under: 3 seasons
Equal: none
Over: 3 seasons
Penn State (-18)
Under: 4 seasons
Equal: none
Over: 6 seasons
Toledo (11)
Under: 2 seasons
Equal: none
Over: none
Arkansas (-16)
Under: 3 seasons
Equal: none
Over: 6 seasons

Here, at the height of the Ten Year War between Ohio State and Michigan, we see that those two Big Ten dominating powers may have been a bit overrated. In fact they were ranked 19 of 20 possible times in the preseason (Ohio State in 1979 being the exception, though those Buckeyes finished the season ranked number four) with all 18 of those being in the top ten. They were only underrated in three of the 19 observations. The poll got it exactly right two times and overrated them the other 14 times, two of which saw them finish the season unranked. Arkansas was this decade’s swing from underrated to overrated while Oklahoma went the other way on the strength of two national titles, two second place finishes and three thirds. Remarkably the preseason poll predicted Oklahoma’s final position correctly five times during this decade, a record tied by Florida State in the 1990s.

1980s

Most Underrated
Most Overrated
Miami (Fla.) (63)
Under: 8 seasons
Equal: none
Over: 2 seasons
Oklahoma (-51)
Under: 2 seasons
Equal: 1 season
Over: 7 seasons
BYU (30)
Under: 4 seasons
Equal: none
Over: 3 seasons
Ohio State (-35)
Under: 3 seasons
Equal: 1 season
Over: 5 seasons
Georgia (28)
Under: 5 seasons
Equal: none
Over: 2 seasons
Nebraska (-33)
Under: 2 seasons
Equal: 1 season
Over: 7 seasons
Oklahoma State (18)
Under: 3 seasons
Equal: none
Over: 1 season
USC (-31)
Under: 2 seasons
Equal: none
Over: 7 seasons
West Virginia (17)
Under: 4 seasons
Equal: none
Over: 1 season
Alabama (-30)
Under: 2 seasons
Equal: none
Over: 7 seasons

With all those points headed to Miami, they had to come from somewhere. Those points came from the elite football schools. The top three overrated teams were more overrated than the number one from the 70s and all five were more overrated than the number one from the 60s. BYU exploded on the strength of the 1984 season in which they started unranked and ended up number one, worth 20 points.

1990s

Most Underrated
Most Overrated
Kansas State (29)
Under: 6 seasons
Equal: none
Over: 1 season
Notre Dame (-43)
Under: 1 season
Equal: 1 season
Over: 8 seasons
Virginia Tech (21)
Under: 5 seasons
Equal: none
Over: 1 season
USC (-30)
Under: 1 season
Equal: none
Over: 7 seasons
Washington State (15)
Under: 3 seasons
Equal: none
Over: none
Miami (Fla.) (-21)
Under: 2 seasons
Equal: 1 season
Over: 7 seasons
Wisconsin (13)
Under: 3 seasons
Equal: none
Over: 3 seasons
Texas A&M (-19)
Under: 4 seasons
Equal: 1 season
Over: 5 seasons
Oregon (13)
Under: 3 seasons
Equal: none
Over: 3 seasons
Penn State (-16)
Under: 5 seasons
Equal: none
Over: 5 seasons

The underrateds turned in much lower totals in the 1990s as elite teams generally dominated (the top five teams in overall points were Florida State, Nebraska, Florida, Tennessee and Michigan) after starting high, justifying their preseason rankings. Florida State actually landed precisely in their preseason place five times during the decade. Miami (Fla.) flip flops to the overrated side after having an incredibly strong run in the 1980s.

Since 2000 (12 seasons)

Most Underrated
Most Overrated
Utah (30)
Under: 4 seasons
Equal: none
Over: none
Florida State (-53)
Under: 3 seasons
Equal: none
Over: 9 seasons
TCU (29)
Under: 5 seasons
Equal: 3 seasons
Over: 2 seasons
Oklahoma (-41)
Under: 2 seasons
Equal: 1 season
Over: 9 seasons
Oregon (29)
Under: 7 seasons
Equal: none
Over: 4 seasons
Tennessee (-33)
Under: 3 seasons
Equal: none
Over: 6 seasons
Boise State (26)
Under: 6 seasons
Equal: none
Over: 4 seasons
Nebraska (-32)
Under: 3 seasons
Equal: none
Over: 7 seasons
Auburn (18)
Under: 6 seasons
Equal: none
Over: 4 seasons
Florida (-29)
Under: 3 seasons
Equal: none
Over: 9 seasons

Oregon pulls off the rarely seen two-decades-in-a-row underrated. Florida State, Nebraska and Florida follow up their runs in the 1990s with more up-and-down performances in the 2000s. Oklahoma surprises in 2000 for the title, but ends up falling short the rest of the period: 9 of the next 11 seasons registered as underperformance.

Informed poll watching

If you really want to use the preseason poll (I don’t recommend it) here are some guidelines for making it useful. Take a look at last year’s final ranking for each team. The higher the final ranking from last year, the more likely it is that that team is overrated in the preseason poll. Poll voters have relatively little data to go on, so they go with last year’s results (a problem not limited to college football observers) when the reality is that teams vary much more widely from year to year.


Outcomes (over/underrated) based on prior year final ranking (note: AP only expanded to 25 teams in 1989)

This graph demonstrates the bias towards last year’s teams present in current year preseason polling. Any team that was ranked at the conclusion of the previous season has more than a 50% probability of being overrated. Those that were ranked in the top five to finish the season are more than 70% likely to be overrated in this season’s poll.

If we apply this logic to the current AP preseason poll, we find a lot of opportunities for skepticism. The final top five from last year – Alabama, LSU, Oklahoma State, Oregon and Arkansas – is very highly rated this year – Second, Third, 19th, Fifth and Tenth. The data suggest that only one team is likely to live up to their ranking.

The only cohort that systematically exceeds their preseason ranking is the group of teams who finished the previous season unranked. Watch out for Texas, Ohio State, Florida and Louisville to outperform their rankings. History suggests that three quarters of all teams who were unranked to finish the previous year will finish higher than their preseason rankings.


2012 Preseason
2011 Final
Team
1
6
USC
2
1
Alabama
3
2
LSU
4
16
Oklahoma
5
4
Oregon
6
19
Georgia
7
23
Florida State
8
12
Michigan
9
9
South Carolina
10
5
Arkansas
11
17
West Virginia
12
10
Wisconsin
13
11
Michigan State
14
22
Clemson
15
NR
Texas
16
21
Virginia Tech
17
24
Nebraska
18
NR
Ohio State
19
3
Oklahoma State
20
14
TCU
21
7
Stanford
22
15
Kansas State
23
NR
Florida
24
8
Boise State
25
NR
Louisville

Check out last year’s final and this year’s preseason to find other potentially overrated teams and get a leg up on identifying possible surprises.

1 comment:

  1. I?d have to check with you here. Which is not something I usually do! I enjoy reading a post that will make people think. Also, thanks for allowing me to comment!
    visit more:https://houseoffuncoins.com/

    ReplyDelete