It’s college football season again, and I am excited about the possibilities. Week one brings an excellent out-of-conference matchup between Alabama and Michigan played at a neutral site in Dallas that, according to preseason polls, features two of the top ten teams in the country. But are Alabama and Michigan really in the top ten, or are they coasting on their considerable reputations?
In evaluating whether teams are properly rated, the first thing that comes to mind is to simply compare preseason and postseason polls. The difference in rankings would show up as a positive (underrated) or negative (overrated) change from preseason to postseason. The weakness with this method is that it treats all spots as the same. A team rated 20th going into the season that finishes 24th would be a -4, just like a team that rated 1st going into the season and finished 5th. To me, the second miss should be more significant because the gap between the 1st and 5th best team in the country should be much larger than that between the 20th and 24th best teams (think of the far right of a bell curve vs. the area closer to the middle). Because of the system that adds weight to movements at the top it will be extremely hard for a team to show up as overrated without being in the top portion of the polls, but that seems like a reasonable proposition as a team starting in the high teens cannot be considered highly rated. Now we just have to think about the best way to systematically account for errors in different ranges of the poll.
In a previous post, I used the average postseason Sagarin points to account for the varying differences between teams from the top (larger gaps) to the bottom of the poll (smaller gaps). While using votes for each team is another option, it relies on the polls themselves – which we are examining for flaws – rather than a reasonably reliable third party. The Sagarin rankings are postseason but, in theory, that’s what the preseason rankings are striving for. Check out the demonstration of the dropoff in points from 1 through 25 displayed nearby (this will help provide context for the numbers in the tables below). Now that we have a methodology, let’s put it to work.