Rating the rater of the raters

A few days ago, I linked to a post that explored the question of what sort of correlation existed between the rating services for high school football players and the players named to college All-American teams. I focused on two things in my post – whether Rivals or Scout proved better at talent evaluation in this context, and the spreadsheet compiled with the article that listed the number of players by school that were named to All-American teams.

What I didn’t address in my post was the larger question of whether there was a correlation between the number of stars awarded to a player and the likely degree of college success. I took a pass on that because the author acknowledged that there were limitations in his research, both in terms of the small sample size (he only looked at 2007) and also because he was unable to find data on groups other than the five star players.

Leave it to Sunday Morning Quarterback to fill in the gaps.

… It’s a shame he didn’t look closer at the Web sites, both of which very clearly do distinguish between the number of five-star, four-star and three-star prospects in any given class, if you add up each level from the sites’ team-by-team breakdowns; subtract those players from the class total, which is also available for each team, and you get the number of players rated two stars and lower. If you do that over the last five years for Rivals, the assumption of a “normal distribution” is fairly destroyed:

  2003 2004 2005 2006 2007 Total % of Total
Total Recruits 2,648 2,644 2,907 2,700 2,812 1,3711 100
Five-Star 32 33 33 37 36 171 1.2
Four-Star 270 239 272 329 340 1,450 10.6
Three-Star 905 607 752 836 911 4,011 29.3
Two-Star or Lower 1,441 1,765 1,850 1,498 1,525 8,079 58.9

In the end, especially if you pull out the punters and kickers from the data (which is justified because it’s clear that those are positions that are poorly researched), it becomes clear that the services do have a clue about their evaluations.

  Number % of Total Odds vs. Rest of Level
Total All-Americans 232 100 1 in 59
Five-Star 19 8.2 1 in 9
Four-Star 53 22.8 1 in 27
Three-Star 81 34.9 1 in 50
Two-Star or Lower 79 34.1 1 in 102

In other words, a typical five-star player is something more than eleven times more likely to see his name appear on an All-American list than a two-star or lower kid.

Again, this just goes to show the difference between recruiting ratings on the macro level versus the micro level. There’s no certainty that any individual kid will live up (or down) to his ranking. But as a rule of thumb, those schools that sign more highly ranked recruiting classes consistently are going to wind up with more talent over the long haul than those that don’t.

Which is why the big schools lay out the big bucks for salaries, recruiting expenses and facilities. You do what you have to do to keep hauling in the big catches.

1 Comment

Filed under Recruiting, The Blogosphere

One response to “Rating the rater of the raters

  1. Chuck

    Excellent read. I’ll suggest that there’s a slight bias that enters the All-American list exactly courtesy of the recruiting rankings. Again, a slight bias.

    If a kid comes in with expectations of greatness and delivers, he’s far more likely to make the list than a sleeper who comes in and performs like an all-american, especially since the 5*s usually land at the higher profile schools.