Rating the rater of the raters

A few days ago, I linked to a post that explored the question of what sort of correlation existed between the rating services for high school football players and the players named to college All-American teams. I focused on two things in my post – whether Rivals or Scout proved better at talent evaluation in this context, and the spreadsheet compiled with the article that listed the number of players by school that were named to All-American teams.

What I didn’t address in my post was the larger question of whether there was a correlation between the number of stars awarded to a player and the likely degree of college success. I took a pass on that because the author acknowledged that there were limitations in his research, both in terms of the small sample size (he only looked at 2007) and also because he was unable to find data on groups other than the five star players.

Leave it to Sunday Morning Quarterback to fill in the gaps.

… It’s a shame he didn’t look closer at the Web sites, both of which very clearly do distinguish between the number of five-star, four-star and three-star prospects in any given class, if you add up each level from the sites’ team-by-team breakdowns; subtract those players from the class total, which is also available for each team, and you get the number of players rated two stars and lower. If you do that over the last five years for Rivals, the assumption of a “normal distribution” is fairly destroyed:

	2003	2004	2005	2006	2007	Total	% of Total
Total Recruits	2,648	2,644	2,907	2,700	2,812	1,3711	100
Five-Star	32	33	33	37	36	171	1.2
Four-Star	270	239	272	329	340	1,450	10.6
Three-Star	905	607	752	836	911	4,011	29.3
Two-Star or Lower	1,441	1,765	1,850	1,498	1,525	8,079	58.9

In the end, especially if you pull out the punters and kickers from the data (which is justified because it’s clear that those are positions that are poorly researched), it becomes clear that the services do have a clue about their evaluations.

	Number	% of Total	Odds vs. Rest of Level
Total All-Americans	232	100	1 in 59
Five-Star	19	8.2	1 in 9
Four-Star	53	22.8	1 in 27
Three-Star	81	34.9	1 in 50
Two-Star or Lower	79	34.1	1 in 102

In other words, a typical five-star player is something more than eleven times more likely to see his name appear on an All-American list than a two-star or lower kid.

Again, this just goes to show the difference between recruiting ratings on the macro level versus the micro level. There’s no certainty that any individual kid will live up (or down) to his ranking. But as a rule of thumb, those schools that sign more highly ranked recruiting classes consistently are going to wind up with more talent over the long haul than those that don’t.

Which is why the big schools lay out the big bucks for salaries, recruiting expenses and facilities. You do what you have to do to keep hauling in the big catches.