What follows is sort of a wonky follow-up to the findings I wrote about in this post, but as the author of the comment there was kind enough to email me a detailed breakdown of how he came up with Georgia’s rankings and as I tend to get into that sort of information, I thought I’d share the information with you.
First, here’s his summary of the data:
The first thing that jumps off the page is that Georgia had a ton of talent from 1998-2004 and the former coach was among the worst in country of getting performance out of that talent (see Perform vs NFL Talent, particularly 1998 and 1999. I also see that the aggregate talent level at Georgia has dropped off (from 200+ talent points down to under 150 talent points), but that its talent ranking remained very solid with the exception of 2005 and 2006. I will detail the formulas below so there is full disclosure, but I wanted to point out the 2008 NFL talent calculation (and thus also the Perform vs NFL talent) will not be complete until the 2011 draft. I include it because it is 80% done.Richt has had 3 top 5 seasons under our performance metric and only 1 season outside the top 20…not bad. Also, he has consistently been top 20 in terms of getting performance out of his talent with 3 years in the top 10. As I noted in my post on your site, Richt has not done a good job of selecting or developing recruits that turn into top NFL talent. His recruiting rankings are high (see Rivals Talent), but as I noted in my post on your site, he ranks 45th in terms of converting Rivals talent into NFL talent…9th in the SEC. In other words, he has to garner more Rivals points to achieve the same NFL points than 44 other teams in the country. He is either not picking the right prospects (ranked high by Rivals, but not legit NFL talents) or not developing them. [Emphasis added.]
I lean towards the latter there, but your mileage may vary, obviously.
Next, the methodology behind his analysis:
NFL Talent–To calculate a score for a given team in a given year, we look at the 3 drafts following a season, giving only 50% credit to the third draft. So for a 2006 team, they get 100% points for the 2007 draft, 100% points for the 2008 draft and 50% points for the 2009 draft. This is why data is not complete for the 2008 and 2009 seasons…there are contributing players on those teams that are yet in the NFL draft. The thought process behind this is to most effectively capture the contribution of key players. It is not perfect…it does not capture true frosh contributors who stay 4 years (captures their soph, jr and sr years, but not their frosh year)…but missing that rare occurrence seems better than including all true frosh, few of whom actually contribute. Draft points awarded as follows: 1st Round- 30 pts, 2nd Round- 20 pts, 3rd Round- 13 pts, 4th Round- 9 pts, 5th Round- 5 ts, 6th Round- 2 pts, 7th Round- 1 pt..Rivals Talent–To calculate a Rivals Talent score for a given year we look at the 5 preceeding recruiting class point totals from Rivals and weight them as follows (taking 2009 as an example): 2009-15%, 2008-20%, 2007-32.5%, 2006-25%, 2005-7.5%. You can see why comparing on the NFL talent and the Rivals talent side by side does not work. Below we detail how weight drafts and recruiting classes to measure “conversion” of Rivals talent into NFL talent.Performance–Our performance metric is weighted towards outstanding performance. We believe that elite performances should greatly outweigh middle of the road performance. The base for our calculation is Sagarin, who we believe does a very good job (not perfect) with his computer rankings. By starting with Sagarin, we can capture strength of schedule, margin of victory and overall wins and losses. We then add emphasis to Sagarin for outstanding performance.The performance points are calculated as follows: Sagarin Composite ranking points for the given year PLUS bonus points for top 10 finishes (final Sagarin) PLUS bonus points for BCS performance PLUS bonus points for wins over top 10 and 30 teams (final Sagarin) MINUS points for losses to non-Top 30 teams (final Sagarin). Bonus points for Top 10 finishes are awarded in this manner: 5 points for 1st, 4.5 points for 2nd, 4 points for 3rd…down to 0.5 points for 10th. BCS performance points are awarded in this manner: 10 points for BCS/AP championship, 3 points for loss in BCS championship game, 2.5 points for regular BCS game win and 1 point for BCS game loss. Bonus points for ranked wins are awarded as follows: 0.5 points for each win over top 10 team and 0.25 points for win over top 30 team (not double counting the top 10 wins). 0.25 points are deducted for each loss to a non-top 30 teamPerform vs NFL Talent–This is a bit complicated. First we take the performance data as calculated for each individual year (same as above). Please note that 2008 is only 80% complete. Then we subtract out the Sagarin composite ranking score for the 20th ranked team for each respective year. The idea is to measure the amount of performance above a certain benchmark…we are not really interested in teams with the 50th best talent producing 40th best performance…to truly over perform, the bar needs to be set and we set it at 20th. To the extent that the number (performance-Sagarin #20) is positive, we then divide that number by the NFL talent points to measure to the over performance relative to talent level (we multiply by 100 to make the numbers more manageable). To the extent the number is negative, we multiply that number by the talent points and divide by 100. If you don’t do that, the more talent an underperforming team has (negative number) the smaller the output.Perform vs Rivals Talent–Same as above, but using Rivals instead of NFLConverting Rivals Talent into NFL Talent–While there is no table on this on the pdf, this is the calculation of Georgia ranking 45th so I thought I would include it. This gets a bit complicated too, but appropriate weighting is necessary because a recruiting class is part of multiple NFL drafts and each NFL draft is comprised of multiple recruiting classes. We have assumed that each recruiting class goes to the NFL 20% after 3rd year, 70% after 4th year and 10% after 5th year. Certainly subject to debate, but moving the percentages around on the margin won’t move the needle materially. Based on these percentages we weighted the 2002-2007 recruiting classes (2008-2010 classes not yet draft eligible) and the 2005-2010 NFL drafts (no one from 2002 class was draft eligible until 2005 draft). The rankings reflect the rankings based on these weightings and will not necessarily tie to the un-weighted data elsewhere in the post. The conversion formula takes the number of weighted NLF draft points divided by the weighted number of Rivals points and multiplies by 1000 (to get to an easy number without a bunch of decimals). Basically….”how many NFL draft points produced for each Rival point”
Finally, here’s his chart that lays out the numbers.
University of Georgia NFL Talent Rivals Talent Performance Perform vs NFL Talent Year Points Rank Points Rank Points Rank Points Rank 2001 223 3 82.97 20 0.22 20 2002 245 3 98.22 5 5.99 9 2003 218 6 95.17 5 5.13 9 2004 196 8 86.6 11 2.23 15 2005 98 22 84.82 18 1.54 17 2006 101 19 20.73 7 84.9 17 1.56 17 2007 132 7 20.26 6 95.33 6 9.79 4 2008 137 6 21.28 5 84.81 16 1.55 17 2009 21.62 6 80.08 25
What jumps off the page there is the drop in the NFL talent points beginning in 2005 and 2006. Hmm… I wonder what happened with the program back then. Take a look and let me know what you think.
Many thanks to mb and Heisman Pundit for being kind enough to share the information with me.