Everything you wanted to know about Georgia and talent points but were afraid to ask.

What follows is sort of a wonky follow-up to the findings I wrote about in this post, but as the author of the comment there was kind enough to email me a detailed breakdown of how he came up with Georgia’s rankings and as I tend to get into that sort of information, I thought I’d share the information with you.

First, here’s his summary of the data:

The first thing that jumps off the page is that Georgia had a ton of talent from 1998-2004 and the former coach was among the worst in country of getting performance out of that talent (see Perform vs NFL Talent, particularly 1998 and 1999.  I also see that the aggregate talent level at Georgia has dropped off (from 200+ talent points down to under 150 talent points), but that its talent ranking remained very solid with the exception of 2005 and 2006.  I will detail the formulas below so there is full disclosure, but I wanted to point out the 2008 NFL talent calculation (and thus also the Perform vs NFL talent) will not be complete until the 2011 draft. I include it because it is 80% done.

Richt has had 3 top 5 seasons under our performance metric and only 1 season outside the top 20…not bad.  Also, he has consistently been  top 20 in terms of getting performance out of his talent with 3 years in the top 10.  As I noted in my post on your site, Richt has not done a good job of selecting or developing recruits that turn into top NFL talent. His recruiting rankings are high (see Rivals Talent), but as I noted in my post on your site, he ranks 45th in terms of converting Rivals talent into NFL talent…9th in the SEC. In other words, he has to garner more Rivals points to achieve the same NFL points than 44 other teams in the country.  He is either not picking the right prospects (ranked high by Rivals, but not legit NFL talents) or not developing them. [Emphasis added.]

I lean towards the latter there, but your mileage may vary, obviously.

Next,  the methodology behind his analysis:

NFL Talent–To calculate a score for a given team in a given year, we look at the 3 drafts following a season, giving only 50% credit to the third draft.  So for a 2006 team, they get 100% points for the 2007 draft, 100% points for the 2008 draft and 50% points for the 2009 draft.  This is why data is not complete for the 2008 and 2009 seasons…there are contributing players on those teams that are yet in the NFL draft.  The thought process behind this is to most effectively capture the contribution of key players.  It is not perfect…it does not capture true frosh contributors who stay 4 years (captures their soph, jr and sr years, but not their frosh year)…but missing that rare occurrence seems better than including all true frosh, few of whom actually contribute.   Draft points awarded as follows: 1st Round- 30 pts, 2nd Round-  20 pts, 3rd Round- 13 pts, 4th Round- 9 pts, 5th Round- 5 ts, 6th Round- 2 pts,  7th Round- 1 pt..
Rivals Talent–To calculate a Rivals Talent score for a given year we look at the 5 preceeding recruiting class point totals from Rivals and weight them as follows (taking 2009 as an example):  2009-15%, 2008-20%, 2007-32.5%, 2006-25%, 2005-7.5%. You can see why comparing on the NFL talent and the Rivals talent side by side does not work.  Below we detail how weight drafts and recruiting classes to measure “conversion” of Rivals talent into NFL talent.
Performance–Our performance metric is weighted towards outstanding performance.  We believe that elite performances should greatly outweigh middle of the road performance. The base for our calculation is Sagarin, who we believe does a very good job (not perfect) with his computer rankings. By starting with Sagarin, we can capture strength of schedule, margin of victory and overall wins and losses.  We then add emphasis to Sagarin for outstanding performance.The performance points are calculated as follows:  Sagarin Composite ranking points for the given year PLUS bonus points for top 10 finishes (final Sagarin) PLUS bonus points for BCS performance PLUS bonus points for wins over top 10 and 30 teams (final Sagarin) MINUS points for losses to non-Top 30 teams (final Sagarin). Bonus points for Top 10 finishes are awarded in this manner: 5 points for 1st, 4.5 points for 2nd, 4 points for 3rd…down to 0.5 points for 10th.  BCS performance points are awarded in this manner: 10 points for BCS/AP championship, 3 points for loss in BCS championship game, 2.5 points for regular BCS game win and 1 point for BCS game loss.  Bonus points for ranked wins are awarded as follows: 0.5 points for each win over top 10 team and 0.25 points for win over top 30 team (not double counting the top 10 wins).  0.25 points are deducted for each loss to a non-top 30 team
Perform vs NFL Talent–This is  a bit complicated. First we take the performance data as calculated for each individual year (same as above).  Please note that 2008 is only 80% complete.  Then we subtract out the Sagarin composite ranking score for the 20th ranked team for each respective year. The idea is to measure the amount of performance above a certain benchmark…we are not really interested in teams with the 50th best talent producing 40th best performance…to truly over perform, the bar needs to be set and we set it at 20th.  To the extent that the number (performance-Sagarin #20) is positive, we then divide that number by the NFL talent points to measure to the over performance relative to talent level (we multiply by 100 to make the numbers more manageable). To the extent the number is negative, we multiply that number by the talent points and divide by 100.  If you don’t do that, the more talent an underperforming team has (negative number) the smaller the output.
Perform vs Rivals Talent–Same as above, but using Rivals instead of NFL
Converting Rivals Talent into NFL Talent–While there is no table on this on the pdf, this is the calculation of Georgia ranking 45th so I thought I would include it.  This gets a bit complicated too, but appropriate weighting is necessary because a recruiting class is part of multiple NFL drafts and each NFL draft is comprised of multiple recruiting classes.  We have assumed that each recruiting class goes to the NFL 20% after 3rd year, 70% after 4th year and 10% after 5th year. Certainly subject to debate, but moving the percentages around on the margin won’t move the needle materially.  Based on these percentages we weighted the 2002-2007 recruiting classes (2008-2010 classes not yet draft eligible) and the 2005-2010 NFL drafts (no one from 2002 class was draft eligible until 2005 draft). The rankings reflect the rankings based on these weightings and will not necessarily tie to the un-weighted data elsewhere in the post.  The conversion formula takes the number of weighted NLF draft points divided by the weighted number of Rivals points and multiplies by 1000 (to get to an easy number without a bunch of decimals).  Basically….”how many NFL draft points produced for each Rival point”

Finally, here’s his chart that lays out the numbers.

University of Georgia
NFL Talent Rivals Talent Performance Perform vs NFL Talent
Year Points Rank Points Rank Points Rank Points Rank
2001 223 3 82.97 20 0.22 20
2002 245 3 98.22 5 5.99 9
2003 218 6 95.17 5 5.13 9
2004 196 8 86.6 11 2.23 15
2005 98 22 84.82 18 1.54 17
2006 101 19 20.73 7 84.9 17 1.56 17
2007 132 7 20.26 6 95.33 6 9.79 4
2008 137 6 21.28 5 84.81 16 1.55 17
2009 21.62 6 80.08 25

What jumps off the page there is the drop in the NFL talent points beginning in 2005 and 2006.  Hmm… I wonder what happened with the program back then.  Take a look and let me know what you think.

Many thanks to mb and Heisman Pundit for being kind enough to share the information with me.

23 Comments

Filed under Georgia Football

23 responses to “Everything you wanted to know about Georgia and talent points but were afraid to ask.

  1. 69Dawg

    WGAS just win the games.

    Like

    • Chuck

      Exactly. WTH are we supposed to do? Besides the axiom that there a million ways to jigger stats around – notice how many time makes assumptions and extrapolations – and the injury factor noted below, you have to wonder what his agenda is. He admits: “Richt has had 3 top 5 seasons under our performance metric and only 1 season outside the top 20…not bad.” Then, he goes on to point out what he sees as failures on Richt’s part. I don’t see it that way. IIRC, we just bested everyone but Vanderbilt in the academic progress measure, AND we still games, even last year. Stats are only so useful before you start to get tied up in them.

      Like

  2. dudetheplayer

    One of the things that doesn’t seem to be taken into consideration here is the effect injuries can have on player development and draft position. For example, a guy like Jeff Owens (or Trinton Sturdivant) seemed to be elite talents who were on the fast track to 1st round status until they blew out their knees.

    The injuries in 2008 in particular I believe did a number on the performance and development of quite a few guys on our team. Add in poor player evaluation in positioning certain guys (Brandon Miller, Tripp, Samuel, etc.) and a couple of really poor recruiting years (’05, ’06), and this all makes quite a bit of sense to me.

    Like

  3. JasonC

    The link for the chart doesn’t work for me. Anyone else have this problem?

    Like

  4. JC in Powder Springs

    O-line recruiting was pretty bad under Callaway, but has turned around with Searels. So I’d expect the number of NFL contributors to pick up in this area.

    Seems like the recruiting services elevate just about every Dawg recruit to 4 star, so how much weight should one give this aspect of the research? CMR has averaged 10 wins per year – so it’s tough to argue he isn’t getting production out of players. If there’s a problem coaching players up, I’d assign it mostly to penalties, poor tackling, dumb mistakes (i.e. directional kicking), etc. I’d say the Dawgs recruiting has generally been overrated by recruiting services, and CMR has done well as a coach but not so well as a recruiter.

    Like

  5. Mayor of Dawgtown

    The object of recruiting and coaching is to win games and, hopefully, championships–not to amass the largest group of NFL prospects nor to prepare them for the NFL. Football is a team sport. How the pieces mesh is vital to success. If you want a coach who got it backwards–lots of players who ended up in the NFL but who did not win championships–see Donnan, Jim.

    Like

    • dudetheplayer

      That is pointed out in the very first sentence of the quoted piece:

      “The first thing that jumps off the page is that Georgia had a ton of talent from 1998-2004 and the former coach was among the worst in country of getting performance out of that talent (see Perform vs NFL Talent, particularly 1998 and 1999.”

      Like

      • Mayor of Dawgtown

        Exactly. This guy is arguing inferentially that Richt has been not successful because he is sending fewer players to the NFL even though he has won the SEC twice, been to the SECCG 3 times and has had his team consistently highly ranked (2007 #2 in the nation). At the same time the poster is badmouthing (without naming him) Donnan for doing exactly what the poster says is good (ie. collecting star recruits and sending lots of them to the NFL).

        Like

        • That’s not what he’s arguing, Mayor. He’s looking for an explanation (or at least a correlation) as to why the program has underachieved in relation to its recruiting success over the past four or five seasons.

          I ask this question again: who’s the last Georgia defensive player to get drafted by an NFL team in either of the first two rounds?

          Like

          • Dave

            Van Gorder left obv.

            Like

          • JC in Powder Springs

            Was it Sullivan? But Pollack, Davis & Thurman were high picks. Without looking it up, I don’t believe those 3 were highly rated by the recruiting services. The question is – was it BVG’s coaching AND D-recruiting that led to the Dawgs success on the field and in the NFL draft? Both declined significantly after BVG left.

            Considering the results here’s the next question – is Garner really as good a recruiter as billed?

            Like

    • That’s correct as far as it goes. But I would argue that a byproduct of an SEC program on a successful run is the generation of a steady, significant amount of NFL-draft level talent. That hasn’t been the case as of late, especially on the defensive side of the ball since VanGorder left.

      Like

  6. Hobnail_Boot

    OLine, Andy Bailey.

    Like

  7. negative ?

    There isn’t any question that in the last 5 years of Coach Richt’s talent here, that it has not measured up to the talent left here by the former Coach.

    You can sit there and squirm in your seat to make up excuses for this, all you want. What you are left with is, that we haven’t had the talent the last 5 years here that we had in the 1st five years of the Coach Richt Era.

    On Defense, we have turned that around with recruiting both last Defensive Recruiting Class 2010 and next year’s Defensive Recruiting Class 2011.

    On Offense, we have done the exact opposite, falling off the map in Offensive Recruiting Class 2010 and again for 2011, especially when you factor in that what Michael Grant did and what Christian LeMay did, were both 1 and the same and UGA Admissions would not let in Michael Grant – so it is not like they can play favorites for us and let in LeMay for the same offense on the same grounds.

    As for this crap about 10 wins a season B.S., we play 14-game seasons – averaging 10 wins a season has brought us to 38-14 the last 4 years under scrutiny here with this up-coming Crucial Coach Richt Season with 15 seniors and 4 or 5 juniors leaving after this Crucial Season for Coach Richt.

    38-14 is the # 19 team in won / loss record over the last 4 years – since after 2005 season.

    # 11 All-Time in 1-A wins UGA should NOT be # 19 the last 4 years.

    10 wins in 14-game seasons is # 19 the last 4 years

    Something has to be done to turn around the Offensive Recruiting Classes of last year 2010 and next year 2011 – especially in this talent-rich state.

    Offensive Coaching Staff is not doing it. Todd Grantham is. When are you going to bother to fix the URL Link ?

    Like

    • The Realist

      The last line made me chuckle.

      I’m not sure what exactly you want to do to turn around the “Offensive Recruiting Class of last year 2010.” Isn’t that sort of over and done with?

      Too, Grantham hasn’t really done anything yet. Meanwhile, Georgia’s offense was one of the most productive in the SEC last year. Their shortcomings were not on the offense — aside from hemorrhaging turnovers — but were special teams (kickoff coverage) lapses and mind-boggling defensive decisions on personnel and coverages.

      You list a lot of complaints, but you offer no solutions. This is what we call “bitching.” You have nothing discernibly constructive to say, and you just want to whine and bitch and moan until someone, anyone stops ignoring you.

      If you feel the “offensive coaching staff” is incompetent, then lay out a statistical analysis exhibiting such, and offer alternatives. Is there some great, young offensive coordinator that just touches you in the right spot? Then, throw out his name and start the campaign to get him on staff. It won’t work, but at least that will be a positive, constructive response instead of bitching like a bitchy bitch.

      Like

  8. Dave

    A few things this analysis leaves out. #1: Injuries. #2: Star players leaving early. UGA has never kept its stars for four years. Most schools don’t, but a few do (see Fla 2009, where many could have leaped after 2008). At UGA, under Richt, almost always, the best players have left after 3 years, even if it was for a mid-round draft position.

    And yes, I agree with the Senator, a big change happened after 04 and I would bet these stats are correlated to some degree.

    Like

  9. mb

    Couple of things:

    1). I have no agenda with Richt, Donnan or Georgia football. I am a die hard fan of college football and follow another program (not in the SEC or the South). I put this together because the Senator expressed some interest in the analysis.

    2). Some of you are mis understanding the use of NFL draft points as a measure of college talent. I ONLY care about college football, but the best way to determine talent on a college team is by looking at the nfl draft. Sure there are some great college football players that don’t translate to the NFL draft, but that is the exception not the rule. The question with Georgia is why isn’t the Dawgs great recruiting translating to great results on the field. That is what I am looking at. Injuries are certainly part of the equation. The idea is not to amass nfl talent and nothing with it (see Florida state in mid 2000s), the idea is to recruit great talent, develop that talent and win games. The by product of that is your players having success in the. Nfl draft. The other way to do it is to develop a system that works in college football without much recruiting or nfl talent (see boise, utah, west virginia)

    3). There clearly was a drop off in Georgia’s talent mid decade. It has rebounded nicely, but is still nowhere near what it was in the late 90s and early part of the decade

    Hope you found the analysis helpful and best of luck this year. Georgia is my favorite team in the SEC.

    Like

    • Some good points there, mb. One thing I’d add – when you say “The other way to do it is to develop a system that works in college football without much recruiting or nfl talent (see boise, utah, west virginia)”, I’m not sure how viable an approach that is over the long haul in a talent-rich environment like the SEC.

      Like

    • Mayor of Dawgtown

      The idea is to get players that help you win games/championships not to see how many draft choices in a future NFL draft you can acquire. That by itself achieves nothing. Winning is the object. I do not necessarily agree that the”by-product of that is your players having success in the NFL draft.” Just ask Eric Crouch and Charlie Ward, to name only two. There are a host of others. Getting as many top-notch players as possible without figuring out to how the players would fit together has been tried many times in many sports without success (e.g. New York Mets circa 1990s). While the post may be an interesting academic exercise to some, its premise is fatally flawed. Sorry.

      Like

      • mb

        Mayor—Thanks for the response. I think you are missing my point. Of course winning is the object….we have no disagreement there. The point is NOT to acquire NFL talent for the sake of acquiring NFL talent…the point is to pull together the best possible talent for COLLEGE football.

        The vast, vast majority of supremely talented college players play in the NFL…there are of course exceptions to that rule. Those few players that meet are great in college, but NFL caliber (such as Charlie Ward) do not move the needle much in the calculation.

        Take Ward as an example. That 1993 Florida St team had 244 talent points…MASSIVE TALENT. If you add in Ward as a 1st rounder (to equate to his college impact), it would take FSU to 274 points. Point is FSU was incredibly talented and it showed up in the numbers, even without Ward. That is why looking at the NFL draft is a very accurate measure for how talented a particular college team is. The analysis is not perfect (there is no perfect way to evaluate college talent IMO), but it does shed some light on how talent impacts performance in college football…IMO.

        Let’s look at Georgia…..from 1998-2004, Georgia was ~200 talent points or above 5 times and never below 160 talent points. Since then, Georgia has yet to top 150 talent points (although the 2008 team has a shot to do based on the 2011 draft). Now put that in context with what it takes to get to the BCS Title Game (winning in college football).

        From 1998-2007 (full data available), 87% of the BCS Title Game participants have had over 160 talent points. The 4 teams that did not have 160 talent points were LSU 2007 (2 losses in regular season, 147 points), Virginia Tech 1999 (Michael Vick system, 128 points), Nebraska 2001 (option attack, 82 points) and Oklahoma 2000 (amazing coaching job, 79 points). As I said in the original post, it it possible to achieve good success in CFB without elite talent if you have a system designed around that (this is is what Virginia Tech 1999 and Nebraska 2001 had)…but it is hard to sustain that model. Outside of that we are left looking at LSU (hard to get to the title game with 2 losses) and just a great story in Oklahoma.

        So with Georgia’s talent level now consistently below 160 points, it just makes it that much harder to get to the title game….not impossible, but really, really hard. Even harder when you consider that ~75% of the title game participants have over 200 talent points. That is why the data shows me that Richt is not recruiting or developing enough talent. JMO.

        Like

    • mb

      I agree that it is nearly impossible to win consistently in the SEC without real talent. No doubt.

      Like