A fix for the Coaches’ Poll?

Whatever you do after you read this post, do not ask me in the comments section how I got here. Because I have no clue how I stumbled on this.

That being said, I think I may have figured out a way to reduce significantly the bias/conflict of interest problems that plague the Coaches’ Poll.

To start with, I’m forced to get a little esoteric with you. I’m talking “mechanism design” economics theory here. That’s a theory that tries to bring back certain efficiencies to a market (such as insurance, where people may seek coverage without disclosing risks) where parties don’t have equal levels of knowledge about a matter. How do you find a way to share that information efficiently?

What intrigued me about that linked article was this little tidbit – Mechanism design has also been used to refine voting procedures in NCAA football rankings…” I googled the hell out of that fragment, but couldn’t find anything direct on it. But I did find an interview with Roger Myerson, one of the three economists who won a Nobel Prize for their work in this area, who had this to say about applying the concepts of mechanism design to voting:

… If there are 10 candidates on the ballot running, say, in a Democratic primary, you can vote for several of them, but you can’t give more than one vote to any one of them. And the person who gets the most votes, most approval votes, was approved by the largest majority, gets the bonus of winning the votes of the state.

And with that, the light bulb went off over my head. Instead of letting each coach rank in order the top 25 schools and then add up the ranked totals, why not simply have them each turn in his list of the ten best schools in the country – unranked? The teams would then be ranked in order of those which received the most votes. With 600 some odd votes for teams, you’d still be able to construct a top 25 fairly easily, I would think. (And if not, you could always let the coaches vote on a bigger group, such as a top twelve or fifteen.)

The beauty of this approach is that it minimizes, if not eliminates, the effect of any individual coach’s bias or conflict. Go back and look at Tony Barnhart’s analysis of the final regular season coaches’ poll of 2007. Remember stuff like this?

… Stoops, understandably, had his team ranked No. 1 after beating Missouri twice this season. He had three Big 12 teams in his top four (Oklahoma, Missouri, Kansas). He had LSU at No. 6.

… Somebody at Georgia made Wyoming’s Joe Glenn mad at some point. He had Georgia No. 10 on his ballot.

… You got to give the Big Ten coaches credit for one thing. They all stick together. Seven of them vote in the poll and they all had Ohio State No. 1, even Michigan’s Lloyd Carr, who has announced his retirement.

… Hal Mumme of New Mexico State continues to be our resident contrarian in the coaching fraternity. He had undefeated Hawaii No. 1 on his ballot.

With approval voting, none of those individual selections would impact the final poll, because none of the coaches would be able to vote like that.

Also, think about how the psychology of voting changes with this approach. Once a coach knows he can’t game the system with an absurdly high vote for a school (maybe to enhance the chances for a BCS crasher like Hawai’i or to improve a school’s strength of schedule by lifting other schools’ rankings), it forces him to face his vote in a more rational context. In other words, unless enough of his peers agree with his evaluation of a particular school’s merit, his vote won’t have sufficient impact. That should reduce his incentive to rig the voting.

One thing you’d have to be careful about would be to make sure that one conference wasn’t overrepresented in the voting (see Barnhart’s notes about the seven SEC coaches’ voting, and, more particularly, about the seven Big Ten coaches’ voting).

Another benefit from this approach is that I have to believe it’s much easier for a head coach to provide an honest assessment of the ten best teams in the country than to sit down and try to rank the top 25 in order. That should also increase the accuracy of the results in that they’re more likely to come directly from the coaches than from a surrogate.

I’d be interested in anyone’s thoughts on this. Have I missed something obvious?

26 Comments

Filed under College Football

26 responses to “A fix for the Coaches’ Poll?

  1. Spence

    I think as long as they have to publish their list to the public then your method really could do a lot to eliminate the nonsensical voting we see every year. The only thing a coach could try to do is, as you suggest, stack a conference, but if the list is published then he’ll have to explain why Kentucky (for example) is more deserving than an OSU or USC or whomever was left off.

    I think there’s some real merit in this, though I wonder if a team like Hawaii, whom everyone last year would have certainly put in their pre-Sugar top 5 but almost never at #1 would reap extra-ordinary benefits. In other words, in that scenario there’s almost no way that UH wouldn’t a consensus #1 (using your method) while almost nobody really thought they should be the best team in the country. The effect would be to take the Boise States of the world and turn them into consensus #1′s or force coaches to leave them out of the top 15 altogether.

    The only thing I could think that could solve this would be to ask coaches to vote for a #1 team within the list, and to be voted #1 from the pooling mechanism you’d have to have some percentage of #1 votes. Or perhaps you could do this method and the old fashioned method of ranking all of them and weigh them together somehow.

    That’s the only thing I can think of.

  2. I think you have to be careful not to make the coaches vote for too many teams. You want the ballot list to be exclusive enough so that it’s sufficient to differentiate between 25 schools.

    That being said, I went and looked at the final coaches’ ballots with regard to your point about Hawai’i. Hawai’i received seven votes placing it fifth or higher, while eighteen coaches had it outside the top ten.

    By comparison, Georgia had no votes outside the top ten and forty ranking it in the top five.

    So I’m not sure your concern would play out, because ordinarily, most coaches voting rationally would have more concerns about Hawai’i than they would about Georgia being one of the ten best.

    I suspect that where more of the hue and cry will come from will be over a school like Ohio State, which will probably appear on every top ten ballot, even though every SEC partisan at this point thinks the Buckeyes belong in the Capital One Bowl, at best. ;)

  3. scdawg

    Let’s take last year as the example. I think most people thought USC and LSU were #1 and #2. Even those who thought other teams were better certainly had LSU and USC in the top ten, right?

    How does it work to figure out who is #1 or 2 when all the coaches vote several of the same teams in the top 10, 25, whatever. I think this would work well for figuring out who the best 25 teams are, particularly 7-25, but not as well to figure out who are the top 6–I think everyone has them in the top 25. Am I missing something?

  4. Good question. Maybe what you have to do is call for two rounds of ballots from each coach, consisting of their top five and their top ten, for example, so that you generate some differentiation in the overall vote on the top ten and top 25.

  5. BTW, I went back and checked the ’07 ballots. Only five teams were listed in the top ten on all of the coaches’ ballots.

    Southern Cal wasn’t one of them.

  6. Andrew

    Great idea Senator, but scdawg has a point. If you have the two round voting that you called for, if there were clearly three top teams in the nation, their number of votes would not be different by much. But you could still get the same bias if the teams are close enough so that the number of votes is within five. I feel like I’m playing devil’s advocate, though, this idea is better than what we’re using now. Great Post

  7. Spence

    Thanks for finding out the info about Hawaii. I think it proved that your system may be better and that the collective wisdom of the coaches is a better measure.

    That being said, I’d be curious to know how Boise State was voted prior to their magical bowl game and afterwards, and how your system would change those results. Or perhaps BYU when Darth Meyer ran them.

    Agree, great post.

  8. Keep in mind as you parse this that I’m not proposing to do away with the other components of the BCS calculations.

    In other words, even if there’s a four way tie at the top of the coaches’ poll, there’s still the Harris Poll and the computers left to factor into the equation.

  9. Dawg93

    First time poster, long-time reader here . . . .

    An interesting idea that probably could be tweaked into something that would be better than the current system. For instance, why don’t ALL coaches vote every year? Is it really that much of a burden for them to turn in a ballot each week? I would think not if we go to a system similar to this one, where you don’t really have to think TOO hard about your top 5, 10, or 15. I would think the more coaches that vote, the better the quality of the top 25 that results from that vote.

  10. Spence

    Understood, though that would be interesting to apply to the AP poll as well.

  11. hoodawg

    Senator, could you solve the “SCDawg problem” by having the coaches rank the teams 1-10, but treat all top-10 votes equally except to break ties? In other words, if you did have a broad consensus that three teams all deserve to be in the top 3, and thus those got equal number of “approval votes,” you then look deeper to see where those individual teams were ranked on the ballots. You wouldn’t avoid the bias issues entirely, but you’d minimize the cases in which they become relevant.

    Of course, you could just get over it if there were ties, as well, and allow the other elements of the BCS process to work them out. If three teams tied at the top of the coaches rankings and the computers or Harris polls sorted it out, I think you just get comfortable with that.

    Great post.

  12. JasonC

    So did Hal Mumme have the Dogs ranked #1 after the Sugar Bowl?

  13. NM

    Hey, we talked about approval voting in my poli sci class last fall! It’s a good idea but every method can be gamed…

    Can the coach vote for fewer than 10? If so, that’s an easy way to game the system. If Stoops, say, votes for OU and only OU, then it’s likely OU would be #1 with X votes (unanimously approved) but all the other good teams would be tied at 2 with X-1 votes. So then everyone starts voting for just one team and it becomes like the presidential election.

    Another way to game it is this: Let’s take 2004 as an example. Obviously, everyone put OU, USC, and Auburn in their top 10. But let’s say Coach Tub leaves out the Trojans and the Sooners. He could still fill out a complete top 10, but just include the actual #11 and #12 teams in his ballot instead. All of a sudden, guess who’s #1. Of course, if you make ballots public, that’s unlikely to happen, but it might be worth it. Or, at the very least, Tubbs convinces a friend to leave out one or the other and claim “weak Pac10 schedule” or something.

    There’s potential here, though and the Coaches Poll definitely needs reforming.

  14. Can the coach vote for fewer than 10?

    I’d already thought of that. The answer is no.

    … Of course, if you make ballots public, that’s unlikely to happen, but it might be worth it. Or, at the very least, Tubbs convinces a friend to leave out one or the other and claim “weak Pac10 schedule” or something.

    Given what’s at stake, I’d want things set up so that a coach caught trying to game the results would suffer – at least by having his ballot nullified.

    Also, don’t forget the other components of the BCS calculations would still be in play.

  15. Great post Senator – the ultimate answer is to combine this with something I mentioned a while back. No team should be allowed into the BCS champ game without a conference championship game. In essence, if the big east, big 10, and pac 10 expanded to include 12 and a championship game, we would have 60 BCS schools and a perfect voting allotment for your scheme. I’ve long said 60 or so all we need in D- 1A anyways. If that is too little, then combine some of the mid-majors and have a go with 7 conferences and 72. This would be sooo much better than any playoff imo.

  16. peacedog

    “Have I missed something obvious?”

    You mean besides the coaches poll being an absurdly bad idea in the first place?
    ;)

  17. peacedog

    My only other comment would simply regard the # of teams to vote for. I’ve seen structures liked this used elsewhere. I wonder, is there a sweet spot with the # of votes casts versus the # of desired results (in this case 25)?

  18. I thought about that, too.

    I guess it comes down to what you want the coaches’ poll for. If it’s strictly utilitarian, to fill in one-third of the BCS calculations, than a top 10 ballot is all you need. But if you’re looking to create a top 25 ballot by approval voting, you probably need to have the coaches cast their votes for 15-18 schools.

    Personally, I’m leaning towards the utilitarian approach. There’s no magic about a top 25 – in fact, it used to be a top 20 poll until fairly recently. And the smaller the number of schools the coaches have to validate, the more I trust their judgment.

  19. Pingback: Rashida Jones’ Goodies « Third Saturday in Blogtober

  20. Thomas Brown

    You disagree with your own “fix” for why the Pac-10 ended up with only 1 team in the Top 15 of every poll, while The SEC cheated to have 5 of the Top 15 by direct comparison.

    If you take the top 10 and add them up, you still have the same top 10 and have not determined who the 1 team is who should be given the chance to play against Number 1 to be the MNC.

    So, you say do it again.

    Take the top 5 and add them up.

    Geez Louise.

    You still end up with the same top 5.

    And, only get to pick 1 to play Number 1.

    The fix is not to throw the baby out with the bath water, but to have Plus One +1.

    Give someone else a chance besides 1-team conferences like the Pac-10 and Big 10 and instead of 2 team conferences like the Big XII.

    And, have them actually play a top team in the nation to remain at the top of the rankings for the choice of which 2 square off for the MNC.

    Plus One +1 fixes everything.

    Your solution to the meaningless USA Today Coaches’ Poll, you agree yourself, is wrong.

  21. Thomas, this isn’t a playoff proposal.

    All I’m trying to do here is come up with a vehicle to reduce the bias inherent in the coaches’ vote. That bias is a problem as long as the coaches’ poll is part of the BCS math – whether that poll is used to determine the two best schools, four best or whatever. So, no, Plus One doesn’t fix everything.

  22. Thomas Brown

    Your concept was to have the top 10 teams voted on, disregard where they are ranked (Number 1 is equal to Number 10, which is absurd.), and total the total number of votes for that team.

    You still have the same 10 teams.

    So, you disagree with your own concept and say : Well, then make it 2 rounds of this voting and the 2nd round will be only the top 5 teams.

    You still have the same 5 teams now.

    So, you disagree with the whole entire concept of your solution for the USA Today Coaches’ Poll and say well then at this point (with the top 10 teams still being the same top 10 teams, and with the top 5 teams still being the same top 5 teams), oh ok go ahead and see where they are ranked.

    You said Bluto to not allow them to rank them. Are you proposing now a 3rd round of votes where now the coaches are asked to turn in their actual rankings ? You said they could not rank the top 10, just list the teams with no order.

    You have gone full circle in your presented solution back to where the coaches send in their votes for the top teams and rank them.

    Jeez Louise.

    The object of all this is to determine the top 2 teams that then will play each other in the BCS Mythical National Championship Game, which is a playoff game of only 2 deserving teams.

    That’s a playoff Bluto.

    If there is some bias in the crappy polls which is all there is in the BCS current system any longer, you still have the bias with your solution.

    Why not accept that there is bias in every single one of the BCS included polls ?

    Take the top 4.

    1 vs. 4.
    2 vs. 3.
    Winner of 1 vs. 4 against Winner of 2 vs. 3.

    It is very clear to everyone in the season just ended, that Ohio State would have beat no team in the top teams if it was the top 4.

    Just as it is very clear to everyone that Southern California too, did not deserve to be included in the top 4 before the MNC Game.

    Give someone else a chance besides Oklahoma, who like Ohio State and Southern California, plays cupcake SoS every year.

    1 vs. 4 2 vs. 3 winners to play each other fixes it.

    It is just your opinion that Plus One +1 does not fix the bias.

    Sure it does. And, all of this is a discussion of a playoff; don’t delude yourself when you disagree with your own solution 3 times in this one thread.

  23. Thomas, I’m not sure I understand your argument here.

    The coaches would turn in one ballot of their top ten teams. On that same ballot, they would also designate the five best of those ten. The latter vote would only come into play if there were a tie involving any of the top five teams after the approval votes were tallied. If there were no tie, there would be no reason to count them.

    I have no idea what you’re talking about with regard to a third round.

    I didn’t expect much of a difference in the final results from last year, because I was still using the pool of votes from the coaches based on last year’s method. As I explained in my other post on the subject, I would expect the coaches to vote differently under an approval voting format, because no one could use an individual vote for a school to game the results.

    And as for your belief that the Plus One fixes the bias of the coaches’ poll, I don’t see it. You still have to develop a way to select the four teams.

  24. Thomas Brown

    No it doesn’t matter what the bias is, built into every college football poll, when you pick 4 teams. 4 is 100 percent better than the current only Pick-2 system. Obviously, Ohio State has not been one of the top 2 teams the last several years. Were they to be one of the top 4, they would have Lost last year to all the other teams and would not have made it to the Mythical Pick-2 BS BCS National Championship Game, which is just only a beauty contest of 1-team conferences like the Big 10 and Pac-10, who do not want to play a game at the end of the season against the 2nd best team in their conferences because they might Lose.

  25. Pingback: Winning Cures All