As I’ve mentioned, one big target for the Mumme Poll is taking on the bias/conflict of interest problems that plague the Coaches Poll. As Tony Barnhart illustrated in his post about the last regular season Coaches Poll vote in 2007 (it’s linked at the Mumme Poll page, above), it wasn’t just Hal Mumme’s blatant attempt to prop Hawaii’s position up in the poll results that was a concern, it was also things such as how all the coaches in a given conference voted with regard to its members.
Obviously, our voters don’t have the money stakes in the vote that the coaches do in theirs, but the Mumme Poll is a fan-based operation, so bias is still an issue to be evaluated. It’s just that you have to take out the financial motivation and substitute an emotional one in its place. And as I think about it, this has the potential to be one of the weak spots of the MP as an evaluation tool.
Here’s why: while you might see, say, every coach in the Big Ten look favorably upon Ohio State in a given year in an (I’ll be charitable and call it unconscious) effort to enhance that school’s chances of appearing in a BCS game, they’re not voting that way because they’re Buckeye homeboys. In other words, they’re not a coordinated bloc that can be counted on merely because of institutional loyalty. On the other hand, our Ohio State voters do have that loyalty.
Why does that matter? Well, remember that one thing approval voting is structured for is to reduce the impact that any one vote could have on the overall result. Hal Mumme voted Hawaii number one because he believed that, given the design of the Coaches Poll which assigns more weight to a higher ranking, his vote alone would have an impact. With approval voting, unless enough of his peers would vote the same way on Hawaii as Mumme, his attempt to manipulate the vote wouldn’t succeed, and knowing that was unlikely, Mumme’s own vote would have been different as a result.
What you have to worry about with our voters is that the psychology may be different. Those Ohio State voters aren’t thinking about whether they can manipulate the results to get their beloved school into a BCS game with a big payoff; they just love their school. And “they” is a key – it’s not like they’re voting as a coordinated, planned bloc, but you still have to be concerned that a group bias could come into play.
Now I’m not suggesting that’s happened, of course. And there are two factors that would ameliorate the problem, if, in fact, it existed. First of all, every school is likely to have its share of homers voting. So to some extent, that should even out in the overall voting results. And, in any event, there are a large number of voters who aren’t homers, whether that’s by design or circumstances (I’m looking at us, Georgia voters).
Anyway, it’s something I’ll be taking a look at over the rest of the season. Here’s a breakdown of how some of the voting groups cast their ballots, compared to the overall results:
- Ohio State: 10th in the MP; tied for 6th among OSU-affiliated voters.
- Alabama: 1st in the MP; tied for 1st among Alabama-affiliated voters.
- Florida: 2nd in the MP; tied for 1st among Florida-affiliated voters.
- Texas: 3rd in the MP; tied for 1st among Texas-affiliated voters.
- Penn State: 15th in the MP; 12th among Penn State-affiliated voters.
- Georgia Tech: 18th in the MP, 13th among Tech-affiliated voters.
You can get into sample-size issues with this, so I don’t want to get too crazy with it. But I think you can see that human nature is what it is.