Mumme Poll pondering

In the wake of this week’s results, there are some interesting discussion points out there worth addressing.

Over at the Mumme Poll site, there’s a good discussion thread developing in the comments section about the methodology of the voting, specifically with regard to the tiered structure and the number of slots on the ballot.  Here are a few examples:

It’s interesting to me that, as the season goes on, the top twelve become more and more solid and the bottom 20+ receive very few votes. This was one of my concerns going into the season regarding this poll. It seems like it’s difficult to determine who the 22nd best team is with this method because very few voters think that team is worth of being in the top 12. If 5 people think you’re top 12 material, does that really mean you’re #22? If the number of teams were expanded, it would be easier to determine the worst of the best, if you will…

Maybe a third tier should be added encompassing 13-20?

Is Iowa being ranked above Texas for being in 2 more top 12s while being in 103 less top 5s a good thing for the results, from a science-y view?

I know the top 5′s are just a tie-breaker, but when there’s such a large discrepancy as with Iowa-Texas…it seems like something is off… I wouldn’t mind seeing a system something like top 1-5 (3 pts), 6-10 (2 pts), 11-15 (1 pt).

“Might as well flip a coin” captures an important (and good) feature of this. If 60% of voters put one team in the top 12, and 40% put the other in the top 12, that’s how we rank the teams. #13 makes a good observation; the more it’s “obvious” who the 12 best teams are, the more poorly this procedure does at distinguishing among the top 5 or #20-25. I’ve been thinking lately that the average coach probably could rattle off a top 3 in order without taking too much more effort than a top 12 (not in order). The top 12 is especially relevant for BCS purposes, and makes sense for that reason; I’d kind of like the championship game to be based on votes for the top 1, period. I certainly wouldn’t want it to be based on whether 1 vs. 3 voters decide a team isn’t in the top 12.

Let me say for starters that I wasn’t real thrilled with the idea of having a tier for the top five when I thought my way through how to structure the MP last year.  Ultimately I thought it was useful because with a smallish voting pool the chances for a significant number of ties at the top portion of the rankings were likely and I felt that including the tier would reduce those.  As it turned out, there was some validity to my concern, but the tiering didn’t really do much good in terms of reducing ties during the regular season, as there were three-way ties at the top slot in the last three weeks.

My feeling is that there are two things that will have a much greater impact on reducing ties in the top five than tiering.  One is having a significantly larger voting pool than the one that finished last season.  As we’ve seen in the first two rankings, there has only been one tie in the top five (the Cinci-USC vote this week) so far.  It’s something I’ll carefully monitor over the course of the season.  The second is the nature of the season itself.  I’ve said it before, but I’d love to see what kind of results the Mumme Poll would have generated with the final regular season vote of 2007.  Right now, we’ve got a year where there’s a consensus over the top three teams.

Some of the commentary strikes me as coming from a mindset that’s still used to looking at ranking a team for a specific slot instead of taking a group approval approach to the vote.  I suspect that if this year’s results bore out my expectation that, with a large enough voting pool, tiering isn’t essential in order to generate a credible set of rankings and we chose to do away with tiering in 2010, there would be a significant amount of dissent over that decision.

Ultimately, this gets back to what we’re after with this poll.  I’d probably have different priorities if the MP were to become the sole arbiter of college football’s hierarchy.  Our goal is more modest.  We’re just looking for a way to create a Coaches Poll that’s more credible than the one we’ve currently got.  That means finding a format for generating rankings which (1) reduces the possibility of an individual coach manipulating those rankings with his vote and (2) is less time consuming in the evaluation process in order to make it significantly easier for a coach to retain responsibility for actually casting the vote instead of sloughing it off on another party.

That doesn’t necessarily mean that there won’t be votes that could be considered reaches.  So when I read an observation like cocknfire’s,

I like the Mumme Poll run by the Senator, but what does it say about the persistence of preseason expectations that five people think Oklahoma is one of the Top 12 teams in the country?

I understand it, agree with it on a certain level (I sure didn’t include Oklahoma in my top 12), but ultimately I’m not particularly concerned about it, at least in and of itself.  An individual’s ballot has to be considered in the larger context of approval voting.  Some people may vote as fans, which indicates a certain level of bias, but others are going to cast votes based on arcane formulas or other factors that they objectively believe are valid.  When the dust settles, we have to hope that a credible truth emerges from all of these individual agendas.

Which is why I like this comment from the MP site:

… This past weekend in fact I might have been the ideal test case for this as a mechanism for voting for the coaches’ poll; I was really busy, paid no attention to college football, realized, “Oh, s**t, I have to vote in this poll”, and was fairly crude in how I decided who to vote where. I imagine we’ll all go through some weekends where we know exactly how we want to vote, and others where we kind of wing it; this goes back to my response to #4, that a lovely feature of the system is the distinction between whether 40% or 60% vote you in the top 12. I hope that the probability of my “winging” a team into the top 12 gets higher the better the team really is, even if I don’t have much time to think about it, and I hope that, when I screw up, the rest of you will cancel me out, even if you’re winging it, too.

Using the Sooners as an example here, it’s fair to note that they finished higher in the Mumme Poll this week (#22) than they did in the Harris, AP or Coaches Polls.  But it’s not that significant a gap between them all.  What is significant to me is that none of the Mumme Poll voters who cast ballots including Oklahoma this time have stated an affiliation with the school.  That’s a good start.  My hope is that I learn a lot more from the process as we keep voting.