Why Pollsters Screw Up

In my previous column, I talked about the limitations (and they are many) of the polls. In the Iowa primary, pollsters dramatically underestimated the number of young voters (those under 30), who voted heavily for Barack Obama. In New Hampshire, the pollsters over-compensated and overestimated the potential number of young voters.

If you watched CNN (and other networks) last Tuesday, you saw that they kept waiting for the Obama votes to come in from Hanover (Dartmouth College), Durham (the University of New Hampshire), and other college locations (including tiny Franklin Pierce College). Presumably, they're still waiting. Mrs. Clinton held a good lead throughout the night, college students voted in disappointing (for Obama) numbers, and female students who did vote often went for the female candidate.

One reason the polls messed up lies with the concept of "likely voters," a concept invented by Gallup many years ago. Generally, likely voters are those who've voted with some regularity in past elections. If you've voted before, you're "likely" to vote this time -- right? Well, not exactly.

The pollsters have had a terrible time this year with young voters, especially those who may -- or may not -- vote for the first time. If a pollster has no evidence of past voting practices, it's impossible to tell if a voter is actually going to show up at the polling place. In New Hampshire, the pollsters apparently guessed what might happen, and they guessed wrong.

The media, which has significant influence on pollsters, assumed Obama's victory in Iowa would give him a big "bounce" in New Hampshire. It didn't. Poll evidence (?) suggests Mrs. Clinton generated some votes, not a whole lot, from her tearing up episode. Overall, there was a very high turnout in the Democratic Primary (280,000 voters various 220,000 for Republicans), and that apparently helped Mrs. Clinton, something the pollsters should have picked up before the vote.

In a typical election, voters identified as "likely" tend to vote at a percentage between 70% and 75%. In other words, as many as 30% of likely voters end up not voting. They're sick, or they're out of town, or it snows heavily -- or perhaps they're just not turned on by the current election.

Registered Democrats are more likely to vote than the much-heralded "Independents." Registered Republicans are more likely to vote than either Indies or Democrats. Many of the much-discussed "Independents" are not the dispassionate, reflective souls we sometimes visualize. In fact, many of them have little interest in voting. If they see a snowflake, they're likely to stay home.

Gallup is the gold-standard of polls. It skews its likely voter’s concept slightly toward the Republicans. It does so for reasons stated above -- that Republicans are more likely to vote. Apparently, the Harris Poll does something similar. Traditionally, that approach has made Gallup and Harris the most accurate polls.

However, in 2004 Gallup's Polls through the autumn consistently showed George W. Bush doing well, perhaps winning nationally by millions of votes. Many Democrats and many in the media (generally Democrats) didn't like that and suggested Gallup was shilling for the GOP.

Gallup got nervous. It changed its likely-voter template to include more Democrats. Soon, it showed Bush running neck-and-neck with John Kerry. Also, Gallup indicated Kerry probably would win key states like Ohio and Florida. On the other hand, it also suggested Bush had a good chance of winning Pennsylvania.

On Election Night, Gallup (and many in the media) had a huge quantity of egg on their face. Bush won Ohio and Florida by fairly substantial margins. He lost in Pennsylvania, as most Pennsylvanians -- including this one -- had predicted he would.

What about the pollsters' legendary margin-of-error? As both Iowa and New Hampshire exist, the term is largely mythical. If pollsters really knew with precision what inaccuracy there may be in their polls, they'd presumably correct it. The margin-of-error is mainly a way for them to explain why their polls failed to predict an election accurately. They can always say, "Well, I was within the margin of error." I presume Fortune Tellers could make a similar claim.

Actually, I like pollsters, much as I like, say, polar bears. But I don't want to get overly close to either.

May the best man -- or best woman -- win.