Polls, People, Predictions, and Buffalo

Associated Press-AOL News sponsored a poll in the last week of 2006 that asked people in the U.S. to contemplate what 2007 holds for the country.

Among the results one finds:

  1. 60 % say the the U.S. will be the victim of a terrorist attack.
  2. 60 % say likely that a biological or nuclear weapon will be unleashed somewhere else in the world.
  3. 70 % predict a major natural disaster in the US.
  4. 70 % expect "worsening global warming."
  5. 35 % predict the military draft will be reinstated.
  6. 35 % predict a cure for cancer will be found.
  7. 25 % anticipate the second coming of Jesus Christ.
  8. 19 % think scientists are likely to find evidence of extraterrestrial life.
  9. 29 % think it is likely that the U.S. will withdraw its troops from Iraq.

These figures are from a telephone poll with 1000 respondents. Let's suppose that the poll has only those defects that are endemic to even high quality political polls.

Is it possible to speculate meaningfully on these figures?

The Playing Along Bias

Some folks seem willing to play along, or perhaps willing to spice up the conversation.

How else can you explain people saying "Oh, sure" to items 6, 7, and 8?

Experts would surely put down a big fat zero for the probability of these events.

The Definitional Bias

Many of the questions are flawed for lack of clear definitions.

Consider No. 4 on global warming. This is not an issue that is influenced by a year's worth of data, so if one now thinks that global warming is worsening, then over 2007 it will continue to worsen. This question is not about prediction, it is about your view of a steady state, or at least a state that is not changed by even five or ten years of additional observations (unless they are of new kinds).

No. 9 also has definitional problems. If the question focused on "begin to withdraw troops" then the event is possible. In fact, I'd give this event a 30% chance of happening, and I'd make that number larger on days when I am optimistic. Still, to have the US troops completely withdrawn by the end of the year is virtually impossible. My own view is that we may stay in Iraq as long as we have stayed in Germany.

No. 3 is a pure-play definitional disaster, since we have no definition of a "major disaster."

Virtually every year, some part of the US achieves the status of a "disaster area," yet, unless you have friends or family in Oklahoma, Kansas, and West Texas, it is not a major disaster when a tornado runs through a trailer park on the edge of one of their towns.

A blizzard that kills no one but which shuts down Denver airport for one day has more economic impact.

The "Damn Politicians No Matter What They Do" Bias

Question 5 is interesting. There are no advocates in Congress for the draft, except one or two loonies who occasionally try to grab some cheap press. The military leadership is uniformly opposed to the draft.

It simply is not going to happen. No one who thinks about this can really believe that it could, especially within the span of one year.

The Structural Flaw Bias

Now consider my favorite: "60% say likely that a biological or nuclear weapon will be unleashed somewhere else in the world."

This is a structurally flawed question, since we have people saying what they believe is "likely" and we have no way to understand what they understand by this.

I'd posit a 1% chance of an atomic weapon being used for military or political purpose somewhere during 2007--2010.

I'd increase this to 10% for a biological weapon.

I suspect that I am being a optimist in both forecasts, but my suspicion is not enough for me to offer a revision.

As Long as the Grass Grows ...

Polls are a staple of journalism. People really do want to know what other people think. People may also like the surprise of learning that what they think differs from what other people think.

Polling articles exist to serve this market.

Unfortunately, many newspaper polls are worse than useless. They suggest --- and possibly promote --- a view of the world that is simply false.

Still, one can safely predict that they will be with us as long as the grass grows.

In this respect, polls differ from the buffalo.

Navigation: JMS Home, JMS Rants, Surprise Me!