If you’ve read this column regularly, you know that misrepresentation of polling data is one of my pet peeves.

Over the past few weeks, there have been several polls on a variety of subjects in Maine that could use some analysis. They provide both good and bad examples of how to conduct and release public opinion research and how to report on the results.

Let’s start with the worst. In January, a group called TREES4Maine (short for the Trust for Responsible Economic & Environmental Stewardship for Maine) released a so-called poll of the residents of the Patten area on the desirability of a North Woods national park. It was reported on by the Bangor Daily News.

The group, helmed by former Maine Senate President Charles Pray, claimed that they had “scientifically” determined that 87 percent of people living in and around Patten opposed “the proposal for a national park in the Katahdin area.”

The poll, to put it simply, is bogus. First of all, the group, which has no track record of opinion research, used volunteers who oppose a park to conduct the survey, rather than an independent phone bank or automatic calls. While volunteers can be trained and supervised to make sure they don’t influence the opinions of those they call, the risk is too high to accept these results without some serious caveats.

Second, they surveyed only “46 or 47” of 1,475 local residents. Even for such a limited population, this is way too small a sample. Based on these numbers and ignoring other flaws, the margin of error of this poll (which is never stated in the article) would be more than ± 14 percent, at the 95th percentile.

Most importantly, the group failed to release (or the BDN failed to report) several pieces of vital information about the poll, including full question wording, exact survey methodology and the demographics of those surveyed. We have no idea if the respondents were at all representative of the region.

This is not to say that this poll is necessarily inaccurate. In fact, it’s likely that many or most people in Patten really are opposed to the kind of change that a national park would represent. The truth is that we can make no judgment about accuracy from what we know about the poll.

For the Bangor Daily News to prominently report the 87 percent figure and to allow proponents to tout the poll as “scientific” is bad journalism, especially as it gave basically the same amount of attention to a real poll conducted by a legitimate polling firm (Critical Insights), finding 60 percent of Mainers support a study of a national park.

A better, but still somewhat dodgy, example of polls in the media are recent reports from Washington, D.C., news sources of an internal poll from Rep. Mike Michaud’s re-election campaign showing Michaud ahead of Republican challenger (and current Maine Senate President) Kevin Raye, 55 percent to 32 percent.

Internal polls (those conducted by the campaigns themselves) are notoriously inaccurate. Not because they are inherently biased, but because campaigns usually release them only if they show favorable results.

For a good example, look no further than the last contest between Michaud and Raye, in 2002.

A little more than a month before the election, Michaud’s campaign released a poll (from one of the same pollsters who conducted the latest survey) showing their candidate ahead by 11 points. On Election Day, he won by 4 percentage points.

The Michaud campaign also declined to provide addition information about this poll, including favorability ratings for Raye, full question wording and demographics.

In the D.C. media, it may be enough to note that this is an internal poll and their readers will know how to appropriately interpret the results. If this is reported in the Maine media, however, it should come with more caveats and perhaps even some criticism of Michaud for not releasing the full results.

One final, better example of reporting about public opinion is coverage of a poll showing 54 percent of Mainers supporting equal marriage rights, released by supporters of a new referendum on the subject, which was made public as they delivered more than 100,000 signatures to the Secretary of State.

The poll (while technically internal) was released with a full methodological description, had a large sample size and aligned closely with other recent surveys on the subject. Rather than becoming a story in and of itself, or being taken as gospel, It was reported as just one data point within the larger discussion on marriage rights in Maine.

Mike Tipping is a political junkie. He writes the Tipping Point blog on Maine politics at DownEast.com, his own blog at MainePolitics.net and works for the Maine People’s Alliance and the Maine People’s Resource Center. He’s @miketipping on Twitter. Email to writebacktomike@gmail.com

Copy the Story Link