About five years ago, in 2007, I was called by one of the major, national polling organizations (Pew Research) and asked if I would care to answer questions for their survey. It was an interesting experience in hindsight, though boring in the doing of it—and it makes me wonder about the accuracy of such polling on several levels.
First, they called me on a Saturday morning about 10:30 AM. I wonder whether this is necessarily the best time to be calling someone, particularly since it was in August during the summer. Wouldn’t most people be outside mowing the grass, or in their pools, or at the beach or off somewhere doing something on the weekend? Or at the very least in the middle of getting ready to go do something? I wonder how many potential households they miss by their choice of time for calling.
Then, the survey questioning lasted nearly forty-five minutes. It felt like taking a test. It’s been a long time since I had to take tests; I give them to my students, which is entertaining for me. But for the students, it really isn’t much fun. I wonder how many people actually stick with a series of rather dull questions for the full length of the survey? How many just start babbling random answers to get the annoying voice on the other end of the phone to go away?
They asked me what I thought about a variety of important, national questions. They wanted to know which of the various announced presidential candidates for my party was I most likely to vote for if the election was held that day.
How many people were really paying any attention to upcoming primaries nearly a year out?
After a bunch of questions about a bunch of people I had not thought much about yet, the man on the phone finally asked me about the then current occupant of the White House. The insistent voice wondered what I thought of the President, of his various policies, and of the direction of the country. Did I think the future would be good or bad? Did I approve of his handling of the economy? Did I like how the President was fighting the war in Iraq?
Of course, on each question, I had only two possibilities: yes or no. Occasionally, I’d get a multiple choice question. But in no case, was I ever given an essay style query. There was never room to explain that, “well, I like this aspect of the President’s policies, but I wish he’d do such and so about this problem.” You have to give blanket yes or no: no middle, no nuance, no discussion of details. Just true or false. When my tormenter asked about my political beliefs and listed off the multiple choice possibilities, I was dismayed to discover that what I actually am was not one of the listed categories. I pointed this out. “Well, you need to pick one. How about ‘c’? There wasn’t even a “none of the above.” Although my friends may describe me as odd, my ideas do fall under a recognizable label that is not all that obscure–except apparently to whoever at Pew Research came up with the questions.
After all the queries regarding my opinions, I faced a set of demographic questions: whether my telephone is a land line, cell, or VOIP, what my age and ethnicity might be, my marital status, my religious beliefs and how much money I make. They even asked me if I would be willing to be interviewed by a reporter later.
I can’t help but wonder, how “carefully selected” I was if they didn’t know any of that stuff ahead of time. They apparently knew my party registration, since they only asked about the candidates for the party I would likely vote for come the next election; but they had no clue about anything else until they asked.
I understand that polling is supposed to be a scientific endeavor and that the pollsters gather a representative sampling of data, so that from the thousand or so people they talk to, they know they’ve gotten a sense of what the whole three hundred million of us really think.
I am not entirely convinced. It seems to me that they are really just shooting craps, dependent upon whomever they happen to find at home on a Saturday morning with a life that is so empty he’s got nothing better to do than to answer dull questions from a stranger on the far end of a phone line. Admittedly, at the time, my wife was in Australia for three weeks leading a delegation of highschoolers for the People to People Student Ambassadors Program—and I’m an author who works alone from home. And so any excuse to chat with an actual human adult was a welcome thing. But how many people could there really be in my odd situation? And they’re going to base their polls on us?