Problems with surveys

The New York Times comments today on the country of Bhutan and how they measure progress not in terms of GDP but GNH -- Gross National Happiness.

First, congratulations to the Times for picking up on this late-breaking story. </sarcasm>. Bhutan has been doing this for years.

Setting that aside (it's not really important), here's an excerpt from the article:

In the study, the researchers, Sara J. Solnick and David Hemenway, gave the subjects a choice of earning $50,000 a year in a world where the average salary was $25,000 or $100,000 a year where the average was $200,000.

About 50 percent of the participants, the researchers found, chose the first option, preferring to be half as prosperous but richer than their neighbors.

Boy, I guess equality is really important, right? Equal-size slices of the pie are more important than the size of the pie itself?

That may or may not be true, but this often-cited survey doesn't show it. The question is poor; it leads the respondent to make a number of assumptions, which may or may not be true. One is that in the world where the average salary is $200,000, or in any world where the average salary is high, that basic goods will cost more. Similarly, the respondent will likely assume that in the lower-average-salary world, basic goods will cost less. In either case, the person is inferring something about how much money it takes to live in this hypothetical world, so the question is essentially "would you rather live in a world where you had to spend most of your income on essential goods or a world where you spent only a little on essential goods?"

In that case it's easy to see why most people might choose a society where they don't spend that much time/money on essential goods. For fun, let's say that a person in the 200,000/100,000 society spends one month/year on food, rent, etc., and that the person in the 25,000/50,000 society spends 8 months/year on the same.

Now let's ask a different question to these people. Would you rather live in a society where it takes you one month to obtain basic necessities but it takes everyone else one week to do the same, or a society in which it takes you 8 months to obtain those same basic necessities but it takes everyone else 10 months? That's the question that should be asked. When 50 percent of people come back and say "the 8 months/10 months society," let me know, I'll be surprised.

Even if the original question is prefaced by saying that the actual cost of living is the same, you are likely to still get a good percentage saying they'd rather have half the income but live in a world where you make more than everyone else. This can be attributed to people taking the information that is given and ignoring it, since it would be difficult to believe in this case.