Consumer Reports

Dawn Eden and After Abortion Annie are noticing that Consumer Reports coverage of birth control is a little skewed. For most people, Abortion is not a topic to be grouped with birth control. Oddly enough the CEO of Consumer Union, the group that puts out Consumer Reports, is the former head of Maryland's Chapter of Planned Parenthood. So much for objectivity.

Now this isn't a post about birth control or abortion. It is about magazines like Consumer Reports or the firearms equivalent, Gun Tests. Their testing methodology is to spend their own money and purchase one of several models of comparable products like cars or humidifiers or 1911 pattern automatic pistols. Then they perform comparison testing on these products and assume that these tests reflect the overall performance of the entire model line.

Did I mention that for most products, they only use a single example of each model? For those that aren't aware, one is not a statistically significant sample size. Now for some things this won't matter, because the variation between examples of a single model is small. However if you are talking about say the access speed of competing harddrives or the accuracy of competing pistols, those quantities can vary at lot within models.

The other problem is that these folks don't seem to know what they are talking about half the time. The bigger problem is that that half the time seems to coincide nicely with when I know something about their topic. Meaning that I have no reason to believe they aren't full of "it" all of the time. This is a problem I find with a lot of journalists working in technical and near technical fields. For instance, when Consumer Reports comments that the Corvette isn't a good car because it doesn't have good trunk space and gets poor gas mileage, it shows they have no freaking clue why people buy Corvettes. As if all cars should have the performance/comfort mix of a Honda Accord or something. Cars that are built around tight crisp handling tend to be louder and have rougher rides. Its an engineering tradeoff in the fundamental physics of the thing. Grrr.

So that is why, when I want to get reviews of a product, I look to the internet. There are many product review websites. When I'm thinking about buying a gun, I go to a relevant gun forum and look at what people complain about. This way I usually have a wider sample of model performance and user wants. Plus the people tend to know what they are talking about. Horray for the internet!