The story began with a study of 31,209 Seventh-Day Adventists. Researchers questioned them about their consumption of 65 different foods. To the researchers’ surprise, those who ate nuts at least five times a week had only half the risk of fatal heart attacks as those who had nuts less than once a week.
Her analysis of the bias in the study included:
- Unfortunately, we do not know from this account how many of the sixty-four other foods were associated with a lower risk of heart attacks. We do not know if the nut eaters shared other characteristics besides eating nuts that may have explained their lower rate of fatalities. Seventh-Day Adventists do not smoke or drink, which makes them an abnormal population to study. And according to this account, the study was based on their memories, not observation.
In other words, the study was biased. Decision makers who might attempt to make choices based upon this study might not select the important characteristics to modify. Crossen continues with another walnut-cholesterol study.
This time, the researchers put 18 healthy volunteers on two carefully controlled diets for two months. One was a nut-free version of a standard low-cholesterol diet. The other was nutritionally similar, except 20% of the calories came from about 3 ounces of walnuts per day. ...... On the no-nuts diet, the volunteers’ cholesterol levels fell 6 percent. When they switched to the walnut diet, their cholesterol declined an additional 12 percent. Everyone’s cholesterol dropped while eating nuts, and the average decrease was 22 points, from 182 to 160.
Her analysis:
- While not a fatal flaw, eighteen subjects is a very small study. The subjects were put on a low-cholesterol diet, which means their cholesterol was going to drop no matter what. Think about eating three ounces of walnuts every day. It comes to more than fifty pounds a year. ... They lost me. Did all the subjects first eat no-nuts, then the nuts regime? Or were there two groups, one starting with no nuts and one starting with nuts? Did the 22-point cholesterol drop include the decrease attributable to the low-cholesterol diet alone? How long did the study go on -- that is, would the cholesterol level have continued to drop from the low-cholesterol diet with or without the nuts? Those walnuts displaced other food -- was the drop a substitution effect alone?
In other words, because of the bias in which the data were collected and summarized, we actually know nothing from either study. However, upon first reading, it appears as though information is unbiased. It is this subtle bias, that is unintentional to the decision maker, which can cause significant problems for a DSS. |