This article starts by discussing the mysterious nighttime deaths of Hmong refugees, which some attribute to evil spirits. To support this theory, the Standard cites a discredited study by David Phillips, at the University of California, San Diego.
Phillips’ co-authored study was called “The Hound of the Baskervilles Effect,” referring to Sir Arthur Conan Doyle’s story in which Charles Baskerville dies of a heart attack while he is being pursued down a dark alley by a vicious dog:
The dog, incited by its master, sprang over the wicket-gate and pursued the unfortunate baronet, who fled screaming down the yew alley. In that gloomy tunnel it must indeed have been a dreadful sight to see that huge black creature, with its flaming jaws and blazing eyes, bounding after its victim. He fell dead at the end of the alley from heart disease and terror.
Phillips argued that Japanese and Chinese Americans are similarly susceptible to heart attacks on the fourth day of every month because in Japanese, Mandarin, and Cantonese, the pronunciation of four and death are very similar.
Four is an unlucky number for many Asian-Americans, but we see the number 4 every day—in times, addresses, phone numbers, prices, and car odometers. Are Asian Americans really so superstitious and fearful that the fourth day of the month—which, after all, happens every month—is as terrifying as being chased down a dark alley by a ferocious dog?
The Baskervilles study (isn’t the BS acronym tempting?) examined data for Japanese and Chinese Americans who died of coronary disease. A natural test would be a comparison of the number of coronary deaths on the third, fourth, and fifth days of the month. In their data, 33.9 percent of the coronary deaths on these three days occurred on the fourth day of the month, which does not differ substantially or statistically from the expected 33.3 percent. If days 3, 4, and 5 are equally likely days for coronary deaths, we can expect a difference this large more often than not.
So, how did the Baskervilles study come to the opposite conclusion? The authors didn’t report the 33.9 percent figure. Instead, they reported deaths from some kinds of heart disease, but not others. In the International Classification of Diseases, coronary deaths are divided into several categories. In some categories, more than one-third of the deaths occurred on day 4. In other categories, fewer deaths occurred. The Baskervilles study reported results only for the former. They discarded data that did not support their theory.
Phillips coauthored two different studies that used all heart disease categories and a third study that used completely different categories. The only reason for using different categories in different studies is to manufacture support for otherwise unsupported theories.
When we suspect that a researcher made choices after looking at the data, this suspicion can be tested by trying to replicate the results with fresh data. The Baskervilles study used data for the years 1989-1998. When data for 1969-1988 and 1999-2001 were applied to the cherry-picked heart disease categories reported in the Baskervilles study, the results were neither substantial or statistically significant. In the 1969-1988 data, there were more deaths on day 5 than on day 4; in the 1999-2001 data, there were more deaths on day 3. It is also revealing that the authors could have used the 1969-1988 data (and did so in other studies), but chose not to do so in the Baskervilles study. We can guess why.
The only evidence supporting this disparaging theory were obtained by omitting diseases and years that did not support the theory. Without this helpful pruning, there is no evidence that the fourth day of the month is fatal for Asian-Americans. What this study really demonstrates is that support can be found for any theory—no matter how silly—by discarding data that don’t support the silly theory.