They argue that people don’t take hurricanes with female names seriously and consequently don’t prepare properly and are more likely be killed. Their study was published in the Proceedings of the National Academy of Sciences, so it must be correct, right?
Not necessarily. Lots of wacky studies have been published in prestigious journals. The British Medical Journal published a study titled “The Hound of the Baskervilles Effect” claiming that Asian Americans are more susceptible to heart attacks on the fourth day of the month because seeing the unlucky number 4 is like being chased down a dark alley by a ferocious dog. As Dave Barry would say, I’m not making this up.
Nobel Laureate Ronald Coase cynically observed that, “If you torture the data long enough, it will confess.” Researchers seeking fame and funding are easily seduced into ransacking data for attention-grabbing results.
In the natural sciences, provocative claims can be retested in laboratories. In the social sciences, it is not so easy to gather fresh data. What we can do is look for evidence of tortured data. For example, would the results change if questionable data that were included had been excluded, or vice versa?
In the Baskerville study (isn’t the BS acronym tempting?), the authors included heart diseases that supported their preposterous theory and excluded those that did not. They tortured the data.
The Hurricane study has problems, too.
1. Many deadly hurricanes were omitted. One of their sources described Hurricane Bill in 2009:
“Large swells, high surf, and rip currents generated by Bill caused two deaths in the United States. Although warnings about the dangerous waves had been posted along the coast, over 10 000 people gathered along the shore in Acadia National Park, Maine, on 24 August to witness the event. One wave swept more than 20 people into the ocean; 11 people were sent to the hospital, and a 7-yr-old girl died. Elsewhere, a 54-yr-old swimmer died after he was washed ashore by large waves and found unconscious in New Smyrna Beach, Florida.”
Hurricane Bill was not included in the study’s data, evidently because the hurricane didn’t quite make it to shore. But, surely, this episode is evidence that people didn’t take a hurricane named Bill seriously.
The authors wrote that in the data they did analyze, they found “no effect of masculinity-femininity of name for less severe storms.” If they had included hundreds of low-impact storms like Hurricane Bill, they almost surely would have found no relationship at all between names and fatalities. Excluding these data bolstered their conclusion.
Data shouldn’t be omitted just because they show little or no effects. The catastrophic decision to launch the space shuttle Challenger in below-freezing weather was based in part on a flawed statistical analysis that excluded seventeen flights where there had been no O-ring failures. It is generally better to include all the data.
2. In 1950, 1951, and 1952, hurricanes were named using the military phonetic alphabet (Able, Baker, Charlie, . . . ). A switch was made to all female names in 1953. Many feminists decried this sexism, with Roxcy Bolton noting that, “Women are not disasters, destroying life and communities and leaving a lasting and devastating effect.” The switch to the current system of alternating male and female names was made in 1979.
The statistical issue here is that the average number of deaths per hurricane was 29.1 during the all-female era and 16.2 afterward. Perhaps there were more fatalities in earlier years because hurricanes tended to be stronger (the average hurricane category was 2.26 during the all-female era and 1.96 afterward) or because there was less advance warning. What is clear is that we should compare deaths from male and female hurricanes during years when hurricanes had male and female names. This correlation is only 0.01:
Using the 7.25 figure for Sandy, the correlation is –0.03. The more feminine the name, the fewer the fatalities, although it is far from statistically persuasive (p = 0.890).
The reported relationship between between hurricane names and death toll depends on several data ambiguities: the inclusion and omission of storms, the inclusion of years when storms were deadlier and names exclusively female, and the questionable categorization of Sandy as a very feminine name.
Why do journals published flawed research? Some don’t have the resources for careful scrutiny. Some like the publicity that comes with provocative articles. Some welcome confirmation of their prejudices.
Be skeptical, very skeptical, of provocative assertions that you suspect may be based on tortured data.