Hurricanes with male names are at least as deadly as those with female names

Lots of news services have been passing on the startling conclusions of a recent academic paper in The Proceedings of the National Academy of Sciences, a quite high-impact journal, that, and these are direct quotes from the paper in question's abstract:

feminine-named hurricanes cause significantly more deaths than do masculine-named hurricanes. Laboratory experiments indicate that this is because hurricane names lead to gender-based expectations about severity and this, in turn, guides respondents’ preparedness to take protective action.

I'll just show you this graph I made with the data published in the PNAS paper, then explain further below (click to enlarge):

The blue (male) line is almost always above the orange (female) line. What gives?

I was certainly not the only person to be skeptical of the paper's conclusions; Tyler Vigen used his usual satirical approach to good effect, showing how remarkable spurious correlations can be. But the thing is, this idea that people wouldn't take female-named hurricanes seriously as a threat may sound dubious, but it also sounds plausible, and I imagine it pushes some buttons.

I am not at all qualified to comment on the soundness of the social science in this paper, but I do think the data analysis is quite flawed.

First of all: it's kind of a rule that one should compare apples to apples. Before 1979, all hurricanes had female names. While this by itself does not invalidate their hypothesis that, all other things being equal, a feminine-named hurricane will result in more deaths, all other things were not equal before and after 1979. Off the top of my head, back when hurricanes had only female names, meteorologists were not as good at predicting the severity and paths of hurricanes as they later became with the help of experience and, especially, computers, and communications technology was not as good at relaying information and evacuation orders. Neither of these factors were addressed in the paper; isn't it worth looking into whether other factors besides name gender contributed to deaths?

So if we limit our analysis to post-1979, when we can directly compare male and female names of hurricanes, the masculine hurricanes caused more deaths up until 2012's Sandy. At present, the feminine names are barely ahead, 459 to 413. This is at least counter-evidence to the paper's claim that "changing a severe hurricane’s name from Charley to Eloise could nearly triple its death toll." (To be fair, they didn't just look at whether a name was male or female -- I know one male named Sandy -- but how "masculine" or "feminine" a study group considered the names. This doesn't change the fact that they ignored much more plausible reasons for deaths prior to 1979.)

I find it curious that the researchers limited their analysis to American deaths; hurricanes kill a lot more people before they ever reach the United States. Of course, a greater proportion of non-Americans are too poor to shelter or evacuate, but this strikes me as a combination of partial cherry-picking, circular reasoning and insufficient research: they limited their data to people affluent enough protect themselves against a hurricane, and then claimed they died because they didn't protect themselves a hurricane, without actually looking at whether or not they protected themselves against a hurricane.

For example, the second-most deadly hurricane on their list, Diane, killed 200 people in 1955 despite being only a Category One (Five is the strongest) for which evacuation orders are rarely if ever given. The reason it was so deadly is that Hurricane Connie passed through the same areas in Pennsylvania and Connecticut a few days before, saturating the ground so that Diane caused massive floods.

At least they left Katrina off the list; I think most people would agree it's probable that there were some social, economic and political factors that contributed more to its 1,833 deaths than its name. But their reason for considering Katrina an outlier was that it "leads to a poor model fit due to over-dispersion." There's kind of another rule in data analysis: you don't choose your data to fit your model, you choose your model to fit your data.

I will admit that the researchers' laboratory studies succeeded in convincing me that the people they studied (including Amazon Mechanical Turk users, not exactly a representative cross-section of people who might ignore a hurricane) answered questions in such a way that they appeared to assign lower risk to hypothetical hurricanes with more feminine names. It's just rather a stretch to claim that:

(a) This laboratory result is truly an indicator that in a real-world scenario these people would actively ignore the risk of dying in a hurricane; and

(b) That there is any risk-ignoring behavior correlated with hurricane deaths at all. (There very well might be. But the researchers didn't even attempt to find out. There was no historical data, no text mining of contemporary news sources, just a bare minimum of meteorological data, damage and death assessment.)

PNAS is a good journal (and always a barrel of laughs when you say the acronym out loud). I'm sure they'll get it better next time.

UPDATE Randal Olson, who is definitely an expert in such matters, pointed out that a more convincing graph would be one that showed deaths from hurricanes were more frequent in general before 1979 when they started giving them male names. So I whipped one up quick in Excel. Katrina of course incredibly skews the aggregate data, but you can see it was more common for any individual hurricane to have over a handful of deaths before 1979 (click to enlarge)

I didn't control for storm severity as Randal suggested, but I'm reasonably confident it will change nothing: the six storms that caused more than 100 deaths were categories 1, 1, 2, 4, 5 and 5.