Americans generally agree that mass shootings require our attention. They might even agree on the notion that mass shootings require interventions. One thing that really struck me when I started studying mass shootings with Tara Leigh Tober is that there is really less agreement on what exactly a mass shooting is. Not everyone studying mass shootings uses the same definition. And even when they do, they’re not all using the same data.
I just published a new article with Tara Leigh Tober and Melanie Brazzell examining discrepancies between five of the most influential databases relied upon to make claims about mass shootings in the U.S. in The Lancet Regional Health–Americas, “Database discrepancies in understanding the burden of mass shootings in the United States, 2013-2020.”
As a part of an ongoing project, Tara Leigh Tober and I have been putting together different databases used to assess mass shootings in the U.S. Regardless of how you define them, these things happen in the U.S. more than anywhere else. That’s not really in question. But answering the “How much more?” question receives just incredibly varied responses. Depending on which database you use, you could find databases showing that mass shootings happen approximately once a month in the U.S. to databases documenting more than one mass shooting per day in the U.S. It really comes down to a question of what qualities ought to be used to describe a shooting as “mass.”
A good deal of scholarship relies on the definition of a crime operationalized by the FBI referred to as “mass killings.” According to the FBI, mass killing incidents are those in which four or more people are killed by a single perpetrator. Building on this, some suggest that mass shootings are simply mass killings by firearm. (There are a whole host of mass killings involving other weapons. The number of options available for the variable used to identify weapon in the FBI’s Supplemental Homicide Report is horrifying.) As a technicality, the FBI actually now defines incidents in which three or more people are killed as mass killings, lowering their fatality threshold from 4 to 3. Despite this, a majority of scholarship continues to use 4 fatalities as an important and defining quality.
Beyond the numbers of people killed, mass shootings have been defined more narrowly to exclude a host of incidents meeting that definitional parameter but failing others. For instance, a good deal of scholarship on mass shootings excludes: (1) incidents with more than one shooter; (2) incidents that can be attributed to drug or gang-related violence or robberies; (3) incidents that occur as a part of other criminal activity; (4) incidents associated with family or intimate partner violence; (4) incidents that occur in more than a single location (something the FBI classifies as “spree killings”); and (5) unsolved incidents. The table below from our article summarizes some of the database definitional distinctions.
The thinking is that incidents that meet these much more narrow criteria are unique and different in important ways from incidents that might be included if we didn’t narrow the definition so much. The truth is, as far as I can tell, there is very little research examining how and why those incidents differ from others. And even if they do differ in meaningful ways, we have less information than we should because that sort of analysis is framed as “beyond the scope” by definition.
Five of the largest databases that are used by scholars, policymakers, law enforcement agencies, journalists and media professionals, and more to make claims about mass shootings are: (1) Gun Violence Archive, (2) data from theFBI’s Supplementary Homicide Report, (3) the more recently produced “Active Shooter Report” also produced by the FBI, (4) the Mother Jones database produced by Mark Follman, and (5) a database of incidents put together by Everytown for Gun Safety.
For this article, Tara Leigh Tober, Melanie Brazzell, and I downloaded all of the data from each database, looking specifically at years for which all five databases included data (in the article we look at data from 2013-2020). We wrote a script to attempt to identify incidents across databases. [The script did part of the work and we had to do the rest by hand.] We wanted to know, of the 3,155 incidents that were in at least one of these databases between 2013 and 2020, how many were in all five. We were also interested in the overlaps between databases more generally.
The database with the least restrictive definition is Gun Violence Archive. They include incidents with 1-2 shooters involving 4 or more gun-related injuries, not restricted by the number and type of locations, relationships, and motives. This means that the Gun Violence Archive data includes a LOT more incidents. We discovered 2,727 incidents that were in the Gun Violence Archive database, but none of the other four (see Figure 1 from the paper below). If you look along the bottom of the UpSet figure until you see the five dots connected to each other, that is the figure for the number of incidents in at least one of these five databases for the years we examined that were in all five databases… 25 incidents. 25 of the 3,155 incidents that were in at least one database were included in all of them. That’s only 0.008% of the incidents analyzed.
The data are a bit easier to look at if you exclude Gun Violence Archive because the injury (rather than fatality) threshold used to produce that database produces such a larger population of incidents. But, even when excluding GVA data, it was still the same 25 incidents that were included in all of the other four databases (see Figure 2 from the paper below).
Even when we looked only at those incidents meeting the 4+ fatality threshold, it was still the same 25 incidents, even though the total population of incidents decreases substantially if we only look at those incidents in which at least four people were not only shot, but killed (see figure 3 from the article, below).
This is a problem for many reasons. It ought to call into question a good deal of what we claim to know about mass shootings in the U.S. It is not a common practice in mass shootings scholarship to corroborate discoveries with different databases. But, here, we see that only a small number of incidents in some of the databases are in all of them.
In the article, Tara, Melanie, and I recommend a more inclusive definition, using an injury (rather than fatality) threshold of 4, including incidents with 1-2 shooters, and not discriminating based on the number of type of locations or assumptions about motives. We also advocate for including unsolved incidents as well. In a recent report Tara Leigh Tober and I put together for the Mass Casualty Commission in Canada (HERE), we argue for this same definition. We can and should still examine whether and how incidents meeting the narrower criteria differ from others captured by this more broad definition.
More data will allow us to look at different patterns. It will enable us to examine patterns in different types of firearm-related incidents injuring large numbers of people. It will help us to include incidents excluded by the narrower definition that should be examined as “mass shootings” as well, but are sometimes excluded. Consider that the recent subway shooting in NYC in which the shooter released a smoke grenade on a moving train, put on a smoke mask, pulled out a handgun, and fired into the smoke-filled train until the train came to a stop and the next station. It was a horrifying crime. Thankfully, no one was killed. But that also means that, according to many databases, that crime may have been awful, but it wasn’t a “mass shooting.” Our definition captures that crime. Our definition captures Columbine high school (an incident with two shooters). Our definition captures Sandy Hook (sometimes not included for two reasons: fatalities/injuries at multiple locations as well as part of the crime being associated with family violence). Lots of high profile incidents sometimes referred to as “mass shootings” are either not included in databases and scholarship on mass shootings, or are included with with an exception clause (as is the case for Columbine in the Mother Jones database, for instance).
I hope this article reaches a lot of people and helps us to recognize the value in a broader definition for this uniquely American problem. Below is the full citation and link again if you’re interested in reading more (it’s open access) and available for free to all.
Bridges, Tristan, Tara Leigh Tober, and Melanie Brazzell. 2023. “Database discrepancies in understanding the burden of mass shootings in the United States, 2013–2020.” The Lancet Regional Health – Americas 22: 1-8.
Inequality by (Interior) Design 