An explosion of research into the effects of shall-issue laws on violent crime was triggered in 1997 by the publication of analyses using county-level data from 1977 to 1992. Using these data, Lott and Mustard (1997) concluded that states implementing shall-issue laws saw significant decreases in rates of violent crime, murder, rape, and assault. Their "more guns, less crime" conclusion was immediately controversial and led to a proliferation of studies exploring the robustness of the study's findings to alternate model specifications and to improvements or expansions to the data series. The table below lists studies from this early period of responses to Lott and Mustard (1997), as well as their counter-responses. Two important reviews of the scientific literature on gun policy effects—one by the National Research Council (NRC), a part of the National Academy of Sciences (NRC, 2004), and one by the Community Preventive Services Task Force, established by the U.S. Department of Health and Human Services (Hahn et al., 2005)—evaluated this early literature and reached nearly identical conclusions. In their review of existing studies examining shall-issue laws, Hahn et al. (2005) found insufficient evidence for determining the effect of such laws on violent crime. NRC (2004) reviewed much of the same literature and reanalyzed data that were common to many of these analyses: a panel data set originally spanning 1977–1992, then expanded through 2000. After reviewing many of the studies listed in the table below, the NRC (2004) panel, with one member dissenting, concluded:
a These studies are treated in our analysis as being superseded by later studies by the same authors. b This same paper was earlier circulated as Lott, Plassmann, and Whitley (2002). In addition to the sensitivity of results to minor changes in model specification noted by the NRC report, these early studies suffered from multiple serious problems with data and methodology that lead us to discount their value for informing this synthesis of evidence on the effects of shall-issue laws. These problems include the following:
Finally, we regard a majority of these early studies as having been superseded by later work by the same authors that improved upon their earlier contributions to this literature. As a result, we focus on their later efforts to evaluate the effect of shall-issue laws. We first describe studies published since 2004 that aimed to estimate the effects of concealed-carry laws on violent crime using county-level data. We then turn to studies that focused on state-level data, then studies that employed city-level data. We conclude by discussing results from a set of studies in which the objective was not to identify the effects of shall-issue laws but that nonetheless present estimates that may be considered part of the evidence base for how concealed-carry policies influence violent crime outcomes (e.g., some studies of the effects of abortion rates on violent crime include shall-issue laws as a covariate in their models). County-Level StudiesMany important shortcomings of county-level crime data identified through the early studies of shall-issue laws (see the table above) resulted from the fact that large numbers of county police agencies do not report crime statistics to the Federal Bureau of Investigation (FBI). Moreover, the way that county crime statistics address these missing data changed abruptly in the early 1990s, making data from the earlier part of the series not comparable with later data, according to the National Archive of Criminal Justice Data (undated). Nevertheless, several analyses have continued to use county-level crime data to evaluate law effects, or they have used homicide data from the Centers for Disease Control and Prevention (CDC)'s National Vital Statistics System, which has less of a problem with missing data (Loftin, McDowall, and Fetzer, 2008). Roberts (2009) used the FBI's Supplementary Homicide Reports to analyze the effect of shall-issue laws on intimate partner homicide rates using monthly county-level data spanning 1985–2004. The author found that (the more-restrictive) may-issue laws significantly increased intimate partner total homicides by 71 percent compared with shall-issue laws, but may-issue (compared with shall-issue) laws had an uncertain effect on intimate partner firearm homicides. The author also found uncertain effects of concealed-carry bans compared with shall-issue laws on either overall or firearm-related intimate partner homicides. However, neither analysis clustered standard errors at the state level, so serial correlation that was unaccounted for in the panel data likely resulted in underestimated standard errors and correspondingly misleading tests of statistical significance. Aneja, Donohue, and Zhang (2014) analyzed the county-level data set used in NRC (2004), extended through 2006, and state-level data through 2010. The authors corrected the NRC analyses for several errors that they identified, including data-coding errors related to the timing of shall-issue legislation, an endogenous control variable (arrest rate), and a failure to cluster standard errors at the state level. The authors argued that the decision in NRC (2004) not to cluster the standard errors of the county-level analyses at the state level was incorrect and showed that confidence intervals (CIs) were badly misestimated when clustering was not accounted for. In their preferred county-level specification including state trend effects, they found no statistically significant effects of shall-issue laws on either the level or trend of any of seven crime rates, and they found only one suggestive effect across the 14 effects they tested. Moody et al. (2014), responding to an earlier version of the Aneja, Donohue, and Zhang (2014) paper, reestimated their models after adding many more demographic control variables, robbery and assault rates, and a lagged outcome as a predictor meant to capture unmeasured state differences associated with crime rates. Moody et al. (2014) offered statistical tests suggesting that the model with added covariates predicted the data significantly better, which the authors interpreted as evidence that estimates in Aneja, Donohue, and Zhang (2014) suffered from omitted-variable biases. The revised hybrid model results in Moody et al. (2014) suggested that shall-issue laws significantly reduced the trends in rape and murder rates. They found no significant association between shall-issue laws and either assault or robbery. The fact that their model predicted a given outcome better than the Aneja, Donohue, and Zhang (2014) model is not sufficient to demonstrate the claim that the latter's model suffered from omitted-variable bias or that the model preferred by Moody et al. (2014) offered a less biased estimate. An overfit model can predict the data exceptionally well while producing biased and unreliable coefficient estimates. Using county-level panel data spanning 1979–2000, Durlauf, Navarro, and Rivers (2016) examined the sensitivity of analyses that estimate the relationship between shall-issue laws and violent crime. They reported that use of population weights may lead to inefficient estimates and upward biases in estimates of the effect of shall-issue laws on crime. In addition, they found that hybrid or spline models are preferred to dummy models and that models that allow for heterogeneity in the effect of laws (including effects that vary with region, rates of gun ownership, and the level of urbanization in an area) outperform models that do not allow for variation in effects. For the spline model specifications that the authors assessed to perform best for the outcome of violent crime, they estimated that shall-issue laws increase violent crime in the first year after law passage and that violent crime continues to increase in subsequent years. The authors concluded that, overall, there was substantial variation in the estimated effects for each model across the model space analyzed and, thus, there was little evidence that shall-issue laws generate either an increase or a decrease in crime on average. Crifasi et al. (2018b) evaluated the effects of shall-issue laws and four other gun laws on homicides in large, urban counties between 1984 and 2015. Using a Poisson model that included year fixed effects, random effects for counties, and county-level demographic and economic covariates, the authors found that shall-issue laws were associated with a significant increase in firearm homicide rates. Specifically, after implementing these laws, counties would be expected to see 1.04 times more firearm homicides (95-percent CI = 1.02, 1.06). The authors also included a comparison outcome, nonfirearm homicides, on the theory that, if the effect of shall-issue laws is correctly estimated, it should be found only for firearm homicides, not nonfirearm homicides. However, their estimate for nonfirearm homicides was virtually identical to the estimate for firearm homicides (incidence rate ratio [IRR] = 1.03; 95-percent CI = 1.00, 1.06), which raises questions about the model or the authors' theory that nonfirearm homicides should be unaffected by the law. The paper did not describe any corrections for serial correlation in the data used, without which incorrect claims of statistical significance would be expected to proliferate (Schell, Griffin, and Morral, 2018; Aneja, Donohue, and Zhang, 2014; Helland and Tabarrok, 2004). State-Level StudiesHepburn et al. (2004) evaluated the effects of shall-issue laws on homicide rates using data from 1979 to 1998 in a study that came out too late to be reviewed in either the NRC (2004) or the Hahn et al. (2005) reviews of firearm research. Using a negative binomial model with two-way fixed effects and controlling for demographic and economic variables, including a proxy for gun ownership, the authors found uncertain effects for shall-issue laws on state homicide rates. Estimated effects remained uncertain in subgroup analyses of adults aged 25 or older and of white men aged 35 or older (see the first figure below). Rosengart et al. (2005) examined the effect of several state gun laws, including shall-issue laws, on firearm homicides and total homicides using state-level data. One limitation was that the data covered only 1979–1998, and other studies have shown the sensitivity of results to shorter periods, partly because shorter periods include observation of fewer states that have adopted shall-issue laws. The policy variable was specified as a dummy variable (indicating that a shall-issue law was or was not in place). The authors found suggestive effects that shall-issue laws increased firearm and total homicide rates. French and Heagerty (2008) tested the sensitivity of these results and similarly concluded that shall-issue laws had a suggestive effect consistent with the laws increasing firearm-related homicide rates, although estimates varied across specifications. However, the Rosengart et al. (2005) paper, and presumably the French and Heagerty (2008) paper, also had an unfavorable ratio of model covariates to observations (less than one to eight), suggesting that the model may have been overfit, and thus its estimates and their CIs may be unreliable. Martin and Legault (2005) demonstrated that Lott (2000) used incorrect state crime rate estimates that differed substantially from official FBI state estimates. They replicated Lott (2000)'s model despite misgivings about its specification to demonstrate that the effects Lott reported were sensitive to this measurement error. In their replication exercise using state-level crime data from the FBI's Uniform Crime Reports spanning 1977–1992, Martin and Legault (2005)'s estimates showed that shall-issue laws significantly reduced total violent crime and, specifically, aggravated assault. They found only suggestive effects that the laws reduced rates of robbery and murder, as well as uncertain effects on rape (see the second figure below). However, as with Lott (2000), the authors did not statistically adjust for serial correlation in the panel data, and the model's ratio of estimated parameters to observations was less than one to ten, meaning the model may have been overfit, and thus its parameter estimates and their CIs may be unreliable. Grambsch (2008) conducted a state-level analysis of (total) murder rates (relative to the U.S. murder rate) from 1976 to 2001 using the 25 states that passed shall-issue laws between 1981 and 1996. She found a selection effect among states adopting shall-issue laws—namely, that states that passed shall-issue laws in this period experienced an increasing trend in murder rates prior to adoption relative to other states. Her estimates showed that, after controlling for regression to the mean, there was either an uncertain effect or a significant positive effect of shall-issue laws on relative murder rates (i.e., shall-issue laws increased murder rates) depending on the model used. However, the model finding significant effects (the state fixed-effects model) had fewer than ten observations per estimated parameter, meaning the model may have been overfit, which can lead to unreliable estimates and standard errors. Furthermore, neither model included adjustments for serial correlation in the panel data. Using a panel of state data, Lott (2010) provided an update of his earlier analyses examining the effect of shall-issue laws on violent crime. His preferred specification included a set of dummy variables that indicated different time intervals before and after shall-issue legislation was in effect for states that passed such legislation. Many of Lott's modeling results were presented as figures and did not indicate statistical significance. Detailed results were provided only for an analysis of homicide rates. These included information on the statistical significance of each coefficient in the model but not for a test comparing post-implementation time intervals with pre-implementation time intervals. Lott interpreted the pattern of effects as demonstrating that homicides declined significantly after implementation of shall-issue laws, but he did not provide test statistics or sufficient description to clarify what specific effect was observed. The author also included coefficients and their statistical significance from dummy and spline models similar to those from his earlier work, but he did not include standard errors or test statistics. All of the preferred models appear to have had a ratio of estimated parameters to observations that was less than one to ten, meaning the model may have been overfit, and thus the reported estimates and their CIs may be unreliable. Similarly, it does not appear that Lott used any adjustments for serial correlation in his panel data, so some of the effects reported as statistically significant might not be after correcting these analyses (Schell, Griffin, and Morral, 2018; Aneja, Donohue, and Zhang, 2014; Helland and Tabarrok, 2004). DeSimone, Markowitz, and Xu (2013) evaluated the effects of child-access prevention laws on nonfatal injuries using data from 1988 to 2003, but they included sensitivity analyses that controlled for shall-issue laws. Using fixed-effects Poisson regression models, they found that shall-issue laws were significantly associated with firearm assault injuries for children under age 18, as well as for adults. Specifically, their estimate suggests that, after a state implemented a shall-issue law, assault injury rates were more than double what would have been expected without the law (see the second figure below), which would be extraordinary if true. However, the estimated effects of shall-issue laws in this study were based primarily on implementation in one state that changed its law during the study time frame (Arizona); thus, the study offers little evidence that the observed effects are due to the change in the law rather than to other factors affecting the state's assault rate that occurred around the same time the law was changed. Webster, Crifasi, and Vernick (2014) analyzed state-level data from 1999 to 2010, using generalized least-squares regression models to estimate the effect of shall-issue laws on age-adjusted homicide rates. They found suggestive effects indicating an association between the implementation of shall-issue laws and a 10-percent increase in rates of nonfirearm homicide, a 6-percent increase in rates of total homicide, and an 11-percent increase in rates of murder and nonnegligent manslaughter.[1] However, their estimates showed an uncertain association between shall-issue laws and firearm homicide rates. The statistical model used to arrive at these results used a large number of estimated parameters relative to observations (a ratio of about one to eight), meaning the model may have been overfit, and thus its estimates and their apparent statistical significance could provide little generalizable information about the true causal effects of shall-issue laws. Gius (2014) examined the effect of shall-issue laws on gun-related murder rates using state-level data from 1980 to 2009. He found that states with may-issue or more-restrictive policies had higher gun-related murder rates than shall-issue states. Relative to states with shall-issue laws, states with more-restrictive firearm-carry policies had rates of firearm homicide that were 11 percent higher (see the second figure below). However, this model did not statistically adjust for the known serial correlation in these panel data, which has been shown to result in misleadingly small standard errors (Schell, Griffin, and Morral, 2018; Aneja, Donohue, and Zhang, 2014; Helland and Tabarrok, 2004). For this reason, the apparently significant effect observed in this study could be invalid. Using their preferred specification with state-level data from 1979 to 2010 and a dummy, spline, or hybrid specification of shall-issue laws without state trends, Aneja, Donohue, and Zhang (2014) found suggestive evidence that shall-issue laws increase assaults by 8 percent (see the first figure below). In the dummy specification, shall-issue laws significantly increased rape by 12 percent, although estimates of this effect from the spline model were uncertain. The authors also found suggestive evidence that shall-issue laws increased rates of robbery, although estimates again became uncertain in other specifications. Effects of shall-issue laws on murder rates were uncertain. The authors tested the sensitivity of their results to less-parsimonious (including the Lott and Mustard [1997] specification) and more-parsimonious demographic specifications; the inclusion of state-specific time trends; the inclusion or exclusion of years that were likely to be influenced by the crack cocaine epidemic, which affected crime rates; and the specification of the policy variable (dummy, spline, hybrid). The authors noted that their results, which showed that the significance and sign of estimated effects varied substantially depending on the specification employed, underscored the sensitivity of gun-crime modeling estimates to modeling decisions. Moody et al. (2014) and Moody and Marvell (2018a) critiqued several modeling decisions of the Aneja, Donohue, and Zhang (2014) paper, as well as an earlier version of that study (Aneja, Donohue, and Zhang, 2011). Foremost, the studies critiqued the decision to treat models without state-specific trends as the preferred ones.[2] Thus, Moody et al. (2014) reestimated the hybrid models in Aneja, Donohue, and Zhang (2014), incorporating state-specific trends and additional covariates into an analysis of state data. In doing so, the authors found, as they had with their county-level analyses, that their specification improved model fit over that of Aneja, Donohue, and Zhang (2014). They also found that the individual states' trends were jointly significant, which they took as evidence supporting the need for their inclusion in the models of shall-issue law effects. Using hybrid models that included state-specific linear trends, Moody et al. (2014) found that shall-issue laws significantly increased assault rate trends and increased robbery rate levels, but the laws also significantly reduced murder rate trends. In an updated analysis that favored using a series of leading and lagging indicators of shall-issue laws over the hybrid model specification, Moody and Marvell (2018a) found largely uncertain effects of shall-issue laws on violent crime outcomes. As noted earlier, neither study demonstrated that its model estimates were less biased than those in Aneja, Donohue, and Zhang (2014) or that the Aneja, Donohue, and Zhang (2014) model suffered from omitted-variable biases. Furthermore, the state-level analyses of Moody et al. (2014) used a statistical model with a large number of estimated parameters relative to observations (close to one to five), meaning the model may have been overfit, and thus the estimates and inferential statistics may provide little generalizable information about the true causal effects of shall-issue laws. In a series of analyses by John Donohue and colleagues, Donohue, Aneja, and Weber (2019) provided estimates of the effects of shall-issue laws; the study used updated data covering 1977–2014, during which 33 states implemented these laws. The authors' two-way fixed-effects model—controlling for demographic, economic, and law enforcement factors—indicated uncertain effects on the logged murder and firearm murder rates but significant increases in violent crime and property crime generally. Donohue, Aneja, and Weber (2019) also described an assessment of the effects of shall-issue laws that relies on constructing synthetic controls for each state that implemented a shall-issue law. Synthetic controls are weighted combinations of states that never implemented the law or that implemented it more than ten years after the treated state, such that, in the period before a state's passage of the law, the temporal pattern of crime in the synthetic control closely matches that in the state. Repeating this procedure for each of 33 states with shall-issue laws, the authors concluded that violent crime increased over a ten-year period in 23 of 31 states with at least ten years of post-implementation data. In aggregate, the authors estimated that, five years after law passage, states with shall-issue laws had violent crime rates that were 7 percent higher than expected, which rose to 14 percent after ten years. The authors calculated significance levels for these estimates using a permutation test designed to estimate the distribution of treatment effects under the assumption that laws have no real effect. They concluded that, after the seventh year post-implementation, states with shall-issue laws had significantly elevated rates of violent crime. Synthetic control methods are relatively new, and especially when controls are made up of just a few states, as they were in this case, their usefulness for identifying causal effects may be compromised (RAND Corporation, 2018, Appendix A). Barati (2016) explored whether the effect of shall-issue laws depends on whether the legal regime before implementing these laws was one of no issue (i.e., no one is permitted to carry concealed firearms) or may issue. The author used a weighted least squares regression of logged crime rates onto a model with state and year fixed effects; linear state-specific trends; and almost two dozen other social, economic, and legal covariates. When looking at violent crime outcomes, Barati (2016) found only a suggestive effect that the transition from no-issue to shall-issue laws caused a reduction in robberies. However, this model had an unfavorable ratio of estimated parameters to observations (about one to six), meaning the model may have been overfit, and its estimates and CIs may thus be unreliable. Luca, Malhotra, and Poliquin (2017) used data from 1977 to 2014 to evaluate the effects of various firearm laws on homicide rates among adults aged 21 or older. Although the authors' focus was on background check and waiting-period laws, they included model specifications that additionally controlled for concealed-carry and permitting laws. Their analysis was based on log-linear models adjusting for national trends, state fixed effects, and a limited set of state-level time-varying sociodemographic factors; they found that shall-issue and may-issue laws had uncertain effects on total and firearm homicide rates relative to no-issue regimes. Employing similar models but using data from the FBI's Uniform Crime Reports over a shorter time frame (1986 to 2015), Hamill et al. (2019) similarly found uncertain effects of adopting a shall-issue or permitless-carry regime on overall rates of violent crime, homicide, rape, and aggravated assault; findings for robbery rates showed suggestive but small decreases associated with moving from a more restrictive to a more permissive concealed-carry regime (see the first figure below). In contrast, using age-adjusted homicide rates and analyzing a shorter time period (1991 to 2015), Siegel et al. (2017b) found that, relative to may-issue laws, shall-issue laws resulted in significantly elevated rates of total homicide and firearm homicide. A shortcoming of the authors' analysis was that it dropped several years of data for six states after 1998, because the CDC began suppressing homicide counts below ten in that year. Nevertheless, the authors report similar results from sensitivity analyses using a different data source, the Supplementary Homicide Reports database, that does not have the same suppression issues. The authors report using "robust standard errors that account for the clustering of observations, serial autocorrelation, and heteroskedasticity" (p. 1927), but they appear to have used a standard error adjustment that accounted for only heteroskedasticity and not the serial correlation that characterized their state-level panel data. Indeed, in a commentary on this study, Donohue (2017)'s replication of Siegel et al. (2017b)'s analyses produced estimated effects with properly clustered standard errors that were nearly twice as large as those shown in Siegel et al. (2017b)'s main analyses. However, even with the increased uncertainty around the effect sizes, the estimated effects of shall-issue laws on total and firearm homicide rates remained positive and statistically significant. Shi and Lee (2018) estimated a panel data model with interactive fixed effects and spatial dependence in order to evaluate how shall-issue or permitless-carry laws affected crime rates from 1977 to 2012. In contrast with most prior studies of the effects of concealed-carry laws, the authors did not estimate regression models that directly controlled for state-level covariates that likely influence firearm legislation and crime rates (e.g., socioeconomic factors, changes in law enforcement resources). Instead, they accounted for (potentially) nonlinear state-specific time trends as a function of unobserved national time trend factors interacted with state-specific factor loadings that determine the degree to which each state was differentially affected by the time trend factors. Their first-differences models also included a lagged outcome variable and covariates to account for potential spatial spillover effects. Their results were mixed. Some outcomes (e.g., robbery) indicated a significant increase immediately after shall-issue law enactment followed by a declining trend, while other outcomes (e.g., murder) showed significant declines but not until more than five years after law passage. Effects on rape rates and assault rates were uncertain or suggestive, depending on when (i.e., how long after implementation) the effect was assessed. However, for both outcomes showing significant effects, the study's models had an unfavorable ratio of estimated parameters to observations (about one to three for murder rates and one to nine for robbery rates), which suggests that these models may have been overfit and thus produced unreliable estimates and CIs. Finally, two studies estimated how shall-issue laws affected fatal or nonfatal assaults on police officers (Mustard, 2001; Crifasi, Pollack, and Webster, 2016). Mustard (2001) preferred a spline model, estimating the change in trends before versus after implementation of shall-issue laws for the outcome of felonious police deaths per capita or per full-time equivalent police officer from 1984 to 1996. Across multiple specifications (e.g., Poisson, Tobit), the author tended to find that shall-issue laws had uncertain effects, except when the outcome was measured as police deaths per full-time equivalent officer; in that case, shall-issue laws led to a negative shift in trend that was statistically significant. However, this model had an unfavorable ratio of estimated parameters to observations (about one to seven) and did not account for serial correlation within states, which suggests that the estimated effects and associated CIs may be unreliable. Crifasi, Pollack, and Webster (2016) extended the period of study through 2013 and instead evaluated how shall-issue or permitless-carry laws affected fatal or nonfatal assaults on law enforcement officers, measured as a rate per full-time equivalent officer. The authors found uncertain effects of the laws on fatal assaults but a suggestive effect (p = 0.13) consistent with less-restrictive concealed-carry laws resulting in lower rates of nonfatal assault on law enforcement officers. City-Level StudiesKovandzic, Marvell, and Vieraitis (2005) examined the effect of shall-issue laws on violent crime (homicide, robbery, assault, and rape) using panel data from 1980 to 2000 for 189 large U.S. cities. The authors clustered the standard errors at the state level, addressed coding errors in previous research, allowed for a time trend in the effect of shall-issue laws, allowed for city-specific time trends, and conducted analyses that allowed for heterogeneity in the effect of shall-issue laws across states. In their analysis that estimated the average effect of shall-issue laws for all included cities using a dummy model specification, Kovandzic, Marvell, and Vieraitis (2005) found uncertain effects for all of the violent crime outcomes analyzed. These findings were largely consistent when they instead modeled the effects of shall-issue laws as a trend variable, except that their preferred spline models showed effects consistent with shall-issue laws increasing assault rates (a significant effect) and increasing rape rates (a suggestive effect). Their estimates for the effect on assault suggest that shall-issue laws were associated with a 10-percent increase in aggravated assault rates after five years. In examining state-specific effects with their spline models, the authors further found that there were more states where shall-issue laws led to statistically significant increases in crime compared with decreases. However, this study had an unfavorable ratio of model covariates to observations (less than one to ten), meaning the model may have been overfit, and thus its estimates and CIs may be unreliable indicators of the true effects of the laws. La Valle (2013) analyzed data from 56 cities spanning 1980–2010. The author noted that the analyses "include statistical corrections for variation in sample unit independence," but he did not explicitly mention clustering the standard errors at the state level. La Valle (2013) used a dummy variable specification for the concealed-carry law. In his preferred specification (using interpolated control variables for inter-censal years, population weighted analysis, and a one-year lagged outcome as a covariate), he found that shall-issue laws significantly reduced gun homicides by 15 percent and total homicides by 13 percent (see the first figure below). Results were sensitive to specification, however, and other authors (e.g., Kovandzic, Marvell, and Vieraitis, 2005; Durlauf, Navarro, and Rivers, 2016) have expressed concern that weighting gives undue influence to localities with large populations and worsens, rather than improves, standard error estimation. In unweighted analyses using inter-censal years, La Valle (2013) found that shall-issue laws reduced gun homicides but not total homicides. In La Valle and Glover (2012), which used similar data (panel data on 57 cities from 1980 to 2006) and a similar approach, the authors included separate indicators for may-issue and shall-issue states. In the authors' preferred analysis (with interpolated data for controls for inter-censal years and weighting), shall-issue laws were associated with a significant 23-percent increase in the homicide rate, and may-issue laws were associated with a significant 19-percent decrease in the homicide rate (compared with cities that the authors concluded did not have either a may-issue or shall-issue law). Similarly, shall-issue laws were associated with a significant 32-percent increase in the firearm homicide rate, while may-issue laws were associated with a significant 33-percent reduction in the firearm homicide rate. (No estimates for unweighted data with inter-censal years were provided.) The diametric findings from these two studies further highlight the sensitivity of results to model specification, as well as to how shall-issue laws are classified. Other StudiesThree studies that focused on the relationship between unmarried fertility or abortions and violent crime included shall-issue laws as a covariate in their models (Donohue and Levitt, 2001; Lott and Whitley, 2007; Kendall and Tamura, 2010). Using data from 1985 to 1997 and estimating weighted least squares with a logged outcome and state and year fixed effects, Donohue and Levitt (2001) found uncertain effects of shall-issue laws on violent crime and murder rates. Analyzing data over a partially overlapping period, from 1976 to 1998, and using a Poisson model that controlled for state and year fixed effects, state-specific linear trends, and time-varying state covariates, Lott and Whitley (2007) found suggestive or significant effects (depending on specification) indicating that murder rates fell approximately 1 percent faster after the adoption of shall-issue laws relative to the rates in states without such policies. Employing a different model specification over a longer period (1957–2002), Kendall and Tamura (2010) estimated that shall-issue laws had a suggestive but small association with reduced rates of murder and uncertain relationships with rates of rape, robbery, and assault. Zimmerman (2014) extended prior research evaluating the role of private security measures in reducing crime (e.g., see Benson and Mast, 2001). Although the author's focus was on understanding the crime rate implications of changes in employment within four private security occupation groups (security guards, detectives and investigators, security system installers, and locksmiths), he included shall-issue laws as a covariate in the models to account for the potential deterrent effects of allowing private citizens to carry handguns. Estimating linear models with a logged outcome and controlling for state and year fixed effects, state-specific linear trends, a lag of the dependent variable, and time-varying state characteristics, Zimmerman (2014) found that shall-issue laws led to significantly higher rates of murder and assault; estimated effects on robbery rates were suggestive but also consistent with an increase following the passage of shall-issue laws. However, the analyses had a ratio of estimated parameters to observations of less than one to five, and the paper provided no additional evidence to demonstrate model fit. Therefore, in accordance with our review methodology, we discount this evidence because of the possibility that the model was overfit, and thus the estimated effects and their CIs may be unreliable indicators of the true causal effects of the laws. Manski and Pepper (2018) investigated the sensitivity of shall-issue effect estimates to a range of assumptions by comparing property and violent crime rates in two states under progressively less-restrictive assumptions about how the laws' effects may vary over time or between states. This study compared outcomes in just two states, meaning causal effects were not well identified. Moreover, it treated Virginia's shall-issue law as having been implemented in 1989, when we believe the correct date is 1995. For these reasons, we do not review this paper's results. Applying Bayesian model comparison techniques, Strnad (2007) reanalyzed models of the effects of shall-issue laws from Donohue (2004). In contrast to the approach of Donohue (2004) and many others, Strnad (2007) did not assess the evidence for or against shall-issue laws in terms of how frequently estimates of the effect were statistically significant or were found to have positive (as opposed to negative) estimated effects under different model specifications. Instead, he used model comparison techniques to establish which models fit the data best and to evaluate whether evidence favored models with or without shall-issue effects. He concluded that Donohue's models provided much stronger support for a conclusion that shall-issue laws had little or no effect on most outcomes than Donohue (2004) concluded after examining patterns in the direction and significance levels of these effects. The exceptions were murder, which shall-issue laws appeared to cause to decline gradually, and robbery, which appeared to increase or decrease, depending on the state. The figures below display the IRRs and CIs associated with the concealed-carry laws examined in the studies published after the NRC (2004) review. The first figure displays the studies for which we found no serious methodological issues, and the second figure displays the studies for which we did find methodological issues. In these figures, we highlight effect estimates based only on dummy-coded models, for reasons discussed in the full report (see Smart et al., 2020, Chapter Two) and in the first edition of this report (RAND Corporation, 2018, Appendix A). We exclude the estimates from Zimmerman (2014) for having a ratio of estimated parameters to observations of less than one to five and thus serious potential issues with model overfit. Furthermore, Lott (2010), Shi and Lee (2018), and Moody and Marvell (2018a) did not provide enough information for us to calculate IRRs and CIs for their effect sizes of interest, so we do not include these in the figures. In addition, the estimates in Durlauf, Navarro, and Rivers (2016) were available only for the spline specification; Kovandzic, Marvell, and Vieraitis (2005) preferred their own spline model; Moody and Marvell (2009) and Moody et al. (2014) offered only a hybrid model; and Manski and Pepper (2018) and Strnad (2007) did not seek to produce a preferred estimate of the effect of shall-issue laws. Because we could not readily calculate unique effect sizes and CIs for these studies, we do not include them in the figures. This forest plot shows estimates of how this policy affects this outcome, based on the evidence in the studies examined. In particular, the graphic shows the standardized effect sizes (or IRRs) and their 95-percent CIs for each outcome. An effect size of 1.00 indicates that, after a state passes the law, we would expect the outcome (e.g., suicide or firearm suicide) to be unaffected. An effect size of less than 1.00 indicates that the law appears to reduce the outcome. For example, if the effect size were 0.92, we would expect the rate of the outcome to fall to 0.92 times the rate prior to passage of the law. Conversely, an effect size of more than 1.00 indicates that the law appears to increase the outcome by a factor equivalent to the effect size value. When the CIs do not include the value of 1.00, the estimated effect is statistically significant at p < 0.05.
NOTE: This figure includes only the studies reporting dummy-coded law effects published since the NRC (2004) review of gun policy effects. IRR values marked with empty circles indicate that we identified concerns with the study's methodology, and these concerns are described in the text above. Filled circles indicate that we identified no significant methodological concerns. An arrow on either end of a CI indicates that the interval is wider than can be displayed on the scale.
NOTE: This figure includes only the studies reporting dummy-coded law effects published since the NRC (2004) review of gun policy effects. The estimates from Kovandzic, Marvell, and Vieraitis (2005) are from the authors' dummy model specification rather than their preferred spline model. IRR values marked with empty circles indicate that we identified concerns with the study's methodology, and these concerns are described in the text above. Filled circles indicate that we identified no significant methodological concerns. An arrow on either end of a CI indicates that the interval is wider than can be displayed on the scale. ConclusionsBecause so much more study has been done of the relationship between concealed-carry laws and violent crime than of any other gun policy and outcome, there is a much richer evidence base to draw on, including studies raising serious methodological concerns and several that did not raise as many concerns among our methodology review team. Therefore, to focus this review on the best available evidence, we draw our conclusions in this section based just on those 18 studies that did not raise serious methodological concerns. We incorporate all studies that met this criterion in our discussion, but we prioritize findings from studies with a study time frame that extended beyond 2000. We do so because studies omitting more-recent data (1) identify policy effects excluding a large number of states that have enacted shall-issue laws in the past 20 years and (2) have limited post-implementation data to allow these policies to establish their full effects. Total homicides. Of the 18 studies without serious methodological concerns, 16 examined the effects of shall-issue laws on total homicides, and one examined the effects of the laws on fatal assaults of law enforcement officers. Of the eight studies that evaluated shall-issue laws and included data after 2000, five found only uncertain effects of these laws (Donohue, Aneja, and Weber, 2019; Hamill et al., 2019; Luca, Malhotra, and Poliquin, 2017; Crifasi, Pollack, and Webster, 2016; Aneja, Donohue, and Zhang, 2014). Kendall and Tamura (2010) found small suggestive effects consistent with shall-issue laws reducing homicides. Moody et al. (2014) found that shall-issue laws cause a downward trend in homicides, although a subsequent study that included four more years of data found uncertain effects of the law in seven of eight evaluated years, with a single significant negative effect in the seventh year (Moody and Marvell, 2018a). La Valle and Glover (2012) found that shall-issue laws increased homicides significantly relative to having no law for the legal carriage of a concealed firearm (no-issue laws); and La Valle (2013) found that shall-issue or may-issue laws reduce total homicides relative to no-issue laws. This result cannot be used to distinguish the effect of shall-issue laws per se, but it suggests that shall-issue laws, may-issue laws, or both contribute to reducing total homicides. Of the six studies focused on a period prior to 2000, two found that shall-issue laws caused a downward trend in homicides or murders (Strnad, 2007; Plassmann and Whitley, 2003), one found a suggestive negative effect (Olson and Maltz, 2001), and three found uncertain effects (Hepburn et al., 2004; Helland and Tabarrok, 2004; Ludwig, 1998). Because studies with comparable methodological quality reached inconsistent results, we find that the best available studies provide inconclusive evidence for the effect of shall-issue laws on total homicides. Firearm homicides. Eight of the 18 studies examined the effects of shall-issue laws on firearm homicides. Among these eight, six evaluated data past 2000, and there was one suggestive (Hamill et al., 2019) and one significant (La Valle and Glover, 2012) effect indicating that these laws increase firearm homicides. La Valle (2013) found that shall-issue or may-issue laws cause decreases in firearm homicide rates relative to no-issue laws. This result cannot be used to distinguish the effect of shall-issue laws per se, but it suggests that shall-issue laws, may-issue laws, or both contribute to reducing firearm homicides. The two studies evaluating the longest period (1977 to 2014) found uncertain effects of shall-issue laws on firearm homicides (Donohue, Aneja, and Weber, 2019; Luca, Malhotra, and Poliquin, 2017). One study examined the effects of the laws on fatal handgun assaults of law enforcement officers and found uncertain effects (Crifasi, Pollack, and Webster, 2016). Of the two studies focused on a period prior to 2000, one found that shall-issue laws increase firearm homicides (French and Heagerty, 2008), and the other found that the laws decrease firearm homicides (Olson and Maltz, 2001). With seemingly conflicting evidence, we conclude that the best available studies provide inconclusive evidence for the effect of shall-issue laws on firearm homicides. Robberies. Aneja, Donohue, and Zhang (2014) found a suggestive effect that shall-issue laws may increase robbery rates, while Hamill et al. (2019) instead found a suggestive effect indicating that shall-issue laws decrease robbery rates. Five studies, the three most recent of which included data after 2000, found largely uncertain effects of shall-issue laws on robberies (Moody and Marvell, 2018a; Moody et al., 2014; Kendall and Tamura, 2010; Helland and Tabarrok, 2004; Plassmann and Whitley, 2003). Therefore, we conclude that the best available studies provide inconclusive evidence for the effect of shall-issue laws on robberies. Assaults. Aneja, Donohue, and Zhang (2014) found a suggestive effect that shall-issue laws may increase assault rates, and Moody et al. (2014) found that shall-issue laws were associated with a significant upward trend in assault rates. In contrast, Moody and Marvell (2018a) found suggestive effects consistent with shall-issue laws leading to reduced assault rates, and Crifasi, Pollack, and Webster (2016) found that shall-issue laws had a suggestive negative effect on nonfatal assaults of law enforcement officers. Four studies, including two with data extending past 2000 (Hamill et al., 2019; Kendall and Tamura, 2010), found only uncertain effects of shall-issue laws on assault (Hamill et al., 2019; Kendall and Tamura, 2010; Helland and Tabarrok, 2004; Plassmann and Whitley, 2003). Therefore, we conclude that the best available studies provide inconclusive evidence for the effect of shall-issue laws on assaults. Rapes. Aneja, Donohue, and Zhang (2014) found that shall-issue laws significantly increase rates of rape. Moody et al. (2014) found that shall-issue laws produce a significant downward trend on rates of rape. Moody and Marvell (2018a) also found some evidence of significant declines in rape rates, although these effects did not emerge until four years after implementation of the law. Four studies, two of which included data past 2000, found uncertain evidence of an association between shall-issue laws and rape (Hamill et al., 2019; Kendall and Tamura, 2010; Helland and Tabarrok, 2004; Plassmann and Whitley, 2003). Therefore, we conclude that the best available studies provide inconclusive evidence for the effect of shall-issue laws on rapes. Violent crime. Two studies (Donohue, Aneja, and Weber, 2019; Durlauf, Navarro, and Rivers, 2016) aggregated all violent crimes into a single category and found that shall-issue laws significantly increase violent crime rates. Three studies, one of which included data past 2000, found uncertain effects of shall-issue laws on overall violent crime (Hamill et al., 2019; Helland and Tabarrok, 2004; Plassmann and Whitley, 2003). Because evidence for the effect of shall-issue laws on each component of violent crime is inconclusive, it could be argued that these two studies of the effect of these laws on all violent crimes should not suffice to suggest that there is more than inconclusive evidence for such an effect. However, because analyses on all violent crimes may have greater statistical power to detect any such effects, and because our scoring criteria indicate it, we conclude that there is limited evidence that shall-issue laws may increase violent crime. Originally published March 2, 2018 Concealed-Carry Laws Violent Crime
View the full project bibliography |