No compelling evidence of racial bias in sentencing, per new meta-analysis

The question of racial bias in the criminal justice system is a highly debated issue. In recent years, it has become a common belief that the criminal justice system is racially biased, wherein black and Latino defendants receive harsher sentences than whites or Asian defendants. But the empirical research does not fully support these perceptions, according to a recently published meta-analysis (note: a subscription is required for access).

After carefully reviewing 51 studies published between 2005 and 2022, authors Chris Ferguson and Sven Smith of Stetson University found little compelling evidence of racial bias in criminal sentencing, ultimately concluding that “the criminal justice system appears to be remarkably neutral.”

Methods

The authors were interested in assessing whether race was associated with disparities in sentencing. They also analyzed how other factors, such as age or prior convictions, could explain the variation in sentencing outcomes.

The studies varied in their methodology, but all of them used statistical significance tests to determine whether there were differences in sentencing outcomes across races. The significance tests generated two key data points essential for interpretation. The first is the “p-value,” which indicates whether a relationship exists between the predictor variable (race) and the outcome (sentencing decision). If the p-value is 0.05 or less, it is considered “statistically significant” and implies that the sentencing decision is likely related to race.

The second is the “effect size,” which reflects the magnitude of that relationship. There are different methods for calculating the effect size (e.g., F-value, r-squared, Cohen’s d, etc.), but they all measure the same thing: how much variation in the outcome can be attributed to the specified predictor. Generally speaking, effect sizes should be able to explain at least 10% of the variation in an outcome to be considered practically meaningful from a policy standpoint.

This is especially true when studies are conducted on massive samples in the hundreds of thousands. Large samples have high levels of statistical power, which is great for being able to detect a statistically significant effect; however, it also amplifies the statistical power of errors, wherein “noise” effects become statistically significant. So, if the effect size is too small, it’s hard to know whether it exceeds noise effects. Unfortunately, many researchers naively focus on interpreting p-values and reporting significant effects, but leave out important context as it relates to the effect size and the practical utility of their findings. This can lead to false confidence in scholars’ theories.

The same can be said for the current meta-analysis. Given the high level of statistical power, it is difficult to not find statistical significance. Thus, the authors adhered to the 10% cutoff as the minimum for practical significance in their own interpretations as well. After extracting a total of 120 effect sizes from the 51 studies, they converted them all into a type of effect size known as a “correlation coefficient” or “beta” so that they would all be in the same format. Then, the researchers calculated an overall effect size for each racial comparison dyad.

Results

For violent crime and property crime, the differences between races were so small that they were indistinguishable from statistical noise. The effect sizes indicated that less than 6% of the variation in sentencing outcomes could be attributed to any one race. These differences are not substantial enough to be attributed to deliberate or systematic bias.

For drug crimes, the results were suggestive of small disparities when comparing whites to other races. However, the effect sizes indicated that the difference may not be practically meaningful, with race explaining less than 2.2% of the variation regarding adjudication of drug crimes.

Higher-quality studies were less likely to suggest racial disparities. As more situational and contextual factors are controlled for, racial disparities become less pronounced. This is likely because there are factors unrelated to race, such as criminal history or age, that can impact the relationship that also need to be accounted for to get a clearer picture of the impact of race alone. Many of the studies failed to control for these important variables, making them more prone to finding artificial results.

Discussion

It is likely that many people overestimate the racial bias of the criminal justice system. Overall, the results suggest that the presence of bias in the criminal justice system based on race and class remains inconclusive and is likely not as prevalent as previously assumed.

These results are consistent with other data, which suggests that overrepresentation among perpetrators of crime explains incarceration disparities to a greater degree than does racism in the criminal justice system. Differences in crime commission rates will bring people into contact with the criminal justice system at different rates; these are unlikely to relate to race per se but rather to community factors. Although some might express the concern that overpolicing might explain such discrepancies, similar ethnic differences are seen for victims of crime as well, with most violent crime being intra-racial. Thus, it is reasonable to believe that any differences in sentencing between races are attributable to differences in crime commission rates.

All things considered, a lot of the evidence of racial bias in the U.S. criminal justice system has been weak, but this seems to go ignored in favor of the systemic racism narrative. It has been observed that progressive worldviews on race have become status-signaling, and critical evaluation of such beliefs is taboo. It’s quite possible that this has resulted in substantial miscommunication of research data to the general public, including the overstatement of sentencing disparities. In Jason Riley’s review of the meta-analysis, he takes a similar sentiment, implying that the current narrative only “helps activists raise money, and it helps politicians secure votes.”

Conclusion

The findings from this meta-analysis present a crucial contribution to the ongoing discourse surrounding biases in the U.S. criminal justice system, providing a basis for more nuanced discussions. The results suggest that, for most crimes, criminal adjudication is not substantially racially biased. For drug crimes, there appear to be very small race differences, though these effects were so small that they were not practically significant. Overall, perceptions of bias in U.S. criminal adjudications do not seem proportionate to the available evidence.