2021 national crime rates are a mystery as FBI transitions to a new data system

Earlier this month, the FBI released the national-level crime statistics for 2021. According to that data, violent crime, particularly murder, remained a major issue in the United States. Crime remained relatively consistent from 2020 to 2021 with no statistically significant changes between years, though violent crime was still elevated compared to 2019 levels. From 2020 to 2021, national levels of violent crime decreased slightly (-1.0%), largely driven by decreases in robbery (-8.9%). Murders, however, increased (+4.3%). The data is available for download via the Crime Data Explorer, or it can be accessed using a new tool called the Law Enforcement Agency Reported Crime Analysis Tool (LEARCAT). But the data this year may be lower quality than years past, limiting our ability to draw inferences about national-level crime rates.

Policymakers and researchers rely on these data to understand state and national crime trends, but that may be more challenging this year. Unfortunately, the FBI’s plan to modernize its reporting of crime data has not gone according to plan, so it’s hard know how accurate these 2021 estimates are. The new system has advantages over the old system, but it is much more cumbersome and time-consuming to use, which has negatively impacted law enforcement agencies’ willingness to submit their data. And because of these low participation rates, there are huge gaps in nationwide crime statistics for 2021.

In January 2021, the FBI retired the Uniform Crime Report (UCR)’s Summary Reporting System (SRS) and finalized their transition to a new one, the National Incident-Based Reporting System (NIBRS). The transition began in 2015, however, agencies were still allowed to submit data via the SRS prior to the January 2021 deadline. Now that the transition is finalized, agencies are no longer able to use the SRS, and instead are required to submit their data either through NIBRS or not at all. Simply put, 2021 is the first year that crime statistics will be comprised solely from data submitted through NIBRS, making it different from years past.

SRS vs. NIBRS

The SRS came first in 1929, tracking the number of Part I crimes (i.e., murder, robbery, rape, aggravated assault, burglary, motor vehicle theft, larceny, and arson) reported to law enforcement. Its main limitations were that it only focused on Part I crimes and did not capture the details of each incident. It also used the “hierarchy rule” — which essentially means that only the most serious offense from each incident would be counted (e.g., if a murder and robbery occurred in the same incident, then the murder would be tabulated but not the robbery). But the system was simplistic and easy to use, and therefore had decent participation rates (usually ranging from 75%-95%).

In 1982, NIBRS was developed to improve data quality and facilitate more nuanced understanding of crime patterns. In contrast to the SRS, NIBRS extends to additional crimes beyond Part I offenses and captures data from a variety of specific fields, allowing for a more comprehensive analysis. In other words, a researcher could examine the location type of various robberies, weapons used, and glean information about the offenders involved. NIBRS also abandoned the hierarchy rule, so the data account for multiple offenses occurring in a single incident. In theory, this system would be preferable to the SRS, but it suffers from a major drawback: it is burdensome, and substantially increases the workload associated for law enforcement agencies who submit their data. It also doesn’t help that participation is (and always has been) voluntary, and there are no incentives for completing this task every year. With that in mind, it’s no surprise that participation is lacking: agencies need to prioritize crime control over data collection, particularly when violent crime is rampant.

Adoption of NIBRS

Of course, some agencies are more equipped to adopt NIBRS than others. For agencies with Record Management Systems (RMS), data collection is much easier, but it still requires specialized staff to extract that data and convert it into a usable format, as well as ensure that the data fields are congruent with NIBRS categories. The time and expense needed to make these necessary upgrades is a major barrier, particularly for agencies that don’t currently have a sophisticated RMS. As a result, NIBRS implementation has always lagged. Even thirty years after its introduction, NIBRS-contributing agencies represented only one-third of the agencies reporting to the SRS. At such a low rate, the sample could not produce valid national-level crime estimates.

The FBI was rightfully concerned about these low participation rates, which prompted them to partner with the Bureau of Justice Statistics (BJS) to form the National Crime Statistics Exchange (NCS-X) Initiative. NCS-X was a key component of the five-year transition from 2015-2021 and its main goal was to increase the number of agencies contributing to NIBRS. The NCS-X project provided more than $120 million in financial support to assist 72 large police agencies (750+ sworn officers) in their transition to NIBRS. Funds were used to provide technical expertise, data integration support, and other NIBRS-related training. By 2019, NIBRS-contributing agencies (n=8,497) accounted for 51.3% of the agencies that submitted to the SRS. Together, the jurisdictions covered about 146.5 million people, or about 44% of the U.S. population. This was a noticeable increase since 2012, but still was not near the level of national-level representation. BJS expected this estimate to rise to at least 75% by the end of 2021, but as of June 2022,  participation rates were only up to 64.5%.

Regional Differences in NIBRS Adoption

At present, NIBRS-contributing agencies cover about 66% of the U.S. population. This varies notably by region though, with participation being more concentrated in the South and Midwest and least concentrated in the Northeast. These regional disparities also limit the ability to produce valid national and regional estimates.

32 states had excellent participation rates, where NIBRS-contributing agencies represented more than 90% of the state popluation (e.g., Texas, Washington, Montana, Maine, Tennessee, and others). 14 states had fair participation rates, where NIBRS-contributing agencies represented between 40% and 90% of the state population (e.g., Mississippi, Arizona, Wyoming, Maryland, New Jersey, and others). 4 states had very poor participation rates, including New York (19% population coverage), Pennsylvania (17% coverage), California (7% coverage), and Florida (0% coverage). This is pretty concerning, considering that New York, California, and Florida are all major states with large popluations. Moreover, several major agencies (each serving populations of more than 900k) are absent from the data completely, including New York City (as well as Suffolk and Nassau counties), Los Angeles City (and Los Angeles County), San Jose, Phoenix, and three major counties in Florida (Miami-Dade, Orange, and Hillsborough counties).

At present, with so many agencies failing to report, 2021’s annual crime data has significant blind spots. When the FBI calculates national estimates, they are expressed with “confidence intervals” to indicate how precise the estimates are. Narrow confidence intervals mean the estimate is more precise, while wide confidence intervals mean the estimate is more uncertain. Because of the low 2021 participation rates, confidence intervals are very wide in some states, essentially conveying a lot of uncertainty about the estimates.

For an example, see Table 16 of this report and take a look at some of the state-level violent crime estimates for 2021. New York state had an estimated violent crime rate of 101.9 per 100,000, with lower and upper bounds of 44.8 and 158.9 (i.e., the confidence interval). This means that researchers are only confident that the violent crime rate was somewhere in that range, and 101.9 just happens to be the middle of that range. In other words, New York’s violent crime rate was 101.9 +/- 57.1. That’s a pretty big margin. In Alabama, the uncertainty was even worse, with an estimated violent crime rate of 596.3 +/- 260.9. In Indiana, the violent crime rate was estimated at 313.9 +/- 96.6. There were many other states with wide intervals (e.g., Missouri, Louisiana, Nebraska, and others), while several states had no violent crime estimates at all due to poor data quality (e.g., California, Florida, Pennsylvania, and others). Surely, some states have narrow confidence intervals (and thus, more confidence that the estimate is accurate), such as Utah (250.1 +/- 11.0), Idaho (249.5 +/- 9.6), and Massachusetts (304.7 +/- 2.7), among others. Some of these state-level estimates are likely fairly accurate, but the wide disparity between states is concerning and presents significant blind spots in our understanding of nationwide violent crime.

Conclusion

Compared to the SRS, NIBRS tracks information in much greater detail and covers more types of crimes. This is great in theory, but the crime estimates it produces are questionable. Due to low participation rates, the agencies contributing to NIBRS are not representative of our nation as a whole. While the SRS provided less detail, it produced better national-level crime estimates because more agencies participated. Further, the participation rates vary quite a bit between states, with extraordinarily low participation in some major states (e.g. California, New York). Some state-level estimates might be accurate on their own, but as of now, the 2021 data doesn’t tell us much about nationwide crime in the United States. It is unclear what will be done if NIBRS participation does not improve, but hopefully the system will become a more reliable data source in the future.

As a result, policymakers will have to exercise great care when using this limited data. They may wish to consider alternative sources for public safety information, such as the data on national homicide trends provided by Centers for Disease Control, or the data on non-fatal crime and violence provided by the BJS’s National Criminal Victimization Survey.