Early Intervention System for Reducing Adverse Police Incidents

Partner(s): Charlotte Mecklenburg Police Department, Metro Nashville Police Department, San Francisco Police Department, Benchmark Analytics
Status: In Progress
Github Repo: https://github.com/dssg/police-eis
Team: Klaus Ackermann, Jen Helsby, Sam Carton, Kenny Joseph, Ayesha Mahmud, Youngsoo Park, Sumedh Joshi, Jonathan Keane, Josh Mausolf, Lin Taylor, Tom Davidson, Henry Hinnefeld, Ned Yoxall, Allison Weil, Andrea Navarette, Hareem Naveed, Adolfo De Unanue, Joe Walsh, Lauren Haynes, and Rayid Ghani

Background
Adverse incidents between police and the public such as deadly shootings, unnecessary use of force, or instances of racial profiling can cause serious or deadly harm and result in costly litigation. They can also damage police legitimacy and increase mistrust between citizens and the police, leading to further negative interactions. Evidence suggests these events can be prevented by targeting interventions based on an Early Intervention System (EIS) that flags police officers who are at high risk for involvement in such adverse events. Many large police departments have developed “early interventions systems” to try to identify officers most likely to have adverse interactions with the public (adverse interactions include violations of department rules of conduct, unjustified use of force, civilian or officer injury, and discourteous officer behavior). These systems are typically threshold-based, flagging officers when certain complaint or use-of-force thresholds are reached within a given period of time. The main problems: these systems are neither accurate at predicting future risk nor identifying preventable incidents. As a result, these systems do not allow police departments to prioritize officers in need of interventions.

Our Solution
Our work was initiated in March 2015 as part of the White House’s Police Data Initiative launched out of President Obama’s Task Force on 21st Century Policing. As part of this effort, we discussed data-related challenges with several US police departments, and they noted that existing EISs were ineffective. In collaboration with the Charlotte-Mecklenburg Police Department (CMPD) and the Metropolitan Nashville Police Department (MNPD), The Center for Data Science and Public Policy built the first data-driven, machine learning-based Early Intervention System (EIS). It is now deployed and operational at several police departments and licensed to Benchmark Analytics to scale to police departments globally.

By combining comprehensive data from various sources inside a police department (HR, internal affairs, dispatches, arrests, stops, etc.), our data-driven, predictive Early Intervention System can be used to focus interventions (training, supervisor counseling, psychological counseling and Employee Assistance Program referrals) on officers who would most benefit in terms of reducing adverse incidents. This system will allow police departments to:

    • Decrease adverse police incidents with the public – We can improve over the threshold-based systems used in many police departments by identifying and using only the factors that best predict whether an officer will have an adverse incident.
    • Target limited resources on the officers that most need additional support – Supervisors will be able to use this system to see which officers are most in need of counseling, training, or other assistance to best prepare them to deal safely and positively with individuals in their communities and the high stress nature of their job.
    • Reward low-risk officers and model their behavior – Our EIS produces a ranked list of every officer in the department based on their risk of an adverse incident. This allows the department to identify not only officers at high risk but also those with exceptionally low risk.  
  • Evaluate the effectiveness of existing interventions – and develop new ones based on the risk factors for their officers.
  • Identify the “risk” of other types of incidents – Our EIS can be customized to predict the “risk” of any type of event concerning an officer, such as injuries, complaints, or even compliments. The department can determine the specific outcomes that are most important to them, that they want to increase or decrease, and can adjust the system based on the number and type of interventions available.

Data
Our system is able to take data from multiple sources and use it to build predictors that can generate a risk score for an individual officer. In most departments we’ve worked with, we use any data we can get about an officer including officer demographics, training, payroll, on-the-job actions, internal affairs data (complaints, investigations, reviews of incidents, etc.), dispatch data, stops, arrests, adverse incident reports, as well as publicly available data (such as the American Community Survey and open data portals including 311 and 911 calls).

Analysis
Our system uses historical data about officers to build predictive models that can generate accurate risk scores for an officer being involved in an adverse incident (of various types) in the next 3, 6, or 12 months. We explored different machine learning methods and currently use random forests. More details of our methodology are available in the publications listed below.

Results
Compared to existing Early Intervention Systems, our data-driven system can correctly identify 10-20% more officers who go on to have adverse incidents over a 12-month period while reducing the false positives by 50-60%. This improved capability allows police departments to effectively and efficiently identify at-risk officers and provide them the necessary preventative interventions before an adverse incident occurs.

Future Plans and Areas for Improvement
Our EIS was deployed and went live at Charlotte-Mecklenburg Police Department in November 2017, becoming the first data-driven Early Intervention System in use. CMPD built a web interface for the EIS that not only incorporates the risk score with an officer’s history but also collects supervisor feedback on the officer risk as well as intervention information. The EIS will improve based on the feedback from the department supervisors and the IA team, and we will conduct intervention analysis to understand which interventions work, which of them are effective for what types of risks and officers, and the most effective ways to reduce future risk and improve policing. The Metropolitan Nashville Police Department also uses our EIS to notify supervisors across the department of officers at risk of adverse incidents. We recently partnered with Benchmark Analytics to bring our data-driven EIS and predictive models to a usable product that can be deployed at police departments. More details on the details of the partnership are available here. We are working with Benchmark Analytics to launch a Research Consortium comprised of leading academics, scholars, and practitioners in early warning/early intervention/police reform/law enforcement. Members of the Consortium will have access to the longitudinal data set to use for research and analysis in order to inform best practices and support information sharing as well as ongoing research and policy advancement focused on policing.

Awards
The International Association of Chiefs of Police and Laura and John Arnold Foundation recognized our work with the Charlotte-Mecklenburg Police Department with the 2018 Leadership in Law Enforcement Research Award.

Publications
Early Intervention Systems: Predicting Adverse Interactions Between Police and the Public. Jennifer Helsby, Samuel Carton, Kenneth Joseph, Ayesha Mahmud, Youngsoo Park, Andrea Navarrete, Klaus Ackermann, Joe Walsh, Lauren Haynes, Crystal Cody, Major Estella Patterson, and Rayid Ghani. Criminal Justice Policy Review . Vol 29, Issue 2, pp. 190 – 209. March 9, 2017.

Identifying Police Officers at Risk of Adverse Events. Carton, S.*, Helsby, J.*, Joseph, K.* , Mahmud, A.*, Park, Y.*, Walsh, J., Haynes, L., Cody, C., Patterson, E., Ghani, R. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2016).

Building Better Early Intervention Systems. Crystal Cody, Estella Patterson, Kerr Putney, Jennifer Helsby, Joe Walsh, Lauren Haynes, Rayid Ghani, Samuel Carton, Kenneth Joseph, Ayesha Mahmud, Youngsoo Park. Police Chief Magazine. 2016.

Deploying Machine Learning Models for Public Policy: A Framework. Klaus Ackermann, Joe Walsh, Adolfo De Unánue, Hareem Naveed, Andrea Navarrete Rivera, Sun-Joo Lee, Jason Bennett, Michael Defoe, Crystal Cody, Lauren Haynes and Rayid Ghani. 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2018).

Media Coverage
Can Police Use Data Science to Prevent Deadly Encounters? Scientific American
How Science Could Help Prevent Police Shootings. Mother Jones.
U. of C. researchers use data to predict police misconduct. Chicago Tribune
We Now Have Algorithms To Predict Police Misconduct. FiveThirtyEight
US police use machine learning to curb their own violence. New Scientist

Videos
DSSG 2015 https://www.youtube.com/watch?v=eiA74XXGSF8
DSSG 2016 https://www.youtube.com/watch?v=nGoe2dLHogs
DSSG 2018 https://www.youtube.com/watch?v=GZ4MzxdaM_g
KDD 2016 https://www.youtube.com/watch?v=SFGvmrJ5rjM

University of Chicago https://www.youtube.com/watch?v=_p9Cz8MJ0Mw