Risk, Values, and Pretrial Reform

September 21, 2018 | Evident Change

California recently passed legislation to eliminate the use of cash bail, which is a positive and necessary step toward a better justice system. The numerous problems created by cash bail and the unfair burden the cash bail system places on poor defendants—disproportionately people of color—are well documented. California has replaced cash bail with assessment of risk to inform decisions about pretrial release and detention.

Proponents of risk assessment at the pretrial decision point believe that these tools will help to reduce the influence of judicial bias, make pretrial release decisions more consistent, and reduce pretrial detention while having no negative impact on public safety or failure to appear in court. Opponents often claim that that risk assessment endangers civil liberties because the data used to construct these tools are drawn from structurally racist systems and institutions. Thus, using a risk assessment would mean that more people of color will be detained and experience the many negative impacts that come along with pretrial detention. How do we know whose perspective is correct?

Well, if that sounded like a trick question, it is. Of course, nothing about decisions of this magnitude is quite that cut and dried. To answer the question, we have to start back at the beginning of risk assessment, before the tool is even created.

While most people associate risk assessment with science and statistics, creating a risk assessment actually starts with a conversation about values and goals. For pretrial risk assessment, most jurisdictions may share the same goal: to reserve pretrial detention for those who, by the American Bar Association’s standards, “pose a substantial risk of flight, or threat to the safety of the community, victims or witnesses or to the integrity of the justice process.”

The values each jurisdiction uses to decide who poses a flight risk or threat to safety may vary. For example, County A may decide that any future arrest is a threat to public safety, so their risk assessment should thus measure the likelihood of any future arrest. Their instrument will be designed using (or “tuned to”) data associated with that outcome, and the risk levels it produces will reflect that value. County B may establish a threshold of future felony arrest (or violent felony arrest) as the threshold for a threat to public safety, in which case their risk assessment will be tuned to data associated with this outcome.

If County A and County B assessed the same group of arrested individuals using their own validated pretrial risk assessments, County B’s risk assessment will result in a smaller proportion of people being identified as high risk than will County A’s risk assessment. Values also come into play when jurisdictions determine how to use a risk level: Will high-risk defendants be detained? Will they be released with strict supervision and monitoring strategies?

In both scenarios, jurisdictions have created a risk assessment tool based on their parameters for public safety, but the two tools would lead to different aggregate outcomes. It is a strength of risk assessments that they can be tuned in this way, but it is also incumbent on jurisdictions to be intentional about the purpose and aim of the tools they implement. Risk assessment design encompasses critical decisions that affect the lives of people who are arrested and their families, and also determine how resources are used by the justice system. These decisions will also be influenced by the way a jurisdiction’s district attorney’s office typically charges crimes, the way law enforcement interacts with the community, and the way local and state laws differentially affect different communities, just to name a few. Clearly, risk assessments have a complex interaction with pretrial decision making.

So how do jurisdictions know what is important to them? In Evident Change’s work developing risk assessments for social service systems, we help people figure this out by sitting down with a broad range of community stakeholders and talking it out. These stakeholders include members of the community, judges, attorneys, law enforcement, advocacy groups, and others. Including all these people in the discussion is critical to ensuring that everyone affected by these decisions is heard and that the values of one group do not receive more weight than another’s. All groups must find a balance among victim rights, community safety, protecting civil liberties of the accused, and just and equitable decision making.

Research and data serve the foundation for developing and benchmarking these conversations. Existing data should be used to look at what is currently happening in a jurisdiction. Are the pretrial detention numbers far above what the jurisdiction thinks is appropriate or has the resources to support? Are there high rates of disparities that the jurisdiction wants to change? Examining the data shows everyone involved what is currently happening and whether that matches up with the ideals of stakeholders and community values.

One key element of this values conversation is racial equity. Our society has deeply entrenched structural and institutional racism, and we see staggering racial disparities at all decision points in our justice systems. Racial equity is a necessary component of any discussion about risk assessment. Notably, the pretrial release decision is a point at which many different sections of the criminal justice system intersect. It thus has the potential to introduce or mitigate bias and racism from all angles. This makes it especially important to talk about these issues in any construction of a risk assessment to be used pretrial.

Equity—whether racial, economic, or any other kind—must be present in conversations about risk assessment from the very beginning. As noted above, the justice system and the data that feed in and out of it reflect existing bias against poor people, people of color, and other non-privileged groups. That means that if a jurisdiction wants to reduce disparities via the use of a risk assessment, that risk assessment has to be designed in such a way that it can help mitigate existing system bias. Without this priority incorporated into the design, the tool may perpetuate existing bias. Tools can be designed with the express intent to promote equity and help reduce disparities; in a biased system, however, this may result in sacrificing predictive accuracy. These are the tradeoffs that all stakeholders need to discuss and understand.

It is worth noting that risk assessment can and should be part of a broader intentional strategy—across pretrial reform, including the court’s final decision on detention, and across a jurisdiction’s entire approach to criminal justice—to turn disparities around. In fact, without all these components, risk assessments have a limited ability to make a big difference in a system’s disparities. It is also important to note that risk assessments are just one factor in the decision to release or detain someone pretrial—judicial discretion and other factors are also at play and likewise can either mitigate or perpetuate bias and inconsistency.

It is Evident Change’s strong belief that risk assessment can be used for supportive interventions or for diverting people who do not need or benefit from government intervention in their lives away from the system. In the criminal justice system, a risk assessment intended (and used properly) to assist in diverting people away from the system can be a good thing.

The new California legislation, as well as legislation in many other jurisdictions, mandates the use of pretrial risk assessment. Given the importance of this decision point and the concerns with system data described here, Evident Change would only support the use of a pretrial risk assessment tool in the context of explicit standards related to equity, transparency, and validity. We urge the Judicial Council of California and any other bodies charged with overseeing pretrial risk assessment to mandate and monitor minimum standards for design, implementation, and ongoing evaluation to ensure the protection of civil liberties, the reduction of racial disparities, and the effective use of resources. These standards include the following.

• Commit to transparency in the design or selection of a pretrial risk assessment, including the specific outcomes on which the risk assessment is focused, and the characteristics that are included on the risk assessment scale.
• On a monthly basis, evaluate the impact that a risk assessment is having on pretrial liberties (including detention as well as liberties for those operating under restrictions while at home) for populations that are over-policed, over-arrested, and over-charged in relation to populations that are not.
• On a monthly basis, monitor the difference between risk assessment recommendations and actual judicial decisions to promote judicial transparency and accountability.

We urge jurisdictions to be as dedicated as they can to using their data to inform their practices. Though we have suggested that these outcomes be measured on a monthly or quarterly basis, it is feasible to monitor these impacts in real time, given the necessary data.

We have much more to say on the use of risk assessment in pretrial reform, and in the coming months we will share more thoughts and ideas with you, including the role of pretrial services and the court in the actual detention decision, and how risk assessments and data analysis can fit into a broader reform strategy. We look forward to being part of effective and fair pretrial reform and welcome your feedback.