So You Want to Implement a Pretrial Release Assessment
November 9, 2018 | NCCD
Deciding to jail someone while they await trial is a high-stakes decision. Their liberty is temporarily taken from them without their being tried or adjudicated for a crime. Because of this, it is critical that criminal justice agencies make accurate, consistent, and fair decisions pretrial.
There are a variety of ways to support good pretrial decision making. Some agencies use a pretrial release assessment (also called a risk assessment or public safety assessment) to assess potential risk to public safety and the likelihood of failure to appear for a court date. But not all pretrial release assessments are created equal. It is imperative that agencies do their research before adopting a pretrial release assessment to avoid pitfalls like widening the net of people unnecessarily incarcerated before trial, intensifying disproportionality, or compromising public safety. To do pretrial release assessment well, agencies must commit to strong stakeholder involvement, as well as ongoing data monitoring, and validation after a few years.
Stakeholders and agency decision makers should consider the following critical criteria and actions when selecting a pretrial release assessment.
Begin With the Principles of Actuarial Risk Assessment
We have touched on values and risk assessment before. This is where your stakeholder conversations must begin. In addition, consider the four principles of actuarial risk assessment in the context of your stakeholders’ values.
- Validity: A “valid” pretrial release assessment measures what you want it to measure. In other words, what outcome is your jurisdiction seeking to measure with a pretrial release assessment? Do you want to know someone’s likelihood of failure to appear in court? Likelihood of committing an offense while awaiting trial? Risk to public safety? Stakeholders in your community need to have explicit conversations about this and determine what your assessment will measure.
- Equity: If your pretrial release assessment works equally well for different subgroups (e.g., different racial/ethnic groups), it is equitable. Talk with stakeholders about the importance they place on equity in your community. Because pretrial release assessments are based on administrative data that reflect systemic and institutional bias, a high priority on equity must be factored into the design to avoid perpetuating or exacerbating existing racial disparities.
- Reliability/consistency: An actuarial risk assessment is meant to guide a person’s decision making. Ideally, it works the same way no matter who that person is (and what individual biases they may have). This is called reliability. Is your jurisdiction concerned with the consistency of current decision making? Design or look for a tool with a high degree of reliability.
- Utility: Utility refers to how useful a tool is. Can users complete it in the time they have? Do they understand the wording of the items and the definitions? Do they understand how it is laid out on the screen or page? Do the results of the tool provide clear guidance on how the key decision should be made? Known constraints or challenges to utility should be discussed ahead of time if possible.
Other Crucial Considerations
- Distribution: A pretrial release assessment’s distribution refers to the proportions of a group it puts into different risk categories. For example, if a pretrial release assessment classifies 6 of every 10 people as high risk, that will place a large demand on county resources. This distribution should raise questions, since 60% of a given population would be a very large proportion to be considered high risk. Is it in line with your community’s values to detain 60% of people before trial? Will this distribution put workers in the position of overriding assessment results for reasons unrelated to the individual, bringing in the possibility of less reliable and equitable decisions? Inappropriate distribution can also give a low-risk classification to too many people.
- Transparency: The workings of your pretrial release assessment should be transparent. If your jurisdiction decides to develop a customized assessment, plan for how you will share this information with stakeholders. If you decide to investigate off-the-shelf options, have the developers made the workings of the tool public? All your stakeholders, especially those who will be assessed using the tool, deserve to understand exactly what the tool is measuring and what information it relies on to make its classifications. Workers using the tool should also understand how it functions, which helps improve staff buy-in.
- Non-actuarial factors: A well-designed pretrial release assessment strikes a delicate balance between outcomes that can be measured, such as short-term recidivism or failure to appear; legal statue and policy; and public safety values. For example, some pretrial release assessments consider offense severity in the recommendation.
The best pretrial release assessment is one developed and validated on your community’s population. Using a jurisdiction’s own data to create and test an assessment gives the most accurate and meaningful results.
If you want a customized pretrial release assessment but your agency does not have staff experienced in risk assessment development, find a partner who has demonstrated expertise.
Adopting Another Jurisdiction’s Pretrial Release Assessment
If you are considering using an off-the-shelf assessment, ask about validation. Different agencies can have very different populations, so pretrial release assessments aren’t one size fits all. Has it been validated and where? Has it been validated on a population similar to yours? Who is using it? Can you talk to agencies who are using it, read data reports, learn how they are using it?
Look for transparency. Is the developer forthcoming about what factors go into the risk classification and limitations of development and use? Will this information be available to the people using the tool or seeing its results?
Plan a “risk fit” test. If your jurisdiction elects to adopt an existing pretrial release assessment, plan to do a “risk fit.” This test takes your existing data and applies the pretrial release assessment to it to simulate how the tool would work in practice for your population. If you don’t have enough data to do a simulation, you could pilot the assessment with a randomized target group to see how it works.
Whichever path you choose—customized or off-the-shelf—be sure to engage with community stakeholders who are involved with the population you are working with. Even if every stakeholder doesn’t have decision-making power, keep them informed and involved.
Monitor Data and Be Ready to Change Gears
Ongoing data monitoring is a must. Plan to look at how the assessment is working in practice. Commit to changing things if something looks off; or example, your jurisdiction is detaining far greater numbers of people from a particular racial/ethnic group. Something needs to change, whether it is the tool itself or something about practice.
Because the pretrial decision point is one that can take away a person’s liberty, drive people further into the system, and lead to other negative outcomes, NCCD recommends data monitoring at least monthly.
Plan Ahead to Validate
Work with your research team or a trusted partner to ensure quality data collection from the start so that your pretrial release assessment can be validated down the road. This applies to both customized and adopted assessments. Populations served by an agency can change over time—demographically, by offense type, and more—so what works well now may not work well in three years. Your pretrial release assessment may need an adjustment by then, or it may be working as intended. Again, given what is at stake, you need to know.
A pretrial release assessment has the potential to greatly improve the justice process. However, ensuring improvement requires rigorous research on the part of local experts and stakeholders. Selecting the best pretrial assessment for your jurisdiction will keep agency resources focused on public safety, while also supporting civil rights for all community members.