/ Uncategorized / By

When Some Information Is Better Than None: Predicting Risk

February 27, 2018 | Colleen Kerwin, NCCD

Colleen Kerwin

We think of risk as the likelihood of a particular event taking place. Insurance companies are concerned about your risk of experiencing a car accident; doctors want to know your risk of developing heart disease. At NCCD, we are interested in the risk of individuals reentering the child protection, adult protection, and justice systems. We often refer to this as risk of recurrence—but is it really? Recurrence, in the context of our work, means a child experienced maltreatment, again, or an individual committed a crime, again. However, many instances of child maltreatment and crime go unreported. How can we quantify the risk of recurrence when we cannot measure the true incidence of recurrence?

The truth is we can’t. While this is an unsatisfying reality, it doesn’t stop us from using the information we do have to assist agencies in making good decisions. It also means we have a responsibility to be transparent in what we can do, which is to classify families and individuals by their likelihood to reenter systems—not by their likelihood of recurrence.

Those who work in the fields of child protection, juvenile justice, adult corrections, and adult protection understand the biases that play into individuals experiencing maltreatment versus being involved in child protection, or in committing crime versus being arrested. These biases are important and real and should be considered when determining risk. However, we can certainly do something with the information we have available, even if it is not perfect.

Imagine a group of doctors who work together in family practice. These doctors have seen the negative effects of the flu this season and want to determine if they can make any changes to their practice to prevent the flu for at least some individuals next year. The doctors gather files for all the clients they saw in the past six months, identify which ones developed the flu and which ones did not, and try to tease out signs that an individual is more likely to get the flu or better practices to use with individuals who come in with the flu. This seems like a great initiative: Use the information you have about the population you serve to make better decisions. However, it is also easy to poke holes in the doctors’ well-meaning initiative.

It is likely many clients did get the flu but never went to the doctor and elected home remedies instead. It is also possible that some clients got the flu, and their symptoms were so severe they went to urgent care instead of waiting for an appointment with their primary doctor. The doctors will never be able to truly examine all individuals who experienced the flu—but that doesn’t mean their initiative is a bad idea. Through their efforts, the doctors can identify (1) how best to assess the risk of the flu among the individuals they are likely to see and (2) best practices to use when a client presents with the flu. It is important to understand the limitations of their efforts and research, but these limitations should not be a barrier to leveraging the information available to make good decisions.

At NCCD, we run into the same issues and limitations working with systems. A child protection system will never have information on every family that maltreated a child in the jurisdiction. However, using the available information on system-involved families, these agencies can determine which families are most likely to reenter the system and therefore can benefit the most from supports and services.

Colleen Kerwin is a researcher for NCCD.