Using the SDM Model to Challenge Racial Bias

Using the SDM Model to Challenge Racial Bias

May 7, 2024 | Evident Change

People at computer discussing racial bias

Racial equity is central in Evident Change’s work, as shown in data analysis, policy recommendations, and practice recommendations shared with numerous child welfare agencies over decades. We believe a multi-pronged approach is necessary to make a meaningful impact on racial bias and disparities, and the Structured Decision Making® (SDM) tools are an important part of that.

Racial bias within data and assessment tools is a complicated subject, just as it is in social systems’ decisions generally. Current and historical racial bias exists on many levels: individual, interpersonal, institutional, and systemic. Decisions made in child welfare and other systems cut across these levels, which makes solving the problem of bias a complex endeavor.

Multiple perspectives exist on the impact of tools on racial bias. On the positive side, a robust research base shows that using data to drive decisions in child welfare can be a powerful strategy to address inequities. Data-driven tools work against racial inequities by encouraging greater equity and consistency in decision making.

Tools also generate data that can illuminate where system disparities exist. Once agencies know where disparity and disproportionality appear in the system, they can strategize about how to improve their practice or create culturally responsive prevention and intervention strategies. Communities can also use knowledge of disparities, examining their local laws and practices about when people should make reports to child protection and what other ways exist (or could be created) to meet families’ needs in the community. In this way, data-driven tools are an important complement to other strategies to address racial inequities.

However, concerns also exist that tools created using systems data could worsen racial disparities precisely because systems data contain racial bias. These data are in part the product of existing racial inequities in society. Some organizations and advocates contend that risk assessment tools should not be used for these reasons.

One way to consider the merits of these tools in counteracting racial bias is to think about how decisions are made in their absence. In many child welfare systems, the status quo is often subjective decision making by individuals. In a system with hundreds, sometimes thousands, of individuals making decisions, it can be hard to establish consistency or equity in the way families are treated. In contrast, the risk factors included on a data-driven assessment tool are easily shared, can be tested for equity, and are more consistent.

Other questions to consider relate to the design and operation of assessment tools. Both of these phases can include measures to reduce the impact of bias in the data and in practice. Important questions include:

  • Who is—and is not—part of the tool design team? What perspectives and experiences are present in the discussion? What perspectives aren’t in the room?
  • When these tools are completed and in use, who can see and understand their inner workings? Is information about the tool’s functioning available for workers and families to view, or is it secret? Are the tool’s outcomes continuously monitored? Is the tool updated regularly to ensure that it is functioning equitably?

Evident Change is dedicated to equitable treatment and outcomes for children and families, and we believe that data—both quantitative and qualitative—are critical to better understanding and decision making in child welfare. We also believe that systems change requires input from communities and individuals who experience these systems.

These beliefs underlie the measures we take to retain the benefit that tools offer while reducing the potential harm of using biased data. When agencies invite us to help them create assessment tools, we actively seek to include community members in the design process—in particular, members of communities that have been disproportionately impacted by the system. Through the collaborative design process, we aim to create greater transparency, equity, and accountability in the factors that drive decisions and their results. Inclusively designed, data-informed tools—like the SDM® tools—can help to mitigate individual bias in the decisions that affect families.

Some of the ways Evident Change works to support racial equity in systems include the following.

  • Participatory assessment design
  • Testing tools to learn about their impacts for different subgroups
  • Testing tools across users to ensure that different workers arrive at the same result given the same information
  • Providing ways to use agency data to show where disproportionality exists so agencies can work toward solutions
  • Training and coaching people on racial equity in outcomes and in their agency culture

Child protection workers must make complex, serious decisions that have lasting impact on families. These decisions, when made in the absence of tools like the SDM tools, are subject to an increased likelihood of personal bias and inconsistency. SDM tools support equitable assessment in child protection, particularly when utilized with quality assurance, regular reviews, and a sound practice model.

Evident Change’s work in child protection is grounded in a deep sense of humility and responsibility to the people affected by the child protection system. It has always been our goal to support our client partners’ capacity to treat children and families equitably. We welcome leadership and partnership from people who are part of disproportionately impacted communities to continue to improve SDM tools and child protection practice.

If you would like to know more about racial equity in Evident Change’s research and assessment tools, please contact us.