FAQ: Making Comparisons

RSP takes into account some of the factors over which states have little control and that are not accounted for in the observed performance.

To calculate RSP, a statistical model is used to compare a state’s observed performance to its expected performance. The result of the statistical model is a ratio that is applied to national performance to show how different a state was relative to national performance—not relative to the state’s observed performance.

For example, State A’s observed performance for permanency in 12 months for children entering foster care is 38.5%. That is, for all children who entered care during the year, 38.5% of those children achieved permanency within 12 months of entering foster care. State A has an unusually large proportion of younger children in its child welfare population, and we know that younger children are less likely than older children to achieve permanency in 12 months. Because more of the younger children achieved permanency than we otherwise would have expected, the statistical model takes that factor for State A into consideration when calculating State A’s RSP of 51.2%.

A state’s RSP cannot be compared to its observed performance. RSP takes into account some of the factors over which a state has little control and calculates performance relative to national performance. RSP has no meaning relative to the state’s observed performance.

In our example, State A has an unusually large proportion of younger children in foster care, so we account for that along with other factors when making adjustments to calculate the state’s RSP of 51.2%. When State A’s RSP interval of 50.1% to 52.9% is compared to national performance of 42.7%, State A performed better than the nation.

On the other hand, let’s say that State A has an unusually large proportion of older children in care instead. As a result, we expect more children who enter foster care in State A to achieve permanency within 1 year of entry than if State A’s case mix had more younger children. As a result, State A’s performance is adjusted to calculate an RSP of 30.1% and an RSP interval of 28.9% to 30.7%. When that interval is compared to the national performance of 42.7%, State A performed worse than the nation.

In both scenarios, the observed performance for State A was the same. The RSP changed depending on the proportion of older or younger children in State A’s foster care population.

A state’s observed performance on the statewide data indicators should not be compared to the national performance. The observed performance does not take into account some of the factors over which states have little control. The goal of RSP is to provide states with a fairer way of comparing performance to national performance.

For example, State A’s observed performance of 38.5% does not account for the high proportion of younger children in its population so comparing it to national performance of 42.7% would not be a fair comparison. Comparing the two would make it appear State A’s performance is worse than it is, based on State A having a different case mix than the nation.

RSP should only be used to compare a state’s performance to national performance. It also allows a better way to compare across state performance. It helps identify areas of strength and improvement needs on the CFSR safety and permanency data indicators relative to national performance.

A state’s observed performance is used to determine how the state performs based on its actual child population, to identify how the state performs on an indicator over time, and to determine how performance on one indicator (e.g., permanency in 12 months) relates to another indicator (e.g., reentry to foster care). Observed performance data also helps identify areas of strength and improvement needs on the CFSR safety and permanency data indicators and is used to produce supplemental context data to help understand state performance information included in CFSR Data Profiles.