Flaws in How the Feds Hold the States Accountable for Foster Care

A new brief from Chapin Hall presents their critique and criticism of the federal government's system for evaluating the performance of the states' child welfare agencies. Summarizing the shortcomings they find that the federal Child and Family Service Reviews (CFSR):

• Overlook important variation among states in the demographics of the children and families served.
• Fail to account for systemic state differences in caseload inclusion criteria, and the inherent practice and policy conflicts between measures.
• Draw on data derived from a database (AFCARS) that was not designed to measure longitudinal performance, and is still not of the quality to justify imposing fiscal penalties.
• Count/weight states equally despite enormous differences in the size of the child population.
• Employ a complicated statistical method, principal components analysis (PCA), in the absence of any evidence that such a method is in any way required or superior to simpler and more-transparent approaches to measurement.
• Make many arbitrary and statistically inappropriate decisions in the use of the PCA procedure, thereby undermining the ranking of states that the method produced.
• Arbitrarily set the national standard at the 75th percentile, and then rely on ill-conceived rules that adjust the standard to a different level.

State policymakers can use their own policies for holding their child welfare systems accountable.