Autistic children are less likely to “flourish” than non-autistic children, according to a 2019 study. But a recent re-analysis of the data, published in February in Autism Research, has revealed evidence of potential measurement bias.

The new research underlines the importance of validating quality-of-life measures among children with disabilities — and engaging with the autistic community to “build those measures effectively,” says lead investigator Samantha Ross, assistant professor of adapted physical education at West Virginia University in Morgantown.

The 2019 work relied on data from the caregivers of 812 autistic children and 34,171 controls, collected as part of the 2016 National Survey of Children’s Health (NSCH), which is administered annually to gauge children’s health and well-being. The team used 10 survey items they considered indicative of flourishing — a concept drawn from the positive psychology literature — including questions about a child’s ability to finish tasks, stay calm in the face of challenges, and make and keep friends. They then used a statistical analysis to group related items into three domains: social competence, behavioral control and school motivation. Autistic children scored lower than non-autistic children on the social competence and behavioral control domains, the study suggested.

Autistic and non-autistic children with similar flourishing scores should have similar scores on each of the 10 items, Ross says. Instead, the researchers found that caregivers of autistic children had answered questions about social competence and behavioral control in a different pattern than those of non-autistic children with comparable overall scores had — evidence of measurement bias.

The re-analysis provides “a nice cautionary note,” says Claudia Hilton, associate professor of occupational therapy at the University of Texas Medical Branch in Galveston, who led the 2019 study. “It’s a sign of how we’re evolving our science.”

After Ross and her team controlled for measurement bias, the flourishing gap between autistic and non-autistic children narrowed in all three subdomains. The remaining differences between groups “should be interpreted with caution,” according to the re-analysis.

Researchers can’t draw the same kind of firm conclusions about their data if “the scale is not behaving quite the same way” in different groups of children, warns Judith Miller, associate professor of psychology at the Children’s Hospital of Philadelphia (CHOP) in Pennsylvania, who was not involved in either study.

If they find a significant amount of bias, researchers should critically consider if they should be using that measure, says Ellie Kaplan-Kahn, a postdoctoral researcher at CHOP who was not involved in either study.

“When you’re doing secondary data analysis,” Hilton notes, “you don’t get to write the questions.” She agrees that some of the NSCH items — such as one asking if a child has difficulty making and keeping friends — might not be as sensitive a flourishing indicator for autistic children, whose friendships may look different from those of non-autistic children.

“We should see variability in children with autism,” Ross explains. If autistic children are always expected to receive lower scores in the social domain, “we’ve created a disability penalty,” she says.

Going forward, researchers need to seek input from autistic people about whether a measure developed in the general population is meaningful to them, Miller says.

Ross’ team is interviewing caregivers and autistic youth, asking them to narrate their thought processes while answering questions for the NSCH. Their goal is to discover whether autistic youth receive lower flourishing scores because they’re experiencing difficulties or because the survey items don’t effectively capture their experiences. “We need to ensure our survey tools are trustworthy before assuming that autism traits result in deficits,” she says.

Cite this article: https://doi.org/10.53053/MNNT4651