Whether a potential treatment for autism works in a clinical trial depends on how the trial measures success. There are several ways to gauge a person’s response to a treatment — from interviews with parents to questionnaires filled out by teachers, clinicians and caregivers. It is unclear which of these ‘outcome measures’ reflects improvements that are important to people with autism.
To help identify the best measures, Thomas Willgoss and his colleagues asked people on the spectrum and their families to describe their autism features and the impact of those features. The researchers plan to use the insights from these interviews to evaluate how well existing outcome measures reflect the features that are important to people with autism. They also hope to identify challenges, such as sensory sensitivities and difficulties with daily living, that trials should track and potentially treat. They published their findings in September in Autism Research1.
We asked Willgoss, research scientist at F. Hoffmann-La Roche, a pharmaceutical company headquartered in Switzerland, about the usefulness of this ‘patient-centered’ approach.
Spectrum: Why did you seek input from people with autism and their parents?
Thomas Willgoss: As new treatments are developed for autism, it’s really important that we identify outcome measures that are valid, reliable, sensitive and, perhaps most importantly, grounded in the experiences of people with autism and their families. These measures might be interviews with parents or questionnaires.
To identify good outcome measures, we first need to build an understanding of what it is like to have a specific condition. We do this by developing a list of the concepts that individuals with that condition tell us are important, and then try to understand how these different concepts could relate to each other. We call this a conceptual model.
We developed a conceptual model of the impact of living with autism. We hope it will help us test the ‘content validity’ of outcome measures — that is, the extent to which a measure captures the experiences of the people being assessed. It can also flag elements of the experience of having autism that could be targeted with new treatments.
S: How did you develop this model?
TW: It was an iterative process. Our first step was to review published papers describing the impact of various autism features on daily life, as reported by people with autism and their caregivers around the world. From this, we created a list of 108 concepts, such as resistance to change, problems conforming to social norms, difficulty concentrating or following instructions, and being bullied.
We organized the concepts into 14 domains, which include deficits in nonverbal or verbal communication, difficulties with social interactions, sensory sensitivities, repetitive behaviors, cognitive impairment and behavioral problems. Many of the domains relate to the autism features described in the DSM-5, the current version of the “Diagnostic and Statistical Manual of Mental Disorders.” We used this information to develop a framework for our conceptual model.
We then interviewed 10 people with autism, who ranged in age from 13 to 31, as well as 26 parents of people with the condition. We asked about each individual’s experiences around the concepts we had identified, such as social interaction or sensory sensitivities. We also asked open-ended questions, such as, “What does a good day look like?” and “What does a bad day look like?” These interviews allowed us to get detailed accounts of how some features associated with autism affect school, work or daily life.
We transcribed recordings of each interview and coded the concepts each person discussed. We refined our initial model by including only the concepts that were mentioned by 25 percent or more of the people we interviewed.
Finally, we asked seven clinical experts to review the model. They tweaked some of the wording and helped us organize the content. The final model groups 47 concepts into four domains: core symptoms, associated symptoms, impact on the individual with autism and impact on family members or caregivers.
S: What were the most interesting things you learned?
TW: Difficulties with conversations and with social situations were among the most frequently reported concepts. Accordingly, conversational skills and expressing or processing emotions were the aspects that parents and people with autism would most like to see improved.
Another key takeaway is the multifaceted nature of the autism experience — that autism has an impact on many aspects of people’s lives.
Our model also highlights the importance of daily-living skills and independence, as well as success in school, work and social life. These things aren’t necessarily the target of new therapies, but we should try to measure them as part of a holistic patient-centered approach.
S: What are you working on now?
TW: We’re using our model to evaluate existing outcome measures. So far, we’ve used it to evaluate the Vineland, a clinician interview with a caregiver. The Vineland’s socialization and communication domains appear to capture the core social-communication concepts highlighted in our model and provide a fairly comprehensive assessment of the impact on daily-living skills. Hopefully, researchers will use our model to evaluate the validity of other outcome measures.