Researchers have defined a change in score of about 4.5 points on a popular autism rating scale as a mark of meaningful improvement — a measure that could help scientists gauge the effectiveness of autism treatments in clinical trials.
To address the problem, researchers aimed to define a minimum level of clinically meaningful improvement on the Childhood Autism Rating Scale (CARS). Clinicians use the 60-point rating scale to assess the severity of an autistic person’s traits, such as the ability to communicate with others or adapt to change. The widely used test is suited to a broad age range, has been translated into several languages and is relatively quick and easy to administer.
To establish the minimum value, the researchers used a process called ‘expert knowledge elicitation,’ in which a statistician collaborates with experts during a workshop to convert their opinions into a range of values and probabilities. The approach is most often used when researchers don’t have enough data to answer a question directly.
“The idea is really to translate qualitative knowledge into a statistical model,” says Lucie Jurek, a child and adolescent psychiatrist at Le Vinatier Hospital in Bron, France, who co-led the work.
Before the workshop, the experts — five autism researchers and clinicians from four countries — completed a refresher training on the CARS and took an online course on the expert elicitation process. They also read a summary of findings from four previous clinical trials that had used the CARS ad hoc to gauge an autism treatment’s effectiveness.
The experts then met for the one-day workshop with two facilitators — one who guided the workshop and one who ran statistical software. One of the facilitators asked each expert to record what they thought represented the minimum clinically meaningful improvement in a person’s CARS score, for autistic people of any age and trait severity. Instead of guessing a single number, the experts were asked to provide the lowest and highest plausible values, as well as a likely median.
The experts then shared their responses, discussed their differences and came to an agreement on the plausible range and median. The person running the software created a model based on the answers and adjusted it until the experts thought it accurately represented their beliefs.
The resulting model suggests that, on average, clinicians judge a minimum improvement of about 4 points on the CARS to be a meaningful change, the researchers reported in April in European Child and Adolescent Psychiatry. But given the inherent uncertainty in the elicitation process, an improvement of 4.5 points may be a more appropriate benchmark, they say.
The findings jibe with results from previous clinical trials, in which researchers considered a meaningful improvement to be in the range of 4 to 8 points. The result of the elicitation process may be on the lower end of this range because the researchers were looking to define the minimum clinically meaningful change.
Going forward, scientists could use the CARS alongside other outcome measures in clinical trials, the researchers say.