Vin Ganapathy

Several self-report surveys allow adults in the general population to assess any traits of autism they may have, such as difficulties with social interactions or communication. In the past decade, it has become commonplace for researchers to rely on these tests to measure autism symptoms in research participants.

A new study challenging the validity of these questionnaires has touched off a debate, with the developers of one test saying the researchers misused it.

The study, published 17 December in the Journal of Autism and Developmental Disorders, suggests that some self-administered questionnaires for adults don’t distinguish between traits of autism and conditions that often co-occur with the disorder.

Takeshi Nishiyama and his colleagues at the Aichi Medical University in Japan compared the reliability and validity of four self-reports that measure autism symptoms in adults: the shortened and full-length versions of both the Autism-Spectrum Quotient (AQ) and the Social Responsiveness Scale2-Adult Self-report (SRS2-AS), the Subthreshold Autism Trait Questionnaire (SATQ) and the Broader Autism Phenotype Questionnaire (BAPQ).

Different aims:

All of these measure autism traits, but each was designed for a slightly different purpose. The AQ measures a wide range of autism symptoms to identify people in the general population with the disorder. The SRS2-AS does the same, but focuses on social behaviors and was adapted from a version developed for children.

In contrast, the BAPQ specifically probes for mild forms of autism-like personality traits and behaviors in first-degree relatives of people of autism — what’s known as the broad autism phenotype (BAP). Finally, the SATQ looks for subtle autism-like symptoms in the general population.

The researchers translated the surveys into Japanese and provided them to 3,147 university students and working adults from the general population and 60 people of similar ages diagnosed with autism. They also administered a battery of questionnaires measuring symptoms characteristic of other conditions, including general psychological distress and psychotic tendencies.

Individuals diagnosed with autism tended to score higher on the autism questionnaires than those in the general population, indicating that they had more, or more severe, symptoms.

An analysis of the results revealed, however, that the participants’ answers on some versions of the AQ were inconsistent with their responses to other questions about the same symptoms. This suggests that these questions do not measure traits of autism, the researchers say.

Also, eight questions on the SRS2-AS and one on the BAPQ seemed to measure psychological distress and psychotic tendencies rather than autism. High scores on the SRS2-AS often aligned with high scores on other measures of these symptoms. The SATQ performed well on these and other tests of validity.

Experts take issue with some of these findings. In a response to the study published 20 February in the same journal, Joseph Piven and Noah Sasson contended that Nishiyama’s team “inappropriately evaluated” the BAPQ.

Piven and Sasson, who developed the BAPQ and have tested its efficacy, note that their questionnaire was not designed to be used for individuals with autism, nor to measure symptoms of autism in the general population. Because it was tailored to measure the more subtle characteristics of BAP, they say it was incorrect to use it as Nishiyama’s team did.

Responding to this criticism, Nishiyama and his colleagues countered that the BAPQ performs just as well as the other questionnaires in measuring autism traits in the general population and in individuals with autism.

Without further research, the validity of BAPQ and other self-report tests for detecting autism traits may remain unresolved. Additional studies not only would help clarify what each of these surveys is capable of measuring but also may indicate how appropriate each questionnaire is for different populations.