Algorithms generated through machine learning can sort through observations of children’s behavior in short home videos to determine if the children have autism, a Stanford study has shown.
Short home videos can be used to diagnose autism in children, according to a new study from the Stanford University School of Medicine.
The research, which was published online Nov. 27 in PLOS Medicine, expands on a 2014 feasibility study on the topic by the same researchers. In the new study, the scientists employed machine learning to determine which features of children’s behavior should be rated to evaluate autism, using computers to whittle down a long list of behavioral features to those most relevant to the diagnosis. They also devised an algorithm that weights each feature to provide an overall diagnostic score for each child.
“Across the United States, the average waiting list to get access to standard-of-care can last up to a year,” said the study’s senior author, Dennis Wall, PhD, associate professor of pediatrics and of biomedical data science at Stanford. “Using home videos for diagnosis has the potential to streamline the process and make it far more efficient.”
Home videos offer another potential advantage for diagnosing behavioral and developmental disorders such as autism. “Home video catches the child in his or her natural environment,” Wall said. “The clinical environment can be stark and artificial, and can elicit atypical behaviors from kids.”
Value of early diagnosis
Autism is a developmental disorder characterized by restricted interests, repetitive behaviors and difficulty forming social connections. Previous research showed that behavioral therapies for autism work best when started before age 5, but long waitlists for testing make it difficult for families to access timely treatment. Current diagnostics are time-intensive, requiring one-on-one assessment with an autism specialist. Clinicians spend a few hours per patient assessing dozens of aspects of the child’s behavior.
In the new study, the researchers devised and tested eight machine-learning models for diagnosing autism from short videos. Each model consisted of a set of algorithms that included five to 12 features of children’s behavior and produced an overall numerical score indicating whether the child had autism.
To test the models, the researchers asked families recruited through social media and autism listservs to submit brief home videos, and received 116 videos of children with autism (average age 4 years, 10 months) and 46 videos of typically developing children (average age 2 years, 11 months) that met their criteria: The videos were 1 to 5 minutes long, showed the child’s face and hands, showed direct social engagement or opportunities for engagement, and showed opportunities for use of objects such as toys, crayons or utensils.
Nine video raters received brief instruction on how to evaluate each video, answering 30 yes/no questions about whether children in the videos exhibited certain behaviors such as using expressive language, making eye contact, expressing emotion and calling attention to objects. All of the yes/no questions were based on behavioral characteristics used in standard autism screening tools.
All nine raters scored 50 of the videos, and the researchers used these results to determine that three raters were the minimum number needed to generate a reliable score. The remaining videos were randomly assigned to the raters, with three raters scoring each video.
On average, watching and scoring the videos took the raters 4 minutes each. The data for each video, consisting of the 30 yes/no answers to questions about the child’s behavior, was fed into the eight mathematical models.
One model, a logistic regression model that used five behavioral characteristics, performed best, identifying autism with 88.9 percent accuracy overall, including correctly labeling 94.5 percent of children with autism and 77.4 percent of children without autism.
To validate their findings, the researchers repeated the experiment with an additional 66 videos — 33 featuring children with autism and 33 with children who did not have autism. The same model again performed best, with correct identification of 87.8 percent of children with autism and 72.7 percent of children without autism.
“We showed that we can identify a small set of behavioral features that have high alignment with the clinical outcome, that nonexperts can rapidly and independently score these features in a virtual environment online in minutes, and that the model we used to combine those features is effective in producing a score that matches the clinical outcome,” Wall said. The final scores are not just a “yes or no” autism diagnosis, he added; instead, the numerical scores may hold information about the severity of the disorder and be of value for tracking progress over time.
Providing tool for pediatricians
Wall hopes simple scoring systems for home videos will help streamline the process of autism diagnosis. “This could be used in general pediatric settings such as well-baby checkups,” he said, adding that video scores could be plotted over time and compared with the general population, similar to how a child’s height and weight are plotted on a growth chart.
“Our long-term dream is that a tool like this will give general pediatricians more confidence in making diagnostic decisions about autism and other developmental disorders,” he said. For a very young child — at an age when autism can be difficult to distinguish from normal development — the doctor’s decision might be to engage in watchful waiting, but with the advantage of having a video score as a baseline for later evaluations. In other cases, it might be clear that a child needs to immediately begin autism treatment, or needs to be referred to a specialist for a more detailed diagnostic evaluation.
The researchers are now repeating their investigation with home videos of young children in Bangladesh to see how well their mathematical models translate across cultures.
The study’s lead author is Qandeel Tariq, a data analyst in Wall’s lab. Other Stanford authors are Jena Daniels, a former clinical operations manager; clinical research coordinator Jessey Nicole Schwartz; bioengineering graduate student Peter Washington; and postdoctoral scholar Haik Kalantarian, PhD. Wall is a member of Stanford Bio-X, the Stanford Child Health Research Institute and the Wu Tsai Neurosciences Institute.
The research was funded by the National Institutes of Health (grants 1R01EB025025 and 1R21HD091500), the Hartwell Foundation, the Bill and Melinda Gates Foundation, the Coulter Foundation, the Lucile Packard Foundation for Children’s Health, and program grants from Stanford’s Precision Health and Integrated Diagnostics Center, Beckman Center, Bio-X, the Predictives and Diagnostics Accelerator Program and the Child Health Research Institute. The research also received support from David Orr, Imma Calvo, Bobby Dekesyer and Peter Sullivan.
Stanford’s departments of Pediatrics and of Biomedical Data Science also supported the work.
