Google Glass lives on as a device to teach autistic children

The technology could help the children learn how to recognise emotions and engage directly with those around, in addition to measuring changes in behaviour

By Cade Metz

Published: Jul 18, 2019

Google Glass lives on as a device to teach autistic children

Esaïe Prickett, wearing Google Glass, at his home in Morgan Hill, Calif., July 10, 2019. Prickett, who has autism, tested an app on the computerized glasses in a clinical trial meant to help him learn how to recognize emotions and make eye contact with those around him. (Cayce Clifford/The New York Times)

SAN FRANCISCO — When Esaïe Prickett sat down in the living room with his mother, father and four older brothers, he was the only one wearing Google Glass.

As Esaïe, who was 10 at the time and is 12 now, gazed through the computerized glasses, his family made faces — happy, sad, surprised, angry, bored — and he tried to identify each emotion. In an instant, the glasses told him whether he was right or wrong, flashing tiny digital icons that only he could see.

Esaïe was 6 when he and his family learned he had autism. The technology he was using while sitting in the living room was meant to help him learn how to recognize emotions and make eye contact with those around him. The glasses would verify his choices only if he looked directly at a face.

He and his family tested the technology for several weeks as part of a clinical trial run by researchers at Stanford University in and around the San Francisco Bay Area. Recently detailed in The Journal of the American Medical Association, Pediatrics, the trial fits into a growing effort to build new technologies for children on the autism spectrum, including interactive robots and computerized eyewear.

The Stanford study’s results show that the methods have promise and indicate that they could help children like Esaïe understand emotions and engage in more direct ways with those around them. They could also measure changes in behavior, something that has historically been difficult to do.

Experts believe that other new technologies may help in similar ways. Talking digital assistants like Amazon’s Alexa, for example, could help children who misuse their pronouns. But even as these ideas spread, researchers warn that they will require rigorous testing before their effects are completely understood.

Catalin Voss started building software for Google Glass in 2013, not long after Google unveiled the computerized eyewear amid much hullabaloo from the national media. An 18-year-old Stanford freshman at the time, Voss began building an application that could automatically recognize images. Then he thought of his cousin, who had autism.

Growing up, Voss’ cousin practiced recognizing facial expressions while looking into a bathroom mirror. Google Glass, Voss thought, might improve on this common exercise. Drawing on the latest advances in computer vision, his software could automatically read facial expressions and keep close track of when someone recognized an emotion and when they did not.

“I was trying to build software that could recognize faces,” Voss said. “And I knew that there were people who struggled with that.”

At the time, the brief moment Google Glass spent in the national spotlight was already coming to an end. Google stopped selling the device to consumers amid concerns that its built-in camera would compromise personal privacy.

But Google Glass lived on as something to be used by researchers and businesses, and Voss, now a doctoral student, spent the next several years developing his application with Dennis Wall, a Stanford professor who specializes in autism research, and others at the university.

Their clinical trial, conducted over two years with 71 children, is one of the first of its kind. It spanned everything from severe forms of autism, including children with speech impairments and tactile sensitivities, to much milder forms. Children who used the software in their homes showed a significant gain on the Vineland Adaptive Behavior Scales, a standard tool for tracking the behavior of those on the autism spectrum, Voss said.

Jeffrey Prickett, Esaïe’s father, said he had been drawn to the study because he had known it would appeal to his son, who enjoys using iPad apps and watching movies.

“He does fine interacting with people,” Prickett said. “But he does better interacting with technology.”

Prickett found it hard to judge whether the Google device helped his son recognize emotions, but he saw a marked improvement in Esaïe’s ability to make eye contact.

Heather Crowhurst, who lives near Sacramento, California, said she had experienced something similar with her 8-year-old son, Thomas, who also participated in the trial. But Thomas was not entirely captivated with the digital therapy. “It was kind of boring,” he said.

The concern with such studies is that they rely on the observations of parents who are helping their children use the technology, said Catherine Lord, a clinical psychologist at the University of California, Los Angeles, who specializes in the diagnosis and treatment of autism. The parents are aware of the technological intervention, so their observations may not be reliable.

Still, the Stanford team considers its study a first step toward wider use of this and other technologies in autism. It has licensed the technology to Cognoa, a Silicon Valley startup founded by Wall. The company hopes to commercialize the method once it receives approval from the Food and Drug Administration, which oversees the use of medical devices in the United States. That may still be years away.

Other companies are taking a different approach. Brain Power, a startup in Massachusetts that has built similar software for Google Glass, is selling its technology to local schools. The company considers it a teaching tool, not a medical device.

Patrick Daly, assistant superintendent of the school district in North Reading, Massachusetts, is testing Brain Power’s technology after watching its effect on his 9-year-old son, who is on the spectrum. The district intends to test the technology over the next few years.

Previously, the district tried to teach similar skills through iPad computer tablets. Daly sees Google Glass as a big improvement.

“It can actually maintain eye contact,” he said. “They are not looking down while they try to learn an emotion.”

Robokind, a startup in Dallas, applies the same philosophy to different hardware. The company spent the past several years designing a robot that attempts to teach many of the same skills as technologies built for digital eyewear. Called Milo, the doll-like, 2-foot-tall robot mimics basic emotions and tries to make eye contact with students. It also asks questions and tries to engage students in simple conversations.

Robokind has sold hundreds of the robots to schools for testing. Each one costs $12,000, plus more than $3,500 for additional software.