Virtual reality program aimed at helping people with Asperger’s

A team of doctoral candidates and a University of Houston professor are developing various augmented reality and virtual reality programs, which include programs designed to help people with Asperger’s identify emotions, teach people what to do in disasters and create 3-D models of MRI data.

The programs are being developed primarily for the Microsoft HoloLens, an augmented reality headset that places holograms and information over the physical world, said computer science professor Chang Yun. Some, however, are made in virtual reality using the HTC Vive. 

The team is working alongside CERT, a division of FEMA that helps educate the public about disaster preparedness, Yun said.

“In a nutshell, what CERT does is teaching civilians how to do particular tasks to both save themselves and their family in times of emergency,” said Brian Holtkamp, a doctoral candidate working on a program to teach people how to properly use a fire extinguisher. “So hurricanes, tornadoes, those type of natural disasters or man-made disasters.” 

The fire extinguisher program is in VR and will go into testing next month, he said.

“We put them into a house that has fire, and you have to go extinguish them or see if you can extinguish them,” Holtkamp said.

Another program the team is working on aims to help people with Asperger’s or high functioning autism converse with others. It uses the Microsoft HoloLens and tells the user what emotion — happy, sad, angry, neutral or surprised — is displayed on the face whoever they are talking to.

“If we can rely on it, imagine how much accessibility we can give those patients, so they can practice at least in their homes or even with their friends,” said Mohammed Alshair, a doctoral candidate working on the project. “They can wear it or just look and know this person is getting frustrated, getting mad, getting angry, and over time, they get better.” 

They are working with doctors from the University of Texas Health Science Center to test the viability of such a program.

Yun has been working for several years to find ways to help those with Asperger’s or high functioning autism, previously by using Kinect-based games at the UH Clear Lake Center for Autism and Developmental Disabilities.

“The high functioning autistic patients or Asperger’s (patients) having problems recognizing social cue, like the facial expression,” Yun said. “So they fail conversing with other people, time after time after time. In the end, what happens is people, after enough defeat, they give up. Now we are providing a tool, so they can understand how other people are feeling, at least facially.”

Five years ago, a family with a son who has autism came to Yun. The son’s mother told Yun that teaching him to program would be more than enough — more so than earning a degree.

“That kind of do things on you, literally,” Yun said. “Seeing mother beg me to death. After then, just chances came by, and I met this faculty director leading UHCL’s Autism Center.”

MRI data typically come in slices with parts of the brain shown in each slice, Yun said. The team has an AR program that creates a 3-D hologram of the brain in the physical world, so doctors and physicians can walk around and view the hologram and any tumors or blood vessels connected to it as large as they want.

“Say a doctor needed to find a tumor in the lower portion of the brain. We can take that and just put a little color on that, it’ll pop straight out,” Holtkamp said. “Surgery is obviously something very serious at that point in time, so minimizing potential problems is a huge win.”

Yun said he is optimistic about how prevalent augmented reality will be 10 years from now with its ability to overlay more information on top of anything and as AR headsets become smaller and cheaper.

“This is like iPhone One. Nobody has concept of what smartphone is all about. Now everybody knows it’s an essential,” Yun said. “Ten years from now, this will be more like an essential. All of this information that you need to know and want to know, in addition. Have a imagination of what this type of device can do for you to better your living.”

Augmented reality is a new computing territory, and the HoloLens is the only mass market AR headset. Virtual reality has been around in some form for several years. The team is dealing with the initial problems and finding the solutions for AR to pave the way for those who want to build on it in the future.

“If enough people are impressed, screw it. We are going to make this work,” Alshair said. “I’m one of them. Annoying, yes, I’m cursing Microsoft everyday for like why, why, why. But at the end, I’m not choosing not to, because it’s worth the trouble.”

[email protected]