A face to face conversation contains a lot more than words. Studies show that as much as 65 percent of the information exchanged in a normal conversation is non-verbal. So which senses are used instead? Certain information about mood and nuances in the language can be perceived through the tone of voice and vocal pitch, but when it comes to the ability to read and interpret another person’s emotion, eye sight is invaluable.
”My goal, what I want to do, is to help visually handicapped people take active part in society.” This says Shafiq ur Réhman, a doctoral student working on the project Tactile Video –Computer and camera become eyes for the blind, at Umeå University. The objective of the project is to develop an advanced system where visual information from a camera is transformed to emotional Braille code so blind and visually impaired can perceive the part of social life they are now missing.
The camera reads the facial expressions and transfers the information to the visually handicapped person through tactile stimulation, for example through the backrest of a chair.
“Being able to see, we probably don’t realize how dependent we are of our own and other people’s emotions” says Réhman, “it is natural for us to look the person we are talking to in the eyes and if they start to look away or yawn we understand that they are losing interest and stop talking. A blind person will not get the unspoken confirmation that someone is listening or paying attention and that is a huge social handicap.”
The project just started and is supposed to be running for six years, but Réhman says some prototypes are already finished.
“We have a number of products that are almost finished, but then there is always room for development and improvements within everything from software and electronics to sensors and the actual chair.”
For the time being, the work is focused on camera modules.
“We want to create a miniature camera that can fit in a pair of regular size glasses” says Réhman, “the camera used today makes the glasses huge.”
He also says they are working on a type of jacket which has the sensors sowed into the back, so you will not have to depend on sitting in the chair. There is also a handheld camera, resembling a flash light, which the visually handicapped will point at the person they are talking to. Information from the camera will then be received through pulses from a plate in the handle.
The camera can identify six different emotions, happiness, sorrow, anger, disgust, fear and surprise. Réhman explains that the system will correct itself, should there be uncertainties, that the advanced algorithms behind the technique will compensate any errors that may occur.
Certain tests have been made, but not yet with blind or visually impaired.
“They tend to be restrictive when it comes to new technology and aids” says Réhman “they want to be part of society, not stand out like lab rats with ungainly aids that draw attention. That’s why we want to develop the technique first and offer it to them later.”
In spite of the positive results, the project is facing a number of challenges. One is how to transform the visual information from the camera to tactile signals without losing any of the details. Another problem is the dime delay, you want the computer to work in real time so the visually handicapped will perceive the facial expressions right away. Also, a code system for emotions must be developed.
No easy tasks, but Shafiq ur Réhman is as positive as he is passionate about his work.
“Emotions are tremendously important to verbal communication, this technique can change the lives of plenty of people” he concludes.
CMTF, c/o Tillämpad fysik och elektronik
Umeå universitet, 901 87 Umeå
Tel: +46 (0)90-786 96 38.
E-post: Britt Andersson.
