Story |

AI for Good

Story by
Julio Kongyu

Julio Kongyu

Vice-President of Huawei's European Public Affairs and Communication Office

There’s been a lot of talk about the problems Artificial Intelligence might cause “if let loose” in the workplace and home, or in terms of personal data and privacy. The potential for AI to improve society, however, is really quite breathtaking. This is why Huawei has decided to highlight how AI can be used to do good, beginning with these apps to improve the everyday lives of people who are visually or hearing impaired.

Huawei’s Facing Emotions app, which can be downloaded from Google Play and Huawei’s own AppGallery, helps visually impaired people ‘see’ emotions through the power of sound.

The debut of Facing Emotion follows Huawei’s StorySign, another app which taps AI tools, such as image recognition and optical character recognition (OCR), to translate popular children’s books into sign language.

Such apps are only the first to provide evidence of the marvellous potential for AI to work for the better of society and which are being highlighted in “AI for Good”, a global campaign by developers to promote the far-reaching benefits of the technology.

A new world with smiles

A simple smile is something we all take for granted. But imagine a world without smiles? It is the world of the visually impaired; unable to see faces and read emotions, they are often literally left in the dark. Now Huawei’s Facing Emotions app allows blind people to ‘see’ or experience emotions, on the faces of people they are speaking to, using the power of AI.

Facing Emotions uses the powerful camera and AI of the HUAWEI Mate 20 Pro, to translate seven universal human emotions into seven unique sounds, making it possible for the blind and visually impaired to have a richer experience and understanding when communicating with others. Using the back cameras, the application scans the face of the person with whom the blind person is talking, and identifies elements of the face: eyes, nose, brows and mouth, along with their position in relation to each other. AI then processes the identified emotion into a defined sound heard on the phone (or in the earphone the blind person is wearing). All this happens in a real time and in offline mode. It’s only possible thanks to the powerful processor and AI that comes in the Mate 20 Pro.

To develop Facing Emotions, Huawei partnered with the Polish Blind Association and created a key group of blind testers who were involved in every key step of the app development process. We interviewed blind community representatives about their needs and based the app on those insights. It was the input of visually impaired people that helped shape the final user experience, as well as its functionalities, colours, and sounds.

Huawei used a blind composer to create the sounds

A key component of this process were the sounds themselves. They were created by blind composer Tomasz Bilecki to capture each of the seven emotions. The sounds had to be short, concise, simple and non-invasive for the users and their environments. Tomasz’s unique perspective on the nature of sound allowed him to convey the emotions to other visually impaired people.

And, during the app development process, we also learned that visually impaired people would ideally love a hands-free solution so that they would have easy access to it, since their hands are often used for navigating or holding a cane. As part of the campaign, award-winning designer Janek Kochanski developed a special 3D printed phone holder for the Mate 20 Pro that makes it easier for visually impaired people to use the app and their phone in general.

Working with deaf children

Meanwhile, StorySign leverages AI tools such as image recognition and optical character recognition (OCR) to translate popular children’s books into sign language, offering millions of early learners the chance to enjoy books before they are able to read standard written text themselves.

StorySign was developed by Huawei in collaboration with the European Union for the Deaf, publisher Penguin and animation specialists Aardman.

Using image recognition technology, StorySign is able to detect words even if the phone is positioned at an awkward angle in relation to the book, while optical character recognition enables greater accuracy. The app is available for download from Google Play and the Huawei AppGallery in 10 European markets.

With some 460 million people with disabling hearing loss globally, of whom 34 million are children, there is clearly a large demand for such “AI for Good” applications and great scope for them in general, to help create a better society.

See the StorySign video

Watch the short documentary film on Facing Emotions

*Huawei’s DigitALL Chinese New Year Reception in Brussels on 7 February will showcase such ideas for creating an inclusive digital future. Find out more