Helen

This article is about idea we are working on and prototype demo we have presented at Intel India Innovation and career fair 2016. This idea helps to solve communication problem being faced by specially challenged peoples. 

Helen is an idea proposed by us to solve the communication problem being faced by blind, deaf and deaf mute people and make them enable to communicate with normal people.

Around Globe about there are about 70 million – Deaf and Deaf-mute and 39 million – Blind that gives us motivation to develop this product.

Our final proposed idea is wearable device for specially able people and it’s work is in progress but for prototype purpose we have an android app ready that can show the functionality that device will perform.

Objective

Communication is playing vital part in the personality of individual and best form of it is F2F communication. For f2f communication, what comes to mind first is words for communicating, voice quality, proper eye contacts while communicating etc. It is not the case for each and every human being. There are some special people who can’t communicate like other people with their natural senses like speech, hearing and seeing. Those special people communicate with their own sign language. For normal people it’s not easy to understand their language. Numbers saying that in the world there are approximately 70 million people in the category of deaf and deaf-mute, 39 million in the category of blind, same for India numbers are in millions. What a huge number! For deaf and deaf-mute it is difficult to understand without sign language when someone is communicating through speech and for blind people there are limited materials available in braille at common places. How much it is difficult for them to read something in daily life what we can read easily from different sources. We use to read newspaper, books daily from electronics media but for them it is not possible. Large number of special people face these difficulties daily and we have no proper solution yet! So we are proposing the affordable & reliable product that can help deaf, deaf-mute and blind people to communicate with normal world.

Approach

For difficulties faced by special people one device/medium is needed which can process or communicate on behalf of them.

1. Blind – Real time Image to Speech (Read Newspaper, Books, Surf Internet )
2. Deaf – Real time Speech to Text (Screen shows text of what speaker speaking)
3. Deaf-mute – Real time Speech to Text and Gesture to Speech (Camera that understands gesture and converts into words and speaks for them)

All these solution are available individually but are not being properly integrated to use reliably for them in daily life. So we are planning to make one common device that can perform this tasks reliably. 

Prototype

Our final product will be wearable specs with inbuilt mini camera, mic, speaker and computational device and we have started working on it, but for testing and to see usefulness and limitations of our idea, we have created one android application with similar features. 

Android application have three buttons each one for one for blind, deaf and deaf-mute person. We have implemented Speech to Text and Image to speech inside app and gesture recognizance we are still working. 

For speech to text recognizance we have used android’s default API functions that supports offline English speech recognizance and converts it to the text. The main challenge we are trying to solve is real-time translation of speech to text and ability to understand words from different language. 

 

Image to speech recognizance is implemented in app as a OCR (Optical Character Recognizance) and Text to speech converter.

For OCR we have used trained library provided by google that has ability to recognize English characters offline.

We are still working on gesture recognize and understand that gesture and convert it into speech, we have started work on that using openCV and python.

Idea Published by: Jigar J. Soni, Kruti H. Vadhavaniya

Download Android App here