IIT Madras: Wearable devices with AI and ML for people with hearing & motor impairment

Wearable devices with latest sensor technologies that can assist people with hearing impairment and motor disabilities to communicate independently and enhance their quality of life — researchers at the Indian Institute of Technology (IIT) Madras are working Vibe and iGest for the hearing impaired and persons with motor disabilities,

Sony Pictures Networks India (SPNI) is supporting these projects at IIT Madras through its CSR initiatives.

The devices are being developed by the Centre for Rehabilitation Engineering and Assistive Technology (CREATE), a multidisciplinary translational research and educational initiative of IIT Madras. They were conceived as an outcome of interactions researchers’ interactions with NGOs and inclusive schools.

CREATE sets out to meet the social need for indigenous development of low-cost technologies that address the needs of people with different abilities in India.

Anil Prabhakar, head of CREATE and faculty, Department of Electrical Engineering, IIT Madras, highlighted the need for indigenous development of assistive devices.

“People with hearing impairment are often excluded from the mainstream and even inclusive education because of the non-availability of affordable and sustainable assistive devices and systems. Imported devices cannot be afforded by most people,” Prabhakar said.

“We have kept the cost of the product low at less than Rs 5,000 to make it an affordable device for its basic functionality. The advance of technology and the advent and availability of low-cost microcontrollers and sensors allows us to come up with this unique low-cost device,” he sai.d

The two major projects being developed by CREATE are ‘Vibe’ and iGest’ for the hearing impaired and for persons with motor disabilities, respectively. Both devices are embedded systems that will bring the latest developments from IoT and ML to wearable assistive devices. Such wearable devices will have rechargeable batteries and communicate with a mobile phone over Bluetooth.

Elaborating on the impact envisaged from these CSR projects, Mr. Rajkumar Bidawatka, Company Secretary, Compliance Officer and Head CSR, Sony Pictures Networks India, said, “Inclusivity is an essential aspect of the culture at Sony Pictures Networks India (SPNI). We are proud to extend our support to IIT Madras for this project, which is an effort to ensure that the needs of differently-abled communities are addressed in the most convenient, affordable and accessible manner. At SPNI, our aim is to support initiatives that result in a tangible impact to communities who benefit the most from such innovations.”

Both Vibe and iGest are embedded systems that will bring the latest developments from IoT and ML to wearable assistive devices. They will have rechargeable batteries and can be used to communicate with a mobile phone over Bluetooth.

Vibe is a compact, wearable device that vibrates for acoustic sounds around a person with hearing impairment. It features a multitude of sound patterns that are recognised using a microphone and voice recognition modules. Vibe will alert a person with hearing impairment about a specific sound such as a doorbell, alarm, or a crying child. It is a simple way of providing vibration input for pre-identified surrounding sounds, with each such sound corresponding to a specific vibration and blinking LEDs to alert the user.

iGest will function as an alternative and augmentative communication device for persons with cerebral palsy. It will recognise the gestures of those with limited motor skills and convert them into audio output through a smartphone. It aims to address issues of speech impairment and motor impairments faced by persons with cerebral palsy.

iGest, which borrows on commercially available fitness sensors, will be designed using an inertial motion unit. For persons with cerebral palsy, movements can be much slower than normal people and also less repetitive. Hence, iGest will be designed around available Edge ML microcontrollers that provide machine learning (ML) capabilities to IoT devices.