USING MM-WAVE RADAR AND AI FOR DEVELOPMENT OF NEW GESTURE LANGUAGE / GESTAR

Abstract

Gestures are the primary actions that manifest various human expressions and these manifestations are independent of age, gender, and health conditions. Considering the importance of gestures for communication with the surrounding environment, automatic detection of gestures using the intelligent electronic system is an interesting domain having application in diverse fields. As a fundamental use case, a “touchless“ system is an important application of the intelligent electronic gesture recognition approach. A touchless system is effective to communicate with a machine without having physical contact with it, a possible way to avoid the spread of viruses and bacterias such as COVID-19. Depending on the ambient conditions, the SARS CoV-2 virus has been found alive for hours or even days on surfaces, creating a huge risk of transmission through touching of these fomites followed by touching the mouth, nose, or eyes. In public places, these surfaces could range from elevator controls, doorknobs, banks, hospitals & restaurant service counter machines, ATMs, point-of-sale (POS), sanitizing and vending machines, and many more.

To prevent surface transmission, touchless sensing and operation of devices in the above scenarios through micro-gesture recognition (MGR) is a need of the hour. Therefore, this project aims to utilize the latest mm-wave radar technology at 60GHz and augment it with the power of AI to firstly develop a library of micro-gestures and secondly to integrate it with some public use devices to demonstrate touchless sensing.

Primary Goals

The project output is in the form of a production-ready prototype with the following objectives:

    1. To develop a hardware-software framework for obtaining radar data.
    2. To formulate a library of unique gestures suitable for detection using mm-wave radar.
    3. To develop a machine learning pipeline for gesture recognition and classification.
    4. To integrate the radar hardware-software system with few public use devices.
    5. To test and customize the product for the use-cases of public interest such as touchless systems and assistive living devices for blind people.
    6. To demonstrate a working system to potential customers at an open-day.
    7. To submit research papers in conferences and journals.

Principal Investigator

Co-Principal Investigators

Dr. Hammad M. Cheema
hammad.cheema@rimms.nust.edu.pk

Industrial Collaborator(s)

Company: Renzym (Pvt.) Ltd

Name of Person: Yasir Javed

yasir@renzym.com

Co-Funding Provided (if any):  Rs 0.5 Million in-kind

Scientific Field(s)

Artificial intelligence, radar-based monitoring, gesture recognition

Duration

12 months

Funding

6 Million PKR