Overview / Usage

Most of the studies on emotion recognition problem are focused on single-channel recognition or multimodal approaches when the data is available for the whole dataset. However, in some practical cases, data sources could be missed, noised or broken. The main goal is to find a solution for reliable recognition of emotional behavior when some data is unavailable. I describe the problem, present an emotion dataset and suggest a baseline solution for a given data. It is based on naive decision-level data fusion via recurrent neural networks (Long short-term memory, LSTM). I will classify 4-seconds intervals, for which features for missed data is replaced with zeros. Using above suggested approach demonstrated I will classify on a 6-class problem (angry, sad, disgusted, happy, scared and neutral state). I will also compare the performance of different sets of modalities.

Methodology / Approach

In this project, I will predict one emotion label for each frame in the whole video. The predictive model will be trained on estimated features for 4 modalities: voice, body data (Heartbeat and Respiration Rate), face, and eyes. A frequency of expected labels should be 100 frames per second​. To achieve this :
● I will implement ​Eulerian Video Magnification​ framework. This will help us extract ​Heartbeat and Respiration rate​.
● Extract other necessary features using best techniques.
● Implement Proposed Model (you can find it in the proposed model section below).
● Make an Interactive demo of the proposed project.
● If time permits I will also be making an API so that anyone can use our model in the future.

Technologies Used

Intel optimized libraries, Intel DevCloud, Movidius NCS

Repository

https://github.com/Ujjwal-9/Emotion-Recognition

Other links

• Link to Proposed Project [WIP]