Facial Emotion Recongiton
- 0 Collaborators
This project aimed at developing a model capable of identifying basic human emotions in highly occluded facial images in real time. The developed model possesses rotational invariance and effective mapping of pose relations which are achieved through the use of Hinton's Capsule Networks. ...learn more
Project status: Under Development
Intel Technologies
Intel Integrated Graphics
Overview / Usage
The project is aimed at the development of a facial emotion recognition system that is capable of identifying human emotions in highly occluded facial images in real time. The model was developed earlier using the convolutional neural networks and was further improved by using Capsule networks that added the power of rotational invariance. The major problem with CNNs is that they cannot achieve rotational invariance which is important in this task as images with partially rotated faces or side faces are many. The idea that Change of the facial orientation does not change the classified emotion is guaranteed by rotational invariance which is achieved through Capsule Networks and its powerful vector representations. Facial Emotion Recognition systems find a lot of applications in the field of security, medical diagnosis, entertainment, etc.
Methodology / Approach
The main idea was to compare the performance of Convolutional networks and Capsule networks on the task of facial image data and classification of emotion. First, I implemented several CNN architectures of a varied number of layers and other hyper-parameters and tuned my model for the task of emotion recognition. The database considered throughout the project is FER2013 which consists of around 35000 grayscale images covering 7 basic emotions. Further, the CNN architectures were improved by trying the addition of efficient depth separable layers or making a fully convolutional layer model. The accuracy did rise by a certain percentage. But still, the model was limited to frontal full facial images and struggled against the side faces. So then I implemented a Capsule network-based emotion recognition system which achieved rotational invariance and achieved a significant upsurge in classification accuracy. I am still working on the model to make it more robust even on other known datasets.
Technologies Used
Several handcrafted algorithms like Haar Cascade, Fisher projection, etc. along with algorithms for hyper-parameter tuning were used. Several libraries of Python programming language like OpenCV, Tensorflow, Keras (Tensorflow API), numpy, sklearn, etc. were used for the accomplishment of this task.
The capsule network implementation was done in PyTorch because of some issues with Tensorflow-GPU.