Overview / Usage

Our machine learning model can take in a live video feed of people swimming in a swimming pool and can detect whether anyone is drowning and if so can identify where in the video the person who is drowning is located. We can then draw a boundary box around the location in which the drowning victim is and alert the person who is associated with the video feed about the drowning.

Methodology / Approach

We first had to build our own dataset of videos of people drownings as there were no available data sets that contained this action information. To do this we found all the videos we could that contained people drowning and then we isolated the action of drowning from the video using VoTT and iMovie. We also used actions of those who were not drowning as a separate class and these actions were taken from the same dataset from people who were in the same pool as those who were drowning. We then tested multiple models on microsoft azure on their data science virtual machine to try and correctly detect the action of drowning from a video until we landed on using a 3 dimensional convolutional net that was capable of analyzing spatiotemporal information. To implement this model we used the microsoft cognitive toolkit (CNTK) on the virtual machine. We used to send out the real time notification when a drowning was detected.

Technologies Used

Microsoft Azure, Microsoft CNTK, OpenCV, Tensorflow, Scikit-Learn

Repository

https://github.com/sswarnakar/LifeGuard.IO