Overview / Usage

Image analysis has the potential to transform a wide variety of industries - from insurance companies to law enforcement to self-driving cars. However, understanding the content of videos is particularly difficult as the context of an individual frame in a video depends on the series of frames both prior and following. Despite an overwhelming need to automate video analysis, few techniques exist to optimize and analyze these data. Here, I present a use case where I apply a hierarchical recurrent neural network technique to detect whether or not dashboard camera footage contains a car accident. This approach uses a recurrent neural network to deconstruct the content of each individual video and its constituent time-dependent sequence of images. An additional recurrent neural network is used to learn patterns across many video examples. I built a dataset by scraping YouTube videos of dashboard camera footage and complemented this with video from VSLab, an academic research collaboration. The dataset consists of over 250 unique video clips (>25,000 individual frames), balanced between positive car crash and negative examples. The model performs with greater than 80% accuracy on a holdout test set of 52 videos and is parallelized for implementation at scale. This tool allows organizations to sift through millions of hours of video and pick out video segments that are relevant and important to their mission. The hierarchical recurrent neural network can be generalized to automate processing and analysis of video footage to be of immense value in other industries and applications. See crashcatcher.site and https://github.com/rwk506/CrashCatcher for more.