Overview / Usage

In 2016, 3,450 people in the US alone were killed due to distracted drivers. In 2015, 3,477 people were killed and 391,000 were injured. If you had a say in the matter, what would you hope the number of casualties to be this year? 1,000? 500? 2,000? If I said, that in 2018, there will only be 3 deaths due to distracted driving, what would you say? That’s great? That it sounds impossible? What if those 3 people were your family members? When it’s personal, the only number that should be acceptable is 0.

Methodology / Approach

To solve the problem described above I modified an Android launcher to use three sensors (capacitive matrix, rear phone camera, and GPS) to determine in which of the three areas of the car the user is located. The Android Launcher that I modified, KISS, is an open source launcher.

Once the Android launcher is instantiated, it automatically takes pictures every 10 seconds using the rear camera without user interaction. The images are then forwarded to the local Tensorflow installed in the phone, then are classified into a set of pre-trained categories. If the accuracy of the images is below 80% or the system has taken more than 10 pictures, all of them with an accuracy over 80%, then the pictures are forwarded to the Azure IoT hub. A set of three serverless functions then classifies the images and sends the answer back to the end user.

Once the system receives the results from the cloud, it compares both Tensorflow results: the ones obtained from the local Tensorflow, and the ones obtained from the cloud. If there are any discrepancies, the result defaults to those from the cloud since the Tensorflow model has higher accuracy than the one in the phone.

Once the system has figured out the physical location of the user inside the car, the capacitive matrix obtains the number of fingers on the back of the phone and then either enables or disables certain apps of the phone. If the user tries to open one of the distracting applications before the results from the cloud are received, the application will solely rely on the results from the phone. Once the results from the cloud are received, if they are different than those of the phone, and the phone had allowed the application to be opened, the application will now be killed due to the initial false positive.

The are two reasons why I decided to have two Tensorflow models (one in the cloud and one on the phone). The first reason is that the Tensorflow model on the phone uses a very lightweight algorithm that can provide results very quickly, which are needed in order to make decisions on enabling or disabling applications. The downside is that the accuracy is lower, which introduces false positives. This means that in some cases the applications would be incorrectly enabled for the driver or disabled for the passengers. To reduce the number of false positives, I used the Azure serverless functions to run a much slower yet much more accurate algorithm (table below). The second reason why I wanted to have two Tensorflow models was so I could reduce the amount of data that is sent over the network and not overuse the user's data plan.

Technologies Used

Tensorflow, Android, Arduino and Intel Powered PC

Repository

https://github.com/egibert/TextAndDrive

Other links

• Azure IoT on Serverless Hackathon Winner