Overview / Usage

The impact of the COVID-19 pandemic has caused a global economical recession, and Latin American countries are not an exception. However, even worse, low unemployment rates in this region have caused a dramatic rise in delinquency and crime. Day by day, Latin Americans have a high perceived level of insecurity, as they do not only fear becoming victims of homicide but also of other common crimes, such as assault or rape. For example, thieves are used to using motorcycles to wait and assault people in public places or when they are opening the door of their house. On the other hand, there is a common belief that police forces are corrupt and slow to respond to emergencies (usually more than 1 hour). This has been the motivation for many entrepreneurs to start their private security companies, which in turn demand new monitoring and surveillance tools to offer a prompt service.

With SafetyTracker, we aim at proposing a stereo vision surveillance system that monitors an area and notifies the authorities (public or private) when there is potential assault or presence of guns/white arms. Currently, the surveillance device is connected to a central web platform, which helps authorities monitor multiple areas in several georeferenced places in real-time. The end device consist of an OAK-D camera, a Jetson Nano, the Intel Movidius Neural Compute Stick, and a 3D printed case. Going forward we plan to install and offer the surveillance service as a monthly payment service. The system considers different types of notifications such as text messages through Telegram, visual notifications, and sound alerts.

Methodology / Approach

Although technology is a significant part of the SafetyTracker value proposition, what makes our project unique is its approach to provide real-time tools to the final users. Our solution monitors a place and calculates the risk of assaults continually based on a sudden change of the distance between people and the presence of guns/white arms. Once the risk overpasses a threshold, the end device sends notifications to the respective authorities. Currently, we have four alert channels: by sending messages with a chatbot on Telegram, by activating an audible alert, by showing the risk in a display integrated into the embedded device, and finally, by displaying the risks and calculations in real-time on a web platform. This also allows authorities to monitor several areas in multiple georeferenced places through a responsive web interface.

Our approach can be summarised as follows:

1. Design and implement the end device with SolidWorks and 3D printing.
2. Write a web crawler that gathers data of people, guns, white arms from google images.
3. Label our dataset and create .xml file corresponding to each image.
4. Create train and test data directories. Create a script for generating the train.csv and test.csv for the data. Generate the train.record and test.record files.
5. Train/test a model for people detection and another model for gun/white arm detection with Tensorflow on Intel DevCloud.
6. Optimize both models with OpenVino to create the *.xml and *.bin files for deployment on the OAK-D and Jetson Nano.
7. Inside the OAK-D and Jetson Nano, recognise people and calculate the crime risk according to depth/distance maps on the Oak-D. Send/show notifications when the risk is higher than the threshold.
8. Develop a central platform for data collection and real-time monitoring with MeanJS.

Since a sudden distance change between people may mean not mean an assault, we are currently collecting stereo vision data with the OAK-D for body pose tracking, so that the system can detect precisely crime behaviour. We have also moved some components to Movidius NCS for better inference performance per frame.

Finally, it is essential to mention that stereo vision with OAK-D is more precise for distance calculation than regular 2D images, mainly when the scene has occlusions.

Technologies Used

Hardware

• Intel Movidius NCS
• OAK-D AI Kit
• Jetson Nano
• 3D printing

Software

• Intel DevCloud
• Intel OpenVino
• MeanJS
• TensorFlow 2.0
• Telegram Bot API