Overview / Usage

• Overview
  Measure the distance to the object with RealSense D435 while performing object detection by MobileNet-SSD(MobileNetSSD) with RaspberryPi3 boosted with Intel Neural Compute Stick.
  "USB Camera mode" can not measure the distance, but it operates at high speed.
  And, This is support for MultiGraph and FaceDetection, MultiProcessing, Background transparentation.
  And, This is support for simple clustering function. (To prevent thermal runaway)
  Performance measurement result each number of sticks. (It is Detection rate. It is not a Playback rate.)
  The best performance can be obtained with QVGA + NCS1 x5 Sticks or NCS2 x2 Sticks.
  However, It is important to use a good quality USB camera.
  　
• Environment
  • RaspberryPi3 + Raspbian Stretch (USB2.0 Port) or RaspberryPi3 + Ubuntu Mate or PC + Ubuntu16.04
  • Intel RealSense D435 (Firmware Ver 5.10.6) or USB Camera or PiCamera
  • Intel Neural Compute Stick v1/v2 x１piece or more
    • OpenCV 3.4.2 (NCSDK)
    • OpenCV 4.0.1-openvino (OpenVINO)
  • VFPV3 or TBB (Intel Threading Building Blocks)
  • Numpy
  • Python3.5 (Only MultiStickSSDwithRealSense.py is multiprocessing enabled)
  • NCSDK v2.08.01 (It does not work with NCSDK v1.　v1 version is here)
  • OpenVINO R5 2018.5.445
  • RealSenseSDK v2.16.5 (The latest version is unstable)
  • HDMI Display
    　
• Usage
  https://github.com/PINTO0309/MobileNet-SSD-RealSense#firmware-update-with-windows-10-pc
  https://github.com/PINTO0309/MobileNet-SSD-RealSense#1ncsdk-ver-not-compatible-with-ncs2
  https://github.com/PINTO0309/MobileNet-SSD-RealSense#2openvino-ver-corresponds-to-ncs2
  　
• Execute the program
  https://github.com/PINTO0309/MobileNet-SSD-RealSense#execute-the-program
  　
• Future tasks
  • Docker compatible
  • multiple models
  • Individual identification

Methodology / Approach

RaspberryPi3(Raspbian Stretch) or Ubuntu16.04/UbuntuMate + Neural Compute Stick(NCS/NCS2) + RealSense D435(or USB Camera or PiCamera) + MobileNet-SSD(MobileNetSSD)

• MultiProcess
• MultiThread
• MultiRequest
• MultiModel (VOC + FaceDetection)
• MultiClustering

Technologies Used

• MobileNet-SSD
• Realsense D435 (Depth Camera)
• USB Camera
• Semantic Segmentation
• OpenVINO R5

Repository

https://github.com/PINTO0309/MobileNet-SSD-RealSense

Other links

• Boost RaspberryPi3 with Neural Compute Stick 2 (1 x NCS2) and feel the explosion performance of MobileNet-SSD　(If it is Core i7, 21 FPS)
• Detection rate approx. 30FPS RaspberryPi3 Model B(plus none) is slightly later than TX2, acquires object detection rate of MobilenetSSD and corresponds to MultiModel VOC+WIDER FACE
• Intel also praised me again ヽ(ﾟ∀ﾟ)ﾉ Yeah MobileNet-SSD(MobileNetSSD) object detection and RealSense distance measurement (640x480) with RaspberryPi3 At least 25FPS playback frame rate + 12FPS prediction rate
• [24 FPS, 48 FPS] RaspberryPi3 + Neural Compute Stick 2, The day when the true power of one NCS2 was drawn out and "Goku" became a true "super saiya-jin"
• [24 FPS] Boost RaspberryPi3 with four Neural Compute Stick 2 (NCS2) MobileNet-SSD / YoloV3 [48 FPS for Core i7]

Collaborators