Real-time object detection by combination of depth camera and VPU, implementation of high-speed transparentation and distance measurement

Katsuya Hyodo

Katsuya Hyodo

Nagoya, Aichi

RaspberryPi3(Raspbian Stretch) or Ubuntu16.04/UbuntuMate + Neural Compute Stick(NCS/NCS2) + RealSense D435(or USB Camera or PiCamera) + MobileNet-SSD(MobileNetSSD) - MultiProcess - MultiThread - MultiRequest - MultiModel (VOC + FaceDetection) - MultiClustering ...learn more

Project status: Under Development

RealSense™, Internet of Things, Artificial Intelligence

Groups
Internet of Things, DeepLearning, Movidius™ Neural Compute Group

Intel Technologies
OpenVINO, Movidius NCS

Code Samples [1]Links [5]

Overview / Usage

  • Overview
    Measure the distance to the object with RealSense D435 while performing object detection by MobileNet-SSD(MobileNetSSD) with RaspberryPi3 boosted with Intel Neural Compute Stick.
    "USB Camera mode" can not measure the distance, but it operates at high speed.
    And, This is support for MultiGraph and FaceDetection, MultiProcessing, Background transparentation.
    And, This is support for simple clustering function. (To prevent thermal runaway)
    Performance measurement result each number of sticks. (It is Detection rate. It is not a Playback rate.)
    The best performance can be obtained with QVGA + NCS1 x5 Sticks or NCS2 x2 Sticks.
    However, It is important to use a good quality USB camera.
     
  • Environment
    • RaspberryPi3 + Raspbian Stretch (USB2.0 Port) or RaspberryPi3 + Ubuntu Mate or PC + Ubuntu16.04
    • Intel RealSense D435 (Firmware Ver 5.10.6) or USB Camera or PiCamera
    • Intel Neural Compute Stick v1/v2 x1piece or more
      • OpenCV 3.4.2 (NCSDK)
      • OpenCV 4.0.1-openvino (OpenVINO)
    • VFPV3 or TBB (Intel Threading Building Blocks)
    • Numpy
    • Python3.5 (Only MultiStickSSDwithRealSense.py is multiprocessing enabled)
    • NCSDK v2.08.01 (It does not work with NCSDK v1. v1 version is here)
    • OpenVINO R5 2018.5.445
    • RealSenseSDK v2.16.5 (The latest version is unstable)
    • HDMI Display
       
  • Usage
    https://github.com/PINTO0309/MobileNet-SSD-RealSense#firmware-update-with-windows-10-pc
    https://github.com/PINTO0309/MobileNet-SSD-RealSense#1ncsdk-ver-not-compatible-with-ncs2
    https://github.com/PINTO0309/MobileNet-SSD-RealSense#2openvino-ver-corresponds-to-ncs2
     
  • Execute the program
    https://github.com/PINTO0309/MobileNet-SSD-RealSense#execute-the-program
     
  • Future tasks
    • Docker compatible
    • multiple models
    • Individual identification

Methodology / Approach

RaspberryPi3(Raspbian Stretch) or Ubuntu16.04/UbuntuMate + Neural Compute Stick(NCS/NCS2) + RealSense D435(or USB Camera or PiCamera) + MobileNet-SSD(MobileNetSSD)

  • MultiProcess
  • MultiThread
  • MultiRequest
  • MultiModel (VOC + FaceDetection)
  • MultiClustering

Technologies Used

  • MobileNet-SSD
  • Realsense D435 (Depth Camera)
  • USB Camera
  • Semantic Segmentation
  • OpenVINO R5

Repository

https://github.com/PINTO0309/MobileNet-SSD-RealSense

Collaborators

There are no people to show.

Comments (0)