Deep Learning Based Real-time Object Detector for DJI Drones

Demonstration Video for Visual Detector

This project involves real-time object detection for DJI drones from the ground station using TensorFlow Object Detection API. The TensorFlow Object Detection API is an open source framework built on top of TensorFlow making it easier to construct, train and deploy object detection models. This results in machine learning models capable of localizing and identifying multiple objects in images streaming from DJI drones to the ground station with more computational power.

The ground station code is available at the GitHub repo to apply object detection for streaming videos from DJI drones.

Requirements

Configuration at DJI GO

  1. General Settings -> Select Live Broadcasting Platform -> Customer 
  • rtmp://<ip address>/live/djidrone
  • Be sure to substitute the IP address of your computer running docker for <ip address>

Command at the Ground Station

  1. python object_detection_multithreading.py --width=712 --height=400
  2. python object_detection_multithreadng.py Optional arguments (default value):
  • Device index of the camera --source=0
  • Width of the frames in the video stream --width=480
  • Height of the frames in the video stream --height=360

Thanks

Special thanks to  our sponsors and team members. It was a really cool endeavor and an unforgettable experience that is almost impossible to get anywhere else

Hazel Zhu and UAS Hobbyist Team

Views: 1280

Comment by Mathew Wright on September 25, 2018 at 8:14am

Hi Hazel, 

Cool project. I've recently done something similar on a TX2 with caffe instead of tensorflow; due to memory allocation issues with tensorflow.

What type of model were you running on the pc? Standard coco mobile net? Did you test max FPM at all? If so how many FPM were you able to process?

I'm guessing you didn't run into any bottle necks on your machine and if you did they were due to the GPU not the CPU? 

Good work here!

Comment by Hazel Zhu on September 25, 2018 at 9:38pm

Thanks for your comments. Below are a few collected data:

   Actual model used: ssd_mobilenet_v1_coco_11_06_2017

   Elapsed time (total): 285.38

   Approx. FPS: 16.81

   GPU usage: 13~26%

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

Season Two of the Trust Time Trial (T3) Contest 
A list of all T3 contests is here. The current round, the Vertical Horizontal one, is here

© 2018   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service