Demonstration Video for Visual Detector
This project involves real-time object detection for DJI drones from the ground station using TensorFlow Object Detection API. The TensorFlow Object Detection API is an open source framework built on top of TensorFlow making it easier to construct, train and deploy object detection models. This results in machine learning models capable of localizing and identifying multiple objects in images streaming from DJI drones to the ground station with more computational power.
The ground station code is available at the GitHub repo to apply object detection for streaming videos from DJI drones.
Requirements
- ubuntu 16.04 LTS (Intel i7 CPU and GeForce GTX 1060)
- Nginx-1.7.5 and Nginx-rtmp-module
- Anaconda / Python 3.5
- TensorFlow 1.2
- OpenCV 3.0
Configuration at DJI GO
- General Settings -> Select Live Broadcasting Platform -> Customer
- rtmp://<ip address>/live/djidrone
- Be sure to substitute the IP address of your computer running docker for <ip address>
Command at the Ground Station
- python object_detection_multithreading.py --width=1280 --height=720
- python object_detection_multithreadng.py Optional arguments (default value):
- Device index of the camera --source=0
- Width of the frames in the video stream --width=480
- Height of the frames in the video stream --height=360
Thanks
Special thanks to our sponsors and team members. It was a really cool endeavor and an unforgettable experience that is almost impossible to get anywhere else
Hazel Zhu and UAS Hobbyist Team
Comments
Thanks for your comments. Below are a few collected data:
Actual model used: ssd_mobilenet_v1_coco_11_06_2017
Elapsed time (total): 285.38
Approx. FPS: 16.81
GPU usage: 13~26%
Hi Hazel,
Cool project. I've recently done something similar on a TX2 with caffe instead of tensorflow; due to memory allocation issues with tensorflow.
What type of model were you running on the pc? Standard coco mobile net? Did you test max FPM at all? If so how many FPM were you able to process?
I'm guessing you didn't run into any bottle necks on your machine and if you did they were due to the GPU not the CPU?
Good work here!