Machine-vision prototyping with HackflightSim + OpenCV

This video shows a test flight of the HackflightSim quadcopter flight simulator, with a new feature: a simulated "companion board" (Raspberry Pi, BeagleBone, ODROID) running machine vision code written in Python with the OpenCV library. This software allows rapid prototyping of machine-vision algorithms before deploying and testing them on an actual vehicle. In this simple demonstration, we are running a threshold-based detector for blue components in the image: non-blue pixels appear as black in the detector, and blue pixels appear as white. Clicking the Play button on the simulator starts the simulation and then launches a Python script that reads successive images from a simulated belly camera on the vehicle. By grabbing the image bytes directly from the V-REP simulator using a C++ plugin (rather than the slower vision sensor built into V-REP), we get an on-the-fly resizable camera image and realistic update rates of 35-40 frames per second. The C++ firmware flying the simulated vehicle is the same as what we run on our actual quadcopters, and the Python code running the machine vision algorithm can likewise be deployed on an actual companion board with no modification.

Views: 1339

Comment by vorney thomas on July 8, 2016 at 6:29am

can you post some link or code to show the details of your work that be involved in machine vision and object detection?

Comment by Simon D. Levy on July 8, 2016 at 11:00am
Comment by Sergey Kirillov on July 8, 2016 at 1:24pm

Nicely done.

I'm doing very similar things with PX4 stack and builtin Gazebo/ROS simulation.

Comment by Simon D. Levy on July 8, 2016 at 1:58pm

Thanks, Sergey!  I'd love to see some videos :^)


You need to be a member of DIY Drones to add comments!

Join DIY Drones

© 2020   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service