OpenMV and ArduPilot tracking a colored ball

OpenMV is my favorite computer vision camera, and it combines a built-in OpenCV-like machine vision library with a microPython interface, all in one little board for $65. It also speaks MAVLink, so Patrick Poirier was able to do the above with just these extra lines of code to the standard built-in color tracking example:

while(True):
clock.tick()
img = sensor.snapshot()
for blob in img.find_blobs([thresholds[threshold_index]], pixels_threshold=100, area_threshold=20, merge=True):
img.draw_rectangle(blob.rect())
img.draw_cross(blob.cx(), blob.cy())
send_landing_target_packet(blob, img.width(), img.height())
print("TRACK %f %f " % (blob.cx(), blob.cy()))

and then send the result to Pixhawk via MAVLink:


def send_landing_target_packet(blob, w, h):
global packet_sequence
temp = struct.pack("<qfffffbb",
0,
(((blob.cx() / w) - 0.5) * h_fov)/2.25,
(((blob.cy() / h) - 0.5) * v_fov)/1.65,
0, #int(z_to_mm(tag.z_translation(), tag_size) / -10),
0.0,
0.0,
0,
MAV_LANDING_TARGET_frame)

Views: 847

Comment by Gary McCray on April 23, 2017 at 6:15pm

Cool,

I can put a red ball on my hat and get it to follow me around, of course it seems to want to land there as well, that could be a problem.

Also probably don't want to use at Christmas, Rudolph shows up and there goes your drone.

Seriously, great result and super simple script.

You really can do a lot with not much Python.

Best,

Gary


Developer
Comment by John Arne Birkeland on April 24, 2017 at 1:59am

The OpenMV seem like a great tool for learning. But be aware that the limited resources means you will likely not be able to progress very far past academic computer vision tasks like the classic red ball tracking. The key for most 'real world' scenarios is to have optic flow and robust feature detection (blob has limited usefulness) running at high frame rates, to get smaller variation between frames resulting in more predictable tracking.

But I think the OpenMV can be a nice 'gateway drug' to get you started. :) Just be aware that pretty soon you will find yourself looking at much more expensive solutions like Nvida Tegra and friends.

Comment by Patrick Poirier on April 24, 2017 at 3:41am

Thanks Chris for posting my garage sessions :-)

Spent a couple of cold winter week-ends experimenting in what I called the  "ThunderDrone".

Quad 450 + BBBMINI + PX4FLOW + LidarLite V3 and different sensors.

This OpenMV is really easy and fun to work with, thanks to kwagyeman & iabdalkader,  the IDE offers a lot of tools and features making it ideal for fast prototyping. The really cool stuff is that you can remotely edit scripts within OpenMV during tests and reflash and reboot the vision system without having to restart the flight controler-telecom-mission planner.

Next test will be using April Tags to do precision landing and start playing with "PokeTag" chasing. I have to thanks the ArduPilot developers as well for supporting and enhancing features.

This is an entry level system that I would recommend to anyone interested to machine vision.

John Arne, I suspect that Intel's Euclid, will be an interesting avenue as well.


3D Robotics
Comment by Chris Anderson on April 24, 2017 at 7:35am

John Arne, you might want to give OpenMV a closer look. It does everything you mentioned. It's got optic flow, feature detection (and it's got the best April Tag implementation I've tried), and about 30 other standard CV routines, and it runs at 30FPS.Here's a link to its libraries, which you can see cover a lot of the OpenCV territory. 

In my experience it performs at about the level of a RaspberryPi 3 running OpenCV, despite having less computing power, which is probably due to the optimized built-in libraries.


Moderator
Comment by Bill Piedra on April 24, 2017 at 1:32pm

Did you get your OpenMV camera yet?  I ordered mine more than a month ago but haven't heard anything.

Comment by Patrick Poirier on April 24, 2017 at 3:33pm

Hello Bill,

Yes I received it last week and its already flying ;-)


3D Robotics
Comment by Chris Anderson on April 24, 2017 at 5:31pm

Also note that JeVois, which is a similar integrated OpenCV board, is also now shipping for just $49. Lots of cool features, including optical flow and road tracking, built into that one, too. 

Comment by acchkr zhang on June 23, 2017 at 1:54am

Hi,

((blob.cx() / w) - 0.5) * h_fov)/2.25,
(((blob.cy() / h) - 0.5) * v_fov)/1.65,

 how to fix2.25 and 1.65?

I tried a apriltag as target,but it seems the angle is small for vehicle to the center,i changed some values,but no use,

thanks.

Comment by Patrick Poirier on June 23, 2017 at 4:37am

Hello,

For April Tags, you may change the lens for a wider angle (OpenMV store offers a IR filtered Wide Angle). 

You can reduce the TAG size as well or you can build a "tag mosaic"  like Fnoop as done with Aruco Tags

shown here: 

Hello,

For April Tags, you may change the lens for a wider angle (OpenMV store offers a IR filtered Wide Angle). 

You can reduce the TAG size as well or you can build a "tag mosaic"  like Fnoop as done with Aruco Tags

shown here: https://github.com/fnoop/vision_landing

Good Luck an lease report progress ... or fail, that is a different, but necessary form of progress ;-)

Comment by acchkr zhang on June 23, 2017 at 5:30pm

@Patrick Poirier,

Thanks a lot,I'll try it.

Comment

You need to be a member of DIY Drones to add comments!

Join DIY Drones

© 2017   Created by Chris Anderson.   Powered by

Badges  |  Report an Issue  |  Terms of Service