I made this video by running the video through RoboRealm and using the optical flow module. I captured the x,y vector variables and overlayed them, as well as the flow vector. This software could on a fast machine rx video and output these vectors back to a UAV. The idea is that the vector direction and length gives an idea of the rate of pitch and roll. Watch the video and tell me what you think of the idea.
Harry,
you ever see one of the oil/water wave machines ? rectangular clear tube with blue oil and water that rocks end to end making wave forms in the tube .
My PhD thesis, which I just finised writing, discusses ways of controlling aircraft with vision, and in particular optic flow. We demonstrated fully autonomous flights, including take off, landing, altitude regulation, attitude stabilisation and obstacle avoidance with a 400-g flying wing, using only 7 mouse sensors, two gyros and a diff pressure sensor. A video of some of this is available here:
Really cool stuff! Does it compute rotation (curl) as well as translation? I imagine that you would need the rotational measurement for roll and translation for pitch and yaw.
Cool idea, but I think perhaps you need to fuse a few ideas:
use one computed frame from the difference of sky and ground with white and black for stabilization and knowing yaw and roll, and a second one using something more like what happened in the lower part of the screen with the roads and such - this way it can also pick out and follow or navigationally use the road direction if a map or directions are programmed.
I think the most interesting aspects of optical flow would be in crowded environments rather than high up. Making a small and slow plane be able to fly low between high buildings and such...
nice, mmmm maybe a pre processing job like add contrast in the image, will return a much more clear image so software can detect sky and ground with less error.
That is a promising approach. Looks like RoboRealm is computing perhaps 6 frames/second @ 320x240. It should be possible to do this onboard with an adequate processor - I have optimized motion estimation code for the Blackfin processor that I extracted from an MPEG encoder that should be able to compute motion vectors in a similar way, and have integration of this code one of my TODO list items, though it's been on the list for over a year.
Mouse sensors have been used quite effectively outdoors for obstacle avoidance and automated landing. In particular, BYU has done some nice work in this area, e.g. http://video.google.com/videoplay?docid=-1952814402978315740
For technical papers, do a google search on Griffiths, Beard and BYU
Optical flow is indeed very cool and has been used on quite a few indoor MAVs. I don't know whether it's ever been used on a n outdoor/bigger one, however. Might have something to do with unpredictability of the magnitude of the flow due to uncertain distance to objects, and thus control issues, but I don't know.
Replies
you ever see one of the oil/water wave machines ? rectangular clear tube with blue oil and water that rocks end to end making wave forms in the tube .
http://lis.epfl.ch/research/projects/microflyers/videos/optiPilot_o...
More will be published soon.
http://diydrones.com/profiles/blogs/software-based-horizon-finder
tehre are a lot of videos too,
http://www.mil.ufl.edu/~nechyba/mav/
use one computed frame from the difference of sky and ground with white and black for stabilization and knowing yaw and roll, and a second one using something more like what happened in the lower part of the screen with the roads and such - this way it can also pick out and follow or navigationally use the road direction if a map or directions are programmed.
I think the most interesting aspects of optical flow would be in crowded environments rather than high up. Making a small and slow plane be able to fly low between high buildings and such...
quite similar to what an thermopile see itself,
Mouse sensors have been used quite effectively outdoors for obstacle avoidance and automated landing. In particular, BYU has done some nice work in this area, e.g.
http://video.google.com/videoplay?docid=-1952814402978315740
For technical papers, do a google search on Griffiths, Beard and BYU
We were planning to use optical flow with an optical mouse chip on our blimp, but got sidetracked:
http://diydrones.com/profiles/blogs/705844:BlogPost:35372