I made this video by running the video through RoboRealm and using the optical flow module. I captured the x,y vector variables and overlayed them, as well as the flow vector. This software could on a fast machine rx video and output these vectors back to a UAV. The idea is that the vector direction and length gives an idea of the rate of pitch and roll. Watch the video and tell me what you think of the idea.

Views: 472

Attachments:

Reply to This

Replies to This Discussion

Optical flow is indeed very cool and has been used on quite a few indoor MAVs. I don't know whether it's ever been used on a n outdoor/bigger one, however. Might have something to do with unpredictability of the magnitude of the flow due to uncertain distance to objects, and thus control issues, but I don't know.

We were planning to use optical flow with an optical mouse chip on our blimp, but got sidetracked:
http://diydrones.com/profiles/blogs/705844:BlogPost:35372
That is a promising approach. Looks like RoboRealm is computing perhaps 6 frames/second @ 320x240. It should be possible to do this onboard with an adequate processor - I have optimized motion estimation code for the Blackfin processor that I extracted from an MPEG encoder that should be able to compute motion vectors in a similar way, and have integration of this code one of my TODO list items, though it's been on the list for over a year.

Mouse sensors have been used quite effectively outdoors for obstacle avoidance and automated landing. In particular, BYU has done some nice work in this area, e.g.
http://video.google.com/videoplay?docid=-1952814402978315740
For technical papers, do a google search on Griffiths, Beard and BYU
nice, mmmm maybe a pre processing job like add contrast in the image, will return a much more clear image so software can detect sky and ground with less error.
Good point. There should be some computational less complex transforms than optical flow which would help in locating the horizon optically. A basic high pass filter is probably a good starting point - just run a basic edge detection filter with a low threshold, and the areas with lots of noise probably correlate well with the ground regions, while low noise areas correlate with the sky regions,
thats what i get just clicking in the greyscale button, and then adding lot of contrast
quite similar to what an thermopile see itself,


Not certain exactly how you would do this in RoboRealm, but let me suggest the following -
1. turn off the optical flow
2. use a histogram to establish the mean brightness
3. use image segmentation, thresholding the image so everything above the mean is white and everything below the mean is black (similar to what you did with the contrast function)
4. display the results

This will be much faster to compute than optical flow.
Cool idea, but I think perhaps you need to fuse a few ideas:
use one computed frame from the difference of sky and ground with white and black for stabilization and knowing yaw and roll, and a second one using something more like what happened in the lower part of the screen with the roads and such - this way it can also pick out and follow or navigationally use the road direction if a map or directions are programmed.

I think the most interesting aspects of optical flow would be in crowded environments rather than high up. Making a small and slow plane be able to fly low between high buildings and such...
I'm still playing with this in roborealm(different video) and I found if I increase the size of the spacing and use diagonal axis, the vector reacts in sync with the motion of a plane better. Plus, the processing is much faster. The vector seems to be indicating a few things though. The real short jumpy vectors are when pitch and roll is absent, no objects near. The long stable vectors indicate some constant condition like a steady turn or steady climb or decent. When the long ones get jumpy, then there's a good chance there's an object. Sometimes vertical vectors are when wings are level and it's pitching. What seems important is how far off vertical or horizontal the vector is and what quadrant.

These are just observations. I'm going to try it on some other videos.
I downloaded the AVRstudio when I programmed the ATtiny45 on the Ardupilot. Every time I loaded that program it mentioned WinAVR plugin was missing, so I went looking for that. It's a suite of compiling tools for AVR. I then went and looked for some stuff on optical flow and found a project which uses an ATmega164p and included source code. I managed to fumble around and finally got AVRstudio to compile the code without errors. I got a whole mess of warnings which I dont understand, but no errors.

Here is the link:
http://open.grcnz.com/trac/low-cost-altimetry
The BYU UAV folks, as mentioned above, seemed to be get better results with the mouse sensors than the grcnz.com experiment. However, I suspect the full frame video approach is still the right way to go.
Really cool stuff! Does it compute rotation (curl) as well as translation? I imagine that you would need the rotational measurement for roll and translation for pitch and yaw.
The link above has a manual which says it will output Raw mouse motion messages. The original project is meant as an altimeter, the meat of it is that they interfaced an optical mouse. Cutting the code down so it only does what's needed is what I'm after, but I'm functionally code illiterate at the moment.

RSS

Contests

Season Two of the Trust Time Trial (T3) Contest has now begun. The third round was a reliablilty/aerial photography round for both planes and copters, which is now closed. Stay tuned for the next round, beginning soon.

A list of all T3 contests is here

 

© 2012   Created by Chris Anderson.

Badges  |  Report an Issue  |  Terms of Service